Using LongMemEval to Improve Agent Memory

Sam Bhagwat, the co-founder and CEO of Mastra, discusses the LongMemEval benchmark for agent memory and the process of optimizing Mastra's memory layers. He defines memory as the compression of chat messages and the ability to search them effectively. Sam explains the subtasks within memory, including information extraction, multi-session reasoning, temporal reasoning, knowledge updates, and the ability to recognize missing information. He details Mastra's two main memory types: semantic recall and working memory, and how they were implemented and improved. Sam shares the initial benchmark results and the iterative steps taken to enhance performance, such as generating tailored templates, refining working memory updates, correcting date-related bugs, and restructuring data presentation. The improvements led to state-of-the-art accuracy, demonstrating the importance of continuous evaluation and iteration in developing AI agent frameworks.