Token-Smart Agents: Self-Editing Memory, History Compaction (Part 2)

In the previous article, we gave a gentle introduction using simple examples to explain the core idea behind self-editing memory. Now we'll build a production-ready AI agent that:

Uses AdalFlow for multi-step tool use
Maintains persistent memory across multiple conversations/sessions
Automatically summarizes long histories to avoid prompt bloat
Exposes memory tools (remember/recall/jot/counter) your model can call

Architecture Overview

A Python project with:

JSONMemoryStore: Thread-safe, file-backed memory store
HistoryCompactor: Auto-summarizes long chats into a long-term session summary
Memory-aware system persona injection on every agent run
A set of tools that read/write memory
An AdalFlow Agent + Runner driving the tool-calling loop

The Memory Store

We want agent knowledge to outlive the Python process and be safe under concurrent access.

Key ideas:

short_term: injected verbatim into the prompt (fast recall, higher token cost)
long_term: durable facts + an evolving summary created from older history (compact)
global: rare, cross-session settings (e.g., shared counters, feature flags)

History Compaction

When the chat history grows beyond a threshold (e.g., 18 turns), summarize everything except the last few messages using a light model.

Save that summary to long_term.summary, and keep only the last few turns in memory.

This approach gives the agent a kind of semantic spine—a long-term understanding of what's been happening—while keeping recent messages word-for-word.

System Persona Injection

Each run, you render a snapshot of memory into the system message:

GLOBAL MEMORY (rare, shared settings)
SESSION LONG-TERM SUMMARY (dense history)
SHORT-TERM FACTS (current working memory)
LONG-TERM FACTS (durable preferences/config)
SESSION_ID (current session id)

Memory Tools

The model never updates memory implicitly—it must call a tool. That makes state changes auditable and policy-guarded.

remember(session_id, key, value, scope): write a fact/preference
recall(session_id, key): fetch a fact
jot(session_id, note): append a note (free-form)
counter(session_id, op): example of global shared state

The Agent + Runner

Agent: knows which tools exist and how to call the model
Runner: executes the multi-step loop: think → decide tool → call → observe → continue → final answer

Example Session

agent_step("Please remember my favorite model is gpt-4.", session_id="A")
# 🤖 Agent: Your favorite model, gpt-4, is stored in long-term memory.

agent_step("What is my favorite model?", session_id="A")
# 🤖 Agent: Your favorite model is gpt-4.

agent_step("Set my locale to en_US", session_id="B")
# 🤖 Agent: The locale is set to en_US.

agent_step("What is my favorite model?", session_id="B")
# 🤖 Agent: Your favorite model is not stored in the memory.

Each session is like a separate notebook—session A has the favorite model, session B only has the locale.

Summary

This article turns an LLM into a smart, self-learning agent:

AdalFlow plans multiple steps and uses tools
File-backed memory keeps short-term separate from long-term
History compaction automatically summarizes old messages
One system persona keeps the model on track