How do you manage AI agent memory in production?

01Two kinds of AI agent memory, and a few more

LangGraph's persistence documentation draws a useful line. Short-term memory is handled by checkpointers, which save a thread's graph state. You select a thread by passing a thread ID in the configuration. Long-term memory, such as facts about a user that should survive across conversations, lives in a separate store. In a real product there are a few more layers around those two:

The last row matters most in established products. Anything that changes during the day should be fetched from the system of record when it is needed, not remembered from earlier in the conversation.

02Do not keep production memory in process

The LangGraph documentation is direct about in-memory savers: they keep checkpoints in RAM and lose all state when the process restarts. It recommends a persistent option such as a Postgres checkpointer for production. In a containerized deployment, processes restart all the time: on every deploy, on scaling events, on health check failures. An agent that forgets every open conversation on each deploy is not ready for customers.

There are also small details that only show up at scale. The same documentation notes that thread IDs should stay under 255 characters with the Postgres checkpointer because of a column limit. Build thread IDs from short, stable identifiers, such as a tenant key and a conversation key, rather than long strings assembled from user input.

03The connection layer is where it breaks

A checkpointer writes after each step of the graph, so a busy agent makes many short database writes. If those writes share a single long-lived connection, that connection will eventually be closed by the database, a proxy or an idle timeout, and the next write fails in the middle of a customer's conversation. To the user, the agent simply forgets what they said.

The psycopg connection pool documentation explains why a pool alone is not enough. The pool does not actively check its connections, so a broken connection can be handed to a client unless you configure a check. Connections that exceed a maximum lifetime are replaced, idle connections are closed after a set time, and failed connection attempts are retried with exponential backoff. Our checklist for a checkpointer in production:

1. 01

   Use a connection pool, not a single connection, and size it for real concurrency

   workers multiplied by concurrent graph runs.

2. 02

   Configure a health check when a connection is taken from the pool.

3. 03

   Set the maximum connection lifetime below the timeouts of your database, proxy and network.

4. 04

   Handle reconnects in the agent service, and make each graph step safe to retry.

5. 05

   Run the checkpointer's table setup during deployment, not on the first request.

6. 06

   Monitor pool wait time, failed checkouts and checkpoint write latency next to model latency.

7. 07

   Load test long conversations, not only single questions, because that is where connection and memory problems appear.

None of this changes the model. All of it changes whether the product works on a busy afternoon.

04Keep context small on purpose

More memory is not better memory. If every turn sends the full transcript, cost grows with each message and the model starts to lose track inside a long context. Add a summary step that compresses older turns once a thread passes a set length, keep the last few turns verbatim, and put fresh retrieved data next to them. For live facts, ground the answer in a fresh lookup rather than in what the conversation said earlier.

Test the summary step like any other component. Take long real conversations, run the summary, and check that the facts a user would expect the agent to remember are still there. Those include the product they asked about, the budget they gave and the person they want to reach. A summary that is short but drops the budget is worse than no summary, because the agent now answers confidently from an incomplete picture. The cost side is covered in how to reduce AI infrastructure cost in production.

05Retention, reset and privacy

Memory is data, and it needs the same rules as any other customer data:

• Decide how long threads are kept, and delete or archive them on that schedule.
• Reset idle sessions in the client, so a shared computer does not continue someone else's conversation.
• Re-check identity and role on a fresh profile before sensitive actions, and refuse stale or unexpected sessions.
• Keep tenants apart in every query that reads thread state.
• Redact customer data and tokens before logs leave the production environment.

06What we see in production

We built a multi-tenant AI sales assistant for dealer websites in partnership with a dealership software company. Its conversation state lives in a PostgreSQL checkpointer behind a LangGraph agent that uses tools for inventory search and lead capture. Keeping that memory stable in production took a connection pool and reconnect handling, not a change to the model. Early tests with dealers surfaced context loss, which we addressed by simplifying the flow and later rebuilding it as a tool-using agent.

The widget keeps session persistence with an idle reset, dealer admins see chat history with AI summaries, and token usage is logged per conversation. The stack runs on AWS with RDS PostgreSQL, Redis and background workers, and it was moved from our AWS account to the client's own account. It is live with dealers in the U.S. and Canada, with leads and appointments flowing into the client's CRM.

In a multi-tenant AI sales assistant platform for a retail sales services company, we kept memory per thread and added a summary node so long chats stay cheap. In an AI operations assistant for a short-term rental platform, answers are grounded in fresh profile and property data rather than a stale snapshot. Voice calls re-check the user's role on a fresh profile and refuse stale sessions, and a runbook requires redacting customer data and tokens before logs are shared.

Frequently asked questions