LLM Memory: How Parametric, Contextual, and External Memory Work
TL;DR:
- LLM memory covers the ways information can persist or become available to a language model.
- Parametric memory is encoded in model weights. Contextual memory is information available during the current inference.
- External memory stores information outside the model so it can be retrieved across separate calls or sessions, providing one route to LLM long-term memory.
- LLM memory management differs by memory type: parametric memory changes through training or editing, while contextual and external memory are managed at the application level.
LLM memory refers to the ways information can persist or become available to a large language model. Some knowledge is encoded in the model's parameters during training. Other information comes from the current context or is retrieved from external systems and supplied for a specific request.
Where information is stored, how long it persists, and how it can be changed are important factors in the discussion about long-term memory. Expanding a context window increases how much information a model can access during an inference, while preserving information across separate inference calls requires a different memory mechanism.
In this guide, we'll examine parametric, contextual, and external memory, how each is managed, and how RAG relates to external memory.
What Is LLM Memory?
LLM memory is information-bearing state that can influence a model's output. Three types of LLM memory are central to this article: parametric, contextual, and external memory.
| Where is it stored? | Persistent? | How does it change? | What does it provide? | |
|---|---|---|---|---|
| Parametric memory | Model weights | Persists across inference calls | Pretraining, fine-tuning, PEFT, or model editing | Learned knowledge, patterns, and behavior |
| Contextual memory | Current context and inference-time state | Temporary | Context assembly, truncation, compression, or retrieval | Instructions, recent interactions, and information available during inference |
| External memory | Databases, documents, vector indexes, graphs, or other application storage | Persists independently of the model | Application-level writes, updates, and deletion | Information that can be retrieved across separate calls or sessions |
The storage-and-persistence framework outlined in this recent LLM memory research paper also includes procedural/episodic memory, which encompasses retained experiences, interaction histories, and learned procedures. But because those concepts are more commonly developed within experience-based and agent memory architectures, we'll focus here on the three categories that most directly explain where information available to an LLM is stored.
All three forms of LLM memory can contribute to the same response — a model can combine knowledge encoded in its weights with the current conversation and information retrieved from an external system.
Parametric Memory: Knowledge in Model Weights
Parametric memory is information encoded in an LLM's learned parameters. During pretraining, the model adjusts its weights as it learns statistical patterns from large datasets. Those parameters later influence its predictions without requiring access to the original training examples.
This is how an LLM can answer many questions in a closed-book setting. Factual associations, linguistic patterns, and learned behaviors are represented within the model and become available through inference.
How parametric memory changes
Much of a model's parametric knowledge is established during pretraining, while further training can modify those representations.
- Fine-tuning adjusts model parameters using additional data to specialize the model for a domain, task, or behavior. Parameter-efficient fine-tuning (PEFT) methods such as LoRA update a smaller set of parameters or low-rank adaptations while leaving most of the base model unchanged.
- Recent research has examined LoRA as a parametric memory mechanism, including how much information low-rank adaptations can encode and how reliably a model can recall it later.
- Model editing targets narrower changes, attempting to modify specific learned associations while limiting effects on unrelated model behavior.
The limits of parametric memory
Parametric memory can preserve broadly reusable knowledge and learned behavior, but several properties make frequently changing or source-sensitive information difficult to manage this way:
- Information encoded in model weights doesn't behave like a database record. A fact isn't stored as a discrete record that can simply be inspected, timestamped, attributed to a source, or deleted independently of the surrounding knowledge.
- Changes can interfere with existing knowledge. Research on lifelong model editing has found that directly changing model parameters can interfere with pretrained knowledge or earlier edits, while retrieval-based methods can struggle to generalize edited knowledge to new queries.
- New information requires a model-level update. Incorporating knowledge into a deployed model's parameters requires additional training, adaptation, or editing rather than a direct change to an individual record.
Information that changes frequently or requires explicit provenance and direct revision is therefore generally easier to maintain outside the model.
Contextual Memory: Information Available at Inference
Contextual memory is the information available to an LLM while it generates a response. It can include system instructions, the current prompt, conversation history, retrieved documents, and tool output.
The application can change this information between requests without modifying the model itself, making contextual memory suitable for recent, task-specific, or dynamically retrieved information.
Its capacity is constrained by the context window, which limits how much input and generated content the model can process during an inference. Increasing the amount of context doesn't guarantee that every token will influence the response equally — long-context evaluations have found that models often use information at the beginning or end of a long input more reliably than information that gets "lost in the middle."
Context and inference caches
Transformer inference can also use a key-value (KV) cache to retain attention calculations for tokens the model has already processed. This reduces repeated computation during generation and improves inference efficiency.
A KV cache is computational state rather than independently addressable application memory. Its contents don't automatically carry into a later session as persistent knowledge.
Conversation history can create a separate impression of persistence at the application level. A chat can appear continuous because earlier messages are supplied to the model again on subsequent requests. If that history is no longer included, it's no longer part of the model's contextual memory.
How LLMs Get Long-Term Memory
LLM long-term memory is information that can influence a model across separate inference calls or sessions. That persistence can come from either model parameters or external storage.
Parametric long-term memory
Knowledge encoded in model weights persists whenever those parameters are loaded. Information learned through pretraining or later adaptation can therefore influence unrelated sessions without an external lookup.
Changing that knowledge requires a model-level update, and facts are difficult-to-impossible to inspect or revise directly.
External long-term memory
Applications can preserve information independently of the model in databases, documents, vector indexes, knowledge graphs, or other storage systems. When it's needed again, retrieval selects relevant records and adds them to the model's context for the current request.
This is why a large context window and long-term memory solve different problems. The context window determines how much information the model can process during an inference; long-term memory determines whether information is still available after that context ends.
| Contextual memory | Long-term memory | |
|---|---|---|
| Persistence | Current inference or reconstructed conversation | Across separate calls or sessions |
| Location | Model context | Model parameters or external storage |
| Updates | Change the supplied context | Update parameters or external records |
| Capacity | Limited by the context window | Determined by the model or external storage |
| Typical role | Current instructions, conversation, and retrieved information | Learned knowledge and information needed again later |
External long-term memory lets applications preserve user context, project state, and interaction history across sessions. When an autonomous agent can write to that persistent state and retrieve it during later tasks, it becomes part of an agent memory architecture.
Parametric vs External Memory
Both parametric and external memory can preserve information across inference calls. However, their storage and update mechanisms lead to different design constraints:
| Parametric memory | External memory | |
|---|---|---|
| Storage | Model parameters | Databases, documents, indexes, graphs, or other external storage |
| Access | Available directly during inference | Requires retrieval |
| Updating | Training, PEFT, or model editing | Direct changes to stored information |
| Freshness | Depends on model updates | Can change independently from the model |
| Provenance | Difficult to trace to an individual source | Can preserve sources and metadata |
| Deletion | Difficult to verify for an individual fact | Records can be removed explicitly |
| Capacity | Constrained by model architecture and training | Can expand with the storage system |
| Latency | No external retrieval step | Retrieval adds work before generation |
The distinction becomes especially salient in use cases that involve frequently-changing information. Product data, policies, project state, newly published material, and user-specific records can all be updated externally without retraining the model. Timestamps, permissions, and source references can also be stored alongside the records, including temporal information used to track when knowledge was valid or changed.
Many applications combine the two: the model contributes learned knowledge and language capabilities, while external storage supplies information that is newer, private to the application, or maintained independently from model training.
How LLM Memory Management Works
LLM memory management refers to how information is added, maintained, accessed, and changed across model parameters, current context, and external storage. Each form of memory requires different operations.
Updating parametric memory
As we've already mentioned, updating parametric memory requires modifying the model itself. Because those changes can interact with knowledge encoded in the weights, evaluation has to check both whether the intended information was incorporated and whether unrelated behavior changed as a side effect.
Removing parametric knowledge is harder to verify. Learned information is distributed across model parameters, so deleting a training example or modifying an association doesn't guarantee that the information will be irrecoverable.
Maintaining external memory
External memory can be maintained independently from the model. Applications can attach metadata to stored information, index it for retrieval, revise records as the underlying information changes, and enforce permissions, retention policies, and deletion requirements.
Only a subset of this information ever needs to enter the model's context for a request. Retrieval selects the relevant records, allowing the persistent store to hold far more information than a single context window can accommodate.
Controlling contextual memory
Contextual memory management determines what reaches the model within the available token budget.
Context engineering governs that selection, including how conversation history is shortened or summarized, which retrieved passages are included for the current query, and how space is allocated among instructions, tool output, and the current request.
Is RAG a Form of LLM Memory?
Retrieval-augmented generation (RAG) gives an LLM access to information stored outside its parameters. A typical RAG pipeline retrieves relevant passages from documents, databases, or search indexes and adds them to the model's context before generation.
In that sense, RAG can provide access to external memory. The underlying corpus persists independently from the model and can be updated without retraining it. But a conventional RAG system is pre-prepared and doesn't necessarily record prior interactions, update existing records as new information arrives, or preserve an evolving history of what the application has retained.
Vector similarity, lexical search, metadata filtering, graph traversal, and hybrid retrieval can all select persistent and relevant information for the current context. But what distinguishes a memory system is its ability to accumulate, update, and retrieve information across interactions rather than query a fixed corpus.
RAG is therefore one retrieval pattern that can access external memory — among several patterns a persistent memory system can use.
Persistent Memory Must Preserve Context
Once information persists outside the model, retrieving the right memory is only part of what determines whether it will also be helpful in a later query. A record can become difficult to interpret when newer information has superseded it, its relevance depends on another entity or event, or its original source is no longer attached.
External memory therefore has to retain enough surrounding context for the LLM to recover why the information is relevant when it's retrieved. Relationships and provenance become especially important as memory accumulates across sessions and individual records become part of a larger history.
We've built cognee to combine semantic retrieval with graph relationships and relational storage, so retrieved information can carry its connections and source history with it. The full memory architecture can also run on a single Postgres instance, keeping vector, graph, session, and relational data together while the memory itself stays independently updateable.
🧠 Build persistent LLM memory with cognee.
Keep context available across sessions and retrieve it with its relationships and provenance intact.
Try cognee today with your free cognee Cloud key or explore the docs to see how the memory layer works.
FAQ
Answers to the most common questions from this guide.
Can an LLM learn from a conversation without retraining?
During standard inference, information from a conversation can influence the model through its current context without changing the underlying model weights. An application can also save information externally and retrieve it in later sessions.
Persistently changing the model's parametric memory requires a model-level update such as fine-tuning, PEFT, or model editing. Test-time training research is exploring architectures that perform learned updates to internal state during inference, but this isn't how most deployed LLM applications currently preserve conversational information.
What happens when retrieved information conflicts with an LLM's parametric memory?
The model can receive contextual information that contradicts knowledge encoded in its parameters, and there is no universal rule guaranteeing which source will dominate the response.
Research into these knowledge conflicts shows that model behavior varies according to the task and how conflicting information is presented. Models can accept coherent external evidence while also showing a measurable bias toward information consistent with knowledge already encoded in their parameters.
For applications using external memory or RAG, successful retrieval therefore doesn't guarantee that the model will follow the retrieved evidence over its parametric knowledge.
Can external memory become parametric memory?
Yes. Information stored externally can later be incorporated into model parameters through further training, fine-tuning, PEFT, model editing, or other consolidation methods.
That process isn't automatic in conventional LLM applications. External memory normally stays independently stored and is retrieved when needed, while incorporating selected information into parametric memory requires an additional model-update process.
Recent research into selective memory consolidation is exploring ways to internalize selected external information in model parameters while leaving other knowledge in external storage.
