You type eleven words and press send. What arrives at the model might be eight thousand tokens, of which your eleven words are the last few. Everything else was assembled by the agent in the milliseconds between your keystroke and the request leaving.
The Short Answer
No. Your raw prompt is almost never what the model receives.
Your message is usually preserved word for word, but it is placed at the end of a much larger structure the agent constructs for every single request. The model reads that whole structure as one input.
What Actually Reaches the Model
| Component | Supplied by | Typical size |
|---|---|---|
| System instructions | Developer | Hundreds to thousands of tokens |
| Tool definitions | Developer | Grows with tool count |
| Retrieved documents | Retrieval layer | Often the largest component |
| Stored facts or memory | Memory store | Small to moderate |
| Conversation history | Agent | Grows every turn |
| Your message | You | Often under 1% |
The proportion is worth absorbing. In a mature agent your actual words are a small fraction of the input. Most of what the model reads is scaffolding — which is why agent behaviour is shaped far more by prompt assembly than by the wording of any individual message.
A Concrete Comparison
What you typed:
Summarise the third one and compare it to the first.
Roughly what the agent sent, in structure:
SYSTEM You are an assistant that... Output format: ... Constraints: ... Current date: ... TOOLS search_documents(query: string, limit: integer) fetch_record(id: string) RETRIEVED CONTEXT [document excerpts pulled for this turn] CONVERSATION HISTORY user: Show me the available reports assistant: Here are five reports... user: What does the second one cover? assistant: The second report covers... USER Summarise the third one and compare it to the first.
Your sentence only means something because of the history above it. “The third one” resolves against a list from three turns ago. Strip the history and the message is unanswerable — which is exactly what would happen, since the model is stateless and would otherwise arrive at your message with no idea what “third” refers to.
The Components
System instructions
Sent on every request, typically first. They define role, behavioural rules, output format, refusal boundaries and any environment facts the model needs — the current date being a common one, since the model has no clock.
Because they are resent every turn, system instructions are the most reliable place to put constraints that must not be lost. Anything stated only in a passing user message can be truncated away later.
Tool definitions
Schemas describing each available tool. These are what the model reads when deciding whether to request one — see who decides tool calling. Their descriptions function as prompt engineering, not documentation.
Retrieved context
Where RAG output lands. The agent runs a search, takes the top results, and inserts them. This is frequently the largest single component and the one most likely to crowd out everything else.
Memory
Facts stored externally from earlier sessions, inserted as text. As covered previously, this is external storage feeding the prompt rather than model state.
Conversation history
Prior turns, resent in full because the model retains nothing. This is what grows unboundedly and forces the trade-offs discussed below.
Roles Are Markers, Not Channels
Modern APIs structure input as a list of messages with roles — system, user, assistant. This looks like separate channels. It is not.
Before inference, everything is flattened into a single token sequence. Role boundaries become special tokens or formatting the model was trained to recognise. There is one input, read start to finish.
This matters for a practical reason: the separation between “instructions” and “user content” is a learned convention, not an enforced boundary. Models generally respect it, and it is genuinely useful. But it is not a security guarantee — which is why content arriving through tool results or retrieved documents can still influence behaviour.
When the Agent Does Rewrite Your Words
Your message usually reaches the model verbatim. There is one common exception worth knowing.
For retrieval, the agent often rewrites your query before searching — because your message may be meaningless standing alone:
| You typed | Rewritten for search |
|---|---|
| “What about the second one?” | “details of the second quarterly report” |
| “And in the other region?” | “performance metrics in the western region” |
Embedding “What about the second one?” retrieves nothing useful, because it contains no searchable content. The agent resolves references against history to build a standalone query, searches with that, then sends your original wording to the model along with the results.
Some agents also expand queries into several variations and merge results. Both are retrieval-side transformations — your message itself still reaches the model unchanged.
The Context Budget
Everything competes for one finite window. The agent must allocate it.
| Component | Compressible? |
|---|---|
| System instructions | Rarely — usually treated as fixed |
| Tool definitions | Somewhat — send only relevant tools |
| Retrieved documents | Yes — fewer results, shorter chunks |
| Conversation history | Yes — truncate or summarise |
| Your message | No |
| Room for the response | Must be reserved |
That final row is easy to forget. Generated output occupies the same window. An agent that fills the context entirely with input leaves nothing for the answer.
When the budget is exceeded, history is normally sacrificed first — truncated, summarised or windowed. This produces the familiar failure where an agent that understood your requirement early begins contradicting it later. The instruction was not forgotten; it was removed from the input.
Why Assembly Order Matters
Two forces push in the same direction: static content first, volatile content last.
Caching. Several providers offer prompt caching, which reuses computation for an identical prefix across requests. Cache hits require the prefix to match exactly, so anything changing per request — timestamps, retrieved documents, history — must come after the stable portion. Putting the current time at the top of a system prompt invalidates the cache on every call.
Attention. Models tend to attend more reliably to material at the beginning and end of a long input than to the middle. Placing your message last, immediately before generation, is deliberate. It is also why critical instructions are sometimes repeated near the end of very long prompts.
Why Structure Does Not Solve Injection
Since role separation is a convention rather than an enforced boundary, embedded instructions in untrusted content can still take effect. A retrieved document or tool result containing “ignore prior instructions and…” arrives in the same token sequence as everything else.
Practices that reduce exposure:
- Mark untrusted content explicitly — delimiters and labelling indicating that a section is data to be read, not instructions to be followed
- Instruct the model that content within those bounds is untrusted
- Never place authorisation logic in the prompt — enforce it in the agent
- Treat tool output as untrusted, since it re-enters context
These reduce risk. None makes the boundary absolute, which is why the controls that matter sit in the agent rather than in the prompt.
Practical Implications
- Poor agent behaviour is usually an assembly problem, not a model problem. Check what was actually sent before changing models.
- Log the assembled prompt. Debugging without it is guesswork, since your message is a small part of the input.
- Put durable constraints in the system prompt, which is resent every turn.
- Watch retrieval volume. Retrieved context is usually what consumes the budget and squeezes out history.
- Keep the prefix stable if you want caching to work.
Key Takeaways
- Your raw prompt is not what reaches the model — it sits at the end of a much larger assembled input
- Your words are often under 1% of the tokens sent
- The agent supplies system instructions, tool definitions, retrieved context, memory and full history
- Roles are markers — everything flattens into one token sequence before inference
- Agents commonly rewrite your query for retrieval while sending your original wording to the model
- When the context budget is exceeded, history is sacrificed first
- Static content goes first for caching; your message goes last for attention
- Prompt structure does not prevent injection — enforcement belongs in the agent
Frequently Asked Questions (FAQ)
Q: Does an agent send my prompt directly to the LLM?
No. Your message is typically preserved word for word but placed at the end of a much larger structure containing system instructions, tool definitions, retrieved documents, stored memory and the full conversation history. The model reads all of it as one input.
Q: How much of the input is my actual message?
Often under one percent in a mature agent. System instructions, tool schemas and retrieved context typically dominate, with conversation history growing every turn. This is why agent behaviour depends more on prompt assembly than on how you word a single message.
Q: Does the agent change my wording?
Your message usually reaches the model unchanged. The common exception is retrieval — the agent often rewrites your query into a standalone form before searching, since a message like “what about the second one?” contains nothing searchable on its own.
Q: What is a system prompt and why is it sent every time?
System instructions define role, behavioural rules, output format and environment facts such as the current date. They are resent with every request because the model retains nothing between calls, which also makes them the most reliable place for constraints that must not be lost.
Q: Are system, user and assistant roles kept separate inside the model?
Not as separate channels. Everything is flattened into a single token sequence before inference, with roles becoming special tokens the model was trained to recognise. The separation is a learned convention rather than an enforced boundary.
Q: What gets cut when the context window fills?
Conversation history first, through truncation, summarisation or windowing. Retrieved context may also be reduced. System instructions and your current message are normally preserved, and room must be reserved for the response itself.
Q: Why does the order of prompt components matter?
Prompt caching requires an identical prefix across requests, so anything changing per call must come after stable content. Models also attend more reliably to the beginning and end of long inputs than the middle, which is why your message is placed last.
Q: Why does my agent ignore an instruction I gave earlier?
Most often because that turn was truncated or summarised out when the context budget was exceeded. The instruction is no longer in the input. Moving durable constraints into the system prompt, which is resent every turn, resolves this.
Related Reading: