---
title: "A 1-million-token context window is not a memory strategy. Cache the stable prefix."
description: "AWS brought Kimi K3 to Bedrock with a 1-million-token context window and explicit prompt caching. Long-context agents should separate durable reusable context from changing task state instead of resending everything."
url: "https://devencelab.com/insights/2026/09/21/a-1-million-token-context-window-is-not-a"
date: "2026-09-21"
section: "Insights"
tag: "Model Releases"
author: "Devence Lab"
reading_time: "2 min read"
site: "Devence Lab"
license: "Readable and quotable with attribution to the canonical URL."
---

# A 1-million-token context window is not a memory strategy. Cache the stable prefix.

AWS brought Kimi K3 to Bedrock with a 1-million-token context window and explicit prompt caching. Long-context agents should separate durable reusable context from changing task state instead of resending everything.

AWS made Kimi K3 available on Amazon Bedrock on 18 September with a 1-million-token context window and explicit prompt caching. AWS says Kimi K3 is the first open-weight model on Bedrock to support that explicit caching mode.

The tempting design is to treat the larger window as permission to resend an agent's entire working history on every turn. The caching mechanics point to a better production pattern: separate stable context from volatile task state, then measure whether the stable portion actually earns cache hits.

## Context capacity and context reuse are different controls

AWS documents explicit cache breakpoints after at least 1,024 tokens. Tokens written into the cache cost more on the write, remain cached for at least 30 minutes, and matching reads are discounted. AWS also says cached input tokens do not count against input-tokens-per-minute quotas.

That makes prompt structure an infrastructure decision. Repository instructions, tool definitions, policy text and reference documents can form a stable prefix. User requests, tool results and changing execution state belong after it. Mixing both into one constantly changing prefix reduces the chance that later calls match the cached content.

> A large context window tells an agent what it can carry; a cache strategy decides what it should pay to carry repeatedly.

## Cache hit rate belongs beside model quality

A model migration test should therefore record more than task accuracy and total tokens. Capture cache-write tokens, cached-read tokens, hit rate, time to first output and cost per completed task. A one-million-token limit can accommodate a large repository or document set, but capacity alone says nothing about whether repeatedly transmitting that context is economical.

The same measurement exposes poor context architecture. If a supposedly stable system prefix changes on most calls, the application will repeatedly pay the cache-write cost. If stable context is too fragmented, teams may save tokens while adding orchestration complexity that does not improve task completion.

## Residency choices must survive the optimisation

AWS offers Kimi K3 through global and US cross-Region inference profiles. It says the global profile costs approximately 10% less than a geographic profile, while the US profile keeps processing within the US geography. That means routing policy remains a deployment constraint even when caching makes a workload cheaper.

The practitioner change is to benchmark context architecture as part of model adoption. Define the stable prefix, place explicit breakpoints around content that genuinely repeats, keep changing state outside that prefix, and measure cost per successful task with and without caching. Then choose the inference profile that satisfies residency requirements. A larger window is useful capacity; it becomes an operating advantage only when the application controls what is reused.

## Sources

- [Introducing Kimi K3 on Amazon Bedrock](https://aws.amazon.com/blogs/machine-learning/introducing-kimi-k3-on-amazon-bedrock/) - AWS Machine Learning