--- title: "AWS's Wordle RL gains expose the gap between training reward and agent readiness" description: "AWS reports an 84% held-out Wordle solve rate after multi-turn RL, versus 61% before training under one evaluation setting. Reward design and serving configuration still determine whether gains are real." url: "https://devencelab.com/insights/2026/10/23/awss-wordle-rl-gains-expose-the-gap-between-training" date: "2026-10-23" section: "Insights" tag: "Agentic AI" author: "Devence Lab" reading_time: "3 min read" site: "Devence Lab" license: "Readable and quotable with attribution to the canonical URL." --- # AWS's Wordle RL gains expose the gap between training reward and agent readiness AWS reports an 84% held-out Wordle solve rate after multi-turn RL, versus 61% before training under one evaluation setting. Reward design and serving configuration still determine whether gains are real. An AWS demonstration published on 23 October 2026 reports that multi-turn reinforcement learning improved a GPT-OSS-20B Wordle agent's held-out solve rate from 61% to 84% at its default reasoning effort. At the low reasoning effort used during training, a separate evaluation comparison rose from 15% to 69%. Those figures are useful precisely because AWS also shows that an apparently weaker training-validation curve did not predict the final serving result. The experiment trained a LoRA adapter over 100 steps using Amazon SageMaker AI's multi-turn RL service. An agent hosted on Bedrock AgentCore played Wordle, while the policy model selected guesses and a reward function scored the outcome of each game. AWS used 640 training words and 128 held-out words. This is a controlled demonstration, not evidence that equivalent gains transfer to enterprise agents. ## Reward design can teach the wrong behaviour AWS structured the reward so solving the word always outranked losing, and solving in fewer guesses outranked a slower success. Small shaping incentives encouraged informative openings, new-letter discovery and reduction of the remaining candidate set without changing that outcome ordering. The team also tested reward boundaries so that future edits would not silently make an unsuccessful trajectory more attractive than a successful one. That design choice applies to service agents too. Rewarding the number of tool calls may train unnecessary activity, while penalising every additional step can promote premature decisions. An external acceptance test must check completion and correctness independently of the training objective, especially when the agent can act on customer records or infrastructure. > A rising training reward is only progress when the trained agent also succeeds under the conditions in which it will actually run. ## Sampling configuration changed the apparent result AWS says the training job's own validation ended around a 37% solve rate with low-effort greedy settings, while subsequent evaluations of the trained model reached 84% at medium reasoning effort and temperature 1.0. Different sampling configurations and evaluation runs explain why these values should not be compared as if they came from the same benchmark setting. The code repository also warns that counting successful-looking log lines in CloudWatch had overstated success by more than twice, because the count mixed training and validation trajectories and omitted failures. Operators should derive completion rates from an authoritative evaluation set, with failed and interrupted episodes in the denominator. ## A demonstration is an acceptance-test template Production teams can reuse the discipline without reproducing the Wordle task. Freeze tool interfaces, initial model and dataset splits; maintain isolated rollouts; version reward code and evaluation rules; and compare baseline and trained agents at the intended serving settings. Track invalid actions, abandoned tasks, timeouts, latency and cost per independently verified completion. The Wordle result does not make multi-turn RL universally cost-effective. It shows that the environment, reward and evaluation settings can each change the observed outcome. A specialist agent should advance to deployment only when a held-out test proves real task improvements without introducing unacceptable errors or tool-authorisation failures. Produced through Devence Lab's automated research and publishing workflow, without prepublication human review. Devence Lab retains editorial responsibility. Sources are linked; report factual errors to contact@devencelab.com. ## Sources - [Multi-Turn Reinforcement Learning on Amazon SageMaker AI: A hands-on guide](https://aws.amazon.com/blogs/machine-learning/multi-turn-reinforcement-learning-on-amazon-sagemaker-ai-a-hands-on-guide/) - AWS Machine Learning Blog - [sample-multi-turn-rl-wordle — project and results](https://github.com/aws-samples/sample-multi-turn-rl-wordle) - AWS Samples - [Best practices for multi-turn reinforcement learning in Amazon SageMaker AI](https://aws.amazon.com/blogs/machine-learning/best-practices-for-multi-turn-reinforcement-learning-in-amazon-sagemaker-ai/) - AWS Machine Learning Blog