Teams training agents with reinforcement learning rarely fail because the infrastructure broke. They fail because the reward function taught the model something nobody intended, and every training curve looked fine while it happened. Amazon has published a detailed walkthrough of how to write those reward functions for multi-turn agents on Nova Forge, and the most useful part is the account of a run that quietly stopped learning.
The mechanism matters for anyone choosing where scoring code runs. For multi-turn training, Nova Forge executes your reward logic inside your own environment through a capability Amazon calls Bring Your Own Orchestration, while the service coordinates rollouts, message passing, and conversation state across turns, according to Amazon’s walkthrough of custom Nova Forge rewards. Amazon also says a serverless multi-turn reinforcement learning option is now generally available for teams that would rather not operate that environment themselves, per the same post.
There is a hard technical reason the orchestration path exists rather than a simpler one. Single-turn reinforcement fine-tuning registers the reward as an AWS Lambda function, but multi-turn conversations and long-running scoring run past the 15-minute Lambda invocation limit, so Nova Forge delegates each rollout to a customer-managed container instead, as Amazon’s Nova reinforcement learning documentation describes. That container returns an aggregate reward per sample plus an optional list of per-component scores.
Amazon’s worked example is a collaborative-coding task built over 500 unique programming problems, used to train Nova Lite 2.0 with Group Relative Policy Optimization and Low-Rank Adaptation on SageMaker HyperPod, according to the AWS post. The model receives an under-specified request while a simulated user privately holds the full specification and reveals details only when asked. Guessing should produce failing code; asking should surface the missing detail.
The instructive failure came from a reasonable-looking earlier design. That version granted the asking bonus only when the final code also passed tests, and added a term rewarding shorter conversations. Training collapsed toward guessing, Amazon reports in the same walkthrough. Its stated explanation is the part worth internalizing: a reward term influences learning only through the variation it creates within a group, so a component that scores identically across every rollout contributes nothing to the gradient no matter how heavily it is weighted.
The fix was structural rather than numerical. Amazon’s four-component reward credits asking on its own instead of gating it behind correctness, requires the model to eventually commit code so “ask forever” is not a winning strategy, and applies an explicit penalty for guessing immediately, per the AWS post. Each component is reported separately so a dead signal is visible instead of hidden inside a healthy-looking aggregate.
Because the correctness component executes model-generated code, Amazon treats that output as unvalidated. Its harness withholds credentials and network access, applies CPU and 2 GB memory limits, runs in a temporary directory, and uses a per-run random sentinel so the model cannot forge a result by writing the expected marker to stderr, as shown in the AWS reward-function code. It also validates the number of tests actually run against the number expected, closing the trick of diluting a score with trivially-passing tests.
The deployment scaffolding is public. Amazon has released the CDK infrastructure for this setup at aws-samples/sample-nova-multi-turn-rl-infra, where uploading a JSONL file to the training-data prefix triggers a Step Functions pipeline, and the documented defaults require ml.p5.48xlarge capacity with a minimum of 10 instances for four generation replicas. That quota requirement is the practical gate on whether a team can reproduce the example at all.
The broader signal is that reward engineering, not orchestration, is now the scarce skill in agentic training. The same shift shows up in production agent builds on Amazon’s stack, such as how LendingTree built a multi-agent mortgage assistant on Amazon, where what the system is scored on determines what it actually does. Amazon’s advice reduces to instrumenting every reward component and distrusting an aggregate that looks healthy.
