RadixArk has open-sourced Miles, a framework for reinforcement-learning post-training of large language and vision-language models. Its design connects SGLang for rollout generation, Megatron-LM for distributed training, Ray for orchestration, and PyTorch as the common programming layer. The project’s launch article presents the result as a systems stack for making large training runs more composable and easier to extend, not as a new reinforcement-learning algorithm Miles: A PyTorch-Native Stack for Large-Scale LLM RL Post-Training.
That distinction is important. Reinforcement-learning post-training has at least two workloads with different demands. Rollout workers generate samples by running the current policy, while trainers consume those samples and update the policy. The system also has to move samples and weights, maintain consistent model behavior across the rollout and training paths, place processes on available accelerators, record useful diagnostics, and recover from failures.
How the Miles stack is divided
SGLang handles high-throughput rollout generation. Megatron-LM supplies the distributed training backend and its parallelism and checkpoint machinery. Ray represents long-lived processes such as trainer ranks, rollout servers, routing proxies, and asynchronous workers as actors, then provides placement, lifecycle management, scheduling, supervision, logs, and dashboard visibility Miles architecture overview.
PyTorch remains the layer used for model components, autograd, distributed primitives, data types, profiling, and extensions. The public RadixArk Miles repository includes the framework package, plugins, examples, tests, deployment assets, and separate entry points for standard and asynchronous training. The repository also says Miles is forked from and co-evolves with the slime framework, context that is more precise than presenting the entire stack as newly created in isolation.
Miles follows what the project calls a “small-core, many-edges” philosophy. Rollout logic, rewards, losses, sample filters, metrics, training hooks, and model-specific components can be attached through user-supplied modules. The point is to keep frequently changed research or product logic outside the distributed training backend rather than requiring every experiment to maintain a private fork of Megatron-LM.
Rollout and training can run together or apart
Miles supports colocated layouts, where rollout and training phases share a set of nodes, and disaggregated layouts, where each phase has its own resources. It also supports asynchronous operation. In that mode, rollout actors stream samples to a queue and the trainer consumes them at its own pace instead of forcing both phases to stop at the same iteration boundary.
Fresh weights have to reach rollout workers after training updates. The project says Ray coordinates the control path while bulk tensor data moves over dedicated NCCL/RDMA channels. That architecture keeps large transfers away from the ordinary Python control path. It describes how the transfer is organized; it does not by itself establish a universal throughput improvement for every cluster or model Miles on moving data across the RL pipeline.
The SGLang project also lists Miles in its reinforcement-learning framework integration roadmap, alongside work on rollout-engine interfaces, weight synchronization, sampling, log-probability semantics, and mixture-of-experts alignment SGLang development roadmap. That upstream view helps readers distinguish current integration work from claims that every planned optimization is already complete in every Miles configuration.
Why mixture-of-experts alignment matters
Mixture-of-experts models route tokens through selected experts. If the rollout system and trainer make different routing decisions for the same samples, the policy that generated the data can diverge from the policy used to compute the update. Miles describes Rollout Routing Replay as a way to preserve those decisions across the boundary.
The framework also exposes model specifications as plugins rather than long-lived backend forks. This allows architecture-specific PyTorch components to enter Megatron’s model pipeline while using the existing autograd, mixed-precision, and checkpoint behavior. Teams should still verify support against the exact model revision, hardware, and configuration they intend to use.
Low precision is a capability, not a free guarantee
Miles advertises unified recipes for BF16, FP8, MXFP8, and INT4 quantization-aware training across rollout and training, plus LoRA support in both paths. Those options may reduce resource use or improve utilization in an appropriate setup, but lower precision can change numerical behavior. PyTorch’s own numerical accuracy guidance warns that reduced-precision computation can introduce errors and that results are not guaranteed to be bitwise identical across platforms or implementations.
Before enabling a recipe, operators should compare reward trajectories, log probabilities, routing decisions, evaluation scores, stability, memory use, and end-to-end wall-clock performance against a higher-precision baseline. zbrandco’s model quantization explainer provides background on why storing or computing with fewer bits can save resources while changing approximation error.
Researchers also need a truthful baseline for the model they are adapting. The open models worth running in 2026 overview can help frame model selection, but a model’s general reputation is not a substitute for task-specific evaluation inside the Miles stack.
What operators should verify
Miles brings several difficult pieces into one project, but the launch materials do not establish a fixed development-time saving, automatic recovery deadline, universal accuracy result, or guaranteed speedup. Those outcomes depend on the model, reward pipeline, cluster topology, networking, precision recipe, workload mix, and operational discipline.
A credible evaluation should begin with a reproducible small run, record exact software and model revisions, test checkpoint restore, simulate worker failure, inspect profiler traces, and compare synchronous and asynchronous layouts under the same task. Only measured results from that matched setup should become performance claims.
The practical value of Miles is architectural: it gives teams an open stack in which rollout, training, orchestration, weight synchronization, low precision, observability, and extension points are designed to work together. Its production authority still has to be earned by the evidence generated on the operator’s own hardware and workload.
