Skip to main content
GRPO training loop: query q enters the Policy Model, which samples G outputs o_1 to o_G; a frozen Reward Model scores each output into r_1 to r_G; Group Computation normalizes the scores into advantages A_1 to A_G; a frozen Reference Model provides a per-token KL penalty fed back to the Policy Model Editable Mermaid source: images/grpo-loop.mermaid.md
Found an error on this page? Report an issue.