> ## Documentation Index
> Fetch the complete documentation index at: https://aegean.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# RL for Reasoning in Small LLMs

> GRPO-based fine-tuning of DeepSeek-R1-Distill-Qwen-1.5B for mathematical reasoning — AAAI 2026.

<Note>
  This lab is under construction. Track progress in [AURA-654](https://aegean-ai.atlassian.net/browse/AURA-654).
</Note>

This lab is based on the AAAI 2026 paper [*Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn't*](https://arxiv.org/abs/2503.16219) and the accompanying [open-rs repository](https://github.com/knoveleng/open-rs).

You will fine-tune `DeepSeek-R1-Distill-Qwen-1.5B` using Group Relative Policy Optimization (GRPO) on a compact mathematical reasoning dataset, reproducing the three experiments from the paper.

## Key results

| Benchmark | Baseline | After GRPO |
| --------- | -------- | ---------- |
| AMC23     | 63%      | 80%        |
| AIME24    | —        | 46.7%      |

Training runs on 4× NVIDIA A40 GPUs (48 GB VRAM) in under 24 hours at a cost of \~\$42.

## Resources

* [open-rs repository](https://github.com/knoveleng/open-rs)
* [arXiv paper](https://arxiv.org/abs/2503.16219)
* Models: [Open-RS1](https://huggingface.co/knoveleng/Open-RS1), [Open-RS2](https://huggingface.co/knoveleng/Open-RS2), [Open-RS3](https://huggingface.co/knoveleng/Open-RS3)
* Datasets: [open-s1](https://huggingface.co/datasets/knoveleng/open-s1), [open-deepscaler](https://huggingface.co/datasets/knoveleng/open-deepscaler), [open-rs](https://huggingface.co/datasets/knoveleng/open-rs)

***

<Callout icon="pen-to-square" iconType="regular">
  [Edit this page on GitHub](https://github.com/aegean-ai/eaia/edit/main/src/aiml-common/lectures/reasoning/open-rs-lab/index.mdx) or [file an issue](https://github.com/aegean-ai/eaia/issues/new/choose).
</Callout>