fix: nightlies using v1 can't use model_save_format=safetensors

[terrykong]

As title describes.

Previously the error encountered was:

Saving checkpoint for step 10...
Traceback (most recent call last):
  File "/opt/nemo-rl/examples/run_grpo_math.py", line 235, in <module>
    main()
  File "/opt/nemo-rl/examples/run_grpo_math.py", line 218, in main
    grpo_train(
  File "/opt/nemo-rl/nemo_rl/algorithms/grpo.py", line 889, in grpo_train
    policy.save_checkpoint(
  File "/opt/nemo-rl/nemo_rl/models/policy/lm_policy.py", line 697, in save_checkpoint
    raise ValueError(
ValueError: safetensors is only supported with DTensorPolicyWorkerV2 (_v2=true).
Traceback (most recent call last):
  File "/opt/nemo-rl/examples/run_grpo_math.py", line 235, in <module>
    main()
  File "/opt/nemo-rl/examples/run_grpo_math.py", line 218, in main
    grpo_train(
  File "/opt/nemo-rl/nemo_rl/algorithms/grpo.py", line 889, in grpo_train
    policy.save_checkpoint(
  File "/opt/nemo-rl/nemo_rl/models/policy/lm_policy.py", line 697, in save_checkpoint
    raise ValueError(
ValueError: safetensors is only supported with DTensorPolicyWorkerV2 (_v2=true).

Summary by CodeRabbit

• New Features

  • Added an optional checkpointing setting to specify the model save format in LLM training recipes. Defaults to current behavior (no format specified) and has no impact unless configured.

• Chores

  • Updated example recipes to include a placeholder for model save format, improving discoverability of the option.

[coderabbitai]

📝 Walkthrough

Walkthrough

Added a new YAML key checkpointing.model_save_format: null to three LLM recipe config files. No other fields or logic altered.

Changes

Cohort / File(s)	Summary
Checkpointing config field addition examples/configs/recipes/llm/grpo-deepscaler-1.5b-8K.yaml, examples/configs/recipes/llm/grpo-gemma3-27b-it-8n8g-fsdp2tp8-actckpt-long.yaml, examples/configs/recipes/llm/grpo-gspo-deepscaler-1.5b-8K.yaml	Add checkpointing.model_save_format: null to explicitly include the model save format key without changing existing behavior.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Pre-merge checks and finishing touches

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check	✅ Passed	The title succinctly describes the primary change of preventing nightlies using the v1 policy from using model_save_format=safetensors, aligning directly with the pull request’s objective and avoiding extraneous details.
Docstring Coverage	✅ Passed	No functions found in the changes. Docstring coverage check skipped.
Test Results For Major Changes	✅ Passed	The PR only adjusts three recipe configuration files by setting the checkpointing model_save_format to null, which is a minor configuration tweak rather than a major feature, breaking change, or significant refactor; therefore, per the check instructions, the absence of explicit test results in the description is acceptable and the check passes.

✨ Finishing touches

🧪 Generate unit tests

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch tk/v1-nightly-no-safetensor

---

📜 Recent review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5166d74 and fff6320.

📒 Files selected for processing (3)

• examples/configs/recipes/llm/grpo-deepscaler-1.5b-8K.yaml (1 hunks)
• examples/configs/recipes/llm/grpo-gemma3-27b-it-8n8g-fsdp2tp8-actckpt-long.yaml (1 hunks)
• examples/configs/recipes/llm/grpo-gspo-deepscaler-1.5b-8K.yaml (1 hunks)

🧰 Additional context used

📓 Path-based instructions (4)

examples/configs/recipes/**/*.yaml

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

examples/configs/recipes/**/*.yaml: Recipe YAMLs under examples/configs/recipes/** are runnable snapshots and may omit documentation
When adding support for a new model, add a recipe YAML under examples/configs/recipes/ in the appropriate domain (llm/ or vlm/) with the correct name

Files:

• examples/configs/recipes/llm/grpo-deepscaler-1.5b-8K.yaml
• examples/configs/recipes/llm/grpo-gspo-deepscaler-1.5b-8K.yaml
• examples/configs/recipes/llm/grpo-gemma3-27b-it-8n8g-fsdp2tp8-actckpt-long.yaml

examples/configs/recipes/llm/*.yaml

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

LLM recipe YAML filenames must follow: --ng-[-modifiers][-long][.vN].yaml

Files:

• examples/configs/recipes/llm/grpo-deepscaler-1.5b-8K.yaml
• examples/configs/recipes/llm/grpo-gspo-deepscaler-1.5b-8K.yaml
• examples/configs/recipes/llm/grpo-gemma3-27b-it-8n8g-fsdp2tp8-actckpt-long.yaml

examples/configs/recipes/**/*.{yaml,sh}

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

Known exception: Deepscaler recipes may encode context length in place of the cluster tuple (e.g., grpo-deepscaler-1.5b-8K.*); allowed but document intended hardware in the script

Files:

• examples/configs/recipes/llm/grpo-deepscaler-1.5b-8K.yaml
• examples/configs/recipes/llm/grpo-gspo-deepscaler-1.5b-8K.yaml
• examples/configs/recipes/llm/grpo-gemma3-27b-it-8n8g-fsdp2tp8-actckpt-long.yaml

examples/configs/recipes/**

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

Place recipe YAMLs under examples/configs/recipes//

Files:

• examples/configs/recipes/llm/grpo-deepscaler-1.5b-8K.yaml
• examples/configs/recipes/llm/grpo-gspo-deepscaler-1.5b-8K.yaml
• examples/configs/recipes/llm/grpo-gemma3-27b-it-8n8g-fsdp2tp8-actckpt-long.yaml

🧠 Learnings (1)

📓 Common learnings

Learnt from: CR
PR: NVIDIA-NeMo/RL#0
File: CODING_GUIDELINES.md:0-0
Timestamp: 2025-09-20T14:58:45.492Z
Learning: Applies to examples/configs/recipes/**/*.yaml : When adding support for a new model, add a recipe YAML under examples/configs/recipes/ in the appropriate domain (llm/ or vlm/) with the correct name

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)

• GitHub Check: Lint check
• GitHub Check: Post automodel integration comment / Comment on PR
• GitHub Check: Post submodule check comment / Comment on PR

🔇 Additional comments (3)

examples/configs/recipes/llm/grpo-gemma3-27b-it-8n8g-fsdp2tp8-actckpt-long.yaml (1)

8-8: Explicitly clearing safetensors works for v1 policies.
Setting model_save_format to null forces the config back to the default (PyTorch weights) so _v2: false runs avoid the ValueError while leaving the rest of the recipe untouched. Nice tight fix.

examples/configs/recipes/llm/grpo-gspo-deepscaler-1.5b-8K.yaml (1)

13-13: Consistent override keeps v1 runner on safe defaults.
Matching the other recipes, this null override prevents safetensors from being requested in the v1 flow while reusing the shared base config. Looks good.

examples/configs/recipes/llm/grpo-deepscaler-1.5b-8K.yaml (1)

11-11: Override removes the safetensors requirement for v1 nightlies.
This keeps the recipe aligned with _v2: false workers by letting them fall back to the default checkpoint format. Change is clean and targeted.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}