[qgallouedec]

Follow-up to #7597, as suggested in #7597 (review).

SDFT and SDPO build their teacher from the student's path, like the GRPO/DPO reference model. With a student passed as an instance, model_init_kwargs is empty, so the teacher loads in fp32 with the default attention whatever the student uses. This applies to the "base" and "ema" teachers (the SDFT and SDPO defaults). Same two setdefault lines as #7597.

Unlike GRPO, both took args.model_init_kwargs by reference, so setdefault would write the dtype into the user's config. I took GRPO's copy line too.

New test in each trainer: a bf16, eager-attention student instance gets a matching teacher. Fails on main (fp32 teacher), passes here.

---

Note

Low Risk
Narrow initialization fix for separate teacher loading; explicit kwargs still override defaults, with regression tests only.

Overview
When SDFT and SDPO construct a separate teacher (base / ema paths that reload from the checkpoint), they now copy model_init_kwargs before applying defaults so setdefault does not mutate the user’s training config.

If the student is passed as an already-loaded model (empty model_init_kwargs), the teacher is loaded with the student’s dtype and attn_implementation instead of defaulting to fp32 and the framework default attention.

Adds test_teacher_model_like_model_instance in both trainer test suites to assert a bf16, eager-attention student gets a matching teacher.

^{Reviewed by Cursor Bugbot for commit 98a3666. Bugbot is set up for automated code reviews on this repo. Configure here.}

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-09T15:46:29.656259Z	98a3666	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[DimensionSTP]

@qgallouedec, #6829 fixes a different branch of this teacher setup. This PR corrects dtype and attention settings when loading a separate teacher. The pure-LoRA EMA branch instead shares the student model and returns before that loading code.

#6829 creates the teacher adapter before ZeRO-3 preparation, clones gathered student adapter state, and gathers teacher parameters while writing EMA updates. Those changes are not covered by the separate-teacher loading fix here.

The current #6829 head, ab8cdacd, incorporates main d9b9f39b5. The official two-rank SDPOTrainer.train() checks pass with DeepSpeed 0.19.3, checkpointing off and non-reentrant checkpointing on, including active distillation and exact EMA/teacher-state checks. Could you review #6829 as the shared-adapter counterpart to this separate-teacher fix?