OPD losses error for Megatron when i use megatron for compute losses the code position…
- User
OPD losses error for Megatron when i use megatron for compute losses the code position is: /verl/trainer/distillation/megatron/losses.py Line 248 teacher_topk_log_probs_cp_split, *_ = preprocess_bshd_engine(teacher_topk_log_probs, pre_process=True)
the teacher_topk_log_probs is [b, length, 64] but preprocess_bshd_engine this function need [n, length] dim, so error happens
Interface
No new interfaces are introduced.
- Assistant
Let me look at how the function is used and check for related tests.
- Assistant
Now let me verify megatron is available so I can test:
- Assistant
Megatron isn't installed here, so I'll test with a stubbed
megatron.coremodule: - Assistant
The package
__init__pulls heavier megatron deps. I'll loadutil.pydirectly: - Assistant
CI CPU tests use
*_on_cpu.pypattern. I'll add a regression test that skips when megatron isn't importable: - Assistant
All 3 pass with real megatron-core. Now run the existing distillation CPU test and lint checks:
- Assistant
The exact reported failure path now works end-to-end, including backward. Let me clean up temp files and show the final diff:
- Assistant
Fixed. The root cause:
preprocess_bshd_enginehardcoded a 2-D(batch, seqlen)output buffer, while the Megatron OPD loss (verl/trainer/distillation/megatron/losses.py:291) feeds it teacher top-k nested tensors shaped(b, seqlen, topk). The thd path already handled this (it buildsshape = list(input_ids.shape[1:])atutil.py:378), so I brought the bshd path to parity.Changes
verl/models/mcore/util.py(preprocess_bshd_engine) — no interface changes:- Allocate the output buffer with trailing dims preserved:
(batch, local_max_seqlen, *input_ids.shape[2:]) - Pad per-sequence buffers in the CP branch with trailing dims
- Replace the two
position_ids.expand_as(input_ids_bshd)calls with explicitexpand(batch_size, local_max_seqlen)—expand_aswould otherwise break (or produce a wrong 3-Dposition_ids) on 3-D inputs
tests/models/test_mcore_bshd_engine_trailing_dims_on_cpu.py(new) — regression tests covering 2-D backward-compat,(b, seqlen, topk)with TP alignment padding, and the zigzag CP split with trailing dims (verifies rank r holds chunks r and 2·cp−1−r, positions included). Skips when megatron-core isn't installed.Verification
- Installed
megatron-core— all 3 new tests pass - End-to-end repro of the reported failure:
compute_forward_kl_topk(..., data_format="bshd")with teacher tensors[b, len, 64]on a 1-proc gloo group — forward + backward succeed, outputs shaped(b, seqlen)with finite grads - Existing
tests/workers/test_distillation_topk_symmetry_on_cpu.py— 4 passed pre-commit runon both files — all hooks pass (ruff, mypy, license, etc.)
One heads-up per the repo's contribution policy: before opening a PR, run the duplicate-work checks (
gh pr list --repo verl-project/verl --state open --search "bshd distillation"etc.), and note the PR description must state AI assistance was used along with the test commands above. - Allocate the output buffer with trailing dims preserved:
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- fable-5Compare side by side →