ISSUE DESCRIPTION OPD losses error for Megatron when i use megatron for compute losses…
- User
ISSUE DESCRIPTION
OPD losses error for Megatron when i use megatron for compute losses the code position is: /verl/trainer/distillation/megatron/losses.py Line 248 teacher_topk_log_probs_cp_split, *_ = preprocess_bshd_engine(teacher_topk_log_probs, pre_process=True)
the teacher_topk_log_probs is [b, length, 64] but preprocess_bshd_engine this function need [n, length] dim, so error happens
Interface
No new interfaces are introduced.
ADDITIONAL ADVICE
Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use
git checkout <file>to restore the file to its original state. You can also reset all changes in the repository usinggit reset --hardcommand. Additionally, you can usegit stashandgit stash popto temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you callsubmit) usinggit statusorgit diffcommands. However you don't need to usegit addorgit commitbefore submitting your changes. In fact,submitonly submits currently unstaged changes because it usesgit diffto get the changes that need to be submitted.CHECKLIST
Before submitting your solution, please go over the following checklist and make sure you've done everything:
- If an error was reported in the issue description, I have successfully reproduced it.
- If an error was reported in the issue description, I have confirmed that my fix resolves the error.
- I have thought about edge cases and made sure my fix handles them as well.
- I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.
Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!
(Current directory: /verl, current file: none) bash-$
Assistant
I'll start by exploring the relevant file.
Assistant
Now let me apply the fix to support extra trailing dims.
Assistant
Now fix the
position_idsexpansion which assumes 2D input.Assistant
Checklist review:
- Reproduced the reported error: Yes. With a stubbed
megatron.core.parallel_state(megatron isn't installed here), callingpreprocess_bshd_enginewith a jagged nested tensor of shape(b, seq, topk)— exactly whatverl/trainer/distillation/megatron/losses.pypasses forteacher_topk_log_probs/teacher_topk_ids— raisedRuntimeError: expand(torch.FloatTensor{[3, 4]}, size=[3]) ..., because the function only allocated a(batch, seqlen)buffer. - Confirmed the fix resolves the error: Yes.
preprocess_bshd_enginenow derivestrailing_dims = input_ids.shape[2:]and allocates(batch, local_seqlen, *trailing_dims)buffers (both the output buffer and the per-sequence CP padding buffer), andposition_idsis explicitly expanded to(batch, local_seqlen)instead ofexpand_as(input_ids_bshd). I verified with a script that for cp∈{1,2}×tp∈{1,2} and every cp_rank, the 3-D result equals stacking per-column 2-D results (so zigzag CP splitting, masks and position ids are unchanged/correct), and then ran the fullcompute_forward_kl_topk(megatron losses) end-to-end on CPU with a gloo process group for bothbshdandthdformats — shapes(bsz, seqlen)are produced and backward gives finite gradients. - Edge cases: 2-D inputs (normal token ids) behave exactly as before (
trailing_dims == ()),need_roll, FP8 padding, and NPU mask paths untouched. I also generalized the analogous CP-alignment padding inpreprocess_thd_engine(d.numel()→d.shape[0]with trailing dims preserved), which was silently wrong for N-D nested inputs (padding was skipped whenseqlen*topk >= align_size); this was caught by my per-column equivalence test for the thd path. Integer/float dtypes both work (ids vs. logprobs). - Ran existing tests:
tests/workers/test_distillation_topk_symmetry_on_cpu.pypasses (4 passed). Other mcore tests require megatron/GPUs and cannot run in this environment.ruff check/ruff formatare clean on the changed file. Temporary scripts were removed; onlyverl/models/mcore/util.pyis modified.
- Reproduced the reported error: Yes. With a stubbed
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →