ISSUE DESCRIPTION OPD losses error for Megatron when i use megatron for compute losses…
- User
ISSUE DESCRIPTION
OPD losses error for Megatron when i use megatron for compute losses the code position is: /verl/trainer/distillation/megatron/losses.py Line 248 teacher_topk_log_probs_cp_split, *_ = preprocess_bshd_engine(teacher_topk_log_probs, pre_process=True)
the teacher_topk_log_probs is [b, length, 64] but preprocess_bshd_engine this function need [n, length] dim, so error happens
Interface
No new interfaces are introduced.
ADDITIONAL ADVICE
Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use
git checkout <file>to restore the file to its original state. You can also reset all changes in the repository usinggit reset --hardcommand. Additionally, you can usegit stashandgit stash popto temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you callsubmit) usinggit statusorgit diffcommands. However you don't need to usegit addorgit commitbefore submitting your changes. In fact,submitonly submits currently unstaged changes because it usesgit diffto get the changes that need to be submitted.CHECKLIST
Before submitting your solution, please go over the following checklist and make sure you've done everything:
- If an error was reported in the issue description, I have successfully reproduced it.
- If an error was reported in the issue description, I have confirmed that my fix resolves the error.
- I have thought about edge cases and made sure my fix handles them as well.
- I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.
Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!
(Current directory: /verl, current file: none) bash-$
Assistant
I'll start by exploring the repository structure and understanding the issue.
Assistant
Let me find the file.
Assistant
Now let me look at the
preprocess_bshd_engineandpreprocess_thd_enginefunctions.Assistant
Let me look at the full
util.pyto understandpreprocess_bshd_engineandpreprocess_thd_enginebetter. Let me also look at howcompute_forward_kl_topkis called.Assistant
Let me look at the test file to understand the expected input shape.
Assistant
Let me look at
preprocess_thd_engine.Assistant
Let me look at the distillation losses.py to understand the flow and how data is prepared.
Assistant
Let me look at how
teacher_logprobs/teacher_idsare prepared and where the logits processor calls compute_topk_loss.Assistant
Let me look at the full
preprocess_bshd_enginefunction once more, carefully.Assistant
Let me find all usages of
preprocess_bshd_engineto understand the impact of changes.Assistant
Let me look at how
preprocess_bshd_engineis used inmodel_forward.py.Assistant
Let me write a quick test script to understand the shape behavior of jagged nested tensors with trailing dims.
Assistant
Let me look at the megatron transformer_impl.py to understand the bshd data flow for distillation.
Assistant
Let me check whether megatron is available and run the existing test setup to understand the environment.
Assistant
Megatron isn't installed. Let me write a reproduction script that mocks the
mpufunctions to testpreprocess_bshd_enginedirectly with a 2D-element nested tensor (simulating the bshd distillation path).Assistant
Let me make a more complete mock of megatron to bypass the import chain.
Assistant
I've reproduced the error. The root cause is that
preprocess_bshd_enginedoesn't preserve the trailing dense dimension (e.g.,topk) of the jagged nested tensor, whilepreprocess_thd_enginedoes. Let me examine the exact lines to fix.Assistant
Now I understand the issue completely. Let me implement the fix in
preprocess_bshd_engineto preserve the trailing dense dimension (likepreprocess_thd_enginedoes). First, let me add thetrailing_shapecomputation and update theinput_ids_bshdcreation.Assistant
Let me verify the changes look correct.
Assistant
Now let me run the reproduction script again to verify the fix works for the 2D case, and also add a check for the 1D case to ensure I didn't break it.
Assistant
The fix works for both 1D and 2D cases. Now let me test the CP (context parallel) branch with cp_size=2 to ensure it handles 2D elements correctly too.
Assistant
The CP branch works correctly too. Let me also test the
need_roll=Truepath with 2D elements to ensure the model forward path (labels) isn't broken.Assistant
Let me re-examine the
compute_forward_kl_topkin losses.py to confirm the data flow and assertion ordering.Assistant
The data flow is confirmed. Now let me try to run the full
compute_forward_kl_topkend-to-end with a single process to validate the bshd path. First, let me check if I can install megatron-core.Assistant
Megatron-core is installed. Let me verify the import works and check the existing test can at least be collected.
Assistant
Megatron imports work. Now let me write a single-process end-to-end test that runs
compute_forward_kl_topkwithdata_format="bshd"and compares against the FSDP reference implementation.Assistant
The assertion fails because my test passes student logits in the wrong shape for bshd. For bshd, the model produces logits as
(B, S, V)(not packed(1, B*S, V)like thd). Let me fix the test to use the correct shapes per format.Assistant
The VP (megatron) bshd path now passes its assertion. The error moved to the FSDP reference. Let me look at the FSDP
compute_forward_kl_topk.Assistant
The FSDP reference always packs into
(1, total_nnz, ...). Let me rewrite the test to flatten both outputs for comparison since the shapes differ between bshd (VP keeps(B,S)) and the packed FSDP reference.Assistant
Both bshd and thd paths work end-to-end and match the FSDP reference. Let me run the existing relevant tests to make sure nothing is broken. Let me find them.
Assistant
The existing test passes. Let me check if there are GPUs available to run the multi-process TP test.
Assistant
No GPUs. The multi-process TP test requires CUDA. My single-process gloo test already validates the megatron KL loss path. Let me search for any other tests touching
preprocess_bshd_engineor the model_forward bshd path.Assistant
No tests directly test
preprocess_bshd_engine. Let me verify thatinput_ids.shape[2:]returns concrete ints (not symbolic) that can be safely passed totorch.zeros.Assistant
trailing_shapereturns concrete ints. Now let me check ifpostprocess_bshd_enginealready supports 3D (to confirm consistency), and run any related mcore model tests.Assistant
Confirmed —
postprocess_bshd_engine(line 720) and the thd variants already support trailing dims via*output.shape[2:]. My fix makespreprocess_bshd_engineconsistent. Let me run a broader set of related tests.Assistant
Let me check the tests/utils/megatron directory and the distillation tests directory.
Assistant
Let me verify the full module imports correctly with the installed megatron, and run a final comprehensive validation.
Assistant
Let me run all my validation scripts and the existing test one final time to confirm everything works.
Assistant
All tests pass. Let me clean up the temporary test files and review the final diff.
Assistant
The diff is clean and minimal. Let me do a final check of git status and confirm only the intended file is changed.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →