Skip to content
TheChatArchiveThe Chat Archive
DeepSeekdeepseek/deepseek-v4-pro· deepseekHugging Face ibragim-bad/swe_rebench_07_2026_trajectories

ISSUE DESCRIPTION OPD losses error for Megatron when i use megatron for compute losses…

  1. User

    ISSUE DESCRIPTION

    OPD losses error for Megatron when i use megatron for compute losses the code position is: /verl/trainer/distillation/megatron/losses.py Line 248 teacher_topk_log_probs_cp_split, *_ = preprocess_bshd_engine(teacher_topk_log_probs, pre_process=True)

    the teacher_topk_log_probs is [b, length, 64] but preprocess_bshd_engine this function need [n, length] dim, so error happens

    ADDITIONAL ADVICE

    Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use git checkout <file> to restore the file to its original state. You can also reset all changes in the repository using git reset --hard command. Additionally, you can use git stash and git stash pop to temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you call submit) using git status or git diff commands. However you don't need to use git add or git commit before submitting your changes. In fact, submit only submits currently unstaged changes because it uses git diff to get the changes that need to be submitted.

    CHECKLIST

    Before submitting your solution, please go over the following checklist and make sure you've done everything:

    • If an error was reported in the issue description, I have successfully reproduced it.
    • If an error was reported in the issue description, I have confirmed that my fix resolves the error.
    • I have thought about edge cases and made sure my fix handles them as well.
    • I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.

    Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!

    (Current directory: /verl, current file: none) bash-$

  2. DeepSeek Assistant

    Let me start by understanding the issue. The problem is in /verl/trainer/distillation/megatron/losses.py at line 248, where teacher_topk_log_probs has shape [b, length, 64] (3D), but preprocess_bshd_engine expects [n, length] (2D). Let me explore the repository structure and understand the code.

  3. DeepSeek Assistant

    Let me look at the specific file mentioned in the issue.

  4. DeepSeek Assistant

    Now let me look at the preprocess_bshd_engine and preprocess_thd_engine functions to understand their expected input shapes.

  5. DeepSeek Assistant

    Let me look at how compute_forward_kl_topk is called to understand the data flow better

  6. DeepSeek Assistant

    Let me look at the test file and the calling code to understand the expected data

  7. DeepSeek Assistant

    Let me look at how compute_forward_kl_topk is called in production code:

  8. DeepSeek Assistant

    Let me look at the FSDP version for comparison:

  9. DeepSeek Assistant

    Now let me look at `preprocess_thd_engine

  10. DeepSeek Assistant

    Let me look at the preprocess_thd_engine function:

  11. DeepSeek Assistant

    Let me trace where teacher_logprobs and teacher_ids are created:

  12. DeepSeek Assistant

    Let me look at how teacher outputs are padded:

  13. DeepSeek Assistant

    Let me find the correct path.

  14. DeepSeek Assistant

    Now let me look at how the outputs are batched in agent_loop.py

  15. DeepSeek Assistant

    Now I understand the issue. Let me create a reproduction script first to confirm the error.

  16. DeepSeek Assistant

    I've confirmed the error. Now let me fix preprocess_bshd_engine to handle multi-dimensional nested tensors. Let me look at the full function one more time

  17. DeepSeek Assistant

    Let me open the file and see the function:

  18. DeepSeek Assistant

    Let me use the right command format.

  19. DeepSeek Assistant

    Let me see the remaining lines:

  20. DeepSeek Assistant

    Now I have a clear understanding of the issue. The preprocess_bshd_engine function only handles 2D nested tensors [b, jagged] (scalars), but it receives 3D nested tensors [b, jagged, topk] from the teacher log probs. I need to modify it to preserve trailing dimensions, similar to how preprocess_thd_engine already does.

    Let me make the fixes

  21. DeepSeek Assistant

    Let me use replace instead since it's more reliable with the exact text matching

  22. DeepSeek Assistant

    Let me try with heredoc syntax:

  23. DeepSeek Assistant

    Let me use a Python script to make the changes instead:

  24. DeepSeek Assistant

    Let me verify the changes:

  25. DeepSeek Assistant

    Let me verify that expand_as would fail with mismatched dimensions:

  26. DeepSeek Assistant

    Let me check all callers of preprocess_bshd_engine to understand usage:

  27. DeepSeek Assistant

    Let me look at how preprocess_bshd_engine is used in model_forward.py to understand position_ids usage:

  28. DeepSeek Assistant

    I need to fix the expand_as calls for position_ids to work with multi-dimensional input_ids_bshd. Let me update both expand_as calls:

  29. DeepSeek Assistant

    Let me verify the final state of the changes:

  30. DeepSeek Assistant

    Now let me test with the reproduction script to confirm the fix works:

  31. DeepSeek Assistant

    Good, the basic logic works. Now let me run the existing tests to make sure nothing is broken:

  32. DeepSeek Assistant

    Let me run the distillation-related tests:

  33. DeepSeek Assistant

    Let me also run any other related tests I can find:

  34. DeepSeek Assistant

    The megatron test can't run because it requires megatron.core which is not installed in this environment. That's expected - it's a distributed test.

    Let me run any other unit tests that might be affected:

  35. DeepSeek Assistant

    The timeout option isn't there. Let me just run the tests without timeout.

  36. DeepSeek Assistant

    Let me verify the full diff of my changes:

  37. DeepSeek Assistant

    Let me also think about whether the _VocabParallelKLDivergence can handle the 3D output from preprocess_bshd_engine. Let me trace through the forward function

  38. DeepSeek Assistant

    That's expected - the random indices go out of bounds. Let me fix the test:

  39. DeepSeek Assistant

    All changes are verified and working. Let me run the distillation tests one more time to be sure:

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.