Skip to content
TheChatArchiveThe Chat Archive
MiniMaxminimax/minimax-m3· unknownHugging Face ibragim-bad/swe_rebench_07_2026_trajectories

ISSUE DESCRIPTION OPD losses error for Megatron when i use megatron for compute losses…

  1. User

    ISSUE DESCRIPTION

    OPD losses error for Megatron when i use megatron for compute losses the code position is: /verl/trainer/distillation/megatron/losses.py Line 248 teacher_topk_log_probs_cp_split, *_ = preprocess_bshd_engine(teacher_topk_log_probs, pre_process=True)

    the teacher_topk_log_probs is [b, length, 64] but preprocess_bshd_engine this function need [n, length] dim, so error happens

    Interface

    No new interfaces are introduced.

    ADDITIONAL ADVICE

    Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use git checkout <file> to restore the file to its original state. You can also reset all changes in the repository using git reset --hard command. Additionally, you can use git stash and git stash pop to temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you call submit) using git status or git diff commands. However you don't need to use git add or git commit before submitting your changes. In fact, submit only submits currently unstaged changes because it uses git diff to get the changes that need to be submitted.

    CHECKLIST

    Before submitting your solution, please go over the following checklist and make sure you've done everything:

    • If an error was reported in the issue description, I have successfully reproduced it.
    • If an error was reported in the issue description, I have confirmed that my fix resolves the error.
    • I have thought about edge cases and made sure my fix handles them as well.
    • I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.

    Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!

    (Current directory: /verl, current file: none) bash-$

  2. MiniMax Assistant

    Let me start by exploring the repository structure to understand the codebase.

  3. MiniMax Assistant

    Now let me look at the preprocess_bshd_engine function to understand its input expectations.

  4. MiniMax Assistant

    Let me see how preprocess_thd_engine handles similar input:

  5. MiniMax Assistant

    Let me look at how teacher_topk_log_probs is constructed in the codebase:

  6. MiniMax Assistant

    Let me look at the FSDP version to see how it handles CP:

  7. MiniMax Assistant

    Now I see the FSDP version uses values().unsqueeze(0) to convert the nested tensor to a dense tensor, then uses slice_input_tensor to split it along dimension 1 (sequence).

    Let me look at slice_input_tensor to understand it:

  8. MiniMax Assistant

    Let me look at existing tests to understand the expected behavior better:

  9. MiniMax Assistant

    Let me check the test_distillation_topk_symmetry_on_cpu.py:

  10. MiniMax Assistant

    Now let me check the _nested_from_rows to understand the format:

  11. MiniMax Assistant

    Let me find where the teacher_topk_log_probs is constructed:

  12. MiniMax Assistant

    Let me look at where teacher_logprobs are constructed:

  13. MiniMax Assistant

    Let me look at how the teacher_logprobs becomes a nested tensor:

  14. MiniMax Assistant

    Let me look at the megatron CP code:

  15. MiniMax Assistant

    Let me look at preprocess_bshd_engine more carefully, specifically the CP splitting part:

  16. MiniMax Assistant

    Let me look at preprocess_thd_engine more carefully:

  17. MiniMax Assistant

    Let me check the nested tensor API:

  18. MiniMax Assistant

    Let me check where these preprocess functions are used:

  19. MiniMax Assistant

    The preprocess functions are used in:

    1. model_forward.py - for general model forward, where input is 1D (token IDs).
    2. model_forward_fused.py - for fused model forward, where input is 1D (token IDs).
    3. trainer/distillation/megatron/losses.py - THIS IS WHERE THE BUG IS. The input is 2D (topk logprobs/ids).
    4. router_replay_utils.py - need to check.

    Let me look at router_replay_utils.py:

  20. MiniMax Assistant

    So preprocess_thd_engine is used in router_replay_utils.py where layers_topk_idx is a nested tensor that includes a feature dim. Let me check this:

  21. MiniMax Assistant

    OK, in router_replay_utils.py, layers_topk_idx is converted to a dense tensor first (line 248: gather_from_sequence_parallel_region(layers_topk_idx, ...).unsqueeze(0).contiguous()), so it's not a nested tensor by the time preprocess_thd_engine is called.

    But wait, line 290 checks if layers_topk_idx.is_nested:. Let me look more carefully:

  22. MiniMax Assistant

    Let me look at the existing test for this code:

  23. MiniMax Assistant

    Let me first write a reproducer to confirm the bug:

  24. MiniMax Assistant

    The megatron module is not installed. Let me check what's available:

  25. MiniMax Assistant

    Megatron isn't installed. Let me read the preprocess function and analyze it carefully to confirm the bug:

  26. MiniMax Assistant

    Let me trace back to understand the data shape better:

  27. MiniMax Assistant

    Let me look at the data structure more carefully. Let me create a comprehensive test script to understand:

  28. MiniMax Assistant

    Let me also check the layers_topk_idx case:

  29. MiniMax Assistant

    Let me check the router_replay_utils.py to see if it would also have this issue:

  30. MiniMax Assistant

    Looking at this, the comment says "shape [bs, max_seq_len, layer_num, topk]". But if layers_topk_idx.is_nested is True, it means the data is a nested tensor with a jagged dim at position 1 (the seq_len dim). The inner shape would be (seq_len_i, layer_num, topk).

    So this code path has the same bug. However, the actual data shape might differ - let me check more carefully:

  31. MiniMax Assistant

    Let me look at the structure of the losses file to understand how to add the fix:

  32. MiniMax Assistant

    Let me write a test to confirm the issue with bshd:

  33. MiniMax Assistant

    Now let me also test the thd function:

  34. MiniMax Assistant

    Now I have a clear picture. The issue is specifically with preprocess_bshd_engine. The thd function already handles 2D nested tensors correctly. Let me implement the fix:

  35. MiniMax Assistant

    Now I have a clear understanding of the issue and the fix. Let me apply the fix to preprocess_bshd_engine in util.py:

  36. MiniMax Assistant

    Now let me apply the fix:

  37. MiniMax Assistant

    Let me now also think about the thd function with cp_size > 1. Let me write a test for the cp_size > 1 case:

  38. MiniMax Assistant

    The thd function works for 2D nested tensors. So I only need to fix the bshd function. Let me apply the fix:

  39. MiniMax Assistant

    Now let me fix the seq_padded allocation:

  40. MiniMax Assistant

    I have a duplicate seq = input_ids[i] line. Let me fix that:

  41. MiniMax Assistant

    Let me delete the duplicate line:

  42. MiniMax Assistant

    The fix is applied. Now let me also test the fix by simulating the function logic:

  43. MiniMax Assistant

    Now let me verify the fix is properly in the file:

  44. MiniMax Assistant

    Let me also verify the thd function works for 2D:

  45. MiniMax Assistant

    Now let me check the postprocess_bshd_engine function to make sure it works with 2D outputs:

  46. MiniMax Assistant

    The postprocess function should also work for 2D. Let me now also think about whether the thd function needs a similar fix:

  47. MiniMax Assistant

    Let me also think about whether there are any other places that might have issues. Let me look at the need_roll case for thd:

  48. MiniMax Assistant

    Let me now also check the existing test setup and run a quick check that the code is syntactically valid:

  49. MiniMax Assistant

    Let me also look at the existing test for the megatron KL loss to see if it tests the BSHD case:

  50. MiniMax Assistant

    The existing test only tests the "thd" case. Let me look at the data flow more carefully:

  51. MiniMax Assistant

    Let me also run the existing test to make sure I didn't break anything:

  52. MiniMax Assistant

    Let me also update the docstring of the function to reflect the new behavior:

  53. MiniMax Assistant

    Let me use a different approach with replace:

  54. MiniMax Assistant

    Let me use a heredoc to handle the special characters:

  55. MiniMax Assistant

    Now let me also verify the syntax is OK:

  56. MiniMax Assistant

    Now let me also write a more comprehensive test to ensure the fix works in all scenarios:

  57. MiniMax Assistant

    All tests pass! Let me also verify by looking at the diff:

  58. MiniMax Assistant

    Let me also verify by looking at the issue once more and checking if my fix addresses the issue:

  59. MiniMax Assistant

    Now let me also think about the thd function in the thd case. Let me check if the thd function's existing logic for 2D nested tensors is correct. Let me check what happens when the input is 2D in thd:

  60. MiniMax Assistant

    Test 3 had a wrong expected shape. Let me fix the test:

  61. MiniMax Assistant

    Hmm, the actual shape is [6, 64], not [8, 64]. Let me check the function logic again:

    align_size = tp_size * cp_size * 2 if cp_size > 1 else tp_size
    

    With tp_size=1, cp_size=2, align_size = 122 = 4.

    seqlens_in_batch = input_ids.offsets().diff()
    pad_size = (align_size - seqlens_in_batch % align_size) % align_size
    seqlens_in_batch_padded = seqlens_in_batch + pad_size
    

    seqlens_in_batch = [4, 6] pad_size = (4 - 4%4) % 4, (4 - 6%4) % 4 = 0, 2 seqlens_in_batch_padded = [4, 8]

    cu_seqlens_padded = [0, 4, 12] sum(seqlens_in_batch_padded_cpu) = 4 + 8 = 12 shape[0] = 12 / 2 = 6

    So the actual shape is [6, 64]. OK my test had wrong calculation. Let me also try with tp_size=2:

    With tp_size=2, cp_size=2, align_size = 222 = 8. pad_size = (8 - 4%8) % 8, (8 - 6%8) % 8 = 4, 2 seqlens_in_batch_padded = [8, 8] sum = 16 shape[0] = 16/2 = 8

    So with tp_size=2, the shape would be [8, 64]. Let me update the test:

  62. MiniMax Assistant

    Good, the thd function works correctly for 2D nested tensors. Now let me also run the existing test to make sure my fix doesn't break anything. Let me look at the test:

  63. MiniMax Assistant

    The test requires distributed mode and Megatron. Let me also check if I can run any CPU tests:

  64. MiniMax Assistant

    Let me try running this test to see if it works:

  65. MiniMax Assistant

    The existing tests pass. Let me also run the other related tests:

  66. MiniMax Assistant

    All relevant tests pass. Let me also run the other tests in the workers folder:

  67. MiniMax Assistant

    All 30 tests pass. Let me also try the models tests:

  68. MiniMax Assistant

    Some tests can't run because of missing dependencies. That's not related to my fix. Let me focus on tests that can run:

  69. MiniMax Assistant

    These tests can't run because of missing dependencies. Let me run only the tests that I can:

  70. MiniMax Assistant

    All 88 tests pass. My fix doesn't break any existing tests. Let me also clean up the temporary test files:

  71. MiniMax Assistant

    Good, all temporary files are cleaned up. Let me verify my final fix:

  72. MiniMax Assistant

    Let me also re-verify my fix by looking at the relevant code:

  73. MiniMax Assistant

    The fix is complete. Let me do a final review:

  74. MiniMax Assistant

    Let me do a final check on the diff and review:

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.