Skip to content
TheChatArchiveThe Chat Archive
xiaomi/mimo-v2.5-pro· unknownHugging Face ibragim-bad/swe_rebench_07_2026_trajectories

ISSUE DESCRIPTION OPD losses error for Megatron when i use megatron for compute losses…

  1. User

    ISSUE DESCRIPTION

    OPD losses error for Megatron when i use megatron for compute losses the code position is: /verl/trainer/distillation/megatron/losses.py Line 248 teacher_topk_log_probs_cp_split, *_ = preprocess_bshd_engine(teacher_topk_log_probs, pre_process=True)

    the teacher_topk_log_probs is [b, length, 64] but preprocess_bshd_engine this function need [n, length] dim, so error happens

    Interface

    No new interfaces are introduced.

    ADDITIONAL ADVICE

    Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use git checkout <file> to restore the file to its original state. You can also reset all changes in the repository using git reset --hard command. Additionally, you can use git stash and git stash pop to temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you call submit) using git status or git diff commands. However you don't need to use git add or git commit before submitting your changes. In fact, submit only submits currently unstaged changes because it uses git diff to get the changes that need to be submitted.

    CHECKLIST

    Before submitting your solution, please go over the following checklist and make sure you've done everything:

    • If an error was reported in the issue description, I have successfully reproduced it.
    • If an error was reported in the issue description, I have confirmed that my fix resolves the error.
    • I have thought about edge cases and made sure my fix handles them as well.
    • I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.

    Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!

    (Current directory: /verl, current file: none) bash-$

  2. Assistant

    I'll start by exploring the repository structure to understand the codebase and locate the problematic file.

  3. Assistant

    Let me look at the file mentioned in the issue.

  4. Assistant

    Let me navigate to line 248 to see the problematic code.

  5. Assistant

    Now let me look at the preprocess_bshd_engine function to understand what it expects.

  6. Assistant

    Let me search for the preprocess_bshd_engine function definition.

  7. Assistant

    Let me examine the preprocess_bshd_engine function to understand its expectations.

  8. Assistant

    Let me look at the preprocess_bshd_engine function more carefully, and also the preprocess_thd_engine function to understand the expected input format.

    The issue is that preprocess_bshd_engine expects a nested tensor with 2D elements [n, length] (jagged), but teacher_topk_log_probs has shape [b, length, 64] which is 3D.

    Let me look at what preprocess_bshd_engine does and how it accesses the data.

  9. Assistant

    Let me look at the preprocess_thd_engine function to understand how it handles nested tensors with an extra dimension:

  10. Assistant

    Let me look at the preprocess_thd_engine function to understand its signature and how it handles tensors with extra dimensions.

  11. Assistant

    Let me scroll down to see the full implementation of preprocess_bshd_engine and preprocess_thd_engine for comparison:

  12. Assistant

    Let me look at how the input tensor is expected and used in the function. Let me look at the relevant lines around 600-640:

  13. Assistant

    Let me look at the preprocess_thd_engine function more carefully to understand how it handles the extra dimensions:

  14. Assistant

    Let me look at the preprocess_thd_engine function to see how it handles the shape. In particular, I want to understand how it works with nested tensors that have extra dimensions.

  15. Assistant

    Now let me understand the full picture. Let me look at the compute_forward_kl_topk function and understand the shape expectations:

  16. Assistant

    Let me look at how preprocess_thd_engine handles the inner shape for comparison:

  17. Assistant

    Let me look at the full preprocess_bshd_engine function to understand all the places where it handles the inner tensor:

  18. Assistant

    Now I understand the issue. The preprocess_bshd_engine function creates a 2D output tensor assuming the nested tensors are 1D sequences, but teacher_topk_log_probs and teacher_topk_ids are nested tensors where each element has shape (seq_len, topk) (2D).

    Let me look at how preprocess_thd_engine handles this (lines 378-381):

    • shape = list(input_ids.shape[1:]) - gets the shape of inner tensors (e.g., [topk])
    • shape[0] = sum(seqlens_in_batch_padded_cpu) // cp_size - replaces the sequence dimension

    The preprocess_bshd_engine needs to be fixed similarly to handle nested tensors with extra dimensions.

    Let me also check if there's a non-nested version of the tensors being passed. Let me look at where compute_forward_kl_topk is called:

  19. Assistant

    Let me look at the fsdp version to understand how it handles nested tensors:

  20. Assistant

    Now I understand the issue! Looking at the FSDP version (lines 55-57), it converts the nested tensors to regular tensors by calling .values().unsqueeze(0). The megatron version tries to pass the nested tensors directly to preprocess_bshd_engine, which doesn't handle nested tensors with extra dimensions (like topk=64).

    The preprocess_bshd_engine function creates a 2D output tensor (batch_size, local_max_seqlen) assuming 1D inner tensors, but the nested tensors have inner shape (seq_len, 64).

    Let me fix the preprocess_bshd_engine function to handle multi-dimensional nested tensors. The fix should be similar to how preprocess_thd_engine handles it:

  21. Assistant

    Now I understand the issue. The preprocess_bshd_engine function creates a 2D output tensor but the nested tensors have inner shape (seq_len, topk) (3D). Let me also check how the postprocess_bshd_engine handles the return value since position_ids has expand_as(input_ids_bshd):

  22. Assistant

    Now I understand the full picture. The preprocess_bshd_engine function assumes nested tensors with 1D elements (just token IDs), but teacher_topk_log_probs and teacher_topk_ids are nested tensors with 2D elements (shape (seq_len, topk)).

    Looking at the code:

    1. Line 601: input_ids_bshd = torch.zeros(batch_size, local_max_seqlen, ...) — creates 2D tensor
    2. Line 607: input_ids_bshd[i, :seqlen_i] = input_ids[i] — tries to assign 2D data to 1D slice
    3. Line 612: seq_padded = torch.zeros(max_seqlen, ...) — creates 1D padding tensor

    The fix needs to handle the extra dimensions. Let me check what preprocess_thd_engine does at line 378 for comparison:

    shape = list(input_ids.shape[1:])  # Gets inner shape (e.g., [topk])
    shape[0] = sum(seqlens_in_batch_padded_cpu) // cp_size  # Replace seq len dimension
    

    Now let me implement the fix:

  23. Assistant

    Now I need to fix the padding tensor creation at line 617 to also include inner_shape:

  24. Assistant

    Now I need to think about the position_ids at line 647:

    position_ids = position_ids.unsqueeze(0).expand_as(input_ids_bshd)
    

    If input_ids_bshd is 3D (batch_size, local_max_seqlen, topk), expand_as would try to expand position_ids to 3D, which doesn't make sense. Position IDs should be 2D (batch_size, local_max_seqlen) regardless of extra dimensions. Let me fix that:

  25. Assistant

    Now I also need to fix the other case (cp_size > 1) at line 659 where it also does expand_as:

  26. Assistant

    Let me review the complete modified function to ensure all changes are correct:

  27. Assistant

    Let me also verify the postprocess function handles 3D tensors correctly. Looking at line 710-711:

    mask = attention_mask[i].bool()
    output_new.append(output[i][mask])
    

    For 3D output (batch_size, local_max_seqlen, topk), output[i] would be (local_max_seqlen, topk) and mask is1D (local_max_seqlen,). The indexing should work correctly to select rows.

    Now let me check if there are any tests I should run to verify the fix:

  28. Assistant

    Let me check the test file to understand how the losses are tested:

  29. Assistant

    Let me continue reading the test file to understand how the nested tensors are created:

  30. Assistant

    Looking at the test file, I can see that:

    1. Line 142-143: The test creates nested tensors using self.to_nested() which converts regular tensors to nested tensors with jagged layout.
    2. The test uses data_format="thd" (line 152), which means it's using preprocess_thd_engine, not preprocess_bshd_engine.
    3. The test doesn't test the data_format="bshd" case, which is the one that has the issue.

    So my fix should handle the data_format="bshd" case correctly for nested tensors with extra dimensions.

    Now let me think about edge cases and ensure my fix is correct:

    1. For1D nested tensors (original case), inner_shape would be empty, and the code should work exactly as before.
    2. For2D nested tensors (like teacher_topk_log_probs with shape (batch_size, [seq_len, topk])), inner_shape would be [topk], and the tensors would be created with the correct shape.
    3. The postprocess_bshd_engine function should handle3D tensors correctly since it uses output.shape[2:] when creating temporary tensors (line 719).

    Let me also verify that the position_ids expansion is correct. I changed it from expand_as(input_ids_bshd) to expand(batch_size, local_max_seqlen). This ensures position_ids are always 2D regardless of the input tensor's extra dimensions.

    Let me run a quick test to verify the fix works. First, let me check if there are any syntax errors in my changes:

  31. Assistant

    Let me check if megatron is installed:

  32. Assistant

    Megatron is not installed. Let me just do a basic syntax check on the file.

  33. Assistant

    Let me also check the losses.py file:

  34. Assistant

    Now let me also check if the d.numel() < align_size check needs updating for multi-dimensional tensors in the preprocess_thd_engine function (for reference):

  35. Assistant

    Let me review the complete diff of my changes:

  36. Assistant

    Let me check if there are any other places in the codebase that use preprocess_bshd_engine to ensure my changes don't break anything:

  37. Assistant

    Let me check the usage in model_forward.py to make sure my changes don't break those usages:

  38. Assistant

    Let me check what v is in the logits_processor_args case:

  39. Assistant

    Let me also check if there are any other files that use preprocess_bshd_engine:

  40. Assistant

    Now let me check if there are any other tests that might be affected:

  41. Assistant

    Let me create a small test to verify the fix works correctly. I'll simulate the scenario without requiring megatron:

  42. Assistant

    Let me fix the test script.

  43. Assistant

    Let me create a more comprehensive test that simulates the preprocess_bshd_engine function with my changes:

  44. Assistant

    Let me clean up the test file:

  45. Assistant

    Now let me verify my changes are complete and correct by reviewing the final diff:

  46. Assistant

    Let me now verify the final state of my changes:

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.