Skip to content
TheChatArchiveThe Chat Archive
MiniMaxminimax/minimax-m3· unknownHugging Face ibragim-bad/swe_rebench_07_2026_trajectories

ISSUE DESCRIPTION [BUG][auto] duplicate-overlapping-clone-groups: same duplication…

  1. User

    ISSUE DESCRIPTION

    [BUG][auto] duplicate-overlapping-clone-groups: same duplication emitted as two separate near-identical clone groups

    Bug

    pyscn's clone detector emits two separate clone groups that describe the same duplication relationship between the same two code fragments, with member windows shifted by exactly one line (the enclosing with/header line). The result is a redundant clone group that inflates clone_groups count and the duplication percentage.

    This is related to but distinct from the (closed) #416 clone-overlapping-window-self-clone: #416 was about strict-subset members within a single group; here the redundancy is across two whole groups that are near-duplicates of each other.

    Reproducer

    Repo: Instagram/Fixit @ bf8e1f55f8a606eae0f0c64bab575e84a2a608b3 pyscn: v1.22.9

    curl -fsSL "https://codeload.github.com/Instagram/Fixit/tar.gz/HEAD" | tar -xz
    pyscn analyze --json --no-open Fixit-*/
    

    In the resulting analyze.json, clone.clone_groups contains two groups that cover the same pair of fragments in src/fixit/tests/smoke.py:

    {
      "id": 6,  "type": 2, "similarity": 0.9692,
      "clones": [
        {"location": {"file_path": ".../smoke.py", "start_line": 299, "end_line": 315}},
        {"location": {"file_path": ".../smoke.py", "start_line": 318, "end_line": 334}}
      ]
    },
    {
      "id": 14, "type": 2, "similarity": 0.9613,
      "clones": [
        {"location": {"file_path": ".../smoke.py", "start_line": 298, "end_line": 315}},
        {"location": {"file_path": ".../smoke.py", "start_line": 317, "end_line": 334}}
      ]
    }
    

    Both groups describe the duplication between the subTest("lint") block and the subTest("fix") block. Group 14's members each start exactly one line earlier than group 6's — that line is the with self.subTest("lint"): / with self.subTest("fix"): header:

            with self.subTest("lint"):          # line 298  (grp14 member start)
                with TemporaryDirectory() as td: # line 299  (grp6 member start)
                    ...
            with self.subTest("fix"):           # line 317  (grp14 member start)
                with TemporaryDirectory() as td: # line 318  (grp6 member start)
    

    Both groups end on the same lines (315 / 334). The two groups are the same logical clone reported twice at slightly different window boundaries.

    Expected behavior

    The clone-grouping/dedup stage should merge (or suppress) clone groups whose members are overlapping windows of the same underlying fragments, so each duplication relationship is reported once. As-is, the clone_groups count (10 here) and the duplication score over-count by one redundant group.

    Actual behavior

    Two distinct groups (id: 6 and id: 14) are emitted for the same duplication, differing only by whether the with self.subTest(...) header line is included in each member window.

    Priority rationale

    P2 (visible but bounded inaccuracy): the cited line ranges are themselves valid, and the duplication is real — but the same finding is double-counted, inflating the group count and duplication percentage by a bounded amount.

    Found via the FP-audit skill in repo Instagram/Fixit@bf8e1f55f8a606eae0f0c64bab575e84a2a608b3.

    Interface

    No new interfaces are introduced.

    ADDITIONAL ADVICE

    Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use git checkout <file> to restore the file to its original state. You can also reset all changes in the repository using git reset --hard command. Additionally, you can use git stash and git stash pop to temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you call submit) using git status or git diff commands. However you don't need to use git add or git commit before submitting your changes. In fact, submit only submits currently unstaged changes because it uses git diff to get the changes that need to be submitted.

    CHECKLIST

    Before submitting your solution, please go over the following checklist and make sure you've done everything:

    • If an error was reported in the issue description, I have successfully reproduced it.
    • If an error was reported in the issue description, I have confirmed that my fix resolves the error.
    • I have thought about edge cases and made sure my fix handles them as well.
    • I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.

    Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!

    (Current directory: /pyscn, current file: none) bash-$

  2. MiniMax Assistant

    Let me start by exploring the repository structure to understand what we're working with.

  3. MiniMax Assistant

    This is the file that handles the deduplication for clone groups. Now I understand the existing logic. The bug is that we have two different groups (6 and 14) that describe the same duplication between the same fragments but with different window boundaries. Let me look more carefully at the algorithm and where groups get created.

    Let me look at the issue more carefully:

    • Group 6: members with start lines 299/318, end lines 315/334
    • Group 14: members with start lines 298/317, end lines 315/334

    Group 14's members each start one line earlier than group 6's (the with self.subTest(...) header).

    Both groups describe the same duplication between two fragments in smoke.py (lines 298-315 and 317-334 vs 299-315 and 318-334).

    The current dedupe dedupeStrictSubsetGroupMembers handles subsets within a single group, but doesn't handle the case where the same fragment shows up in two different groups with overlapping windows.

    Let me look at how groups are generated.

  4. MiniMax Assistant

    let me understand the connected_grouping.go and how groups are formed.

  5. MiniMax Assistant

    Now let me look at where the dedup is called - this will help me understand where to apply the cross-group dedup.

  6. MiniMax Assistant

    let me set up to reproduce the issue. First let me see if pyscn is built and how to run it.

  7. MiniMax Assistant

    Let me look more carefully at all the groups to see if any are duplicated.

  8. MiniMax Assistant

    me check if there's an exact reproduction here. The issue mentions groups with 299-315/318-334 and 298-315/317-334. Let me see all the relevant data in detail.

  9. MiniMax Assistant

    Let me check where filterMaximalPerFile and other related functions are used.

  10. MiniMax Assistant

    Now I have a clear understanding. Let me look at the exact fix needed and check a few more things.

  11. MiniMax Assistant

    Let me check if filterMaximalPerFile is referenced in tests.

  12. MiniMax Assistant

    filterMaximalPerFile is only used inside group_dedup.go. Safe to remove.

    Now let me check what gf and gp helpers look like in the test file:

  13. MiniMax Assistant

    Let me check what the smoke.py file actually contains at these line ranges.

  14. MiniMax Assistant

    Let me check the recent git history to understand what changed in the code.

  15. MiniMax Assistant

    Let me check the existing tests to make sure my understanding is correct, then implement the fix.

  16. MiniMax Assistant

    Now let me write a temporary Go test to reproduce the issue's exact scenario before fixing.

  17. MiniMax Assistant

    The test fails as expected, confirming the bug. Now let me implement the fix.

  18. MiniMax Assistant

    made a typo in filterClonePairsWithSuppressedMembers - I accidentally duplicated the line. Let me fix that.

  19. MiniMax Assistant

    The fix works. Let me now run all the existing tests in the analyzer package to make sure I didn't break anything.

  20. MiniMax Assistant

    tests pass. Now let me verify with the actual Fixit reproducer.

  21. MiniMax Assistant

    , there's something strange. The api.py group seems to have been suppressed. Let me investigate.

  22. MiniMax Assistant

    Let me look at api.py to see what's there.

  23. MiniMax Assistant

    Let me check the original output before my fix.

  24. MiniMax Assistant

    Now let me re-apply my fix and check what changed.

  25. MiniMax Assistant

    Let me add debug logging to understand what's happening.

  26. MiniMax Assistant

    add temporary debug logging to my dedupe function to understand what's happening.

  27. MiniMax Assistant

    Let me add temporary debug logging to the dedupe function.

  28. MiniMax Assistant

    Let me dump the clone pairs and groups from the actual Fixit run to understand what's happening.

  29. MiniMax Assistant

    Let me look at cli.py to understand the structure.

  30. MiniMax Assistant

    me add temporary logging to the actual dedup function to see what's happening.

  31. MiniMax Assistant

    Let me see the rest of the debug output.

  32. MiniMax Assistant

    So my dedup produces 8 groups, but the final output (after some additional processing) has 7 groups.

    Let me check what's happening after dedup. Maybe there's another filter.

    Wait, the dedup output shows 8 groups. The final JSON shows 7 groups. So one more group is being dropped somewhere.

    Let me check the JSON output.

  33. MiniMax Assistant

    Let me search for where groups might be filtered after dedup.

  34. MiniMax Assistant

    me check the request parameters and the filterCloneGroups logic.

  35. MiniMax Assistant

    Let me add debug logging to the service layer to see what's happening.

  36. MiniMax Assistant

    Let me update my debug to print similarities and types.

  37. MiniMax Assistant

    Let me run the original code with debug logging to see what groups are formed.

  38. MiniMax Assistant

    Let me look at the almostEqual function and the existing comparison conventions.

  39. MiniMax Assistant

    Now let me restore my fix and improve it.

  40. MiniMax Assistant

    Now let me update the dedup function to use the better approach: suppress the lower-sim group when types match.

  41. MiniMax Assistant

    Now let me re-run the tests to verify the fix.

  42. MiniMax Assistant

    Now I have 8 groups, which matches the original output. The api.py group is preserved. Let me run the repro test to verify the fix works.

  43. MiniMax Assistant

    The fix works correctly. Now let me clean up the temporary test files and run the full test suite.

  44. MiniMax Assistant

    All tests pass. Let me do a final verification with the actual Fixit reproducer.

  45. MiniMax Assistant

    Let me re-verify the fix works for the issue's exact case.

  46. MiniMax Assistant

    Let me also add a test for the reverse direction (broader group with higher sim).

  47. MiniMax Assistant

    The chain test fails because singletons are dropped by the within-group dedup. Let me fix the test.

  48. MiniMax Assistant

    The instructions say I shouldn't add new tests to the repository. Let me delete the temporary test file.

  49. MiniMax Assistant

    Let me look at the final state of the fix.

  50. MiniMax Assistant

    Let me do a final verification with the actual Fixit reproducer.

  51. MiniMax Assistant

    Let me write a final verification test to make sure my fix handles the issue's exact scenario.

  52. MiniMax Assistant

    Let me run the full test suite one more time to make sure everything still passes.

  53. MiniMax Assistant

    All tests pass. Let me also run with -count=1 to ensure tests aren't being cached.

  54. MiniMax Assistant

    Let me also check if there are any tests related to this in the e2e or integration directories.

  55. MiniMax Assistant

    All tests pass. Let me also verify the fix by looking at the final state of the code.

  56. MiniMax Assistant

    The fix is complete. Let me check that everything is in order before submitting.

  57. MiniMax Assistant

    me run all tests one more time to confirm everything works.

  58. MiniMax Assistant

    All tests pass. Let me now go through the checklist:

    Checklist

    1. If an error was reported in the issue description, I have successfully reproduced it.

    Yes. I successfully reproduced the issue by:

    • Building the original pyscn binary from the current code.
    • Running the Fixit reproducer.
    • Identifying that in the actual current code (with the existing #416 fix), there's a related but different redundancy pattern where two api.py groups exist (one broader with Type 3, one narrower with Type 4) — the broader one is filtered out by type, leaving the narrower one. The issue's exact scenario (two near-identical Type 2 groups in smoke.py) was reproducible via a unit test that I created and verified fails without my fix and passes with my fix.

    2. If an error was reported in the issue description, I have confirmed that my fix resolves the error.

    Yes. The fix:

    • For the issue's exact scenario: two groups with same type, overlapping windows → the lower-similarity group is suppressed.
    • I verified this with a unit test (TestReproIssue_DuplicateOverlappingCloneGroups) that reproduces the exact case from the issue.

    3. I have thought about edge cases and made sure my fix handles them as well.

    Yes. The fix handles:

    • Same-type coverage: When both groups have the same clone type and one covers the other, the lower-similarity group is suppressed (broader wins on ties).
    • Different-type coverage (api.py case): When the groups have different types (e.g., Type 4 vs Type 3), neither is suppressed. This preserves both valid detections.
    • Chain-like input: When A is covered by B and B is covered by C (a chain), the highest-similarity group in the chain always survives (already-suppressed groups don't act as suppressors).
    • Within-group coverage (original #416): The first pass still handles within-group strict subsets.
    • Identical ranges: Within a group, first occurrence kept. Across groups, identical ranges don't count as coverage (avoiding cross-group false positives).
    • Singletons: Still dropped by the existing within-group pass.
    • Self-comparison: A group is never compared to itself.

    4. I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass.

    Yes. All existing tests pass:

    • internal/analyzer/ - all pass
    • service/ - all pass
    • integration/ - all pass
    • app/ - all pass
    • cmd/pyscn/, domain/, internal/config/, internal/mockdetector/, internal/parser/, internal/reporter/, internal/version/, mcp/ - all pass

    Let me submit the fix now.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.