Skip to content
TheChatArchiveThe Chat Archive
gpt-5.6-sol· openaiHugging Face ibragim-bad/swe_rebench_07_2026_trajectories

ISSUE DESCRIPTION [BUG][auto] duplicate-overlapping-clone-groups: same duplication…

  1. User

    ISSUE DESCRIPTION

    [BUG][auto] duplicate-overlapping-clone-groups: same duplication emitted as two separate near-identical clone groups

    Bug

    pyscn's clone detector emits two separate clone groups that describe the same duplication relationship between the same two code fragments, with member windows shifted by exactly one line (the enclosing with/header line). The result is a redundant clone group that inflates clone_groups count and the duplication percentage.

    This is related to but distinct from the (closed) #416 clone-overlapping-window-self-clone: #416 was about strict-subset members within a single group; here the redundancy is across two whole groups that are near-duplicates of each other.

    Reproducer

    Repo: Instagram/Fixit @ bf8e1f55f8a606eae0f0c64bab575e84a2a608b3 pyscn: v1.22.9

    curl -fsSL "https://codeload.github.com/Instagram/Fixit/tar.gz/HEAD" | tar -xz
    pyscn analyze --json --no-open Fixit-*/
    

    In the resulting analyze.json, clone.clone_groups contains two groups that cover the same pair of fragments in src/fixit/tests/smoke.py:

    {
      "id": 6,  "type": 2, "similarity": 0.9692,
      "clones": [
        {"location": {"file_path": ".../smoke.py", "start_line": 299, "end_line": 315}},
        {"location": {"file_path": ".../smoke.py", "start_line": 318, "end_line": 334}}
      ]
    },
    {
      "id": 14, "type": 2, "similarity": 0.9613,
      "clones": [
        {"location": {"file_path": ".../smoke.py", "start_line": 298, "end_line": 315}},
        {"location": {"file_path": ".../smoke.py", "start_line": 317, "end_line": 334}}
      ]
    }
    

    Both groups describe the duplication between the subTest("lint") block and the subTest("fix") block. Group 14's members each start exactly one line earlier than group 6's — that line is the with self.subTest("lint"): / with self.subTest("fix"): header:

            with self.subTest("lint"):          # line 298  (grp14 member start)
                with TemporaryDirectory() as td: # line 299  (grp6 member start)
                    ...
            with self.subTest("fix"):           # line 317  (grp14 member start)
                with TemporaryDirectory() as td: # line 318  (grp6 member start)
    

    Both groups end on the same lines (315 / 334). The two groups are the same logical clone reported twice at slightly different window boundaries.

    Expected behavior

    The clone-grouping/dedup stage should merge (or suppress) clone groups whose members are overlapping windows of the same underlying fragments, so each duplication relationship is reported once. As-is, the clone_groups count (10 here) and the duplication score over-count by one redundant group.

    Actual behavior

    Two distinct groups (id: 6 and id: 14) are emitted for the same duplication, differing only by whether the with self.subTest(...) header line is included in each member window.

    Priority rationale

    P2 (visible but bounded inaccuracy): the cited line ranges are themselves valid, and the duplication is real — but the same finding is double-counted, inflating the group count and duplication percentage by a bounded amount.

    Found via the FP-audit skill in repo Instagram/Fixit@bf8e1f55f8a606eae0f0c64bab575e84a2a608b3.

    ADDITIONAL ADVICE

    Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use git checkout <file> to restore the file to its original state. You can also reset all changes in the repository using git reset --hard command. Additionally, you can use git stash and git stash pop to temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you call submit) using git status or git diff commands. However you don't need to use git add or git commit before submitting your changes. In fact, submit only submits currently unstaged changes because it uses git diff to get the changes that need to be submitted.

    CHECKLIST

    Before submitting your solution, please go over the following checklist and make sure you've done everything:

    • If an error was reported in the issue description, I have successfully reproduced it.
    • If an error was reported in the issue description, I have confirmed that my fix resolves the error.
    • I have thought about edge cases and made sure my fix handles them as well.
    • I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.

    Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!

    (Current directory: /pyscn, current file: none) bash-$

  2. Assistant

    I’ll first inspect the repository structure and locate the clone detection/grouping implementation and relevant tests, then reproduce the overlap behavior with the smallest practical fixture before modifying the deduplication logic.

  3. Assistant

    The project is Go-based, with clone behavior likely split between internal, service, and domain. I’ll enumerate clone-specific files and inspect the detector pipeline, especially any existing handling for overlapping/subset windows related to issue #416.

  4. Assistant

    There is already a dedicated group_dedup.go and tests for strict-subset members within a group. I’ll inspect its data model, grouping strategy, and invocation to determine where cross-group near-duplicate suppression belongs without conflating distinct clone relationships.

  5. Assistant

    The current post-pass only compares members inside each group, so it cannot catch the reported shape: two groups each containing the same number of corresponding, nearly nested windows. I’ll now inspect group construction and fragment identity/keying so the new pass can conservatively identify groups whose members admit a one-to-one same-file containment mapping, retain the maximal-window group, and keep pair output/statistics consistent.

  6. Assistant

    A conservative rule should require equal group cardinality and a bijection where every member of one group contains its counterpart in the same file, with at least one strict containment; partial overlap alone is too broad. Before implementing, I’ll verify how detector groups become domain output and how duplication percentage is calculated, because suppressing a redundant group may also require suppressing only the redundant clone-pair edge while preserving fragments used by unrelated findings.

  7. Assistant

    The analysis score is directly based on TotalCloneGroups, so removing the duplicate group addresses both symptoms. Pair filtering should be exact: remove edges wholly represented by a suppressed group only when their endpoints are not members of any retained group, avoiding damage to shared fragments in overlapping clustering modes. I’ll inspect K-core/star behavior and current test helpers, then implement the cross-group pass as a second stage in group_dedup.go.

  8. Assistant

    A pure containment rule across groups could incorrectly erase a legitimate nested clone (for example, duplicated functions and duplicated inner loops). I’ll therefore use a near-identity criterion: corresponding ranges must be in the same file, one direction must consistently contain the other, and each inner range must cover at least 80% of its outer range. This handles boundary shifts proportionally rather than hardcoding one line, while excluding substantially nested findings. I’ll implement deterministic one-to-one matching, prefer the maximal group, and feed suppressed fragments through the existing pair-filter mechanism.

  9. Assistant

    Reviewing the first implementation exposed an important edge case: two near-duplicate groups may share an exact fragment pointer. Marking every member of the removed group as globally suppressed would then remove valid pairs supporting the retained group. I’ll refine the pass to compute retained membership first and suppress only fragments not used by any retained group. I’ll also collapse fully identical cross-groups, which are an even clearer duplicate case, and rely on actual fragment cardinality rather than potentially stale Size metadata.

  10. Assistant

    The combined shell command had a quoting typo and did not execute, so no partial edit occurred. I’ll rerun the same refinement cleanly, then run the focused analyzer/service/app tests.

  11. Assistant

    The focused test command was blocked by the environment because both GOMODCACHE and GOPATH are unset, not by a code failure. Before rerunning with an isolated /tmp Go cache, I’ll replace the recursive matching with a polynomial-time augmenting-path matcher so large clone groups cannot trigger factorial behavior. Then I’ll run focused and full tests.

  12. Assistant

    Dependencies downloaded successfully, but this minimal environment also lacks HOME/GOCACHE; I’ll explicitly set GOCACHE=/tmp/gocache and rerun. After tests pass, I’ll build the CLI and attempt the exact Fixit revision reproducer in /tmp, comparing the reported smoke.py groups before and after the fix if network access is available.

  13. Assistant

    All focused and full Go tests pass. The exact Fixit analysis also completed and reports 8 groups, but my glob omitted the hidden .pyscn directory. I’ll inspect that report now. To satisfy the reproduction checklist rigorously, I’ll also temporarily build the unmodified baseline from Git (restoring the working file immediately), run it against the same checkout, and compare the specific smoke.py groups and totals.

  14. Assistant

    The current HEAD baseline no longer reproduces the v1.22.9 count exactly because intervening grouping changes already combine these fragments into a larger group; both current binaries report 8 groups. Since the issue names v1.22.9 explicitly, I’ll reproduce against that tag using a separate Git worktree, leaving the submitted tree untouched. I’ll also tighten “near-identical” coverage to 90%, which still captures a one-line shift on 17–18-line windows while more safely preserving deeply nested clone findings.

  15. Assistant

    The uploaded clone does not contain the v1.22.9 tag locally, so the detached worktree command failed before changing anything. I’ll check the remote and fetch only that tag if available, then rerun the historical binary.

  16. Assistant

    No Git remote is configured in the uploaded repository. I can fetch the tag directly from the public repository URL without modifying Git configuration, so I’ll do that and repeat the historical reproduction.

  17. Assistant

    Even the exact public v1.22.9 tag in this environment produces a single broad group containing both outer windows (7 total), rather than the reported 10-group output—likely due configuration/environment differences in the audit run. The historical command nevertheless confirms the relevant outer ranges. I’ll now perform a careful code review, run static analysis, rerun all tests after the final 90% threshold change, and inspect the final diff for correctness and unintended repository changes.

  18. Assistant

    Static analysis and the full suite pass. To directly verify the new private helper without adding any repository test file (as prohibited), I’ll use Go’s -overlay facility to inject a temporary /tmp test at build time. It will cover the exact 298/299 and 317/318 ranges, deeply nested groups that must remain distinct, and shared-fragment pair-suppression safety. I’ll remove the temporary overlay afterward and do one final status check.

  19. Assistant

    Final review is complete, with only internal/analyzer/group_dedup.go modified.

    Checklist:

    • If an error was reported, successfully reproduced it. I reproduced the exact reported post-group shape using the same smoke.py ranges (298–315/317–334 and 299–315/318–334) through a temporary Go overlay test. I also ran the full Fixit revision with both current HEAD and public v1.22.9; those builds now/defaulted to broader connected groups in this environment, so they did not independently emit the historical 10-group layout.
    • Confirmed the fix resolves the error. The overlay test verifies the two near-identical groups collapse to the maximal outer-window group and the inner fragments are suppressed from pair output.
    • Thought about edge cases. The implementation requires equal cardinality, a complete one-to-one same-file containment mapping, consistent containment direction, and at least 90% line coverage. It preserves substantially nested clones, handles identical groups, nil locations, arbitrary member ordering, and shared fragments that remain in another retained group. Matching is polynomial-time.
    • Ran affected existing tests. go test ./... passes, including analyzer, service, app, integration, and all other packages. Focused tests, go vet, gofmt, and git diff --check also pass.

    I’ll now submit the single-file fix.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.