ISSUE DESCRIPTION [BUG][auto] duplicate-overlapping-clone-groups: same duplication…
- User
ISSUE DESCRIPTION
[BUG][auto] duplicate-overlapping-clone-groups: same duplication emitted as two separate near-identical clone groups
Bug
pyscn's clone detector emits two separate clone groups that describe the same duplication relationship between the same two code fragments, with member windows shifted by exactly one line (the enclosing
with/header line). The result is a redundant clone group that inflatesclone_groupscount and the duplication percentage.This is related to but distinct from the (closed) #416
clone-overlapping-window-self-clone: #416 was about strict-subset members within a single group; here the redundancy is across two whole groups that are near-duplicates of each other.Reproducer
Repo:
Instagram/Fixit@bf8e1f55f8a606eae0f0c64bab575e84a2a608b3pyscn:v1.22.9curl -fsSL "https://codeload.github.com/Instagram/Fixit/tar.gz/HEAD" | tar -xz pyscn analyze --json --no-open Fixit-*/In the resulting
analyze.json,clone.clone_groupscontains two groups that cover the same pair of fragments insrc/fixit/tests/smoke.py:{ "id": 6, "type": 2, "similarity": 0.9692, "clones": [ {"location": {"file_path": ".../smoke.py", "start_line": 299, "end_line": 315}}, {"location": {"file_path": ".../smoke.py", "start_line": 318, "end_line": 334}} ] }, { "id": 14, "type": 2, "similarity": 0.9613, "clones": [ {"location": {"file_path": ".../smoke.py", "start_line": 298, "end_line": 315}}, {"location": {"file_path": ".../smoke.py", "start_line": 317, "end_line": 334}} ] }Both groups describe the duplication between the
subTest("lint")block and thesubTest("fix")block. Group 14's members each start exactly one line earlier than group 6's — that line is thewith self.subTest("lint"):/with self.subTest("fix"):header:with self.subTest("lint"): # line 298 (grp14 member start) with TemporaryDirectory() as td: # line 299 (grp6 member start) ... with self.subTest("fix"): # line 317 (grp14 member start) with TemporaryDirectory() as td: # line 318 (grp6 member start)Both groups end on the same lines (315 / 334). The two groups are the same logical clone reported twice at slightly different window boundaries.
Expected behavior
The clone-grouping/dedup stage should merge (or suppress) clone groups whose members are overlapping windows of the same underlying fragments, so each duplication relationship is reported once. As-is, the
clone_groupscount (10 here) and the duplication score over-count by one redundant group.Actual behavior
Two distinct groups (
id: 6andid: 14) are emitted for the same duplication, differing only by whether thewith self.subTest(...)header line is included in each member window.Priority rationale
P2 (visible but bounded inaccuracy): the cited line ranges are themselves valid, and the duplication is real — but the same finding is double-counted, inflating the group count and duplication percentage by a bounded amount.
Found via the FP-audit skill in repo
Instagram/Fixit@bf8e1f55f8a606eae0f0c64bab575e84a2a608b3.ADDITIONAL ADVICE
Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use
git checkout <file>to restore the file to its original state. You can also reset all changes in the repository usinggit reset --hardcommand. Additionally, you can usegit stashandgit stash popto temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you callsubmit) usinggit statusorgit diffcommands. However you don't need to usegit addorgit commitbefore submitting your changes. In fact,submitonly submits currently unstaged changes because it usesgit diffto get the changes that need to be submitted.CHECKLIST
Before submitting your solution, please go over the following checklist and make sure you've done everything:
- If an error was reported in the issue description, I have successfully reproduced it.
- If an error was reported in the issue description, I have confirmed that my fix resolves the error.
- I have thought about edge cases and made sure my fix handles them as well.
- I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.
Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!
(Current directory: /pyscn, current file: none) bash-$
- Assistant
I’ll inspect the repository to locate clone grouping/deduplication logic, identify existing tests and conventions, then reproduce the overlap behavior with targeted tests or a temporary fixture. I’ll implement the narrowest general deduplication fix, run relevant tests (and broader tests if practical), review the diff, and only then submit.
- Assistant
The detector already has a post-group pass for strict-subset members within one group, which explains why this bug survives: the two alternative windows are in separate groups. I’ll extend that post-pass with a separate whole-group deduplication step. It will only collapse groups of equal size whose members can be matched one-to-one to overlapping ranges in the same files, retain the more complete (larger-coverage) representative, and suppress discarded members’ pairs consistently with the existing behavior.
- Assistant
The first edit command had a quoting mistake while generating the new Go block, so formatting did not run and the file temporarily contains literal escape markers. I’m correcting that mechanical issue now, then I’ll compile immediately before evaluating behavior.
- Assistant
The targeted analyzer tests now pass, and
git diff --checkis clean. I’m adding no repository tests (per the task constraint); instead I’ll use a temporary test file outside the submitted diff to verify the exact one-line-shifted two-group shape, confirm the larger window is retained, and ensure disjoint ranges are not collapsed.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
Qwen/Qwen3.6-27BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →
- xiaomi/mimo-v2.5-proCompare side by side →
- xiaomi/mimo-v2.5-proCompare side by side →
- xiaomi/mimo-v2.5-proCompare side by side →
- xiaomi/mimo-v2.5-proCompare side by side →
- xiaomi/mimo-v2.5-proCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →