Skip to content
TheChatArchiveThe Chat Archive
DeepSeekdeepseek/deepseek-v4-pro· deepseekHugging Face ibragim-bad/swe_rebench_07_2026_trajectories

ISSUE DESCRIPTION [BUG] apm install fail second time with Content hash mismatch Describe…

  1. User

    ISSUE DESCRIPTION

    [BUG] apm install fail second time with Content hash mismatch Describe the bug I can install my repo (see apm.yml further down) via apm install <org>/<repo>/package/<project> just fine (kinda, see the logs, it looks wonky but maybe it's just a new feature) but the second time I run apm install it will fail consistently with Content hash mismatch.

    To Reproduce Steps to reproduce the behavior:

    1. apm install <repo>
    2. apm install

    Expected behavior A clear and concise description of what you expected to happen.

    Environment (please complete the following information):

    • OS: Windows
    • APM Version: 0.16.0

    Logs

    > apm install --update --force
    [>] Installing dependencies from apm.yml...
    [>] Resolving github-copilot-agent-skills-testing...
    [i] Targets: copilot  (source: apm.yml)
    Partial clone (--filter=blob:none) failed for https://dev.azure.com/<org>/tools; retrying with full bare clone. Server may not
    support filter v2.
      [+] dev.azure.com/<org>/tools/github-copilot-agent-skills/packages/<project> #default @d479f42e
      |-- (files unchanged)
    Partial clone (--filter=blob:none) failed for https://dev.azure.com/<org>/tools; retrying with full bare clone. Server may not
    support filter v2.
      [+] dev.azure.com/<org>/tools/github-copilot-agent-skills/packages/myorg #default @d479f42e
      |-- (files unchanged)
      [+] dev.azure.com/<org>/tools/github-copilot-agent-skills/.apm/instructions/testing.instructions.md #main
      |-- 1 instruction(s) integrated -> .github/instructions/
    Partial clone (--filter=blob:none) failed for https://dev.azure.com/<org>/tools; retrying with full bare clone. Server may not
    support filter v2.
      [+] dev.azure.com/<org>/tools/github-copilot-agent-skills/.apm/skills/run-coverage #default @d479f42e
      |-- (files unchanged)
    Partial clone (--filter=blob:none) failed for https://dev.azure.com/<org>/tools; retrying with full bare clone. Server may not
    support filter v2.
      [+] dev.azure.com/<org>/tools/github-copilot-agent-skills/.apm/skills/update-from-template #default @d479f42e
      |-- (files unchanged)
    Partial clone (--filter=blob:none) failed for https://dev.azure.com/<org>/tools; retrying with full bare clone. Server may not
    support filter v2.
      [+] dev.azure.com/<org>/tools/github-copilot-agent-skills/packages/csharp #default @d479f42e
      |-- (files unchanged)
      [+] dev.azure.com/<org>/tools/github-copilot-agent-skills/.apm/instructions/code-style.instructions.md #main
      |-- 1 instruction(s) integrated -> .github/instructions/
      [+] dev.azure.com/<org>/tools/github-copilot-agent-skills/.apm/instructions/protect-prefixed-agent-files.instructions.md
    #main
      |-- 1 instruction(s) integrated -> .github/instructions/
    Partial clone (--filter=blob:none) failed for https://dev.azure.com/<org>/tools; retrying with full bare clone. Server may not
    support filter v2.
      [+] dev.azure.com/<org>/tools/github-copilot-agent-skills/.apm/skills/git-conventions #default @d479f42e
      |-- (files unchanged)
    Partial clone (--filter=blob:none) failed for https://dev.azure.com/<org>/tools; retrying with full bare clone. Server may not
    support filter v2.
      [+] dev.azure.com/<org>/tools/github-copilot-agent-skills/.apm/skills/update-nuget-packages #default @d479f42e
      |-- (files unchanged)
    Partial clone (--filter=blob:none) failed for https://dev.azure.com/<org>/tools; retrying with full bare clone. Server may not
    support filter v2.
      [+] dev.azure.com/<org>/tools/github-copilot-agent-skills/.apm/skills/write-tests #default @d479f42e
      |-- (files unchanged)
    cache-pin: remote dep '<org>/tools/github-copilot-agent-skills' has no resolved_commit; drift cannot verify its cache freshness.
    Re-run 'apm install' with a pinned ref (commit, tag, or specific branch HEAD).
    cache-pin: remote dep '<org>/tools/github-copilot-agent-skills' has no resolved_commit; drift cannot verify its cache freshness.
    Re-run 'apm install' with a pinned ref (commit, tag, or specific branch HEAD).
    cache-pin: remote dep '<org>/tools/github-copilot-agent-skills' has no resolved_commit; drift cannot verify its cache freshness.
    Re-run 'apm install' with a pinned ref (commit, tag, or specific branch HEAD).
    [i] Trusting direct dependency MCP 'MyOrg.CLI' from '<project>-skills'
    [i] Trusting direct dependency MCP 'NuGet' from '<project>-skills'
    [i] Trusting direct dependency MCP 'Github' from '<project>-skills'
    [i] Trusting direct dependency MCP 'MyOrg.CLI' from 'skills'
    [i] Trusting direct dependency MCP 'NuGet' from 'skills'
    +- MCP Servers (3)
    |  [+] MyOrg.CLI (already configured)
    |  [+] NuGet (already configured)
    |  [+] Github (already configured)
    [*] All servers up to date
      [!] 11 dependencies unpinned: <org>/tools/github-copilot-agent-skills -- add #tag or #sha to prevent drift
      [!] Transitive package 'csharp-skills' declares self-defined MCP server 'NuGet' (registry: false). Re-declare it in your apm.yml or
    use --trust-transitive-mcp.
      [!] Transitive package 'csharp-skills' declares self-defined MCP server 'Github' (registry: false). Re-declare it in your apm.yml
    or use --trust-transitive-mcp.
    [*] Installed 11 APM dependencies in 87.9s.
    
    
    > apm install
    [>] Installing dependencies from apm.yml...
    [i] Targets: copilot  (source: apm.yml)
      [+] dev.azure.com/<org>/tools/github-copilot-agent-skills/packages/<project> @d479f42e (cached)
      |-- (files unchanged)
      [+] dev.azure.com/<org>/tools/github-copilot-agent-skills/packages/myorg @d479f42e (cached)
      |-- (files unchanged)
      [+] dev.azure.com/<org>/tools/github-copilot-agent-skills/.apm/instructions/testing.instructions.md #main
    Content hash mismatch for <org>/tools/github-copilot-agent-skills/.apm/instructions/testing.instructions.md: expected
    sha256:2227579674ab30b5b59f703ac152fa74ac2912d724d04aeffbf3914cac2e6b88, got
    sha256:b82d09e4e7879eeb0e90cc7dfeebdac45de6ebdbcdda7824b40438196692f8f8. The downloaded content differs from the lockfile record.
    This may indicate a supply-chain attack. Use 'apm install --update' to accept new content and update the lockfile.
    [!] Install interrupted after 7.7s.
    

    Additional context apm.yml

    name: <project>-skills
    version: 1.0.0
    description: '<Project> skills and agent for GitHub Copilot.'
    author: Oskar Klintrot
    target: vscode
    dependencies:
      apm:
      - dev.azure.com/<org>/tools/github-copilot-agent-skills/packages/csharp
      - dev.azure.com/<org>/tools/github-copilot-agent-skills/.apm/skills/run-coverage
      - dev.azure.com/<org>/tools/github-copilot-agent-skills/.apm/instructions/testing.instructions.md
      mcp:
      - name: NuGet
        registry: false
        transport: stdio
        command: dnx
        args:
        - NuGet.Mcp.Server
        - --source
        - https://api.nuget.org/v3/index.json
        - --yes
    

    ADDITIONAL ADVICE

    Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use git checkout <file> to restore the file to its original state. You can also reset all changes in the repository using git reset --hard command. Additionally, you can use git stash and git stash pop to temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you call submit) using git status or git diff commands. However you don't need to use git add or git commit before submitting your changes. In fact, submit only submits currently unstaged changes because it uses git diff to get the changes that need to be submitted.

    CHECKLIST

    Before submitting your solution, please go over the following checklist and make sure you've done everything:

    • If an error was reported in the issue description, I have successfully reproduced it.
    • If an error was reported in the issue description, I have confirmed that my fix resolves the error.
    • I have thought about edge cases and made sure my fix handles them as well.
    • I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.

    Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!

    (Current directory: /apm, current file: none) bash-$

  2. DeepSeek Assistant

    Let me analyze the issue:

    1. The user runs apm install --update --force which works fine
    2. The second time they run apm install, it fails with "Content hash mismatch"
    3. The error says: "Content hash mismatch for .../testing.instructions.md: expected sha256:..., got sha256:... The downloaded content differs from the lockfile record."

    This suggests that there's a bug where the hash recorded in the lockfile during the first install doesn't match what's computed during the second install. The hash mismatch occurs on a .instructions.md file.

    Let me explore the repository to understand the structure and find the relevant code that handles content hashing and lockfile operations.

    Let me start by exploring the repository structure.

  3. DeepSeek Assistant

    Let me explore the source code to understand the structure of the project and find relevant files for content hash handling.

  4. DeepSeek Assistant

    Let me explore the apm_cli directory structure.

  5. DeepSeek Assistant

    Let me look for files related to content hash, lockfile, and caching.

  6. DeepSeek Assistant

    Let me look at both these files to understand the content hash logic

  7. DeepSeek Assistant

    The open command might need different syntax. Let me try opening it directly with the open special command.

  8. DeepSeek Assistant

    Let me find the "Content hash mismatch" area in this file.

  9. DeepSeek Assistant

    Let me look at the context around line 777.

  10. DeepSeek Assistant

    Now let me find the _compute_hash function and understand how hashes are

  11. DeepSeek Assistant

    Let me look at the compute_package_hash function in content_hash.py.

  12. DeepSeek Assistant

    Let me open that file.

  13. DeepSeek Assistant

    Let me also look at the integrate.py file that has "Content hash mismatch".

  14. DeepSeek Assistant

    Let me find the "Content hash mismatch" line in integrate.py.

  15. DeepSeek Assistant

    Let me look at the context around line 197 in integrate.py.

  16. DeepSeek Assistant

    Let me look at the instruction integrator to understand how instructions files are handled.

  17. DeepSeek Assistant

    Let me understand how single-file dependencies like .instructions.md files are installed. Let me look at the download and install path logic

  18. DeepSeek Assistant

    Let me look at how dependencies are resolved and where they're installed. Let me check the model for how deps are structured

  19. DeepSeek Assistant

    Let me look at the cache_pin module.

  20. DeepSeek Assistant

    Let me open cache_pin.py to understand how the marker file works

  21. DeepSeek Assistant

    Let me look at how the install path is determined for dependencies and the download mechanism

  22. DeepSeek Assistant

    Let me look at the get_install_path function in the dependency reference model

  23. DeepSeek Assistant

    Let me look at is_virtual_subdirectory() and get_virtual_package_name()

  24. DeepSeek Assistant

    Let me look at the relevant methods

  25. DeepSeek Assistant

    Let me look more carefully at the FreshDependencySource class and how the hash is computed and

  26. DeepSeek Assistant

    Let me look at lines 730-790 where the hash is computed and verified

  27. DeepSeek Assistant

    Let me look at how the lockfile is built and how content_hash is stored

  28. DeepSeek Assistant

    Let me look at how content_hash is set in the LockfileBuilder

  29. DeepSeek Assistant

    Let me look at the CachedDependencySource to see how it computes the hash

  30. DeepSeek Assistant

    Let me look at the CachedDependencySource.acquire() method

  31. DeepSeek Assistant

    Let me continue reading the CachedDependencySource.acquire()

  32. DeepSeek Assistant

    Let me look at how virtual files are downloaded

  33. DeepSeek Assistant

    Let me look at download_virtual_file_package in github_downloader.py

  34. DeepSeek Assistant

    Let me continue reading `download_virtual_file_package

  35. DeepSeek Assistant

    Let me look at the lockfile writing phase

  36. DeepSeek Assistant

    Let me look at the _attach_content_hashes method

  37. DeepSeek Assistant

    Let me look at how the integration template works, specifically whether it modifies the package install path

  38. DeepSeek Assistant

    Let me look at the full FreshDependencySource.acquire() from the beginning

  39. DeepSeek Assistant

    Let me look at the download_raw_file function

  40. DeepSeek Assistant

    Let me look at the download_raw_file method

  41. DeepSeek Assistant

    Let me look at the download strategies for ADO

  42. DeepSeek Assistant

    Let me look at the ADO file download

  43. DeepSeek Assistant

    Let me look at the yaml_to_str function

  44. DeepSeek Assistant

    Let me look at the yaml_to_str

  45. DeepSeek Assistant

    Let me look at the lockfile_match logic in integrate.py more carefully

  46. DeepSeek Assistant

    Let me look at _should_skip_redownload

  47. DeepSeek Assistant

    Let me look at BaseIntegrator to see if it modifies anything in the install path.

  48. DeepSeek Assistant

    Let me look at find_files_by_glob

  49. DeepSeek Assistant

    Let me look at `find_files_by_glob

  50. DeepSeek Assistant

    Let me check how --force affects the install flow

  51. DeepSeek Assistant

    Let me search for tests related to content hash and virtual file packages

  52. DeepSeek Assistant

    Let me look at the test for virtual file package download

  53. DeepSeek Assistant

    Let me check the VIRTUAL_FILE_EXTENSIONS constant

  54. DeepSeek Assistant

    Let me look at the VIRTUAL_FILE_EXTENSIONS constant

  55. DeepSeek Assistant

    Let me check how --update affects the flow

  56. DeepSeek Assistant

    Let me look at how instruction integration works more carefully, specifically what it does with the package path

  57. DeepSeek Assistant

    Let me now look at the full integration template flow to see if anything writes to the install_path after hash computation

  58. DeepSeek Assistant

    Let me look at the LockedDependency class

  59. DeepSeek Assistant

    Let me look at recent git changes related to content hash or virtual file packages

  60. DeepSeek Assistant

    Let me look at sync_markers_for_lockfile

  61. DeepSeek Assistant

    Let me look at sync_markers_for_lockfile

  62. DeepSeek Assistant

    Let me look at the ADO API URL building

  63. DeepSeek Assistant

    Let me look at the ADO API URL builder

  64. DeepSeek Assistant

    Let me look at download_package in github_downloader.py where it decides between virtual file and normal download

  65. DeepSeek Assistant

    Let me look at build_download_ref function

  66. DeepSeek Assistant

    Let me look at build_download_ref

  67. DeepSeek Assistant

    Let me look at detect_ref_change

  68. DeepSeek Assistant

    Let me look at detect_ref_change

  69. DeepSeek Assistant

    Let me create a minimal reproduction script to understand the hash computation

  70. DeepSeek Assistant

    Let me use the venv to run the script

  71. DeepSeek Assistant

    Let me look at LockedDependency.from_dict()

  72. DeepSeek Assistant

    Let me look at the existing lockfile to understand its format

  73. DeepSeek Assistant

    Let me look at the git log for any recent changes related to content hash or virtual file packages

  74. DeepSeek Assistant

    Let me search for any TODO or FIXME comments related to content hash in the codebase

  75. DeepSeek Assistant

    Let me look at to_dependency_ref in LockedDependency

  76. DeepSeek Assistant

    Let me look at `to_dependency_ref

  77. DeepSeek Assistant

    Let me verify that yaml.safe_dump is deterministic

  78. DeepSeek Assistant

    Let me look at `resolve_commit_sha_for_ref

  79. DeepSeek Assistant

    Let me look at `resolve_commit_sha_for

  80. DeepSeek Assistant

    Let me look at the InstallContext to understand how --force and --update affect things

  81. DeepSeek Assistant

    Let me look at _resilient_get to see if there's anything non-deterministic

  82. DeepSeek Assistant

    Let me look at _resilient_get

  83. DeepSeek Assistant

    Let me look at resilient_get in download_strategies.py

  84. DeepSeek Assistant

    Let me check the git log for changes to relevant

  85. DeepSeek Assistant

    Let me check if --force cleans the install path before download

  86. DeepSeek Assistant

    Let me search for tests about content hash mismatch

  87. DeepSeek Assistant

    Let me look at the diff_aware_install tests to understand the expected behavior

  88. DeepSeek Assistant

    Let me write a script to simulate the exact flow and see if I can reproduce the hash mismatch

  89. DeepSeek Assistant

    Let me look at how DependencyReference is parsed for ADO URLs to check if host is included in repo_url

  90. DeepSeek Assistant

    Let me look at how InstalledPackage → LockedDependency conversion works

  91. DeepSeek Assistant

    Let me look at from_installed_packages

  92. DeepSeek Assistant

    Let me look at `from_installed_packages

  93. DeepSeek Assistant

    Let me look at LockedDependency.from_dependency_ref

  94. DeepSeek Assistant

    Let me look at from_dependency_ref

  95. DeepSeek Assistant

    Let me see the rest of from_dependency_ref

  96. DeepSeek Assistant

    Let me check if force affects the download or hash computation

  97. DeepSeek Assistant

    Let me look at the integrate loop to see how dep_key is determined

  98. DeepSeek Assistant

    Let me look at `_dataclass_replace

  99. DeepSeek Assistant

    Let me check if DependencyReference is a dataclass that supports `

  100. DeepSeek Assistant

    Let me search for any commits or PRs related to this issue

  101. DeepSeek Assistant

    Let me check if the resolve phase pre-downloads virtual file packages

  102. DeepSeek Assistant

    Let me check whether --force causes the install path to be cleaned

  103. DeepSeek Assistant

    Now I have a good understanding of the issue. Let me look at the _resolve_download_strategy code path more carefully to understand the flow for virtual file packages from ADO where resolved_commit is None.

    The key issue I see is in src/apm_cli/install/phases/integrate.py. For Azure DevOps virtual file packages, _resolve_commit_sha_for_ref returns None (as seen in src/apm_cli/deps/git_reference_resolver.py lines 239-240). This means resolved_commit is always None for ADO virtual file deps.

    In _resolve_download_strategy, the lockfile_match check at line 115 requires locked_dep.resolved_commit to be non-None. Since it's None for ADO virtual files, the entire cache-hit path is skipped. There's a fallback for registry deps (line 151-160) but not for git-source deps with missing resolved_commit.

    Let me look at the exact lines to craft a fix:

  104. DeepSeek Assistant

    Now I understand the issue. Let me look at the _resolve_download_strategy function more carefully. For Azure DevOps virtual file packages:

    1. _resolve_commit_sha_for_ref in git_reference_resolver.py (lines 239-240) returns None for ADO deps
    2. This means resolved_commit is always None for ADO virtual file packages
    3. In the lockfile, resolved_commit is therefore None
    4. In _resolve_download_strategy, the check at line 115 requires locked_dep.resolved_commit to be truthy, which fails
    5. The elif at line 151 only handles source == "registry", not git-source deps
    6. So lockfile_match stays False, skip_download is False, and the dep is always re-downloaded

    The fix should add a fallback for deps that have content_hash but no resolved_commit (regardless of source), mirroring the registry path. Additionally, there's a scenario where the _resolve_download_strategy doesn't check _should_skip_redownload when resolved_commit is missing for non-registry deps. This means even if the content hasn't changed, the dep goes through FreshDependencySource, where the hash verification can fail.

    Let me add the fallback:

  105. DeepSeek Assistant

    Now let me also look at download_virtual_file_package to see if it cleans the install directory before creating the virtual package. If old files remain, the hash could differ

  106. DeepSeek Assistant

    I see another potential issue: download_virtual_file_package doesn't clean the target directory before creating the virtual package. If there are leftover files from a previous version, they would be included in the hash. Let me add cleanup:

  107. DeepSeek Assistant

    Let me check the safe_rmtree function to make sure it can handle the target_path properly

  108. DeepSeek Assistant

    Let me look at safe_rmtree

  109. DeepSeek Assistant

    Now let me run the relevant tests to make sure my changes don't break anything

  110. DeepSeek Assistant

    Let me run more tests related to the integrate phase and content hash

  111. DeepSeek Assistant

    Let me run the heal chain tests and some more unit tests

  112. DeepSeek Assistant

    Let me run a broader set of tests that might be affected by my changes

  113. DeepSeek Assistant

    Let me verify my changes by looking at the git diff

  114. DeepSeek Assistant

    Let me run a broader test to make sure nothing is broken

  115. DeepSeek Assistant

    Let me run the wave4 pure logic tests

  116. DeepSeek Assistant

    Good, the tests pass. Let me now think about edge cases and verify the fix is complete.

    Let me also check that I haven't introduced any issues with the safe_rmtree call by verifying the import is available at that point in the code:Let me verify the import will work correctly in that context by checking what's already imported at that point in github_downloader

  117. DeepSeek Assistant

    Let me run the download strategies tests to make sure the safe_rmtree change doesn't break

  118. DeepSeek Assistant

    Let me run the install marketplace tests to be

  119. DeepSeek Assistant

    Let me run the test without the filter issue

  120. DeepSeek Assistant

    Let me run the full install marketplace tests to verify

  121. DeepSeek Assistant

    All tests pass. Let me now verify my changes one more time and also check for any potential issues with the safe_rmtree call in the download_virtual_file_package function.

    Let me think about whether the safe_rmtree in download_virtual_file_package could cause issues in specific scenarios:

    1. Normal flow: target_path exists, is a regular directory → delete it → recreate it → fine
    2. First install: target_path doesn't exist → skip cleanup → fine
    3. Symlink attack: target_path is a symlink → exists() returns True → safe_rmtree → ensure_path_within resolves and detects traversal → raises → caught as download error

    Let me verify there's an exception handler:Let me verify the exception handler in FreshDependencySource.acquire() covers errors from download_virtual_file_package

  122. DeepSeek Assistant

    Let me do a final review of both changed

  123. DeepSeek Assistant

    Now let me verify the changes once more and then submit. Let me run one more quick test to make sure the whole test suite for the install phases works

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.