Skip to content
TheChatArchiveThe Chat Archive
fable-5· unknownHugging Face ibragim-bad/swe_rebench_07_2026_trajectories

Automatic managed-ledger offload triggers create redundant retry work Coalesce automatic…

  1. User

    Automatic managed-ledger offload triggers create redundant retry work

    Coalesce automatic offload triggers while an automatic offload is running. There must be at most one active automatic run and at most one follow-up run when additional triggers arrive. A follow-up must still process newly eligible ledgers, and explicit/manual offload futures must retain their existing behavior.

    Interface

    No new public interfaces are introduced.

  2. Assistant

    Gradle is still compiling (daemon startup takes a while on first run). The background task will notify on completion — I'll continue when it finishes.

  3. Assistant

    Waiting for the compile to finish; I'll proceed with the tests when it exits.

  4. Assistant

    Compile succeeded (exit 0). Now run the offload tests that exercise this path:

  5. Assistant

    While the test runs, let me review the full diff:

  6. Assistant

    The diff looks correct. Now waiting for the test results.

  7. Assistant

    Test run in progress — waiting for completion notification.

  8. Assistant

    The test output is buffered through tail, so it will appear at completion. Waiting for the background notification.

  9. Assistant

    OffloadLedgerDeleteTest passed. Run the remaining offload test classes as a final check:

  10. Assistant

    All offload tests pass. Final verification:

  11. Assistant

    Done. The fix is complete and verified.

    Summary

    Problem: In ManagedLedgerImpl.maybeOffload(), every automatic offload trigger that found offloadMutex locked scheduled its own 100 ms retry via maybeOffloadInBackground. With N triggers arriving during a running offload (e.g. rapid ledger rollovers), N independent retry loops piled up, each re-checking policies and re-scanning ledgers — redundant work.

    Fix (managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java):

    • Added deferredAutomaticOffloadPromises (a list, guarded by itself) and an automaticOffloadFollowUpScheduled flag next to offloadMutex.
    • When tryLock fails, the trigger's promise is added to the deferred list; a retry is scheduled only if one isn't already pending — so there is at most one active run (mutex) and at most one follow-up per burst of triggers.
    • New retryDeferredAutomaticOffload() drains all deferred promises, resets the flag, and re-enters maybeOffloadInBackground with a single follow-up promise that fans out its result to every deferred promise. Because the follow-up re-reads policies/thresholds and re-scans the ledgers map from scratch, newly eligible ledgers are still processed. If the mutex is still held when the follow-up fires, the follow-up promise itself is deferred again — still exactly one pending retry.
    • Explicit/manual offload (asyncInternalOffloadPrefix at line 3644) is untouched: it still fails fast with OffloadInProgressException when an offload is running, and caller-supplied futures to maybeOffloadInBackground still complete with the run's result (or exception).

    Verification: :managed-ledger:compileJava builds; test classes OffloadPrefixTest (44 tests — including multipleAutoTriggers and autoTriggerWhileManualInProgress, which exercise exactly this coalescing path), OffloadLedgerDeleteTest, OffloadPrefixReadTest, and OffloadEvictUnusedLedgersTest all pass with 0 failures. git diff --check is clean and the change is left uncommitted in the working tree.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.