Automatic managed-ledger offload triggers create redundant retry work Coalesce automatic…
- User
Automatic managed-ledger offload triggers create redundant retry work
Coalesce automatic offload triggers while an automatic offload is running. There must be at most one active automatic run and at most one follow-up run when additional triggers arrive. A follow-up must still process newly eligible ledgers, and explicit/manual offload futures must retain their existing behavior.
Interface
No new public interfaces are introduced.
- Assistant
Gradle is still compiling (daemon startup takes a while on first run). The background task will notify on completion — I'll continue when it finishes.
- Assistant
Waiting for the compile to finish; I'll proceed with the tests when it exits.
- Assistant
Compile succeeded (exit 0). Now run the offload tests that exercise this path:
- Assistant
While the test runs, let me review the full diff:
- Assistant
The diff looks correct. Now waiting for the test results.
- Assistant
Test run in progress — waiting for completion notification.
- Assistant
The test output is buffered through
tail, so it will appear at completion. Waiting for the background notification. - Assistant
OffloadLedgerDeleteTest passed. Run the remaining offload test classes as a final check:
- Assistant
All offload tests pass. Final verification:
- Assistant
Done. The fix is complete and verified.
Summary
Problem: In
ManagedLedgerImpl.maybeOffload(), every automatic offload trigger that foundoffloadMutexlocked scheduled its own 100 ms retry viamaybeOffloadInBackground. With N triggers arriving during a running offload (e.g. rapid ledger rollovers), N independent retry loops piled up, each re-checking policies and re-scanning ledgers — redundant work.Fix (
managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java):- Added
deferredAutomaticOffloadPromises(a list, guarded by itself) and anautomaticOffloadFollowUpScheduledflag next tooffloadMutex. - When
tryLockfails, the trigger's promise is added to the deferred list; a retry is scheduled only if one isn't already pending — so there is at most one active run (mutex) and at most one follow-up per burst of triggers. - New
retryDeferredAutomaticOffload()drains all deferred promises, resets the flag, and re-entersmaybeOffloadInBackgroundwith a single follow-up promise that fans out its result to every deferred promise. Because the follow-up re-reads policies/thresholds and re-scans theledgersmap from scratch, newly eligible ledgers are still processed. If the mutex is still held when the follow-up fires, the follow-up promise itself is deferred again — still exactly one pending retry. - Explicit/manual offload (
asyncInternalOffloadPrefixat line 3644) is untouched: it still fails fast withOffloadInProgressExceptionwhen an offload is running, and caller-supplied futures tomaybeOffloadInBackgroundstill complete with the run's result (or exception).
Verification:
:managed-ledger:compileJavabuilds; test classesOffloadPrefixTest(44 tests — includingmultipleAutoTriggersandautoTriggerWhileManualInProgress, which exercise exactly this coalescing path),OffloadLedgerDeleteTest,OffloadPrefixReadTest, andOffloadEvictUnusedLedgersTestall pass with 0 failures.git diff --checkis clean and the change is left uncommitted in the working tree. - Added
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- fable-5Compare side by side →