auth: spurious REFRESHGENERATIONS bump on early-cancel in coalescedrunrefreshcmd Summary…
- User
auth: spurious _REFRESH_GENERATIONS bump on early-cancel in _coalesced_run_refresh_cmd
Summary
_coalesced_run_refresh_cmdinsrc/notebooklm/_auth/refresh.py(moved fromsrc/notebooklm/auth.pyin tier-10 PR-B-high #814, but the code is byte-identical to origin/main since tier-9 E #803) can bump_REFRESH_GENERATIONS[refresh_key]even when no real refresh happened, in a narrow cancel-before-await race.Reproducer (race condition)
- Caller invokes
await _coalesced_run_refresh_cmd(refresh_key, path, profile)from the cancel-recovery loop in_fetch_tokens_with_refresh. - Task is cancelled in the window between
try:and the firstwith _REFRESH_STATE_LOCK:block inside_coalesced_run_refresh_cmd— so NO future is registered in_REFRESH_INFLIGHT_BY_LOOP[loop][refresh_key]. - Control jumps to
except asyncio.CancelledError:in the caller. registry.get(refresh_key)returnsNone.- The inner
if inflight is None or inflight.done():is taken; theif inflight is not None:block is skipped;subprocess_excstaysNone;break. - Outer
if subprocess_exc is not None:is False, so no raise. - Generation is bumped at the
with _REFRESH_STATE_LOCK: ... max(existing, refresh_generation + 1)line, even though no subprocess ran. - Then
if caller_cancelled: raise asyncio.CancelledError().
Impact
A concurrent waiter on a different event loop sharing the same
refresh_keywould observe the phantom generation bump on its lock-re-acquire path, treat the storage as freshly-refreshed, and skip its own_run_refresh_cmd— proceeding tobuild_httpx_cookies_from_storageagainst STALE on-disk state.Bounded blast radius: the stale-storage caller will fail again on the next token fetch and trigger its own real refresh. No data corruption. But it's a wasted round-trip.
Suggested fix (from CodeRabbit on PR #814)
Track whether
inflightwas ever non-None during the cancel-recovery loop. Only bump generation ifsubprocess_exc is None AND observed_inflight is True. Or per the alternative suggestion: raise immediately wheninflight is Noneat the top of the cancel handler (the subprocess never registered, so we can't be the leader-cancel-before-await case for an actually-running refresh).Why not fixed in PR #814
That PR's scope was "extract verbatim from auth.py into _auth/refresh.py + facade-forward". Changing async semantics was explicitly out of scope (
must_not_do). Filing here for a dedicated follow-up.References
- CodeRabbit comment: https://github.com/teng-lin/notebooklm-py/pull/814#discussion_r3256142201
- Original code:
src/notebooklm/_auth/refresh.py:487-537(tier-10),src/notebooklm/auth.py:961-1026(tier-9 E origin)
Interface
No new interfaces are introduced.
- Caller invokes
- Assistant
Let me look at
_coalesced_run_refresh_cmdand the surrounding registry helpers. - Assistant
Now let me check how the
auth.pyfacade forwards monkeypatches, since tests patch viaauth_mod: - Assistant
Now add regression tests to the existing race test file:
- Assistant
All 6 pass. Now verify the two new tests actually fail against the pre-fix code (red-gate check):
- Assistant
Only one of the two new tests matched the
-kfilter. Re-run the red-gate check with both: - Assistant
These failures are all in playwright/login CLI tests — likely pre-existing environment issues. Verify against the clean tree:
- Assistant
Identical 11 failures / 21 errors on the clean tree — pre-existing environment issues (Playwright browser not installed), unrelated to this change. Run ruff on the changed files:
- Assistant
Done. The fix is complete and verified.
Changes
src/notebooklm/_auth/refresh.py(_fetch_tokens_with_refresh, cancel-recovery loop):- Snapshot the registry slot before the first
_coalesced_run_refresh_cmdcall (prior_inflight/prior_was_done), taken under_REFRESH_STATE_LOCKper the module's convention. - Raise immediately in the
CancelledErrorhandler when this cycle never registered a future — i.e. wheninflight is None(the exact race in the issue), or when the slot still holds a previous cycle's already-done future (prior_was_done and inflight is prior_inflight). In both cases no subprocess ran on our behalf, so the cancellation propagates without touching_REFRESH_GENERATIONS— closing the phantom-bump path where a cross-loop waiter would skip its own refresh and reload stale storage.
The second condition covers a sibling variant of the same bug class:
_settleintentionally leaves done futures in the registry (PR #621 finding), so an early-cancel could otherwise misread a stale successful future as its own cycle's result and bump anyway. The re-await path for a genuinely in-flight future (the documented cancel-safety design: hold the asyncio lock until the shielded subprocess settles) is unchanged.tests/integration/concurrency/test_refresh_cmd_race.py— two regression tests:test_cancel_before_inflight_registration_does_not_bump— simulates cancel-before-registration (stub raisesCancelledErrorwithout registering); assertsCancelledErrorpropagates and generation stays 0.test_cancel_before_registration_with_stale_done_future_does_not_bump— same, but with a planted done-successful future from a "previous cycle" in the registry.
Verification
- Red gate: both new tests fail on the pre-fix code (phantom bump to generation 1) and pass with the fix.
- All 6 tests in the race suite pass; full suite: 5346 passed — the 11 failures / 21 errors are pre-existing Playwright-environment issues, reproduced identically on clean HEAD.
mypy src/notebooklm: clean;ruff format --check+ruff check: clean.
- Snapshot the registry slot before the first
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- fable-5Compare side by side →
- fable-5Compare side by side →