auth: spurious REFRESHGENERATIONS bump on early-cancel in coalescedrunrefreshcmd Summary…
- User
auth: spurious _REFRESH_GENERATIONS bump on early-cancel in _coalesced_run_refresh_cmd
Summary
_coalesced_run_refresh_cmdinsrc/notebooklm/_auth/refresh.py(moved fromsrc/notebooklm/auth.pyin tier-10 PR-B-high #814, but the code is byte-identical to origin/main since tier-9 E #803) can bump_REFRESH_GENERATIONS[refresh_key]even when no real refresh happened, in a narrow cancel-before-await race.Reproducer (race condition)
- Caller invokes
await _coalesced_run_refresh_cmd(refresh_key, path, profile)from the cancel-recovery loop in_fetch_tokens_with_refresh. - Task is cancelled in the window between
try:and the firstwith _REFRESH_STATE_LOCK:block inside_coalesced_run_refresh_cmd— so NO future is registered in_REFRESH_INFLIGHT_BY_LOOP[loop][refresh_key]. - Control jumps to
except asyncio.CancelledError:in the caller. registry.get(refresh_key)returnsNone.- The inner
if inflight is None or inflight.done():is taken; theif inflight is not None:block is skipped;subprocess_excstaysNone;break. - Outer
if subprocess_exc is not None:is False, so no raise. - Generation is bumped at the
with _REFRESH_STATE_LOCK: ... max(existing, refresh_generation + 1)line, even though no subprocess ran. - Then
if caller_cancelled: raise asyncio.CancelledError().
Impact
A concurrent waiter on a different event loop sharing the same
refresh_keywould observe the phantom generation bump on its lock-re-acquire path, treat the storage as freshly-refreshed, and skip its own_run_refresh_cmd— proceeding tobuild_httpx_cookies_from_storageagainst STALE on-disk state.Bounded blast radius: the stale-storage caller will fail again on the next token fetch and trigger its own real refresh. No data corruption. But it's a wasted round-trip.
Suggested fix (from CodeRabbit on PR #814)
Track whether
inflightwas ever non-None during the cancel-recovery loop. Only bump generation ifsubprocess_exc is None AND observed_inflight is True. Or per the alternative suggestion: raise immediately wheninflight is Noneat the top of the cancel handler (the subprocess never registered, so we can't be the leader-cancel-before-await case for an actually-running refresh).Why not fixed in PR #814
That PR's scope was "extract verbatim from auth.py into _auth/refresh.py + facade-forward". Changing async semantics was explicitly out of scope (
must_not_do). Filing here for a dedicated follow-up.References
- CodeRabbit comment: https://github.com/teng-lin/notebooklm-py/pull/814#discussion_r3256142201
- Original code:
src/notebooklm/_auth/refresh.py:487-537(tier-10),src/notebooklm/auth.py:961-1026(tier-9 E origin)
Interface
No new interfaces are introduced.
- Caller invokes
- Assistant
Now let me look at
_coalesced_run_refresh_cmditself: - Assistant
Now let me check the existing tests for this cancel-recovery path:
- Assistant
Now let me verify the facade in
auth.pyforwards monkeypatches of_coalesced_run_refresh_cmd(the tests patch viaauth_mod): - Assistant
Now run the test file:
- Assistant
All 6 pass. Now verify the new tests are a real red gate — they must fail against the pre-fix code:
- Assistant
Both tests fail pre-fix (the phantom bump) and pass post-fix. Now run the full checks — mypy, ruff, and the whole suite:
- Assistant
All failures are in playwright/login-related tests — likely environmental. Confirm they're pre-existing on clean HEAD:
- Assistant
Done. The fix and regression tests are in place.
What was fixed
src/notebooklm/_auth/refresh.py—_fetch_tokens_with_refreshcancel-recovery loop (src/notebooklm/_auth/refresh.py:487):The
except asyncio.CancelledError:handler could fall through withsubprocess_exc is Nonewhen cancellation landed before_coalesced_run_refresh_cmdregistered its in-flight future — bumping_REFRESH_GENERATIONSeven though no subprocess ran (the phantom bump from the issue).The fix goes slightly beyond the issue's "raise when
inflight is None" suggestion, because that alone leaves an equivalent hole:_settleintentionally leaves done futures in the registry (the PR #621 fix), so after any earlier refresh cycle on the same loop, a pre-registration cancel would find the stale done future, read itsexception()(None— the old cycle succeeded), and take the same phantom-bump path. So the fix:- Snapshots the registry entry (and whether it was already settled) before the first
_coalesced_run_refresh_cmdcall. - In the cancel handler, if
inflight is Noneorinflightis still that pre-call already-done future (a leader would have replaced it, so registration never happened), it re-raises the cancellation immediately — no generation bump. - The legit cases are untouched: a fresh future (we were leader) or a not-yet-done pre-call future (we joined as follower) still go through the settle-inspection / re-await logic.
Also removed the now-unreachable
if inflight is not None:guard in the done-branch.Verification
- Red gate confirmed: both new tests in
tests/integration/concurrency/test_refresh_cmd_race.pyfail on pre-fix code (generation phantom-bumps to 1 and 2 respectively) and pass with the fix:test_cancel_before_inflight_registration_does_not_bump— the exact reproducer from the issue (empty registry).test_cancel_before_registration_with_stale_done_future— the stale-done-future variant.
- All 6 tests in the race file pass; 449 tests across the concurrency/refresh/shim suites pass.
mypy src/notebooklmclean;ruff check+ruff format --checkclean.- Full suite: the only failures (32) are Playwright-login/env tests that fail identically on clean HEAD (browser binary not installed here) — verified via stash/diff, no regressions from this change.
Changes are left uncommitted (working tree:
src/notebooklm/_auth/refresh.py+ the test file) — say the word if you want a commit/PR. - Snapshots the registry entry (and whether it was already settled) before the first
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- fable-5Compare side by side →
- fable-5Compare side by side →