Skip to content
TheChatArchiveThe Chat Archive
fable-5· unknownHugging Face ibragim-bad/swe_rebench_07_2026_trajectories

auth: spurious REFRESHGENERATIONS bump on early-cancel in coalescedrunrefreshcmd Summary…

  1. User

    auth: spurious _REFRESH_GENERATIONS bump on early-cancel in _coalesced_run_refresh_cmd

    Summary

    _coalesced_run_refresh_cmd in src/notebooklm/_auth/refresh.py (moved from src/notebooklm/auth.py in tier-10 PR-B-high #814, but the code is byte-identical to origin/main since tier-9 E #803) can bump _REFRESH_GENERATIONS[refresh_key] even when no real refresh happened, in a narrow cancel-before-await race.

    Reproducer (race condition)

    1. Caller invokes await _coalesced_run_refresh_cmd(refresh_key, path, profile) from the cancel-recovery loop in _fetch_tokens_with_refresh.
    2. Task is cancelled in the window between try: and the first with _REFRESH_STATE_LOCK: block inside _coalesced_run_refresh_cmd — so NO future is registered in _REFRESH_INFLIGHT_BY_LOOP[loop][refresh_key].
    3. Control jumps to except asyncio.CancelledError: in the caller.
    4. registry.get(refresh_key) returns None.
    5. The inner if inflight is None or inflight.done(): is taken; the if inflight is not None: block is skipped; subprocess_exc stays None; break.
    6. Outer if subprocess_exc is not None: is False, so no raise.
    7. Generation is bumped at the with _REFRESH_STATE_LOCK: ... max(existing, refresh_generation + 1) line, even though no subprocess ran.
    8. Then if caller_cancelled: raise asyncio.CancelledError().

    Impact

    A concurrent waiter on a different event loop sharing the same refresh_key would observe the phantom generation bump on its lock-re-acquire path, treat the storage as freshly-refreshed, and skip its own _run_refresh_cmd — proceeding to build_httpx_cookies_from_storage against STALE on-disk state.

    Bounded blast radius: the stale-storage caller will fail again on the next token fetch and trigger its own real refresh. No data corruption. But it's a wasted round-trip.

    Suggested fix (from CodeRabbit on PR #814)

    Track whether inflight was ever non-None during the cancel-recovery loop. Only bump generation if subprocess_exc is None AND observed_inflight is True. Or per the alternative suggestion: raise immediately when inflight is None at the top of the cancel handler (the subprocess never registered, so we can't be the leader-cancel-before-await case for an actually-running refresh).

    Why not fixed in PR #814

    That PR's scope was "extract verbatim from auth.py into _auth/refresh.py + facade-forward". Changing async semantics was explicitly out of scope (must_not_do). Filing here for a dedicated follow-up.

    References

    Interface

    No new interfaces are introduced.

  2. Assistant

    Now let me look at _coalesced_run_refresh_cmd itself:

  3. Assistant

    Now let me check the existing tests for this cancel-recovery path:

  4. Assistant

    Now let me verify the facade in auth.py forwards monkeypatches of _coalesced_run_refresh_cmd (the tests patch via auth_mod):

  5. Assistant

    Now run the test file:

  6. Assistant

    All 6 pass. Now verify the new tests are a real red gate — they must fail against the pre-fix code:

  7. Assistant

    Both tests fail pre-fix (the phantom bump) and pass post-fix. Now run the full checks — mypy, ruff, and the whole suite:

  8. Assistant

    All failures are in playwright/login-related tests — likely environmental. Confirm they're pre-existing on clean HEAD:

  9. Assistant

    Done. The fix and regression tests are in place.

    What was fixed

    src/notebooklm/_auth/refresh.py — _fetch_tokens_with_refresh cancel-recovery loop (src/notebooklm/_auth/refresh.py:487):

    The except asyncio.CancelledError: handler could fall through with subprocess_exc is None when cancellation landed before _coalesced_run_refresh_cmd registered its in-flight future — bumping _REFRESH_GENERATIONS even though no subprocess ran (the phantom bump from the issue).

    The fix goes slightly beyond the issue's "raise when inflight is None" suggestion, because that alone leaves an equivalent hole: _settle intentionally leaves done futures in the registry (the PR #621 fix), so after any earlier refresh cycle on the same loop, a pre-registration cancel would find the stale done future, read its exception() (None — the old cycle succeeded), and take the same phantom-bump path. So the fix:

    1. Snapshots the registry entry (and whether it was already settled) before the first _coalesced_run_refresh_cmd call.
    2. In the cancel handler, if inflight is None or inflight is still that pre-call already-done future (a leader would have replaced it, so registration never happened), it re-raises the cancellation immediately — no generation bump.
    3. The legit cases are untouched: a fresh future (we were leader) or a not-yet-done pre-call future (we joined as follower) still go through the settle-inspection / re-await logic.

    Also removed the now-unreachable if inflight is not None: guard in the done-branch.

    Verification

    • Red gate confirmed: both new tests in tests/integration/concurrency/test_refresh_cmd_race.py fail on pre-fix code (generation phantom-bumps to 1 and 2 respectively) and pass with the fix:
      • test_cancel_before_inflight_registration_does_not_bump — the exact reproducer from the issue (empty registry).
      • test_cancel_before_registration_with_stale_done_future — the stale-done-future variant.
    • All 6 tests in the race file pass; 449 tests across the concurrency/refresh/shim suites pass.
    • mypy src/notebooklm clean; ruff check + ruff format --check clean.
    • Full suite: the only failures (32) are Playwright-login/env tests that fail identically on clean HEAD (browser binary not installed here) — verified via stash/diff, no regressions from this change.

    Changes are left uncommitted (working tree: src/notebooklm/_auth/refresh.py + the test file) — say the word if you want a commit/PR.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.