Skip to content
TheChatArchiveThe Chat Archive
fable-5· unknownHugging Face ibragim-bad/swe_rebench_07_2026_trajectories

auth: spurious REFRESHGENERATIONS bump on early-cancel in coalescedrunrefreshcmd Summary…

  1. User

    auth: spurious _REFRESH_GENERATIONS bump on early-cancel in _coalesced_run_refresh_cmd

    Summary

    _coalesced_run_refresh_cmd in src/notebooklm/_auth/refresh.py (moved from src/notebooklm/auth.py in tier-10 PR-B-high #814, but the code is byte-identical to origin/main since tier-9 E #803) can bump _REFRESH_GENERATIONS[refresh_key] even when no real refresh happened, in a narrow cancel-before-await race.

    Reproducer (race condition)

    1. Caller invokes await _coalesced_run_refresh_cmd(refresh_key, path, profile) from the cancel-recovery loop in _fetch_tokens_with_refresh.
    2. Task is cancelled in the window between try: and the first with _REFRESH_STATE_LOCK: block inside _coalesced_run_refresh_cmd — so NO future is registered in _REFRESH_INFLIGHT_BY_LOOP[loop][refresh_key].
    3. Control jumps to except asyncio.CancelledError: in the caller.
    4. registry.get(refresh_key) returns None.
    5. The inner if inflight is None or inflight.done(): is taken; the if inflight is not None: block is skipped; subprocess_exc stays None; break.
    6. Outer if subprocess_exc is not None: is False, so no raise.
    7. Generation is bumped at the with _REFRESH_STATE_LOCK: ... max(existing, refresh_generation + 1) line, even though no subprocess ran.
    8. Then if caller_cancelled: raise asyncio.CancelledError().

    Impact

    A concurrent waiter on a different event loop sharing the same refresh_key would observe the phantom generation bump on its lock-re-acquire path, treat the storage as freshly-refreshed, and skip its own _run_refresh_cmd — proceeding to build_httpx_cookies_from_storage against STALE on-disk state.

    Bounded blast radius: the stale-storage caller will fail again on the next token fetch and trigger its own real refresh. No data corruption. But it's a wasted round-trip.

    Suggested fix (from CodeRabbit on PR #814)

    Track whether inflight was ever non-None during the cancel-recovery loop. Only bump generation if subprocess_exc is None AND observed_inflight is True. Or per the alternative suggestion: raise immediately when inflight is None at the top of the cancel handler (the subprocess never registered, so we can't be the leader-cancel-before-await case for an actually-running refresh).

    Why not fixed in PR #814

    That PR's scope was "extract verbatim from auth.py into _auth/refresh.py + facade-forward". Changing async semantics was explicitly out of scope (must_not_do). Filing here for a dedicated follow-up.

    References

    Interface

    No new interfaces are introduced.

  2. Assistant

    Let me look at _coalesced_run_refresh_cmd and the surrounding registry helpers.

  3. Assistant

    Now let me check how the auth.py facade forwards monkeypatches, since tests patch via auth_mod:

  4. Assistant

    Now add regression tests to the existing race test file:

  5. Assistant

    All 6 pass. Now verify the two new tests actually fail against the pre-fix code (red-gate check):

  6. Assistant

    Only one of the two new tests matched the -k filter. Re-run the red-gate check with both:

  7. Assistant

    These failures are all in playwright/login CLI tests — likely pre-existing environment issues. Verify against the clean tree:

  8. Assistant

    Identical 11 failures / 21 errors on the clean tree — pre-existing environment issues (Playwright browser not installed), unrelated to this change. Run ruff on the changed files:

  9. Assistant

    Done. The fix is complete and verified.

    Changes

    src/notebooklm/_auth/refresh.py (_fetch_tokens_with_refresh, cancel-recovery loop):

    1. Snapshot the registry slot before the first _coalesced_run_refresh_cmd call (prior_inflight / prior_was_done), taken under _REFRESH_STATE_LOCK per the module's convention.
    2. Raise immediately in the CancelledError handler when this cycle never registered a future — i.e. when inflight is None (the exact race in the issue), or when the slot still holds a previous cycle's already-done future (prior_was_done and inflight is prior_inflight). In both cases no subprocess ran on our behalf, so the cancellation propagates without touching _REFRESH_GENERATIONS — closing the phantom-bump path where a cross-loop waiter would skip its own refresh and reload stale storage.

    The second condition covers a sibling variant of the same bug class: _settle intentionally leaves done futures in the registry (PR #621 finding), so an early-cancel could otherwise misread a stale successful future as its own cycle's result and bump anyway. The re-await path for a genuinely in-flight future (the documented cancel-safety design: hold the asyncio lock until the shielded subprocess settles) is unchanged.

    tests/integration/concurrency/test_refresh_cmd_race.py — two regression tests:

    • test_cancel_before_inflight_registration_does_not_bump — simulates cancel-before-registration (stub raises CancelledError without registering); asserts CancelledError propagates and generation stays 0.
    • test_cancel_before_registration_with_stale_done_future_does_not_bump — same, but with a planted done-successful future from a "previous cycle" in the registry.

    Verification

    • Red gate: both new tests fail on the pre-fix code (phantom bump to generation 1) and pass with the fix.
    • All 6 tests in the race suite pass; full suite: 5346 passed — the 11 failures / 21 errors are pre-existing Playwright-environment issues, reproduced identically on clean HEAD.
    • mypy src/notebooklm: clean; ruff format --check + ruff check: clean.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.