ISSUE DESCRIPTION auth: spurious REFRESHGENERATIONS bump on early-cancel in…
- User
ISSUE DESCRIPTION
auth: spurious _REFRESH_GENERATIONS bump on early-cancel in _coalesced_run_refresh_cmd
Summary
_coalesced_run_refresh_cmdinsrc/notebooklm/_auth/refresh.py(moved fromsrc/notebooklm/auth.pyin tier-10 PR-B-high #814, but the code is byte-identical to origin/main since tier-9 E #803) can bump_REFRESH_GENERATIONS[refresh_key]even when no real refresh happened, in a narrow cancel-before-await race.Reproducer (race condition)
- Caller invokes
await _coalesced_run_refresh_cmd(refresh_key, path, profile)from the cancel-recovery loop in_fetch_tokens_with_refresh. - Task is cancelled in the window between
try:and the firstwith _REFRESH_STATE_LOCK:block inside_coalesced_run_refresh_cmd— so NO future is registered in_REFRESH_INFLIGHT_BY_LOOP[loop][refresh_key]. - Control jumps to
except asyncio.CancelledError:in the caller. registry.get(refresh_key)returnsNone.- The inner
if inflight is None or inflight.done():is taken; theif inflight is not None:block is skipped;subprocess_excstaysNone;break. - Outer
if subprocess_exc is not None:is False, so no raise. - Generation is bumped at the
with _REFRESH_STATE_LOCK: ... max(existing, refresh_generation + 1)line, even though no subprocess ran. - Then
if caller_cancelled: raise asyncio.CancelledError().
Impact
A concurrent waiter on a different event loop sharing the same
refresh_keywould observe the phantom generation bump on its lock-re-acquire path, treat the storage as freshly-refreshed, and skip its own_run_refresh_cmd— proceeding tobuild_httpx_cookies_from_storageagainst STALE on-disk state.Bounded blast radius: the stale-storage caller will fail again on the next token fetch and trigger its own real refresh. No data corruption. But it's a wasted round-trip.
Suggested fix (from CodeRabbit on PR #814)
Track whether
inflightwas ever non-None during the cancel-recovery loop. Only bump generation ifsubprocess_exc is None AND observed_inflight is True. Or per the alternative suggestion: raise immediately wheninflight is Noneat the top of the cancel handler (the subprocess never registered, so we can't be the leader-cancel-before-await case for an actually-running refresh).Why not fixed in PR #814
That PR's scope was "extract verbatim from auth.py into _auth/refresh.py + facade-forward". Changing async semantics was explicitly out of scope (
must_not_do). Filing here for a dedicated follow-up.References
- CodeRabbit comment: https://github.com/teng-lin/notebooklm-py/pull/814#discussion_r3256142201
- Original code:
src/notebooklm/_auth/refresh.py:487-537(tier-10),src/notebooklm/auth.py:961-1026(tier-9 E origin)
ADDITIONAL ADVICE
Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use
git checkout <file>to restore the file to its original state. You can also reset all changes in the repository usinggit reset --hardcommand. Additionally, you can usegit stashandgit stash popto temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you callsubmit) usinggit statusorgit diffcommands. However you don't need to usegit addorgit commitbefore submitting your changes. In fact,submitonly submits currently unstaged changes because it usesgit diffto get the changes that need to be submitted.CHECKLIST
Before submitting your solution, please go over the following checklist and make sure you've done everything:
- If an error was reported in the issue description, I have successfully reproduced it.
- If an error was reported in the issue description, I have confirmed that my fix resolves the error.
- I have thought about edge cases and made sure my fix handles them as well.
- I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.
Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!
(Current directory: /notebooklm-py, current file: none) bash-$
- Caller invokes
Assistant
Let me start by understanding the repository structure and the issue. The issue is about a spurious
_REFRESH_GENERATIONSbump in_coalesced_run_refresh_cmdwhen there's an early cancel. Let me first explore the relevant files.The key file is
src/notebooklm/_auth/refresh.py. Let me look at the structure of the repository first, then examine the relevant code.Assistant
Let me look at the refresh.py file, particularly around lines 487-537 as mentioned in the issue.
Assistant
Let me open the file and look at the relevant section.
Assistant
Now let me look at the
_coalesced_run_refresh_cmdfunction:Assistant
Let me look at the _coalesced_run_refresh_cmd function starting at line 144.
Assistant
Let me look at how
refresh_generationis set in the caller.Assistant
Now let me also look at the global state variables and understand the full picture:
Assistant
Let me continue reading the global state.
Assistant
Now I have a clear understanding of the issue. Let me summarize:
The Problem: In
_fetch_tokens_with_refresh, the while-loop at lines 487-520 calls_coalesced_run_refresh_cmd. If cancellation happens BEFORE_coalesced_run_refresh_cmdcan register its future in the registry (i.e., before line 172), the cancel handler at line 493 findsinflight = None. Sinceinflight is None, theif inflight is not None:block is skipped,subprocess_excstaysNone, and webreak. Then at line 522,subprocess_exc is not NoneisFalse, so we proceed to bump generation at lines 538-543 — even though no subprocess actually ran.The Fix: When
inflight is Nonein the cancel handler, we know the subprocess was never registered. We should setsubprocess_exc = asyncio.CancelledError()so that the subsequentif subprocess_exc is not None:check correctly routes to the error path (no generation bump) instead of the success path.Let me apply this
Assistant
Let me go to the right line:
Assistant
Now I'll apply the fix. I need to add an
elseclause to handle the case whereinflight is Nonein the cancel handlerAssistant
The edit looks correct. Now let me verify the change by tracing through the logic and running existing tests. Let me first check if there are relevant tests.
Assistant
Let me check the concurrency test for the refresh cmd race, which is likely the most relevant.
Assistant
Let me look at the cancellation
Assistant
Let me look at the refresh_cmd_race
Assistant
Let me run the existing relevant tests to make sure my change doesn't break anything
Assistant
Let me install the package
Assistant
Now let me run the relevant tests
Assistant
Let me install the test dependencies
Assistant
Now let me run the tests
Assistant
All 4 tests pass. Let me now create a reproduction script to confirm the issue exists and that my fix resolves it
Assistant
The fix works. Let me verify that the old code would have had the bug by temporarily reverting and running the test
Assistant
Confirmed: the bug exists without the fix (generation bumped to 1). Let me restore the
Assistant
Now let me also run the broader test suite to make sure nothing is broken
Assistant
Let me run all related tests to ensure nothing is
Assistant
Let me also run the cancellation propagation test and the public
Assistant
All tests pass. Let me also run the full unit test suite to make sure nothing else
Assistant
The failures/errors (12 failed, 21 errors) are all in
tests/unit/cli/test_session.pywhich is about Playwright/CLI tests — completely unrelated to my change. All 4673 other tests pass.Let me verify the diff looks correct
Assistant
The change is minimal and correct. Let me verify the final state of the file around the fix
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
Qwen/Qwen3.6-27BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →
- xiaomi/mimo-v2.5-proCompare side by side →
- xiaomi/mimo-v2.5-proCompare side by side →
- xiaomi/mimo-v2.5-proCompare side by side →
- xiaomi/mimo-v2.5-proCompare side by side →
- xiaomi/mimo-v2.5-proCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →