no warning when api-key in arrearage There is no response for chat when api-key in…
- User
no warning when api-key in arrearage There is no response for chat when api-key in arrearage, please give me a warning
Evaluation contract
Treat HTTP 402, normalized billing/quota tokens such as
insufficient_quota,quota_exceeded,quota_exhausted,billing_hard_limit_reached, andpayment_required, and unambiguous billing text such as “exceeded your current quota”, “insufficient balance”, or “payment required” as account-arrearage failures. Surface a non-empty billing-specific message that does not expose the raw provider dump and takes precedence over a configured generic error message. A plain HTTP 429rate_limit_exceeded, timeouts, connection failures, HTTP 5xx responses, successful responses, and ordinary text that merely mentions a quota are not arrearage. Persist only the existing model-error placeholder in session history.Interface
Type: Function Name: is_arrearage_response Path: nanobot/providers/base.py Input: cls, response: LLMResponse Output: bool Description: Detects API-key arrearage, quota, or billing errors that won't clear on retry. Returns True for HTTP 402 status, billing semantic tokens (e.g., 'insufficient_quota'), or matching text markers in the response content.
- Assistant
Now let me see how error responses are surfaced to the user in the runner/loop:
- Assistant
3. Surface a billing-specific warning in the runner (
nanobot/agent/runner.py): - Assistant
Let me verify the placeholder helper stays as the only thing persisted to history:
- Assistant
Now add tests. First check how existing runner tests are structured:
- Assistant
Now add tests — provider-level classification tests and runner-level surfacing tests:
- Assistant
Now add runner-level tests for the warning surfacing:
- Assistant
Pre-existing lint issues in this test file (tests aren't in the project's
ruff check nanobot/scope). I'll auto-fix the whole file since ruff can: - Assistant
Those lint errors are all pre-existing in files I didn't modify (my changed files pass, verified earlier). Now run the full agent test suite:
- Assistant
All done. When the API key is in arrearage, the user now gets an explicit billing warning instead of silence or a generic error.
Changes
nanobot/providers/base.py-
Added
LLMProvider.is_arrearage_response(cls, response)classmethod that detects billing/quota failures which won't clear on retry. It returnsTrueonly for error responses (finish_reason == "error") that have:- HTTP status 402 (Payment Required), or
- billing semantic tokens in
error_type/error_code(insufficient_quota,quota_exceeded,quota_exhausted,billing_hard_limit_reached,payment_required, …), or - unambiguous billing text markers in the content ("exceeded your current quota", "insufficient balance", "payment required", …).
Successful responses and ordinary text that merely mentions a quota never match; plain 429
rate_limit_exceeded, timeouts, connection failures, and 5xx are classified as non-arrearage. Token/marker sets are shared with the existing non-retryable-429 classification (_ARREARAGE_ERROR_TOKENS/_ARREARAGE_TEXT_MARKERSaliases). -
_is_transient_responsenow short-circuits on arrearage, so a 402 body that happens to contain transient-looking words (e.g. "connection") is no longer retried pointlessly.
nanobot/agent/runner.py- In the LLM-error branch (
runner.py:513), arrearage responses now surface a dedicated warning (_ARREARAGE_ERROR_MESSAGE: "⚠️ The AI provider rejected the request due to a billing or quota problem…") that takes precedence over both the raw provider error dump and the configured genericspec.error_message. The raw provider error is logged at warning level for the operator. - Session history behavior is unchanged: only the existing
[Assistant reply unavailable due to model error.]placeholder is persisted.
Tests
tests/providers/test_provider_retry.py: classification tests for 402 / semantic tokens / text markers, negative cases (plain 429 rate limit, timeout, connection, 5xx, success, ordinary quota mentions), and a no-retry-on-402 retry-policy test.tests/agent/test_runner_errors.py: runner tests asserting the billing warning is surfaced (non-empty, no raw dump, beats the generic message), 402 is covered, plain rate limits keep the existing behavior, and history only records the placeholder.
Verified:
ruff checkpasses on all changed files; fulltests/agent(1039),tests/providers+tests/session(549) suites pass. -
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- fable-5Compare side by side →