Skip to content
TheChatArchiveThe Chat Archive
fable-5· unknownHugging Face ibragim-bad/swe_rebench_07_2026_trajectories

no warning when api-key in arrearage There is no response for chat when api-key in…

  1. User

    no warning when api-key in arrearage There is no response for chat when api-key in arrearage, please give me a warning

    Evaluation contract

    Treat HTTP 402, normalized billing/quota tokens such as insufficient_quota, quota_exceeded, quota_exhausted, billing_hard_limit_reached, and payment_required, and unambiguous billing text such as “exceeded your current quota”, “insufficient balance”, or “payment required” as account-arrearage failures. Surface a non-empty billing-specific message that does not expose the raw provider dump and takes precedence over a configured generic error message. A plain HTTP 429 rate_limit_exceeded, timeouts, connection failures, HTTP 5xx responses, successful responses, and ordinary text that merely mentions a quota are not arrearage. Persist only the existing model-error placeholder in session history.

    Interface

    Type: Function Name: is_arrearage_response Path: nanobot/providers/base.py Input: cls, response: LLMResponse Output: bool Description: Detects API-key arrearage, quota, or billing errors that won't clear on retry. Returns True for HTTP 402 status, billing semantic tokens (e.g., 'insufficient_quota'), or matching text markers in the response content.

  2. Assistant

    Now let me see how error responses are surfaced to the user in the runner/loop:

  3. Assistant

    3. Surface a billing-specific warning in the runner (nanobot/agent/runner.py):

  4. Assistant

    Let me verify the placeholder helper stays as the only thing persisted to history:

  5. Assistant

    Now add tests. First check how existing runner tests are structured:

  6. Assistant

    Now add tests — provider-level classification tests and runner-level surfacing tests:

  7. Assistant

    Now add runner-level tests for the warning surfacing:

  8. Assistant

    Pre-existing lint issues in this test file (tests aren't in the project's ruff check nanobot/ scope). I'll auto-fix the whole file since ruff can:

  9. Assistant

    Those lint errors are all pre-existing in files I didn't modify (my changed files pass, verified earlier). Now run the full agent test suite:

  10. Assistant

    All done. When the API key is in arrearage, the user now gets an explicit billing warning instead of silence or a generic error.

    Changes

    nanobot/providers/base.py

    • Added LLMProvider.is_arrearage_response(cls, response) classmethod that detects billing/quota failures which won't clear on retry. It returns True only for error responses (finish_reason == "error") that have:

      • HTTP status 402 (Payment Required), or
      • billing semantic tokens in error_type/error_code (insufficient_quota, quota_exceeded, quota_exhausted, billing_hard_limit_reached, payment_required, …), or
      • unambiguous billing text markers in the content ("exceeded your current quota", "insufficient balance", "payment required", …).

      Successful responses and ordinary text that merely mentions a quota never match; plain 429 rate_limit_exceeded, timeouts, connection failures, and 5xx are classified as non-arrearage. Token/marker sets are shared with the existing non-retryable-429 classification (_ARREARAGE_ERROR_TOKENS / _ARREARAGE_TEXT_MARKERS aliases).

    • _is_transient_response now short-circuits on arrearage, so a 402 body that happens to contain transient-looking words (e.g. "connection") is no longer retried pointlessly.

    nanobot/agent/runner.py

    • In the LLM-error branch (runner.py:513), arrearage responses now surface a dedicated warning (_ARREARAGE_ERROR_MESSAGE: "⚠️ The AI provider rejected the request due to a billing or quota problem…") that takes precedence over both the raw provider error dump and the configured generic spec.error_message. The raw provider error is logged at warning level for the operator.
    • Session history behavior is unchanged: only the existing [Assistant reply unavailable due to model error.] placeholder is persisted.

    Tests

    • tests/providers/test_provider_retry.py: classification tests for 402 / semantic tokens / text markers, negative cases (plain 429 rate limit, timeout, connection, 5xx, success, ordinary quota mentions), and a no-retry-on-402 retry-policy test.
    • tests/agent/test_runner_errors.py: runner tests asserting the billing warning is surfaced (non-empty, no raw dump, beats the generic message), 402 is covered, plain rate limits keep the existing behavior, and history only records the placeholder.

    Verified: ruff check passes on all changed files; full tests/agent (1039), tests/providers + tests/session (549) suites pass.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.