Skip to content
TheChatArchiveThe Chat Archive
gpt-5.6-sol· openaiHugging Face ibragim-bad/swe_rebench_07_2026_trajectories

ISSUE DESCRIPTION Cap eval table page size for non-export requests Summary GET…

  1. User

    ISSUE DESCRIPTION

    Cap eval table page size for non-export requests

    Summary

    GET /api/eval/:id/table accepts any positive integer for limit on normal table requests. The route already intentionally treats export requests differently by using Number.MAX_SAFE_INTEGER when format is set, but non-export page requests currently do not have an upper bound.

    Why this matters

    A very large limit can make a normal table request ask the server to load and serialize a large amount of eval result data in one response. Other list-style APIs in the server already bound their page sizes, so the eval table endpoint should follow the same resource-control pattern.

    Proposed change

    • Add a maximum limit for non-export GET /api/eval/:id/table requests.
    • Keep format=csv / format=json exports unchanged so full-table exports still work as intended.
    • Return validation errors for excessive non-export limits, consistent with nearby API schema validation.

    A cap of 500 or 1000 would likely cover normal UI usage while preventing accidental oversized page requests.

    Validation

    Suggested tests:

    • schema test rejects EvalSchemas.Table.Query.safeParse({ limit: '<cap + 1>' })
    • route validation test returns 400 for excessive non-export limit
    • export route tests continue to use Number.MAX_SAFE_INTEGER when format is present

    Interface

    Type: Constant Name: EVAL_TABLE_MAX_PAGE_SIZE Path: src/types/api/eval.ts Value/Type: constant literal value 1000 (number) Description: Server-enforced upper bound for the limit parameter on non-export eval table page requests. Exports using format (csv/json) are exempt from this cap; UI pagination controls derive their available page-size options from this value.

    ADDITIONAL ADVICE

    Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use git checkout <file> to restore the file to its original state. You can also reset all changes in the repository using git reset --hard command. Additionally, you can use git stash and git stash pop to temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you call submit) using git status or git diff commands. However you don't need to use git add or git commit before submitting your changes. In fact, submit only submits currently unstaged changes because it uses git diff to get the changes that need to be submitted.

    CHECKLIST

    Before submitting your solution, please go over the following checklist and make sure you've done everything:

    • If an error was reported in the issue description, I have successfully reproduced it.
    • If an error was reported in the issue description, I have confirmed that my fix resolves the error.
    • I have thought about edge cases and made sure my fix handles them as well.
    • I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.

    Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!

    (Current directory: /promptfoo, current file: none) bash-$

  2. Assistant

    I’ll first inspect the eval API schemas, route implementation, UI pagination usage, and nearby tests to understand the existing validation conventions and export-specific behavior. Then I’ll implement the shared constant, apply conditional validation without affecting exports, update UI options if applicable, and run focused tests plus type/lint checks.

  3. Assistant

    The schema currently validates only positivity, confirming the reported gap. Since ripgrep is unavailable, I’ll use grep/find to locate the route, schema namespace exports, pagination controls, and focused tests.

  4. Assistant

    The route parses the query before replacing export limits, so placing a simple .max(1000) on limit would incorrectly reject exports that explicitly provide a larger limit. I’ll inspect the exact handler and schema namespace plus current UI pagination implementation to choose a conditional schema refinement that exempts any valid format.

  5. Assistant

    The issue’s interface explicitly requires the UI pagination choices to derive from the new constant. I’ll inspect the select options and import conventions for shared API types. I’ll also check the project’s Zod version/style so the conditional error is attached to limit and produces the expected route error text.

  6. Assistant

    I’ve confirmed the UI currently hardcodes 1000 as its largest choice. Before editing, I’ll run a small direct schema parse to document the reproduction: a non-export limit of 1001 is currently accepted. Then I’ll add the literal exported constant, conditional object-level validation, and generate the selector from a capped options list.

  7. Assistant

    The reproduction shows both cases pass today. I’ll now implement conditional validation at the query object level: limits above 1000 fail only when format is absent, while CSV/JSON queries retain arbitrary positive integer limits that the handler subsequently replaces with Number.MAX_SAFE_INTEGER. The UI selector will map a candidate list filtered by the shared cap and use the constant itself for its maximum option.

  8. Assistant

    Python is not installed, so no files were changed. I’ll apply the same deterministic replacements with Node, then let Biome format the touched files and inspect the resulting diff.

  9. Assistant

    The diff now matches the requested interface and preserves existing selector behavior. I’ll verify boundary and exemption cases directly (1000 accepted, 1001 rejected, large CSV/JSON accepted), then run the schema, route-validation, export-route, and ResultsTable test files.

  10. Assistant

    All focused tests pass, including 13 export tests that assert Number.MAX_SAFE_INTEGER, 26 route-validation tests, and 71 ResultsTable tests. The direct check confirms exact boundary semantics. I’ll now run TypeScript compilation and OpenAPI consistency checks, because object-level refinements and cross-workspace value imports can expose integration issues not covered by runtime tests.

  11. Assistant

    Root TypeScript compilation completed successfully; the combined command failed only because openapi:check requires a generated site/static/openapi.json that is absent in this checkout. I won’t create/commit a large generated artifact solely to repair the environment. I’ll inspect status, run the app’s own type/build check if configured, and review OpenAPI schema registration to ensure the refinement does not impair generation.

  12. Assistant

    Only the two intended source files are modified and the diff has no whitespace errors. I’ll run the app workspace compiler and lint the touched files. I’ll also execute a temporary route-level test without adding repository tests: it will request limit=1001 against the existing validation test setup pattern and confirm a 400 occurs before model lookup. First I’ll inspect that test’s setup so the temporary check uses the same mocks safely.

  13. Assistant

    Both root and app TypeScript checks pass, as does Biome. To explicitly validate the HTTP behavior requested in the checklist, I’ll launch the real app in a short inline script, send a non-export limit=1001 request, assert status 400 and a limit validation message, and close the server. Because validation occurs before database access, this requires no fixture or repository test changes.

  14. Assistant

    The inline command was intercepted by Bash history expansion on !; the app did not run. I’ll rerun with single-quoted source to prevent shell interpretation.

  15. Assistant

    The real route now returns the expected 400 with the error path limit, before any data lookup. I’ll perform final integration checks: architecture boundaries (important for the new app-to-shared-constant import), OpenAPI generation (to ensure Zod refinement conversion succeeds), then remove the generated spec because it was absent before and verify only intended files remain.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.