ISSUE DESCRIPTION Cap eval table page size for non-export requests Summary GET…
- User
ISSUE DESCRIPTION
Cap eval table page size for non-export requests
Summary
GET /api/eval/:id/tableaccepts any positive integer forlimiton normal table requests. The route already intentionally treats export requests differently by usingNumber.MAX_SAFE_INTEGERwhenformatis set, but non-export page requests currently do not have an upper bound.Why this matters
A very large
limitcan make a normal table request ask the server to load and serialize a large amount of eval result data in one response. Other list-style APIs in the server already bound their page sizes, so the eval table endpoint should follow the same resource-control pattern.Proposed change
- Add a maximum
limitfor non-exportGET /api/eval/:id/tablerequests. - Keep
format=csv/format=jsonexports unchanged so full-table exports still work as intended. - Return validation errors for excessive non-export limits, consistent with nearby API schema validation.
A cap of
500or1000would likely cover normal UI usage while preventing accidental oversized page requests.Validation
Suggested tests:
- schema test rejects
EvalSchemas.Table.Query.safeParse({ limit: '<cap + 1>' }) - route validation test returns
400for excessive non-exportlimit - export route tests continue to use
Number.MAX_SAFE_INTEGERwhenformatis present
Interface
Type: Constant Name: EVAL_TABLE_MAX_PAGE_SIZE Path: src/types/api/eval.ts Value/Type: constant literal value 1000 (number) Description: Server-enforced upper bound for the
limitparameter on non-export eval table page requests. Exports usingformat(csv/json) are exempt from this cap; UI pagination controls derive their available page-size options from this value.ADDITIONAL ADVICE
Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use
git checkout <file>to restore the file to its original state. You can also reset all changes in the repository usinggit reset --hardcommand. Additionally, you can usegit stashandgit stash popto temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you callsubmit) usinggit statusorgit diffcommands. However you don't need to usegit addorgit commitbefore submitting your changes. In fact,submitonly submits currently unstaged changes because it usesgit diffto get the changes that need to be submitted.CHECKLIST
Before submitting your solution, please go over the following checklist and make sure you've done everything:
- If an error was reported in the issue description, I have successfully reproduced it.
- If an error was reported in the issue description, I have confirmed that my fix resolves the error.
- I have thought about edge cases and made sure my fix handles them as well.
- I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.
Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!
(Current directory: /promptfoo, current file: none) bash-$
- Add a maximum
- Assistant
I’ll first inspect the eval API schemas, route implementation, UI pagination usage, and nearby tests to understand the existing validation conventions and export-specific behavior. Then I’ll implement the shared constant, apply conditional validation without affecting exports, update UI options if applicable, and run focused tests plus type/lint checks.
- Assistant
The schema currently validates only positivity, confirming the reported gap. Since ripgrep is unavailable, I’ll use
grep/findto locate the route, schema namespace exports, pagination controls, and focused tests. - Assistant
The route parses the query before replacing export limits, so placing a simple
.max(1000)onlimitwould incorrectly reject exports that explicitly provide a larger limit. I’ll inspect the exact handler and schema namespace plus current UI pagination implementation to choose a conditional schema refinement that exempts any validformat. - Assistant
The issue’s interface explicitly requires the UI pagination choices to derive from the new constant. I’ll inspect the select options and import conventions for shared API types. I’ll also check the project’s Zod version/style so the conditional error is attached to
limitand produces the expected route error text. - Assistant
I’ve confirmed the UI currently hardcodes
1000as its largest choice. Before editing, I’ll run a small direct schema parse to document the reproduction: a non-export limit of 1001 is currently accepted. Then I’ll add the literal exported constant, conditional object-level validation, and generate the selector from a capped options list. - Assistant
The reproduction shows both cases pass today. I’ll now implement conditional validation at the query object level: limits above 1000 fail only when
formatis absent, while CSV/JSON queries retain arbitrary positive integer limits that the handler subsequently replaces withNumber.MAX_SAFE_INTEGER. The UI selector will map a candidate list filtered by the shared cap and use the constant itself for its maximum option. - Assistant
Python is not installed, so no files were changed. I’ll apply the same deterministic replacements with Node, then let Biome format the touched files and inspect the resulting diff.
- Assistant
The diff now matches the requested interface and preserves existing selector behavior. I’ll verify boundary and exemption cases directly (1000 accepted, 1001 rejected, large CSV/JSON accepted), then run the schema, route-validation, export-route, and ResultsTable test files.
- Assistant
All focused tests pass, including 13 export tests that assert
Number.MAX_SAFE_INTEGER, 26 route-validation tests, and 71 ResultsTable tests. The direct check confirms exact boundary semantics. I’ll now run TypeScript compilation and OpenAPI consistency checks, because object-level refinements and cross-workspace value imports can expose integration issues not covered by runtime tests. - Assistant
Root TypeScript compilation completed successfully; the combined command failed only because
openapi:checkrequires a generatedsite/static/openapi.jsonthat is absent in this checkout. I won’t create/commit a large generated artifact solely to repair the environment. I’ll inspect status, run the app’s own type/build check if configured, and review OpenAPI schema registration to ensure the refinement does not impair generation. - Assistant
Only the two intended source files are modified and the diff has no whitespace errors. I’ll run the app workspace compiler and lint the touched files. I’ll also execute a temporary route-level test without adding repository tests: it will request
limit=1001against the existing validation test setup pattern and confirm a 400 occurs before model lookup. First I’ll inspect that test’s setup so the temporary check uses the same mocks safely. - Assistant
Both root and app TypeScript checks pass, as does Biome. To explicitly validate the HTTP behavior requested in the checklist, I’ll launch the real app in a short inline script, send a non-export
limit=1001request, assert status 400 and alimitvalidation message, and close the server. Because validation occurs before database access, this requires no fixture or repository test changes. - Assistant
The inline command was intercepted by Bash history expansion on
!; the app did not run. I’ll rerun with single-quoted source to prevent shell interpretation. - Assistant
The real route now returns the expected 400 with the error path
limit, before any data lookup. I’ll perform final integration checks: architecture boundaries (important for the new app-to-shared-constant import), OpenAPI generation (to ensure Zod refinement conversion succeeds), then remove the generated spec because it was absent before and verify only intended files remain.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →
Qwen/Qwen3.6-27BCompare side by side →