Skip to content
TheChatArchiveThe Chat Archive
GLMz-ai/glm-5.2· unknownHugging Face ibragim-bad/swe_rebench_07_2026_trajectories

ISSUE DESCRIPTION Cap eval table page size for non-export requests Summary GET…

  1. User

    ISSUE DESCRIPTION

    Cap eval table page size for non-export requests

    Summary

    GET /api/eval/:id/table accepts any positive integer for limit on normal table requests. The route already intentionally treats export requests differently by using Number.MAX_SAFE_INTEGER when format is set, but non-export page requests currently do not have an upper bound.

    Why this matters

    A very large limit can make a normal table request ask the server to load and serialize a large amount of eval result data in one response. Other list-style APIs in the server already bound their page sizes, so the eval table endpoint should follow the same resource-control pattern.

    Proposed change

    • Add a maximum limit for non-export GET /api/eval/:id/table requests.
    • Keep format=csv / format=json exports unchanged so full-table exports still work as intended.
    • Return validation errors for excessive non-export limits, consistent with nearby API schema validation.

    A cap of 500 or 1000 would likely cover normal UI usage while preventing accidental oversized page requests.

    Validation

    Suggested tests:

    • schema test rejects EvalSchemas.Table.Query.safeParse({ limit: '<cap + 1>' })
    • route validation test returns 400 for excessive non-export limit
    • export route tests continue to use Number.MAX_SAFE_INTEGER when format is present

    Interface

    Type: Constant Name: EVAL_TABLE_MAX_PAGE_SIZE Path: src/types/api/eval.ts Value/Type: constant literal value 1000 (number) Description: Server-enforced upper bound for the limit parameter on non-export eval table page requests. Exports using format (csv/json) are exempt from this cap; UI pagination controls derive their available page-size options from this value.

    ADDITIONAL ADVICE

    Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use git checkout <file> to restore the file to its original state. You can also reset all changes in the repository using git reset --hard command. Additionally, you can use git stash and git stash pop to temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you call submit) using git status or git diff commands. However you don't need to use git add or git commit before submitting your changes. In fact, submit only submits currently unstaged changes because it uses git diff to get the changes that need to be submitted.

    CHECKLIST

    Before submitting your solution, please go over the following checklist and make sure you've done everything:

    • If an error was reported in the issue description, I have successfully reproduced it.
    • If an error was reported in the issue description, I have confirmed that my fix resolves the error.
    • I have thought about edge cases and made sure my fix handles them as well.
    • I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.

    Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!

    (Current directory: /promptfoo, current file: none) bash-$

  2. GLM Assistant

    I'll start by exploring the repository structure to understand the codebase and find the relevant files mentioned in the issue.

  3. GLM Assistant

    I'll start by exploring the repository structure to understand the codebase.

  4. GLM Assistant

    Let me look at the eval.ts types file and find the relevant schema.

  5. GLM Assistant

    Now let me find the route handler that uses this schema and handles the export logic.

  6. GLM Assistant

    Let me look at the route handler in detail.

  7. GLM Assistant

    Let me look at the existing tests for the eval table schema and route to understand patterns.

  8. GLM Assistant

    Let me search more broadly for tests related to the eval schema and route.

  9. GLM Assistant

    Let me look at how the schemas are tested. Let me search for test directories that test the schemas (server-side validation).

  10. GLM Assistant

    No server tests exist. Let me look at where the schema validation tests might be. Let me search the whole repo.

  11. GLM Assistant

    Let me look at how other list-style APIs bound their page sizes, to follow the existing pattern.

  12. GLM Assistant

    Let me check the Zod version and how refinements/issues are used in the codebase.

  13. GLM Assistant

    Let me look at the existing superRefine pattern in src/contracts/shared.ts to follow the convention for addIssue.

  14. GLM Assistant

    Now let me check the existing tests that may reference the limit or page sizes to ensure my change doesn't break them.

  15. GLM Assistant

    Let me check how the frontend makes the table request and what page sizes it uses, to ensure my change won't break the UI.

  16. GLM Assistant

    Let me look at the ResultsTable pagination page-size selector options to understand what page-size options the UI offers.

  17. GLM Assistant

    The UI page-size options are 10, 50, 100, 500, 1000 — so a cap of 1000 matches the max UI option exactly. Now let me look at the OpenAPI query helper to make sure a superRefine-wrapped schema will work for OpenAPI generation.

  18. GLM Assistant

    Let me check whether there's a committed OpenAPI spec and a check test that could be affected.

  19. GLM Assistant

    Now I understand the structure. Let me reproduce/verify the current behavior first with a quick script before making changes.

  20. GLM Assistant

    Confirmed the bug: limit=1001 with no format currently succeeds. Now let me implement the fix by editing src/types/api/eval.ts.

  21. GLM Assistant

    Now let me open the file and make the edit:

  22. GLM Assistant

    Now I'll make the edit to add the constant and the conditional cap:

  23. GLM Assistant

    Now let me check whether EVAL_TABLE_MAX_PAGE_SIZE should be re-exported from the types barrel, and verify the route still works. Let me check the index exports.

  24. GLM Assistant

    The constant is exported at the top level of the file. Now let me verify the fix with a reproduction script covering all the test scenarios.

  25. GLM Assistant

    The ! triggered bash history expansion. Let me write the test to a file to avoid shell quoting issues.

  26. GLM Assistant

    All scenarios pass. Now let me verify the OpenAPI spec generation still works (since the schema is registered there) and run a typecheck.

  27. GLM Assistant

    The spec file isn't at the default path. Let me find where the committed OpenAPI spec is.

  28. GLM Assistant

    There's no committed spec, so openapi:check isn't part of CI here. Let me confirm the spec still generates without error (to a temp path) and inspect the EvalTableQuery portion.

  29. GLM Assistant

    OpenAPI generation works correctly with the superRefine wrapper. Now let me confirm the route still uses Number.MAX_SAFE_INTEGER for exports and returns 400 on schema failure (already in place). Let me view the route section.

  30. GLM Assistant

    The route already returns 400 on schema failure and uses MAX_SAFE_INTEGER for exports — no route changes needed. Now let me run the existing tests that could be affected. Let me check the test setup.

  31. GLM Assistant

    The root vitest config only covers test/**. The app tests run via test:app. Let me run the relevant app tests.

  32. GLM Assistant

    All app tests pass. Let me check the test/ directory for any eval table route/schema tests that might be affected.

  33. GLM Assistant

    There are dedicated tests. Let me look at the most relevant ones: the schema test, validation test, and export test.

  34. GLM Assistant

    Now let me look at the validation and export route tests to understand expectations.

  35. GLM Assistant

    Let me look at the export route test and the routes-eval test to ensure my change doesn't break existing export behavior.

  36. GLM Assistant

    Let me see the full export test, especially around line 350-400, to check for any limit exceeding the cap with format.

  37. GLM Assistant

    Let me run all the relevant test files to confirm my change doesn't break anything.

  38. GLM Assistant

    The one failure is an environment issue — better-sqlite3 native bindings aren't compiled for this Node version (node-v137), unrelated to my change. Let me confirm it fails identically without my changes by stashing.

  39. GLM Assistant

    Confirmed — the eval.filteredMetrics.test.ts failure is a pre-existing environment issue (better-sqlite3 native binding for node-v137), identical with or without my change. Let me try to rebuild better-sqlite3 so I can run the DB-backed route tests too.

  40. GLM Assistant

    Rebuild succeeded. Let me re-run the DB-backed route tests.

  41. GLM Assistant

    All 68 tests pass. Now let me write a temporary test to verify the route returns 400 for an excessive non-export limit (matching the issue's suggested route test), then remove it. Let me check the vitest config include pattern first.

  42. GLM Assistant

    I'll create a temporary test under test/ (to be removed before submitting) to verify the route behavior for excessive non-export limits.

  43. GLM Assistant

    globals: false requires importing beforeEach. Let me fix the import.

  44. GLM Assistant

    All three route-level scenarios pass. Now let me remove the temporary test file.

  45. GLM Assistant

    Only src/types/api/eval.ts is modified. Let me review the final diff.

  46. GLM Assistant

    The diff looks clean. Let me run the repo's linter (biome) on the changed file to ensure formatting compliance.

  47. GLM Assistant

    Biome passes. Let me search the test suite for any test that passes a large limit to the table endpoint without a format, which could now break.

  48. GLM Assistant

    The smoke test uses limit=abc (NaN → 400), unaffected by my change. Let me run the smoke route test and the evalTableUtils test to be thorough.

  49. GLM Assistant

    All 201 tests pass. Let me verify the constant is accessible via the @promptfoo/types/api/eval package path (as the app imports types).

  50. GLM Assistant

    Let me check how the app resolves @promptfoo/types/api/eval to confirm the constant would be importable from the app.

  51. GLM Assistant

    The app resolves @promptfoo to src/, so the constant is accessible. The UI already caps page-size options at 1000, matching the constant. Let me verify the prettified error message includes limit (for consistency with nearby validation that checks error.toContain(field)).

  52. GLM Assistant

    Let me place the temp script in the repo root so module resolution works.

  53. GLM Assistant

    The error message is clear and includes limit. The change is complete and verified. Let me do a final review of the changes before submitting.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.