Skip to content
TheChatArchiveThe Chat Archive
xiaomi/mimo-v2.5-pro· unknownHugging Face ibragim-bad/swe_rebench_07_2026_trajectories

ISSUE DESCRIPTION generator: typed-table emit exceeds SQLite column cap on wide config…

  1. User

    ISSUE DESCRIPTION

    generator: typed-table emit exceeds SQLite column cap on wide config singletons

    Filed by /printing-press-retro from a run of HaloPSA (scorecard 91/100 Grade A, 100% verify pass).

    Summary

    The typed-table emit path generates one column per leaf JSON field in a resource's response schema with no upper bound. On HaloPSA's tenant-global /Control config singleton, this produced a 3,747-column CREATE TABLE that exceeded SQLite's default SQLITE_MAX_COLUMN=2000. Generation passed every quality gate; sync --full --dry-run died cold with SQL logic error: too many columns on control (1). The failure mode is silent until first sync.

    Where to look

    • Component: generator
    • Likely area: Typed-table-emit path in internal/generator/templates/store/ — wherever CREATE TABLE text is composed per discovered resource.
    • Triggered when: A spec resource's response schema flattens to >~1500 leaf fields. Enterprise config singletons are the obvious class.

    What we observed

    Run-1 output (post-generate):

    PASS go mod tidy / govulncheck / go vet / go build
    PASS halopsa-pp-cli --help / version / doctor
    Generated halopsa at .../working/halopsa-pp-cli
    

    First sync --full --dry-run:

    Error: opening local database: running migrations:
    migration failed: SQL logic error: too many columns on control (1)
    

    Per-table column counts (top 5 widest, recorded by Python introspection of internal/store/store.go):

    tablecolumns
    control3747
    tickets847
    projects847
    opportunities847
    client447

    Tickets / projects / opportunities sit just under the 2000 cap but with no headroom; the next major-version Halo response schema could push them over.

    Workaround applied this run: hand-stripped /Control* paths from the spec (5 paths) and added a stub Control schema so residual $ref targets resolved. Regenerated successfully.

    Suspected root cause

    Confident. The typed-table-emit path walks the resource's response schema and emits one column per leaf field with no width sanity check. The store package has no JSON-only fallback for resources whose schema exceeds a safe column count.

    Suggested direction

    In the typed-table-emit path, compute the column count before writing CREATE TABLE. If the count exceeds a threshold (1500 leaves headroom under SQLite's default 2000; tunable), fall back to JSON-only schema: id TEXT PRIMARY KEY, data JSON NOT NULL, synced_at DATETIME. Emit a generation-time log line naming the resource and the fallback (e.g., store-fallback: control (3747 cols) → JSON-only).

    Preserve typed-column behavior below the threshold — the typed columns power offline FTS5 and cross-entity SQL joins; the fallback should only fire on pathological widths.

    Acceptance criteria

    • positive: feed a synthetic spec where one resource has 3000+ leaf fields; generated store.go for that resource has only 3 columns; sync against an empty DB succeeds.
    • negative: feed a normal resource (<1500 cols); typed-table emission unchanged; flat columns still emitted.

    Frequency

    subclass:wide-config-singleton — complete-functional-failure when it fires.

    Complexity

    small (column-count check + JSON-only emit branch).

    Scope boundary

    Does NOT cover per-column truncation, schema-aware projection, or per-API column allowlists. Single threshold + JSON-only fallback only.

    Dependencies

    None.

    Related issues

    None — first sighting of this specific failure mode.

    Artifacts

    ArtifactLink
    Retro documenthttps://files.catbox.moe/ico9uo.md
    Manuscripts (research + proofs)https://files.catbox.moe/rn4hf4.zip
    Generated CLI source codehttps://files.catbox.moe/4v6myv.zip

    Generated by /printing-press-retro · CLI Printing Press

    Evaluation contract

    Use 1500 columns as the safety cap for a generated typed SQLite resource table, including dependent-parent columns. Resources exceeding the cap must retain their per-resource table but fall back to a JSON-only schema containing id, data, and synced_at; do not generate typed-column indexes, FTS structures, or typed upsert code for that fallback table. Batch store generation must dispatch wide resources to the JSON-only path while preserving the existing typed behavior for ordinary narrow resources.

    Interface

    No new interfaces are introduced.

    ADDITIONAL ADVICE

    Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use git checkout <file> to restore the file to its original state. You can also reset all changes in the repository using git reset --hard command. Additionally, you can use git stash and git stash pop to temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you call submit) using git status or git diff commands. However you don't need to use git add or git commit before submitting your changes. In fact, submit only submits currently unstaged changes because it uses git diff to get the changes that need to be submitted.

    CHECKLIST

    Before submitting your solution, please go over the following checklist and make sure you've done everything:

    • If an error was reported in the issue description, I have successfully reproduced it.
    • If an error was reported in the issue description, I have confirmed that my fix resolves the error.
    • I have thought about edge cases and made sure my fix handles them as well.
    • I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.

    Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!

    (Current directory: /cli-printing-press, current file: none) bash-$

  2. Assistant

    I'll start by exploring the repository structure to understand the codebase and locate the relevant files mentioned in the issue. Let's first check what's in the repository root.

  3. Assistant

    Let me explore the repository structure more systematically.

  4. Assistant

    Let me explore the internal directory structure to find the generator templates.

  5. Assistant

    Let me explore the generator directory to find the store templates.

  6. Assistant

    Let me look at the templates directory to find store-related templates.

  7. Assistant

    Let me look at the store templates specifically.

  8. Assistant

    Let me look at the store template and the main generator code.

  9. Assistant

    Let me look at the store template for CREATE TABLE generation.

  10. Assistant

    Let me look at line 279 to see how the CREATE TABLE is generated for each resource.

  11. Assistant

    Now I need to understand how the .Tables data structure is built in the generator code. Let me search for where the Tables field is populated. code</tool_call>

  12. Assistant

    Let me search for the Tables field in the generator code to understand how it's populated.

  13. Assistant

    Let me look at the TableDef structure and the schema building code.

  14. Assistant

    Let me look for TableDef more broadly.

  15. Assistant

    Let me look at the schema_builder.go file to understand the structure and how columns are built.

  16. Assistant

    Now I understand the structure. Let me see the rest of this file to understand the gravity computation and how the columns are added.

  17. Assistant

    Let me continue reading the schema_builder.go to understand the full picture.

  18. Assistant

    Let me continue reading the file to understand the full structure.

  19. Assistant

    Now let me look at hasTypedTable in the template and the upsert code to understand how the dispatch works.

  20. Assistant

    Let me look at the hasTypedTable function implementation.

  21. Assistant

    Let me also look at how the template handles the Upsert code to understand the typed vs JSON-only paths.

  22. Assistant

    Let me look at the domainUpsertMethodName function.

  23. Assistant

    Now let me check how the existing tests work and understand the test structure for schema_builder.

  24. Assistant

    Let me continue reading the schema_builder_test.go to understand how the existing tests work.

  25. Assistant

    Let me continue reading the rest of the schema_builder_test.go.

  26. Assistant

    Let me keep reading to see the findTable helper function and understand the pattern.

  27. Assistant

    Now let me look at how the UpsertBatch path works and what hasDomainUpsert does.

  28. Assistant

    Now let me look at the UpsertBatch template code to understand how it dispatches:

  29. Assistant

    Let me look at the UpsertBatch dispatch to understand how typed vs JSON-only tables are handled.

  30. Assistant

    Now let me look at buildSubResourceTable since sub-resources can also get columns.

  31. Assistant

    Let me look at the buildSubResourceTable function.

  32. Assistant

    Now let me check how logging is done in the generator codebase to understand the pattern for emitting warnings.

  33. Assistant

    Now I have a clear understanding of the code. Let me first look at the existing tests to see how they work before making changes.

  34. Assistant

    Let me check the go environment.

  35. Assistant

    GOPATH and GOMODCACHE are empty. Let me set them.

  36. Assistant

    Let me try running the test again with the environment set.

  37. Assistant

    Need to set GOCACHE and HOME too.

  38. Assistant

    Let me now look at the typed upsert methods in the template to understand what they do for tables.

  39. Assistant

    Let me look at the full migration section of the template more carefully.

  40. Assistant

    Now let me look at the UpsertBatch dispatch and the hasDomainUpsert related code.

  41. Assistant

    Let me look at the remaining template sections including ListField and other generated code.

  42. Assistant

    Now let me look at the test that checks dead table skipping.

  43. Assistant

    Now I have a clear picture. Let me plan my approach:

    1. schema_builder.go: Add a JSONOnly field to TableDef, add a MaxTypedColumns = 1500 constant, and after building typed columns, check if count exceeds the limit. If so, log a fallback message, set JSONOnly = true, reset columns to baseTableColumns, and clear indexes/FTS.

    2. store.go.tmpl: Modify the CREATE TABLE block to emit the table for JSON-only tables too (but without indexes/FTS).

    Let me first look at the template structure around the hasTypedTable gating for the created table vs the Upsert code to make sure I understand the full scope.

  44. Assistant

    Let me now understand the full picture. I need to look at the hasTypedTable function and the template structure to plan my changes. Let me check a few more details:

  45. Assistant

    Now I have a full understanding of the code. Let me implement the fix. My plan:

    1. In schema_builder.go: Add JSONOnly bool field to TableDef, add maxTypedColumns constant, and in BuildSchema when column count exceeds the threshold, set JSONOnly = true and reset columns/FTS/indexes.

    2. In generator.go: Add a hasPerResourceTable template function that returns true for both typed tables AND JSON-only tables.

    3. In store.go.tmpl: Change the CREATE TABLE gate from hasTypedTable to hasPerResourceTable, keeping indexes/FTS inside hasTypedTable only.

    Let me start with the schema_builder.go changes:

  46. Assistant

    Now let me implement the changes. First, let me add the JSONOnly field to TableDef and the maxTypedColumns constant in schema_builder.go:

  47. Assistant

    Now I'll implement the changes. First, let me add the JSONOnly field to TableDef and the maxTypedColumns constant:

  48. Assistant

    Let me use the correct edit syntax:

  49. Assistant

    Let me use the correct syntax for the edit command:

  50. Assistant

    I need to use the edit command with the correct syntax. Let me try the heredoc approach.

  51. Assistant

    Let me try opening the file first, then editing.

  52. Assistant

    Now let me add the JSONOnly field to TableDef and the maxTypedColumns constant:

  53. Assistant

    Let me use sed instead to make the changes:

  54. Assistant

    Let me verify the change was applied correctly.

  55. Assistant

    Now let me add the maxTypedColumns constant and the import for fmt (for logging):

  56. Assistant

    I need to add "fmt" and "os" to the imports and add the maxTypedColumns constant. Let me do this:

  57. Assistant

    Let me verify the changes and add the maxTypedColumns constant:

  58. Assistant

    Now let me add the maxTypedColumns constant after the baseTableColumns variable:

  59. Assistant

    Let me look at lines around 47-52 to add the maxTypedColumns constant:

  60. Assistant

    Now let me add the maxTypedColumns constant after line 51:

  61. Assistant

    Let me verify the changes:

  62. Assistant

    Now I need to add the column count check in BuildSchema after the high-gravity column loop. Let me find the right location:

  63. Assistant

    Let me see more context:

  64. Assistant

    Now I need to add the JSON-only fallback check after line 127 (after the gravity >= 2 block). Let me insert it:

  65. Assistant

    Let me verify the changes:

  66. Assistant

    I notice two issues:

    1. The unicode arrow was written as literal bytes instead of the actual UTF-8 character
    2. The FTS5 logic still runs after the JSON-only check - I need to make sure FTS5 is not set for JSON-only tables

    Let me fix both issues:

  67. Assistant

    Let me check the actual content:

  68. Assistant

    Good, the arrow is now correct. Now I need to fix the FTS5 condition to not set FTS5 for JSON-only tables:

  69. Assistant

    I need to add !table.JSONOnly to the FTS5 condition to prevent FTS for JSON-only tables:

  70. Assistant

    Let me verify the change:

  71. Assistant

    Good. Now I need to update the template and generator.go to handle JSON-only tables properly. Let me first add a hasPerResourceTable template function in generator.go:

  72. Assistant

    Let me look at the code around line 408 to understand how hasTypedTable is defined and add hasPerResourceTable right after it:

  73. Assistant

    Now let me add the hasPerResourceTable function right after hasTypedTable:

  74. Assistant

    Let me verify the changes:

  75. Assistant

    Good. Now I need to update the store.go.tmpl template to use hasPerResourceTable for the CREATE TABLE block, while keeping hasTypedTable for the indexes and FTS5 sections. Let me look at the template structure:

  76. Assistant

    Now I need to modify the template. I'll replace the gate from hasTypedTable to hasPerResourceTable for the CREATE TABLE, and add a nested hasTypedTable check for indexes and FTS5. Let me make this change:

  77. Assistant

    Let me verify the template changes:

  78. Assistant

    The template changes look correct. Now let me verify the code compiles and run the existing tests:

  79. Assistant

    Good, it compiles. Now let me run the existing schema_builder tests:

  80. Assistant

    Good, all existing tests pass. Now let me run the store-related test:

  81. Assistant

    Great, existing tests pass. Let me run a broader set of tests to make sure nothing is broken:

  82. Assistant

    Let me run all generator tests to make sure nothing is broken:

  83. Assistant

    Let me check if this test failure is pre-existing and not related to my changes:

  84. Assistant

    This test failure is pre-existing - it's a govulncheck failure about a Go standard library vulnerability (GO-2026-5856). Not related to my changes.

    Now let me write a reproduction test. Let me first check how the existing tests create synthetic specs:

  85. Assistant

    Now let me first verify my changes are correct by running the existing tests (excluding the pre-existing govulncheck failure). Let me also check if there are any golden file tests that might need updating:

  86. Assistant

    All existing tests pass. Now let me create a reproduction script to verify my fix works correctly. First, let me write a test script:

  87. Assistant

    Wait, the instructions say I shouldn't add new tests to the repository. Let me remove the test file and instead run it from a temporary location:

  88. Assistant

    Let me create a temporary test script outside the repository:

  89. Assistant

    Now let me run this test script:

  90. Assistant

    I need to put the test file inside the package. Let me create it as a test file inside the generator package temporarily:

  91. Assistant

    Now let me run the test:

  92. Assistant

    All three tests pass! The fix is working correctly. Now let me clean up the temporary test file:

  93. Assistant

    Now let me run the broader test suite to make sure my changes don't break anything. Let me run some key tests:

  94. Assistant

    All tests pass. Let me also check the templates test and the mutation_store_writeback test which are very relevant:

  95. Assistant

    Let me also run a broader set of tests, especially the ones related to store generation:

  96. Assistant

    All tests pass. Let me verify my changes are complete by reviewing the final state of each modified file:

  97. Assistant

    Let me also check the generator.go changes and the store.go.tmpl changes:

  98. Assistant

    Let me also verify the template changes are correct:

  99. Assistant

    The template structure looks correct. Let me also run one more test to ensure the generated store.go compiles and the generated code is valid:

  100. Assistant

    Let me run a broader set of tests excluding the known govulncheck failure:

  101. Assistant

    Let me try a different approach:

  102. Assistant

    Let me run more tests to ensure nothing is broken:

  103. Assistant

    All tests pass. Let me now review my complete diff:

  104. Assistant

    The diff looks clean and correct. Let me run a final comprehensive test to make sure everything works:

  105. Assistant

    All tests pass. Let me also check if there are golden file tests that might need updating:

  106. Assistant

    Let me check the golden file to make sure it's not affected by my changes:

  107. Assistant

    Let me run the golden file tests to make sure they still pass:

  108. Assistant

    Let me look for where the golden tests are:

  109. Assistant

    Let me check if there are any golden tests in the main test suite:

  110. Assistant

    Let me verify my diff one final time and make sure it's clean:

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.