Skip to content
TheChatArchiveThe Chat Archive
MiniMaxminimax/minimax-m3· unknownHugging Face ibragim-bad/swe_rebench_07_2026_trajectories

ISSUE DESCRIPTION generator: typed-table emit exceeds SQLite column cap on wide config…

  1. User

    ISSUE DESCRIPTION

    generator: typed-table emit exceeds SQLite column cap on wide config singletons

    Filed by /printing-press-retro from a run of HaloPSA (scorecard 91/100 Grade A, 100% verify pass).

    Summary

    The typed-table emit path generates one column per leaf JSON field in a resource's response schema with no upper bound. On HaloPSA's tenant-global /Control config singleton, this produced a 3,747-column CREATE TABLE that exceeded SQLite's default SQLITE_MAX_COLUMN=2000. Generation passed every quality gate; sync --full --dry-run died cold with SQL logic error: too many columns on control (1). The failure mode is silent until first sync.

    Where to look

    • Component: generator
    • Likely area: Typed-table-emit path in internal/generator/templates/store/ — wherever CREATE TABLE text is composed per discovered resource.
    • Triggered when: A spec resource's response schema flattens to >~1500 leaf fields. Enterprise config singletons are the obvious class.

    What we observed

    Run-1 output (post-generate):

    PASS go mod tidy / govulncheck / go vet / go build
    PASS halopsa-pp-cli --help / version / doctor
    Generated halopsa at .../working/halopsa-pp-cli
    

    First sync --full --dry-run:

    Error: opening local database: running migrations:
    migration failed: SQL logic error: too many columns on control (1)
    

    Per-table column counts (top 5 widest, recorded by Python introspection of internal/store/store.go):

    tablecolumns
    control3747
    tickets847
    projects847
    opportunities847
    client447

    Tickets / projects / opportunities sit just under the 2000 cap but with no headroom; the next major-version Halo response schema could push them over.

    Workaround applied this run: hand-stripped /Control* paths from the spec (5 paths) and added a stub Control schema so residual $ref targets resolved. Regenerated successfully.

    Suspected root cause

    Confident. The typed-table-emit path walks the resource's response schema and emits one column per leaf field with no width sanity check. The store package has no JSON-only fallback for resources whose schema exceeds a safe column count.

    Suggested direction

    In the typed-table-emit path, compute the column count before writing CREATE TABLE. If the count exceeds a threshold (1500 leaves headroom under SQLite's default 2000; tunable), fall back to JSON-only schema: id TEXT PRIMARY KEY, data JSON NOT NULL, synced_at DATETIME. Emit a generation-time log line naming the resource and the fallback (e.g., store-fallback: control (3747 cols) → JSON-only).

    Preserve typed-column behavior below the threshold — the typed columns power offline FTS5 and cross-entity SQL joins; the fallback should only fire on pathological widths.

    Acceptance criteria

    • positive: feed a synthetic spec where one resource has 3000+ leaf fields; generated store.go for that resource has only 3 columns; sync against an empty DB succeeds.
    • negative: feed a normal resource (<1500 cols); typed-table emission unchanged; flat columns still emitted.

    Frequency

    subclass:wide-config-singleton — complete-functional-failure when it fires.

    Complexity

    small (column-count check + JSON-only emit branch).

    Scope boundary

    Does NOT cover per-column truncation, schema-aware projection, or per-API column allowlists. Single threshold + JSON-only fallback only.

    Dependencies

    None.

    Related issues

    None — first sighting of this specific failure mode.

    Artifacts

    ArtifactLink
    Retro documenthttps://files.catbox.moe/ico9uo.md
    Manuscripts (research + proofs)https://files.catbox.moe/rn4hf4.zip
    Generated CLI source codehttps://files.catbox.moe/4v6myv.zip

    Generated by /printing-press-retro · CLI Printing Press

    Evaluation contract

    Use 1500 columns as the safety cap for a generated typed SQLite resource table, including dependent-parent columns. Resources exceeding the cap must retain their per-resource table but fall back to a JSON-only schema containing id, data, and synced_at; do not generate typed-column indexes, FTS structures, or typed upsert code for that fallback table. Batch store generation must dispatch wide resources to the JSON-only path while preserving the existing typed behavior for ordinary narrow resources.

    Interface

    No new interfaces are introduced.

    ADDITIONAL ADVICE

    Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use git checkout <file> to restore the file to its original state. You can also reset all changes in the repository using git reset --hard command. Additionally, you can use git stash and git stash pop to temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you call submit) using git status or git diff commands. However you don't need to use git add or git commit before submitting your changes. In fact, submit only submits currently unstaged changes because it uses git diff to get the changes that need to be submitted.

    CHECKLIST

    Before submitting your solution, please go over the following checklist and make sure you've done everything:

    • If an error was reported in the issue description, I have successfully reproduced it.
    • If an error was reported in the issue description, I have confirmed that my fix resolves the error.
    • I have thought about edge cases and made sure my fix handles them as well.
    • I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.

    Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!

    (Current directory: /cli-printing-press, current file: none) bash-$

  2. MiniMax Assistant

    Let me start by exploring the repository structure to understand the

  3. MiniMax Assistant

    Now let me look at the table structure generator:

  4. MiniMax Assistant

    Now let me find where the Tables/Columns are computed for the generator:

  5. MiniMax Assistant

    Now let me look at the rest of schema_builder.go:

  6. MiniMax Assistant

    Now let me look at the schemaWithDependentParents function and where store_schema_version_test is referenced:

  7. MiniMax Assistant

    Now let me look at the upsert generation:

  8. MiniMax Assistant

    Now let me see the dispatch in UpsertBatch:

  9. MiniMax Assistant

    Now let me look at the test templates to understand the full structure:

  10. MiniMax Assistant

    Now let me find where the schema data is passed to the templates:

  11. MiniMax Assistant

    Now let me understand the schema_builder tests to know what they look like:

  12. MiniMax Assistant

    Now I have a good understanding. Let me look at the existing test patterns:

  13. MiniMax Assistant

    Now let me run a quick check to understand the test pattern (the relevant one — let me look at what isBackfillColumn does):

  14. MiniMax Assistant

    Now I have enough understanding. Let me look at the whole store.go.tmpl template to understand all the places I need to handle the JSON-only fallback:

  15. MiniMax Assistant

    Now I have a clear picture. Let me also check the existing tests:

  16. MiniMax Assistant

    Let me look at the structure of where I might want to make the changes. Let me first write a test to reproduce the error:

  17. MiniMax Assistant

    Now let me think about the design more carefully. The issue says:

    1. We need a 1500-column safety cap.
    2. Resources exceeding the cap must retain their per-resource table but fall back to a JSON-only schema containing id, data, and synced_at.
    3. Do not generate typed-column indexes, FTS structures, or typed upsert code for that fallback table.
    4. Batch store generation must dispatch wide resources to the JSON-only path while preserving the existing typed behavior for ordinary narrow resources.
    5. Emit a generation-time log line naming the resource and the fallback (e.g., store-fallback: control (3747 cols) → JSON-only).

    Looking at the architecture, the right place to make the change is in BuildSchema in schema_builder.go, where we already build the TableDef. We can introduce a "JSON-only" or "wide" flag, or alternatively alter the table's columns.

    Let me look at how the table is consumed to know if we should reset to baseTableColumns + add a flag, or replace columns:

  18. MiniMax Assistant

    Let me check the schema_version_test template for the same:

  19. MiniMax Assistant

    Now let me also see the sync.go.tmpl to understand the full context:

  20. MiniMax Assistant

    Now I understand. Let me think about the design:

    Design Plan:

    1. Add a "wide" / "JSON-only" concept to the TableDef. When a resource's typed columns exceed 1500, the schema builder should:

      • Reduce the columns to just id, data, synced_at (already in baseTableColumns)
      • Drop FTS5/Indexes
      • Set a Wide flag
    2. Update hasTypedTable to return false for wide tables (since they have only 3 columns and aren't typed).

    3. Update the template so that wide tables still get a CREATE TABLE for the JSON-only shape but no typed upsert, FTS, etc.

    4. Emit a generation-time log line store-fallback: <name> (<n> cols) → JSON-only.

    Let me check the gen log pattern:

  21. MiniMax Assistant

    Now let me think about the cleanest approach.

    Approach: Add a Wide bool (or JSONOnly bool) field to TableDef. The schema_builder.go computes the column count and:

    • If len(t.Columns) > 1500, set Wide=true, reset Columns to base columns, drop FTS5/Indexes, and emit a log line.

    Then hasTypedTable needs to also check for !t.Wide. The template will need to emit a CREATE TABLE for the JSON-only shape (so we keep per-resource table but with id/data/synced_at only).

    Let me also check what domainUpsertMethodName does:

  22. MiniMax Assistant

    Good. Now let me also check the templating around the columns to understand how to keep CREATE TABLE working for wide tables but not the upsert:

  23. MiniMax Assistant

    Now let me design the solution. I'll add a Wide field to TableDef. Then in the schema builder, if a table's typed columns exceed 1500 (counting from the base 3 + typed leaves), we mark it Wide, reset its Columns to just the base, drop FTS5/Indexes, and emit a log.

    Let me start by creating a test to reproduce:

  24. MiniMax Assistant

    Now let me first run the existing tests to see what passes:

  25. MiniMax Assistant

    Now I know the test environment works. Let me create a test script to reproduce the issue:

  26. MiniMax Assistant

    Let me update the test to use the existing fields and then proceed with the actual fix:

  27. MiniMax Assistant

    I've reproduced the bug. The control table has 3003 columns (3 base + 3000 typed), which is way above SQLite's default 2000 cap. Now let me implement the fix.

    Implementation Plan:

    1. Add Wide bool field to TableDef in schema_builder.go.
    2. Add MaxTypedColumns = 1500 constant.
    3. In BuildSchema (or a helper function called from it), detect when a resource's typed columns exceed MaxTypedColumns. When that happens:
      • Set Wide = true
      • Reset Columns to base only (id, data, synced_at)
      • Drop FTS5, Indexes, FTS5Fields, FTS5Triggers
      • Emit a stderr log line
    4. Modify hasTypedTable to return false for Wide tables.
    5. Modify the store.go.tmpl template to emit a CREATE TABLE for Wide tables (id, data, synced_at) but not typed columns/indexes/FTS.
    6. The UpsertBatch dispatch should skip the typed-table dispatch for wide tables.

    Let me also look at how the migrations slice is iterated so the CREATE TABLE for wide tables is still emitted:

  28. MiniMax Assistant

    I see — the gate is hasTypedTable which means wide tables won't even get a CREATE TABLE. We need to change the gate to handle wide tables separately. Let me think about this more carefully.

    The instruction says:

    Resources exceeding the cap must retain their per-resource table but fall back to a JSON-only schema containing id, data, and synced_at

    So we need to:

    1. Emit a CREATE TABLE for the wide table (with id, data, synced_at only)
    2. NOT emit typed-column indexes
    3. NOT emit FTS structures
    4. NOT emit typed upsert code

    The simplest approach:

    • Modify hasTypedTable to also return false for wide tables.
    • Add a new predicate hasWideTable (or hasResourceTable) that returns true for tables that need a CREATE TABLE (typed or wide).
    • Use that predicate for the CREATE TABLE gate.

    Now let me look at the existing gate, which I see is {{- if hasTypedTable .}} for the table creation. We need to:

    • Change CREATE TABLE gate to a new predicate that includes wide tables.
    • Keep typed-upsert and FTS5 gated on hasTypedTable only.

    Let me implement this step-by-step. First, let me make changes to schema_builder.go:

  29. MiniMax Assistant

    Now I have a clear picture. Let me think about the order of operations carefully:

    1. Schema builder change: After the existing column-extraction loop, check if len(table.Columns) > MaxTypedColumns. If so, reset to base only, set Wide = true, drop FTS5/Indexes, log a line.

    But the question is: do we want to keep FTS5 info if the resource has few fields? The issue says "preserve typed-column behavior below the threshold". So below the threshold, the resource gets the full typed table with FTS etc. Above the threshold, it gets the JSON-only fallback.

    Let me also look at the parent_id column that gets added later. The schemaWithDependentParents function adds a parent_id column AFTER BuildSchema. This means dependent wide tables will need to be handled too. But for now, let me check if dependent resources are even common in wide configs (the issue is about a config singleton, so likely top-level).

    Let me check:

  30. MiniMax Assistant

    OK so the parent_id column is added to dependent resources. For wide resources, this would push them over the cap. But after we set Wide = true in BuildSchema, parent_id would be added (since depSet only depends on table name, not on whether it's wide). We should also skip this for wide tables — or at least re-evaluate the threshold after parent_id is added.

    Actually, looking at the issue: "Use 1500 columns as the safety cap for a generated typed SQLite resource table, including dependent-parent columns." So the threshold should include parent_id columns.

    Let me think about the cleanest design:

    • Apply the wide check after both BuildSchema and schemaWithDependentParents, not just BuildSchema.
    • Or: Apply wide check in BuildSchema, and then schemaWithDependentParents should skip adding parent_id to wide tables (since their columns are already minimal).

    Actually, the simpler approach is to apply the wide check at the end of BuildSchema but make schemaWithDependentParents aware of Wide tables. Let me re-think.

    Actually I think the cleanest approach is to handle this AFTER all column additions are done. Let me wrap it in a helper function applyWideResourceFallback that takes the schema and applies the threshold check, and modify schemaWithDependentParents (and the call sites) to call it after all column additions.

    Let me trace the order more carefully:

    • schemaWithDependentParents calls BuildSchema(g.Spec), then adds parent_id to dependent tables.
    • This is called from renderVisionAndRootFiles and renderMCPToolFiles.

    So I'll add a helper function applyJSONOnlyFallback(schema) []TableDef and call it after all column additions.

    Let me start the implementation:

  31. MiniMax Assistant

    Oops, I accidentally duplicated baseTableColumns. Let me remove the duplicate:

  32. MiniMax Assistant

    There's a duplicate line. Let me clean it up:

  33. MiniMax Assistant

    Still duplicated. Let me see the file again:

  34. MiniMax Assistant

    Good. Now I need to add the wide-resource check. The right approach is to add a helper function applyJSONOnlyFallback that takes a []TableDef and mutates wide ones. Let me also think about where it should be called from.

    Looking at the data flow:

    1. BuildSchema is called.
    2. schemaWithDependentParents calls BuildSchema then adds parent_id columns.
    3. The result is passed to template rendering.

    The wide check should be applied AFTER all column additions (including dependent parent_id). Let me put the helper in schema_builder.go and call it from schemaWithDependentParents.

    Let me find the end of BuildSchema:

  35. MiniMax Assistant

    Now let me find a good place to add the helper function. Let me put it at the end of the file:

  36. MiniMax Assistant

    Now let me append the helper function:

  37. MiniMax Assistant

    Now I need to update schemaWithDependentParents to call applyJSONOnlyFallback after adding parent_id columns:

  38. MiniMax Assistant

    Hmm. The issue is probably the comment with parentheses — shell thinks ( is a subshell. Let me try with the file content via Python:

  39. MiniMax Assistant

    Good. Now let me also update hasTypedTable to return false for wide tables. Let me look:

  40. MiniMax Assistant

    The ! was eaten. Let me fix that:

  41. MiniMax Assistant

    Now let me update the store.go.tmpl to use the new hasResourceTable predicate for the CREATE TABLE gate:

  42. MiniMax Assistant

    Now I need to update the CREATE TABLE gate. The current structure is:

    {{- range .Tables}}
    {{- if hasTypedTable .}}
      CREATE TABLE ...
      {{- range .Indexes}} ... {{- end}}
      {{- if .FTS5}} ... {{- end}}
    {{- end}}
    {{- end}}
    

    For wide tables, I want to:

    • Emit CREATE TABLE with the JSON-only shape (which is just base columns since wide tables have base columns only).
    • Skip the Indexes loop (since wide tables have no indexes).
    • Skip the FTS5 block.

    So the new structure should be:

    {{- range .Tables}}
    {{- if hasResourceTable .}}
      CREATE TABLE ...
      {{- if hasTypedTable .}}
        {{- range .Indexes}} ... {{- end}}
        {{- if .FTS5}} ... {{- end}}
      {{- end}}
    {{- end}}
    {{- end}}
    

    Let me make this change:

  43. MiniMax Assistant

    Let me verify:

  44. MiniMax Assistant

    Good. Now let me also check the backfillColumns block, which iterates over all Columns. For wide tables, the columns would just be id/data/synced_at, and the isBackfillColumn check would filter those out, so nothing would be emitted. That's fine. Let me confirm:

  45. MiniMax Assistant

    isStoreBackfillColumn returns false for "id", "data", "synced_at" — so for wide tables, the backfill loop will emit nothing. Good.

    Now let me also check the test templates to make sure they handle wide tables correctly:

  46. MiniMax Assistant

    The test TestMigrate_AddsColumnsOnUpgrade only fires for tables with backfill columns (typed columns beyond id/data/synced_at). Wide tables have no backfill columns, so no test will be generated. Good.

    Now let me look at the upsert batch test:

  47. MiniMax Assistant

    Good. The upsert batch test gates on hasTypedTable, so wide tables won't have a typed-populate test. That's the correct behavior.

    Now let me also look at the SearchX functions:

  48. MiniMax Assistant

    This gates on FTS5. Since wide tables have FTS5 = false, no SearchX function will be emitted for them. Good.

    Now let me run the test to see if my fix works:

  49. MiniMax Assistant

    Good, the build works. Now let me run my repro test:

  50. MiniMax Assistant

    My test only uses BuildSchema directly, not schemaWithDependentParents. Let me update the test to also check the wide-resource handling:

  51. MiniMax Assistant

    The fix works at the schema_builder level. Now let me verify the full pipeline by running a test that renders templates:

  52. MiniMax Assistant

    Let me look at minimalSpec to see how to set up a test:

  53. MiniMax Assistant

    Let me write an end-to-end integration test:

  54. MiniMax Assistant

    My extraction logic is off — it's grabbing too much. Let me fix it:

  55. MiniMax Assistant

    The end-to-end test passes. The CREATE TABLE for control only has 3 columns (id, data, synced_at) and there's no typed UpsertControl function. Let me also verify that the generated code compiles and can sync:

  56. MiniMax Assistant

    Let me move the test inside the package:

  57. MiniMax Assistant

    Now let me also test that the generated code compiles. Let me extract and verify:

  58. MiniMax Assistant

    The generated code compiles. Now let me also test that the generated test file compiles and passes:

  59. MiniMax Assistant

    Excellent! The generated store tests pass. Now let me also run the negative test - a normal resource (<1500 cols) should still produce typed output:

  60. MiniMax Assistant

    The negative test passes. The normal resource retains its typed columns. Now let me also clean up these temporary tests since the instructions say I should NOT add tests to the repository. Let me re-read:

    However UNDER NO CIRCUMSTANCES should you modify existing tests or add new tests to the repository. This will lead to an error in the system that evaluates your performance. Instead, you can just create a temporary script, use it to test changes and remove it before submitting.

    Let me clean up the temporary test files. But first, let me also verify that I'm not breaking any existing tests by running the broader test suite:

  61. MiniMax Assistant

    Good, all temporary tests removed. Now let me run the existing tests to make sure I didn't break anything:

  62. MiniMax Assistant

    I see two failures. Let me look at these:

    1. TestGeneratedSyncHonorsHTMLExtraction - this could be related to my change. Let me investigate.
    2. TestGenerateWithNoAuth - this is a known Go vulnerability check failing (GO-2026-5856 in crypto/tls), not related to my change.

    Let me investigate the first failure:

  63. MiniMax Assistant

    Let me check this test to understand what's failing:

  64. MiniMax Assistant

    Let me check if this test was passing before my change:

  65. MiniMax Assistant

    It passes without my change. So my change broke something. Let me restore my changes:

  66. MiniMax Assistant

    OK so my change broke TestGeneratedSyncHonorsHTMLExtraction. Let me investigate what's going wrong. The test sets up an HTML extraction test. Let me look at what's happening:

  67. MiniMax Assistant

    Let me look at the failing test:

  68. MiniMax Assistant

    Let me actually compile and run the failing test to see what's happening:

  69. MiniMax Assistant

    Let me check what's in the generated internal/cli/sync.go to see how the dependent messages resource is handled:

  70. MiniMax Assistant

    Hmm interesting, it passes when run alone. Let me see if there's some race issue:

  71. MiniMax Assistant

    Wait, the same code passes when I use my simplified test but fails when using the original test. Let me look at the difference more carefully. Maybe my check_sync_temp_test doesn't reproduce the failure because of timing. Let me try to reproduce it without my temp tests:

  72. MiniMax Assistant

    Let me look at the generated test to see what messages table looks like:

  73. MiniMax Assistant

    Let me search for the messages-related code in the generated store:

  74. MiniMax Assistant

    I see the issue. The messages table has columns [id, data, synced_at, parent_id] — but the test expects channels_id (the sub-resource FK by parent table name).

    Let me look at the original test expectations:

  75. MiniMax Assistant

    Now let me check the sync.go.tmpl and see how it uses dependentResourceDef to call the typed table:

  76. MiniMax Assistant

    The original table has parent_id column. The test expects channels_id to be in the JSON output. Let me see where the test gets the JSON:

    The test calls db.List("messages", 10). Let me check what List does:

  77. MiniMax Assistant

    List returns from the resources (generic) table. So the test expectation channels_id is on the JSON data in the data column. Now let me look at the sync dependent resource code that populates the messages JSON:

  78. MiniMax Assistant

    So when syncDependentResource runs, it injects obj[parentFKKey] = parentIDJSON (where parentFKKey is channels_id). So the JSON stored should have channels_id.

    But the test is failing. Let me look at the test more carefully:

  79. MiniMax Assistant

    So the test calls syncDependentResource with dependentResourceDef{Name: "messages", ParentTable: "channels", ...}. Looking at the failure, synced count = 0, want 1. So nothing is synced.

    Let me run my dump test in verbose mode and look for "syncDependentResource" code:

  80. MiniMax Assistant

    OK so messages has the typed table. Now let me look at the sync.go (not store.go) in the generated output. Let me re-dump the sync.go instead:

  81. MiniMax Assistant

    Let me look at syncDependentResource's content area to find how parentFKKey is set:

  82. MiniMax Assistant

    OK so parentFKKey = "channels_id". Now let me look at how the items get processed and check if my changes broke something:

  83. MiniMax Assistant

    OK so the dependent resource logic is unchanged. Now let me think about why my change broke this. Let me compare the messages table before and after my change. The issue is that messages is a dependent resource, and the table has parent_id column. With my change, it should still have this column.

    Let me also see what the parent table schema looks like:

  84. MiniMax Assistant

    OK so messages is a dependent resource of channels. Let me look at the test setup:

    if err := db.Upsert("channels", "bad", []byte(`{"id":"bad"}`)); err != nil {
        t.Fatalf("insert bad parent: %v", err)
    }
    if err := db.Upsert("channels", "good", []byte(`{"id":"good"}`)); err != nil {
        t.Fatalf("insert good parent: %v", err)
    }
    

    It inserts parents into channels table, then calls syncDependentResource with ParentTable: "channels". The test wants the messages to be synced.

    Let me look at dependentParentRows:

  85. MiniMax Assistant

    OK so it calls db.ListIDs(parentTable). Let me find that:

  86. MiniMax Assistant

    OK so it looks for a typed table first. The test inserted into the generic resources table only via db.Upsert("channels", ...). So ListIDs should fall through to the resources table.

    Let me see what db.Upsert does:

  87. MiniMax Assistant

    OK so db.Upsert only inserts into the generic resources table. So ListIDs("channels") should fall back to the resources table.

    Let me check if my change has broken the schema_builder such that the channels table is no longer being created. Actually let me think... channels has no typed columns (because there's no types: channels definition). So BuildSchema gives channels a base table (id/data/synced_at, 3 columns). len(columns) > 1500 is false, so it stays typed... wait, but then hasTypedTable is len(t.Columns) > 3 which is 3 > 3 = false. So channels doesn't have a typed table.

    So ListIDs should hit the fallback (generic resources table). OK so this should still work.

    Let me actually run my full dump test to see the dump more clearly:

  88. MiniMax Assistant

    Let me actually compare the original store.go before my changes and after my changes:

  89. MiniMax Assistant

    OK, the messages resource is the same. Let me run the failing test in isolation and add debug output:

  90. MiniMax Assistant

    OK so messages has 3 columns in BuildSchema (no types declared). Let me find the test that runs to see what changed:

  91. MiniMax Assistant

    Interesting! When I run the same test in isolation, it passes! But when I run TestGeneratedSyncHonorsHTMLExtraction directly, it fails. Let me try running both at once:

  92. MiniMax Assistant

    So when TestGeneratedSyncHonorsHTMLExtraction runs first, it fails, but when my reproduction runs first, it passes. The difference is that the test uses runGoCommandRequired(t, outputDir, "test", ...) - let me see what that does and check what's different:

  93. MiniMax Assistant

    Hmm. So the test does go test -mod=mod -run ... ./internal/cli with a separate GOCACHE. Let me look more carefully:

  94. MiniMax Assistant

    Let me see if my reproduction uses the same approach - yes, I'm running go test -v ... but without -mod=mod. Let me try with -mod=mod:

    Actually wait, the difference is that my repro uses go test without mod tidy. The test does go mod tidy first. Let me try that:

  95. MiniMax Assistant

    I reproduced the failure! Now the question is why. Let me also test by reverting my change:

  96. MiniMax Assistant

    Confirmed: my change is causing the regression. Let me restore:

  97. MiniMax Assistant

    Let me investigate. With my change, the test reports synced count = 0. That means syncDependentResource returned a result with Count=0. So either:

    1. dependentParentRows returned 0 rows (can't iterate parents).
    2. The sync was done but the items couldn't be extracted.

    Let me add more logging to my repro:

  98. MiniMax Assistant

    Now I see something interesting! In my output, I see this:

    `CREATE TABLE IF NOT EXISTS "channels" (
        "id" TEXT PRIMARY KEY,
        "data" JSON NOT NULL,
        "synced_at" DATETIME DEFAULT CURRENT_TIMESTAMP
    )`,
    `CREATE TABLE IF NOT EXISTS "json_pages" (
        "id" TEXT PRIMARY KEY,
        "data" JSON NOT NULL,
        "synced_at" DATETIME DEFAULT CURRENT_TIMESTAMP
    )`,
    `CREATE TABLE IF NOT EXISTS "messages" (
        "id" TEXT PRIMARY KEY,
        "data" JSON NOT NULL,
        "synced_at" DATETIME DEFAULT CURRENT_TIMESTAMP,
        "parent_id" TEXT
    )`,
    

    This looks like an array literal where each element is a SQL statement. So the channels table is being created with just id/data/synced_at, and messages is created with id/data/synced_at/parent_id.

    Wait — but this is good. Let me see what's actually going wrong. Let me dump just the channels table CREATE statement (without the trailing ,):

  99. MiniMax Assistant

    Got it! The issue is ListIDs is returning 0 IDs, even though the resources table has rows. Let me look at ListIDs again:

    func (s *Store) ListIDs(resourceType string) ([]string, error) {
        var table string
        err := s.db.QueryRow(
            `SELECT name FROM sqlite_master WHERE type='table' AND name=?`,
            resourceType,
        ).Scan(&table)
        var rows *sql.Rows
        if err == nil && table != "" {
            rows, err = s.db.Query(fmt.Sprintf(`SELECT id FROM "%s"`, strings.ReplaceAll(table, `"`, `""`)))
        }
        if err != nil || table == "" {
            // Fall back to generic resources table
            rows, err = s.db.Query("SELECT id FROM resources WHERE resource_type = ?", resourceType)
            if err != nil {
                return nil, err
            }
        }
        defer rows.Close()
    

    The channels table exists (per the migration), so it tries to query SELECT id FROM "channels" directly. But db.Upsert only writes to the resources table, NOT to the channels table. So the channels table is empty, and ListIDs returns 0.

    This is the regression: Before my change, the migration didn't create a channels table (because it had no typed projection — len(columns) > 3 was false), so ListIDs would fall through to the generic resources table. After my change, the channels table IS created (because hasResourceTable returns true), so ListIDs queries the empty channels table.

    This is a bug in the existing test setup! The test relies on the channels table NOT being created so that ListIDs falls back to resources. With my change, the channels table is created.

    The issue says: "Resources exceeding the cap must retain their per-resource table". So we need to keep the per-resource table. But we need to make sure db.Upsert (which writes only to the generic resources table) keeps working with parent iteration.

    The fix is to update ListIDs (or similar) to handle this case — but the issue is more fundamental. Let me re-read the issue:

    Resources exceeding the cap must retain their per-resource table but fall back to a JSON-only schema containing id, data, and synced_at; do not generate typed-column indexes, FTS structures, or typed upsert code for that fallback table.

    So the fallback is ONLY for resources exceeding the cap. But my change is now creating per-resource tables for ALL resources, including ones that previously had no per-resource table. This breaks tests that relied on ListIDs falling back.

    The original gate hasTypedTable was stricter (only tables with >3 columns AND not sync_state AND not "UpsertBatch"). Tables with exactly 3 columns (like channels) didn't get a per-resource table at all.

    The fix should be more surgical: Only wide resources should get a per-resource table with id/data/synced_at. The current behavior for narrow resources (no table) should be preserved.

    Let me re-think. The key insight is:

    • For narrow resources with no typed projection (channels), no CREATE TABLE is needed.
    • For narrow resources with typed projection (e.g., pages with 100 fields), CREATE TABLE is needed.
    • For wide resources (3000+ fields), CREATE TABLE with id/data/synced_at only is needed.

    So my hasResourceTable predicate needs to be:

    • Returns true if hasTypedTable(t) is true (typed table exists).
    • Returns true if t.Wide is true (wide fallback table needed).
    • Returns false otherwise.

    Let me fix this:

  100. MiniMax Assistant

    Wait, but I have a reference to hasTypedTable which doesn't exist as a function. Let me change this approach — use t.Wide || (len(t.Columns) > 3 && ...) directly:

  101. MiniMax Assistant

    Let me verify:

  102. MiniMax Assistant

    Now let me run the test again:

  103. MiniMax Assistant

    The dependent resource test passes now. Let me run the original failing test:

  104. MiniMax Assistant

    Let me also remove my repro_temp_test.go and run a broader set of tests:

  105. MiniMax Assistant

    Only TestGenerateWithNoAuth is failing now, and it's a pre-existing govulncheck failure unrelated to my change. Let me verify it's pre-existing:

  106. MiniMax Assistant

    Confirmed pre-existing. Let me restore my changes:

  107. MiniMax Assistant

    Now let me run all tests except the govulncheck one to verify my changes don't break anything else:

  108. MiniMax Assistant

    All other tests pass. Let me also test more thoroughly:

  109. MiniMax Assistant

    The test regex doesn't match the right pattern. Let me try again:

  110. MiniMax Assistant

    All non-G tests pass. Now let me re-run my earlier verification tests:

  111. MiniMax Assistant

    Wide resource works. The generated store tests pass. Now let me also verify the negative case (resource below 1500 cols still works):

  112. MiniMax Assistant

    Both positive and negative cases work. Now let me also verify edge cases:

    1. Resource with exactly 1500 columns (boundary).
    2. Resource with 1501 columns (just over).
  113. MiniMax Assistant

    Interesting. With 1500 fields, we get 1503 columns (1500 + id + data + synced_at). So the column count is len(fields) + 3. Let me re-check the threshold:

    The issue says: "Use 1500 columns as the safety cap for a generated typed SQLite resource table, including dependent-parent columns."

    So the cap is 1500 columns total. With 1500 fields, we get 1503 columns total. So 1500 fields is OVER the cap (1503 > 1500). The correct test is:

    • 1497 fields = 1500 columns = exactly at cap (keep typed).
    • 1498 fields = 1501 columns = just over (JSON-only fallback).

    But really the question is what the issue means by "1500 columns":

    • Strictly: 1500 typed columns + id + data + synced_at = 1503 total.
    • Or: Total = 1500 (so 1497 fields).

    The original issue says: "This produced a 3,747-column CREATE TABLE that exceeded SQLite's default SQLITE_MAX_COLUMN=2000." and "Use 1500 columns as the safety cap for a generated typed SQLite resource table, including dependent-parent columns."

    The phrasing "1500 columns as the safety cap" suggests the cap is on the total number of columns (including id, data, synced_at). And "including dependent-parent columns" suggests we should also include the dependent parent column.

    So the cap should be len(t.Columns) <= 1500. Let me check: 1497 fields = 1500 columns total → keep typed. 1498 fields = 1501 columns total → fall back.

    Let me check my logic again. applyJSONOnlyFallback triggers when len(table.Columns) > MaxTypedColumns. With 1503 columns (1500 fields + 3 base), that's 1503 > 1500, so it falls back. Good.

    For 1500 fields: 1500 + 3 = 1503 columns. With my current threshold > 1500, this triggers fallback.

    Actually wait, let me re-read: "Use 1500 columns as the safety cap for a generated typed SQLite resource table, including dependent-parent columns."

    I think the cap is on total columns. So len(columns) > 1500 triggers fallback. With 1500 fields (1503 columns), this falls back. With 1497 fields (1500 columns), this keeps typed. That's what my current code does.

    The test I wrote was wrong — I was checking UpsertItems exists when fields count is 1500, but that's actually over the cap (1503 > 1500). Let me fix the test:

  114. MiniMax Assistant

    The boundary works correctly:

    • 1497 fields = 1500 cols total → keep typed
    • 1498 fields = 1501 cols total → fall back to JSON-only
    • 1499 fields = 1502 cols total → fall back to JSON-only

    Now let me also check the edge case where dependent parent_id adds a column. This should still trigger fallback if total > 1500:

  115. MiniMax Assistant

    Let me look at the right way to set up:

  116. MiniMax Assistant

    OK so the profile field is *profiler.APIProfile. Let me simplify my test - just verify the schemaWithDependentParents logic works:

  117. MiniMax Assistant

    Hmm. The children table only has 4 columns (id, parent_id, data, synced_at) which means the children resource didn't get any typed columns. Let me think about this...

    The issue is that children is a sub-resource, and sub-resources don't get typed columns. The buildSubResourceTable only adds id, parent_id, data, synced_at. So we don't get 1498 fields in children.

    Let me check what's happening:

  118. MiniMax Assistant

    So the children sub-resource (list) only gets base columns from buildSubResourceTable, which is 4 columns (id, parent_id, data, synced_at). The 1498 fields I defined don't get added because sub-resources don't add typed columns in the schema_builder. So this case won't trigger the wide fallback.

    OK so dependent sub-resources don't have typed columns. My fix is sufficient for the existing data flow. Let me think about whether there are other edge cases.

    Wait - what about top-level resources with many fields that ARE also dependents? Let me check the docs:

    Looking at schemaWithDependentParents:

    for _, dep := range g.profile.DependentSyncResources {
        depSet[dep.Name] = true
    }
    

    It uses dep.Name which is the resource name. So if a top-level resource is also a dependent, parent_id gets added.

    But actually, in practice, dependents are typically sub-resources that aren't top-level. The case where a top-level resource is also a dependent is unusual.

    Let me think about another edge case: what if a resource has gravity < 2 (no typed columns) but is also flagged as dependent? It gets parent_id added, going from 3 to 4 columns. That's still well under 1500. Not an issue.

    Now let me clean up my temporary tests:

  119. MiniMax Assistant

    Good, all temp tests removed. Now let me run the broader tests to make sure nothing else is broken:

  120. MiniMax Assistant

    All non-G tests pass. Now let me also run other package tests:

  121. MiniMax Assistant

    Two failures:

    1. pipeline - test failed
    2. pressauth - test failed

    Let me check if these are pre-existing:

  122. MiniMax Assistant

    This is unrelated to my change - it's a $HOME issue. Let me verify:

  123. MiniMax Assistant

    The HOME environment variable is empty. Let me set it:

  124. MiniMax Assistant

    Yeah, the pressauth tests just need HOME set. Not related to my change. Now let me check pipeline:

  125. MiniMax Assistant

    Let me see which test fails:

  126. MiniMax Assistant

    This test uses chmod 0 which fails when running as root. It's unrelated to my change. Let me confirm:

  127. MiniMax Assistant

    Confirmed: pre-existing failure (because we're running as root, chmod 0 doesn't restrict access). Let me restore:

  128. MiniMax Assistant

    Good. Now let me also do a final check to make sure all the relevant generator tests pass:

  129. MiniMax Assistant

    Let me look at the diff of my changes to verify they are all correct:

  130. MiniMax Assistant

    Wait, my test added chromedp/cdproto as a direct dependency! That was from running the wide resource test which used go mod tidy. Let me revert that:

  131. MiniMax Assistant

    Good. Now let me look at the actual diff for the schema_builder.go:

  132. MiniMax Assistant

    The diffs look clean and minimal. Let me also verify the templates still parse properly with a check on the end-to-end pipeline once more:

  133. MiniMax Assistant

    Builds clean. Let me also write one final verification to make sure my fix produces the expected output. Let me create a test that captures the exact assertions from the issue:

  134. MiniMax Assistant

    Both acceptance criteria pass:

    • Positive: 3000+ fields → control table has 3 columns, generated store tests pass.
    • Negative: 100 fields (under cap) → typed Upsert emitted, all 100 columns in CREATE TABLE.

    Now let me also clean up and remove my temp tests:

  135. MiniMax Assistant

    All temp tests removed. Now let me check if there are any other places that might be affected. Let me look at the domainUpsertMethodName calls to ensure my changes don't break anything:

  136. MiniMax Assistant

    domainUpsertMethodName returns "UpsertBatch" for reserved table names. This is a separate code path. Let me check:

  137. MiniMax Assistant

    Looking at domainUpsertMethodName it just returns Upsert + Pascal name. The check domainUpsertMethodName(t.Name) != "UpsertBatch" only matters if the table name happens to Pascalize to "Batch" (giving "UpsertBatch"). That's basically never (you'd need a resource named "batch"). So this check is essentially always true and doesn't matter.

    Now let me also verify that all generated code with wide resources can run a sync. Let me write one final test that does a full integration test:

  138. MiniMax Assistant

    Good. Let me also do a comprehensive final test by running the generator tests except the govulncheck ones (which are unrelated to my change):

  139. MiniMax Assistant

    All non-govulncheck tests pass. The only failure is the pre-existing govulncheck issue.

    Now let me also check that my fix handles the case where a resource's columns go from 4 to 1501 (e.g., a 1497-field resource gets parent_id appended):

  140. MiniMax Assistant

    Excellent! The pivot case works:

    • 1497 fields = 1500 cols total
    • After parent_id added: 1501 cols (over cap)
    • Falls back to JSON-only with 3 columns

    This shows the cap is correctly evaluated AFTER all column injection (BuildSchema + parent_id).

    Let me also check that without parent_id, 1497 fields stays typed (the column count is 1500 = at cap, not over):

  141. MiniMax Assistant

    The at-cap case works:

    • 1497 fields = 1500 cols total (exactly at cap)
    • Stays typed (no JSON-only fallback)
    • The threshold is > MaxTypedColumns, so 1500 is OK, 1501 is not.

    Let me clean up:

  142. MiniMax Assistant

    Now let me also do one more comprehensive run to make sure my final state is good:

  143. MiniMax Assistant

    Let me also verify the change works without the Wide field having any unintended effects. Let me run the whole package's tests one more time:

  144. MiniMax Assistant

    Only the pre-existing govulncheck failure. My change is clean. Let me also look at the final state of my changes:

  145. MiniMax Assistant

    Good. Let me show the final state:

  146. MiniMax Assistant

    The changes look clean and minimal. Let me also double-check that the test files I have left are the original ones, not my temp tests:

  147. MiniMax Assistant

    That's a build artifact from running tests. Not something I created. Let me check git history:

  148. MiniMax Assistant

    That .local directory is not in gitignore but is also untracked. It's not something I introduced — let me clean it up anyway:

  149. MiniMax Assistant

    Now my changes are clean. Let me also do a final verification with all my tests passing:

  150. MiniMax Assistant

    All tests pass. Let me also quickly verify my key files still compile:

  151. MiniMax Assistant

    Let me also do a final sanity check on the generator output:

  152. MiniMax Assistant

    The test says cols=4. Let me investigate - maybe my code counted wrong. Wait, I see - my test is counting ALL non-empty lines including the backtick, "comma", etc. Let me fix the test:

  153. MiniMax Assistant

    Let me write a more robust test:

  154. MiniMax Assistant

    The regex didn't match. Let me look at the actual format:

  155. MiniMax Assistant

    The fix works. The 3747-column control resource is reduced to a 3-column JSON-only table. The store-fallback: control (3747 cols) → JSON-only log line is emitted.

    Let me clean up:

  156. MiniMax Assistant

    All temp tests cleaned up. Let me also clean up the .local directory:

  157. MiniMax Assistant

    The go.mod is dirty from running my test (mod tidy). Let me revert:

  158. MiniMax Assistant

    Final state. Let me also re-run the build:

  159. MiniMax Assistant

    Let me also check if any tests that I broke earlier need re-checking:

  160. MiniMax Assistant

    All tests pass. My changes are complete.

    Now let me do a final review of my checklist:

    1. If an error was reported in the issue description, I have successfully reproduced it.

      • YES. I reproduced the issue: a control resource with 3000 fields produces 3003 columns in CREATE TABLE, which would exceed SQLite's 2000 cap.
    2. If an error was reported in the issue description, I have confirmed that my fix resolves the error.

      • YES. After my fix, the 3000-field control resource produces a 3-column JSON-only table. The generated store code compiles and the generated store tests pass.
    3. I have thought about edge cases and made sure my fix handles them as well.

      • YES. Edge cases I verified:
        • Boundary: 1497 fields (1500 cols total) keeps typed table.
        • Just over: 1498 fields (1501 cols total) falls back to JSON-only.
        • Wide with parent_id: A 1497-field resource with parent_id (1501 cols total) also falls back.
        • Negative case: 100 fields (103 cols total) keeps typed projection with full Upsert and FTS5.
        • Dependent resource: messages table (with 4 columns from base + parent_id) is unchanged.
        • sync_state: never demoted to wide (special-cased).
    4. I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass.

      • YES. I ran:
        • ./internal/generator/ test suite (excluding pre-existing govulncheck failure)
        • ./internal/... test suite (excluding pre-existing HOME-related and chmod-related failures unrelated to my change)

    Let me submit:

  161. MiniMax Assistant

    Wait, go.mod is still dirty. Let me check:

  162. MiniMax Assistant

    Wait, that go build ./... triggered a re-modification of go.mod. Let me check the .local dir is gone:

  163. MiniMax Assistant

    The build modifies go.mod. Let me investigate what's happening. Actually, go build shouldn't modify go.mod. Let me check:

  164. MiniMax Assistant

    The build is promoting chromedp/cdproto to a direct dependency. This is happening because the build is running with the chromedp-related code somewhere. Let me check if my code uses chromedp:

  165. MiniMax Assistant

    So the generated code uses chromedp. When the test builds the wide-resource spec (which doesn't use auth_browser), but the generator package itself imports generated code with chromedp... let me check:

  166. MiniMax Assistant

    The generator package itself doesn't import chromedp. But maybe go build is doing a full resolution that includes indirect deps. Let me try a more targeted build:

  167. MiniMax Assistant

    So even go build ./internal/generator/ triggers the change. This must be because the test files reference it... let me check:

  168. MiniMax Assistant

    So the template uses chromedp. The test build is somehow resolving this. Let me revert go.mod and confirm the test still passes:

  169. MiniMax Assistant

    Tests pass without modifying go.mod. The chromedp change must be from a previous test run that did go mod tidy on a generated project. Let me check git status:

  170. MiniMax Assistant

    Final state is clean. Let me submit:

  171. MiniMax Assistant

    My changes are:

    1. internal/generator/schema_builder.go:

      • Added Wide bool field to TableDef.
      • Added MaxTypedColumns = 1500 constant.
      • Added applyJSONOnlyFallback helper function that demotes wide resources to JSON-only tables (id/data/synced_at only) and emits a stderr log line.
    2. internal/generator/generator.go:

      • Added hasResourceTable template predicate (broader than hasTypedTable).
      • Modified hasTypedTable to return false for wide tables.
      • Updated schemaWithDependentParents to apply the JSON-only fallback after all column additions.
    3. internal/generator/templates/store.go.tmpl:

      • Changed CREATE TABLE gate from hasTypedTable to hasResourceTable.
      • Moved the Indexes and FTS5 blocks inside a hasTypedTable gate so wide tables don't emit them.

    Let me run my final check:

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.