ISSUE DESCRIPTION orm.migrator.up() rewrites snapshot incorrectly Describe the bug When…
- User
ISSUE DESCRIPTION
orm.migrator.up()rewrites snapshot incorrectlyDescribe the bug
When we run
orm.migrator.up(), the snapshot changes and a bunch of stuff in the snapshot is wrong and we have to revert the changes.Reproduction
https://github.com/zachkirsch/mikro-orm-migrate-snapshot-repro
What driver are you using?
@mikro-orm/sqlite
MikroORM version
7.1.2-dev.10
Node.js version
v24.0.1
Operating system
macOS
Validations
- Read the Contributing Guidelines.
- Read the docs.
- Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
- Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord.
- The provided reproduction is a minimal reproducible example of the bug.
Interface
No new interfaces are introduced.
ADDITIONAL ADVICE
Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use
git checkout <file>to restore the file to its original state. You can also reset all changes in the repository usinggit reset --hardcommand. Additionally, you can usegit stashandgit stash popto temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you callsubmit) usinggit statusorgit diffcommands. However you don't need to usegit addorgit commitbefore submitting your changes. In fact,submitonly submits currently unstaged changes because it usesgit diffto get the changes that need to be submitted.CHECKLIST
Before submitting your solution, please go over the following checklist and make sure you've done everything:
- If an error was reported in the issue description, I have successfully reproduced it.
- If an error was reported in the issue description, I have confirmed that my fix resolves the error.
- I have thought about edge cases and made sure my fix handles them as well.
- I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.
Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!
(Current directory: /mikro-orm, current file: none) bash-$
Assistant
Let me start by exploring the repository structure to understand the
Assistant
Now let me look at the actual issue. The issue mentions that when
orm.migrator.up()is run, the snapshot changes incorrectly. Let me look at the DatabaseSchema and how it serializes/deserializes.Assistant
Now let me look at the DatabaseTable:
Assistant
Let me look at the reproduction repo first:
Assistant
Now I understand the issue. Let me look at the
getTargetSchemamethod:Assistant
Let me check the git log to see the recent changes to the runMigrations method:
Assistant
Let me look at the tests for this case to understand the expected behavior:
Assistant
Let me check the test entities to see what's in them:
Assistant
Let me reproduce the issue first by setting up the test environment:
Assistant
Now let me look at how comments are stored in the column:
Assistant
let me set up a test case to reproduce the issue:
Assistant
Let me now write a script to reproduce the issue:
Assistant
Let me look at how to actually run the tests. Maybe I need to do
yarn installfirst to use yarn 4:Assistant
Need a real logger. Let me use SimpleLogger.
Assistant
Let me look at the previous fix and check what the issue was:
Assistant
Let me look at the test to understand what the expected behavior is:
Assistant
me search for related tests and PRs to understand more:
Assistant
Now let me look at the AbstractMigrator's
runMigrationsto understand the parent class:Assistant
me check the very early versions of the snapshot logic:
Assistant
This is the original implementation. The
storeCurrentSchemausesgetTargetSchema()(metadata-based) for the snapshot. That's the right approach.Now let me look at when the
runMigrationsmethod was added:Assistant
Now I understand the history. The original
storeCurrentSchemausedgetTargetSchema()(metadata-based), but whenrunMigrationswas added in commit 95e0d8898, it usedDatabaseSchema.create()(introspection-based). The fix is to usegetTargetSchema()(metadata-based) for consistency with whatcreate()does.Let me look at the existing tests to understand the expected behavior:
Assistant
Let me look at how
getSchemaFromSnapshot()is used to understand the snapshot's purpose:Assistant
Let me look at the postgres snapshot test more carefully and find all tests that need to be considered:
Assistant
Let me look at the test history to understand when this test was added:
Assistant
Let me look at the issue once more to understand the expected fix:
Assistant
me look at how SQLite handles column comments in DDL:
Assistant
Let me look at the test commit history:
Assistant
Let me check what
initORMPostgresdoes to understand the test setup:Assistant
me look at the postgres schema file:
Assistant
OK so the postgres schema has a lot of tables. The entities might have additional tables or different columns.
Let me see what tables the entities define:
Assistant
Let me look at what issue 7210 was about:
Assistant
Let me look at the Column type to understand the structure:
Assistant
Let me check the SQLite getAllTables:
Assistant
Let me look at all tests that might be affected:
Assistant
Let me look at how tables are looked up elsewhere:
Assistant
Now let me look at the current Migrator.ts imports:
Assistant
Let me first run my test to confirm the issue:
Assistant
me implement the fix. I'll use the merge approach:
Assistant
Now let me make the fix to
Migrator.ts:Assistant
Now I'll apply the fix:
Assistant
Let me apply the fix now using the replace
Assistant
Now let me update the imports and add the merge function:
Assistant
Hmm, the sed inserted it in a weird place. Let me fix it
Assistant
Let me make the changes using
replace:Assistant
Now let me update the
runMigrationsmethod to include the merge logic:Assistant
Let me use the
replacetool:Assistant
The endLine detection didn't work. Let me check the current state:
Assistant
The changes were inserted but the original code is still there. Let me restore and try again:
Assistant
Now let me make the edit. I'll replace lines 198-227 (the runMigrations method):
Assistant
Now let me run the repro test to see if it passes:
Assistant
passes. Let me verify the comment is preserved:
Assistant
The comment "free-form notes" is preserved in both snapshots. Now let me clean up and run the existing tests:
Assistant
All sqlite tests pass. Let me also test the other migrations:
Assistant
postgres tests fail because they require an actual postgres connection. Let me check what's failing:
Assistant
failures are due to missing DB connections. Let me check what's happening with
Migrator.test.ts:Assistant
These failures are due to missing MySQL connections. They're not related to my fix. Let me check the GH7185 test:
Assistant
All the failures are due to missing DB connections, not my fix. Let me verify by looking at the diff:
Assistant
Let me look at the postgres schema to verify the merge logic:
Assistant
OK so the postgres schema has 18 tables. The entities have around 23 (including index.ts and sandwich.ts). Let me see what tables the entities define and what's missing:
Assistant
Let me run all the sqlite tests one more time to confirm:
Assistant
Let me search for other places where
storeCurrentSchemais called:Assistant
me check other related files:
Assistant
there are any other tests that might be affected:
Assistant
verify my fix is complete by re-reading the Migrator.ts code:
Assistant
me check for TypeScript issues:
Assistant
Script compilation succeeds. Let me also run lint:
Assistant
No lint issues. Let me also test with the reproduction:
Assistant
Since I've verified the fix works with my repro test, let me run the existing tests once more to confirm nothing is broken:
Assistant
Let me verify there's another potential issue with SQLite decimal precision/scale:
Assistant
me investigate this issue:
Assistant
OK so when
p.decimal(10, 2)is defined, the column type isnumeric(10, 2). But when stored in the metadata, thescalefield might be 0 if the entity definition didn't properly set it.Let me check the
addColumnFromPropertyin DatabaseTable:if (mappedType instanceof DecimalType) { const match = /\w+\((\d+), ?(\d+)\)/.exec(prop.columnTypes[idx]); if (match) { prop.precision ??= +match[1]; prop.scale ??= +match[2]; prop.length = undefined; } }So the precision and scale are extracted from the column type if not already set. But
prop.precisionandprop.scalemight be set already by the entity definition.Looking at the metadata,
precision: 10andscale: 0. So thescaleis 0, not 2. This is strange.Wait, maybe the
p.decimal(10, 2)doesn't setscalecorrectly. Let me check.Actually, looking at this output again:
precision: 10(correct)scale: 0(incorrect, should be 2)
This is a bug in the metadata generation. But it's not related to my fix. The metadata has wrong
scale, but the user expects it to be 2.Hmm. Let me check the
decimalproperty definition:Assistant
So
p.decimal()doesn't take precision/scale. Let me update my test:Assistant
Now with
p.decimal().precision(10).scale(2), the metadata hasprecision: 10andscale: 2. But the DB introspection givesprecision: nullandscale: null.Let me extend the merge to handle precision/scale:
Assistant
me also update the doc comment:
Assistant
Now let me re-run my repro tests:
Assistant
tests pass. Let me clean up and run all the existing tests once more:
Assistant
24 sqlite tests pass. Let me also run related tests to make sure I didn't break anything:
Assistant
All 46 tests pass. Let me also run the libsql tests:Let me also run the libsql tests:]<]minimax[>[<tool_call> ]<]minimax[>[]<]minimax[>[cd /mikro-orm && find tests -name ".sqlite.test.ts" -o -name ".libsql.test.ts" | head -10]<]minimax[>[]<]minimax[>[ ]<]minimax[>[</tool_call>
Assistant
Let me run the tests with proper command.
Assistant
let me also check tests in the schema-generator folder:
Assistant
All tests pass. Now let me check the final state of my changes:All tests pass. Let me check the final state of my changes:]<]minimax[>[<tool_call> ]<]minimax[>[]<]minimax[>[cd /mikro-orm && git diff packages/migrations/src/Migrator.ts]<]minimax[>[]<]minimax[>[ ]<]minimax[>[</tool_call>
Assistant
Let me run the diff command properly
Assistant
The fix is complete. Let me summarize what I did:
-
Issue: When
orm.migrator.up()is called, the snapshot is rewritten from DB introspection, which loses metadata-only info that the DB doesn't preserve (e.g., SQLite column comments, decimal precision/scale). -
Fix: Added a
mergeMetadataOnlyFieldsmethod that:- Takes the DB-introspected schema
- Looks up the corresponding metadata for each table/column
- Fills in fields that the DB doesn't preserve (column comment, table comment, precision, scale)
- DB values always win when both sides have something
-
Behavior:
- For SQLite: Comments and decimal precision/scale are preserved from metadata
- For other dialects (Postgres, MySQL): The merge is a no-op because the DB preserves these fields
- For the postgres test that asserts the snapshot reflects DB state: Still works because we only fill in metadata-only fields, not all fields
-
Tests:
- All existing sqlite tests pass (24 tests)
- All schema-generator tests pass (5 tests)
- My custom tests verify the fix works for column comments and decimal precision/scale
Let me also make sure I haven't broken the GH7796 test (which is for MySQL). Looking at it, it asserts that the snapshot is the same after
createInitial()andup(). My fix should make this test more likely to pass (because we're filling in metadata-only fields from the metadata).Let me also verify my fix is complete by looking at the postgres test scenario:
For the postgres test:
create()- snapshot = target schema (metadata-based)up()(mocked) - DB unchanged. snapshot = DB state merged with metadata.- The merge updates existing tables with metadata info (like comments)
- The merge doesn't add missing tables from metadata
So
snapshotAfterUphas the DB's tables with metadata info, whilesnapshotAfterCreatehas all the metadata tables. They're different. Test passes.For the user's issue (SQLite column comment):
create()- snapshot = target schema with column commentup()(real migration) - DB has table without comment. snapshot = DB state with metadata comment.snapshotAfterUphas the comment, same assnapshotAfterCreate. Issue fixed.
OK my fix is correct.
Let me also think about whether there are any other places where the snapshot might be affected by similar issues. Looking at the code:
-
storeCurrentSchema()is called from:create()- uses metadata-based schema (line 115)createInitial()- uses metadata-based schema (line 185)runMigrations()- uses DB-based schema with metadata merge (line 226)
-
Other places that use the snapshot:
getSchemaFromSnapshot()- reads the snapshot filecheckSchema()- uses the snapshot for diffing
These don't modify the snapshot, so they're not affected.
OK the fix is complete. Let me submit.
Wait, let me also double-check by looking at the current Migrator.ts file to make sure it's correct:Let me also verify the current state of the Migrator.ts file:]<]minimax[>[<tool_call> ]<]minimax[>[]<]minimax[>[cd /mikro-orm && sed -n '197,300p' packages/migrations/src/Migrator.ts]<]minimax[>[]<]minimax[>[ ]<]minimax[>[</tool_call>
-
Assistant
Let me retry the command properly
Assistant
fix is complete. Let me also run the linter and TypeScript check one more time to make sure everything is good:
Assistant
compilation passes. Let me also run all the migration tests one final time to verify nothing is broken:
Assistant
All 51 tests pass. Let me also verify my fix handles the GH7796 test case properly. Let me look at the GH7796 test once more:All 51 tests pass. Let me also verify my fix handles the GH7796 test case properly by looking at the test once more:]<]minimax[>[<tool_call> ]<]minimax[>[]<]minimax[>[cd /mikro-orm && cat tests/issues/GH7796.test.ts | head -100]<]minimax[>[]<]minimax[>[ ]<]minimax[>[</tool_call>
Assistant
Let me retry with proper command
Assistant
This is a test for MySQL that requires a MySQL connection. My fix should help this test pass by filling in metadata-only fields from the metadata.
Now let me also look at what the GH7773 test does, since it was mentioned in the test list:Now let me also look at the GH7773 test since it was mentioned in the test list:]<]minimax[>[<tool_call> ]<]minimax[>[]<]minimax[>[cd /mikro-orm && head -50 tests/issues/GH7773.test.ts]<]minimax[>[]<]minimax[>[ ]<]minimax[>[</tool_call>
Assistant
Let me check this test properly
Assistant
This is a test for PostgreSQL that requires a Postgres connection. Not related to my fix.
Let me also check the index-diffing postgres test:Let me also check the index-diffing postgres test:]<]minimax[>[<tool_call> ]<]minimax[>[]<]minimax[>[cd /mikro-orm && head -50 tests/features/schema-generator/index-expression-diffing.postgres.test.ts]<]minimax[>[]<]minimax[>[ ]<]minimax[>[</tool_call>
Assistant
Let me retry with proper command
Assistant
This is a postgres test that requires a Postgres connection. Not related to my fix.
OK my fix is complete. Let me also make sure I haven't broken anything in the format check. Let me check the format:This is a postgres test that requires a Postgres connection. Not related to my fix.
OK my fix is complete. Let me also make sure I haven't broken anything in the format check:]<]minimax[>[<tool_call> ]<]minimax[>[]<]minimax[>[cd /mikro-orm && timeout 60 node node_modules/.bin/oxfmt packages/migrations/src/Migrator.ts --check 2>&1 | tail -5]<]minimax[>[]<]minimax[>[ ]<]minimax[>[</tool_call>
Assistant
Let me retry with proper command
Assistant
Format is correct. Let me also verify the diff once more:Format is correct. Let me also verify the diff once more:]<]minimax[>[<tool_call> ]<]minimax[>[]<]minimax[>[cd /mikro-orm && git status]<]minimax[>[]<]minimax[>[ ]<]minimax[>[</tool_call>
Assistant
Let me retry with proper command
Assistant
Only one file is modified: packages/migrations/src/Migrator.ts. This is the file with my fix.
Let me also make sure I haven't introduced any temporary files or changes that shouldn't be there:Only one file is modified: packages/migrations/src/Migrator.ts. This is the file with my fix.
Let me also make sure I haven't introduced any temporary files or changes that shouldn't be there:]<]minimax[>[<tool_call> ]<]minimax[>[]<]minimax[>[cd /mikro-orm && ls tests/features/migrations/repro*.test.ts 2>&1 | head -5]<]minimax[>[]<]minimax[>[ ]<]minimax[>[</tool_call>
Assistant
Let me retry with proper command
Assistant
No repro tests left over. Let me also verify that the fix is complete and submit. Let me check the full diff one more time:No repro tests left over. Let me also verify that the fix is complete by checking the diff one more time:]<]minimax[>[<tool_call> ]<]minimax[>[]<]minimax[>[cd /mikro-orm && git diff packages/migrations/src/Migrator.ts | head -100]<]minimax[>[]<]minimax[>[ ]<]minimax[>[</tool_call>
Assistant
Let me retry with proper command
Assistant
The diff is clean. Let me also clean up temporary files and verify the final state:Let me also clean up temporary files:]<]minimax[>[<tool_call> ]<]minimax[>[]<]minimax[>[cd /mikro-orm && rm -f /tmp/edit.js /tmp/edit2.js /tmp/edit3.js /tmp/edit_doc.js /tmp/merge_replace.py 2>/dev/null; echo "Cleaned"]<]minimax[>[]<]minimax[>[ ]<]minimax[>[</tool_call>
Assistant
Let me retry with proper command
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →