ISSUE DESCRIPTION /v1/models: return the full catalog with per-model availability +…
- User
ISSUE DESCRIPTION
/v1/models: return the full catalog with per-model availability + opt-in filter Split out from #242.
Current: `GET /v1/models` returns only models that are connected right now (enabled AND an enabled key can serve them).
Wanted: by default return the whole catalog (one entry per model id), each annotated with whether it's usable, plus an opt-in filter to get just the connected ones.
Per-model annotation:
- `available`: boolean — enabled AND a matching enabled key exists.
- `unavailable_reason`: `null` | `"no_key"` (enabled, no key) | `"disabled"` (catalog row disabled).
Filter:
- `?available=true` (alias `?connected=true`) → only connected models (the previous default behavior).
Notes:
- The extra fields are additive — OpenAI-compatible clients ignore unknown fields.
- Dedup prefers an available instance of a model id over a disabled/keyless one.
- The `auto` virtual model is always `available: true`.
Interface
No new interfaces are introduced.
ADDITIONAL ADVICE
Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use
git checkout <file>to restore the file to its original state. You can also reset all changes in the repository usinggit reset --hardcommand. Additionally, you can usegit stashandgit stash popto temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you callsubmit) usinggit statusorgit diffcommands. However you don't need to usegit addorgit commitbefore submitting your changes. In fact,submitonly submits currently unstaged changes because it usesgit diffto get the changes that need to be submitted.CHECKLIST
Before submitting your solution, please go over the following checklist and make sure you've done everything:
- If an error was reported in the issue description, I have successfully reproduced it.
- If an error was reported in the issue description, I have confirmed that my fix resolves the error.
- I have thought about edge cases and made sure my fix handles them as well.
- I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.
Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!
(Current directory: /freellmapi, current file: none) bash-$
Assistant
Let me start by exploring the repository structure to understand how it's organized and find the relevant code for the
/v1/modelsendpoint.Assistant
It seems I'm already in the repository directory. Let me list the contents.
Assistant
This is a Node.js project. Let me explore the server directory to find the
/v1/modelsendpoint.Assistant
Let me explore the server/src directory to find the models endpoint.
Assistant
Let me find the models endpoint more specifically.
<tool_call> <function=bash_terminal> <parameter=command> find server/src -type f -name "*.ts" | xargs grep -l "models" 2>/dev/null | head -20 </tool_call>
Assistant
I need to run a bash command to search for files containing "models". Let me try a different approach.
Assistant
Found it! There's a file called
server/src/routes/models.ts. Let me open it to understand the current implementation.<tool_call> <function=open> <parameter=command> open server/src/routes/models.ts </tool_call>
Assistant
I need to use the correct tool call format to open the file.
Assistant
Now I understand the current implementation. The
/v1/modelsendpoint currently returns all models without filtering for availability or providing theavailableandunavailable_reasonannotations.According to the requirements, I need to:
- Return the whole catalog by default (one entry per model id)
- Add
availablefield (boolean) — enabled AND a matching enabled key exists - Add
unavailable_reasonfield —null|"no_key"(enabled, no key) |"disabled"(catalog row disabled) - Add an optional filter
?available=true(alias?connected=true) to return only connected models - The
autovirtual model is alwaysavailable: true
Let me also check what the API keys structure looks like to understand how to check if there's an enabled key for a platform.
<tool_call> <function=bash_terminal> <parameter=command> cat server/src/routes/keys.ts </tool_call>
Assistant
I need to use the correct tool call format to run the command. Let me try again.
Assistant
Now I understand the structure better. Let me also check if there are any existing tests for the models endpoint to understand how the current behavior is tested.
<tool_call> <function=bash_terminal> <parameter=command> find server/src -type f -name "*.test.ts" | xargs grep -l "models" | head -10 </tool_call>
Assistant
Let me search for test files related to the models route.
Assistant
Let me search for any specific test file for the models endpoint.
Assistant
The exit code 0 but no output means no files matched the pattern. Let me search in a different way.
Assistant
Let me look at the app.ts to understand how the routes are organized and check for any integration tests.
Assistant
Let me look at the full models.ts file and understand its structure better, then modify it to implement the requirements.
Assistant
Let me look at the full flow test to understand the expected behavior.
Assistant
Let me now look at the full flow test to understand what the expected behavior is when we add a key.
<tool_call> <function=bash_terminal> <parameter=command> sed -n '70,120p' server/src/tests/integration/full-flow.test.ts </tool_call>
Assistant
I keep having issues with my tool calls. Let me be more explicit with the function name.
Assistant
Now let me look at the database schema to understand how models and api_keys are structured. This will help me understand how to check if there's an enabled key for a platform.
Assistant
Now let me look at the models table structure more closely.
Assistant
Now I understand the structure:
modelstable hasplatform,enabledcolumnapi_keystable hasplatformandenabledcolumns
Now I need to modify the
models.tsroute to:- Return all models with
availableandunavailable_reasonfields - Add a filter
?available=true(alias?connected=true) to return only connected models - The
autovirtual model is alwaysavailable: true
Let me check if there's a special
automodel:<tool_call> <function=bash_terminal> <parameter=command> grep -n "auto" server/src/routes/models.ts </tool_call>
Assistant
I keep getting an error with my tool calls. Let me try again with the correct syntax.
Assistant
No "auto" model in models.ts currently. Let me search for it elsewhere to understand how it's handled.
Assistant
The "auto" model is a virtual model that doesn't exist in the catalog. It's handled specially in the proxy route. Let me look at the proxy-auto-model test to understand how it should be returned in the models endpoint.
Assistant
Now I need to understand the expected response format. The test shows it expects an OpenAI-compatible format with
object: 'list'anddata: [...]. Let me look at the current models route more carefully and check how it's mounted.Assistant
The models route is at
/api/models(admin route), but the test is checking/v1/models. Let me search for where/v1/modelsis handled.<tool_call> <function=bash_terminal> <parameter=command> grep -n "/v1" server/src/app.ts </tool_call>
Assistant
Let me try again with the correct bash command to search for /v1 routes.
Assistant
So
/v1/modelsis handled by theproxyRouter. Let me look at the proxy route to understand how it handles the/v1/modelsendpoint.<tool_call> <function=bash_terminal> <parameter=command> grep -n "models" server/src/routes/proxy.ts | head -30 </tool_call>
Assistant
Let me try the correct tool call format again.
Assistant
Good! Now I need to look at the current
/v1/modelsimplementation in proxy.ts to understand how to modify it.<tool_call> <function=bash_terminal> <parameter=command> sed -n '119,200p' server/src/routes/proxy.ts </tool_call>
Assistant
I'm having trouble with the tool call format. Let me try to run the bash command differently.
Assistant
Now I understand the current implementation. The
/v1/modelsendpoint in proxy.ts only returns models that are connected (enabled AND have an enabled key). According to the issue description, I need to:- Return the whole catalog by default
- Add
availableandunavailable_reasonfields to each model - Add a
?available=truefilter (alias?connected=true) to return only connected models - The
autovirtual model is alwaysavailable: true
Let me also check the
/api/modelsendpoint in models.ts to see if it needs similar changes.Let me first check the full proxy.ts file to understand the structure better and find where I need to make changes.
<tool_call> <function=bash_terminal> <parameter=command> sed -n '1,15p' server/src/routes/proxy.ts </tool_call>
Assistant
I keep having issues with the tool call. Let me try to just run the bash commands directly.
Assistant
Let me find the line number for the
/modelsendpoint and then look at what AUTO_MODEL_ID is.Assistant
Let me now understand the current implementation of the models endpoint in proxy.ts. I need to modify it to:
- Return the whole catalog by default
- Add
availableandunavailable_reasonfields - Add an optional filter
?available=trueto return only connected models
Let me also check the api models endpoint in models.ts to see if I need to modify it as well.
<tool_call> <function=bash_terminal> <parameter=command> cat server/src/routes/models.ts </tool_call>
Assistant
Let me try a different approach - I'll read the file first and then make changes to it.
Assistant
Now I understand the current implementation. There are two endpoints:
/v1/modelsin proxy.ts - returns OpenAI-compatible format with just connected models/api/modelsin models.ts - returns admin endpoint with all models but no availability info
According to the issue description, I need to modify
/v1/modelsto:- Return the whole catalog by default
- Add
availableandunavailable_reasonfields - Add a
?available=true(alias?connected=true) filter - The
autovirtual model is alwaysavailable: true
The issue description says "No new interfaces are introduced" and "OpenAI-compatible clients ignore unknown fields". So I should add the
availableandunavailable_reasonfields as additional fields to the existing OpenAI-compatible response.Let me modify the proxy.ts file to implement this change. I'll need to:
- Modify the query to fetch all models with their availability info
- Add the
availableandunavailable_reasonfields - Handle the
?available=truefilter to return only connected models - Make sure the
automodel is alwaysavailable: true
Let me first look at the entire proxy.ts file to understand how to access the database and what ModelListRow type looks like.
<tool_call> <function=bash_terminal> <parameter=command> grep -n "ModelListRow" server/src/routes/proxy.ts </tool_call>
Assistant
I keep having issues with the tool call format. Let me try again with the correct bash command.
Assistant
Let me look at the shared types to understand the ModelListRow type.
<tool_call> <function=bash_terminal> <parameter=command> cat shared/types.ts </tool_call>
Assistant
I need to run the bash command with the proper syntax. Let me try again.
Assistant
Now I have a better understanding. Let me modify the
/v1/modelsendpoint in proxy.ts to implement the required changes. I'll use theedittool to replace the models endpoint function.Assistant
Let me check what's around line 121 to understand the full scope of the function.
Assistant
Now I understand the current implementation. I need to modify this function to:
- Return all models from the catalog
- Add
availableandunavailable_reasonfields - Support
?available=truefilter (alias?connected=true) to return only connected models - The
automodel is alwaysavailable: true
Let me create the modified version of this function. I'll need to:
- Get the query parameter for
availableorconnected - Fetch all models with their availability info
- Map the models to the response format with additional fields
- Filter if
available=trueis passed
Let me first check what the exact line range is for this function.
<tool_call> <function=bash_terminal> <parameter=command> wc -l server/src/routes/proxy.ts </tool_call>
Assistant
Let me count the lines in the proxy.ts file to understand its size.
Assistant
The proxy.ts file has 1108 lines. The models endpoint is on lines 119-171. Let me use the
edittool to replace this function with the new implementation.<tool_call> <function=bash_terminal> <parameter=command> sed -n '117,173p' server/src/routes/proxy.ts </tool_call>
Assistant
I keep having issues with the tool call format. Let me try a different approach - I'll use sed to create a backup of the file and then manually modify it.
Actually, let me try to use the edit tool with a file path option to replace the models endpoint.
Assistant
Now I understand the exact structure. Let me use the
edittool to replace the models endpoint function with the new implementation.Assistant
Now let me use the
edittool to replace the models endpoint in the proxy.ts file with the new implementation.Assistant
Now let me use the
edittool to replace the models endpoint function with the new implementation. I'll replace lines 119-171 with the new code.Assistant
Now let me add the new models endpoint function.
Assistant
Now let me add the rest of the file (from line 173 onwards) to the new file.
Assistant
Now let me replace the original file with the new one.
Assistant
Now let me verify the change was applied correctly by checking the models endpoint.
Assistant
Good! The changes look correct. Let me also check the rest of the function to make sure it's complete.
Assistant
The file looks correct. Now I need to also update the
/api/modelsendpoint in the models.ts file to provide the same availability info. Let me also check if that endpoint needs to be modified.According to the issue description, it mentions
/v1/models, but the/api/modelsendpoint is also exposed. Let me check if there are any tests for the/api/modelsendpoint to see if it needs to be updated as well.Actually, looking at the issue description again, it says:
- "by default return the whole catalog (one entry per model id)"
- The
?available=true(alias?connected=true) filter to get just the connected ones
The issue description seems to focus on
/v1/modelsendpoint. However, since the/api/modelsendpoint also returns model information, it would be good to update it as well to provide consistent availability info.Let me now check if there are any tests for the models endpoint to verify my changes work correctly.
<tool_call> <function=bash_terminal> <parameter=command> cat server/src/tests/routes/proxy-auto-model.test.ts | head -100 </tool_call>
Assistant
The tool call command was not properly formatted. Let me try again with the correct syntax.
Assistant
Now let me check if there are any specific tests for the
?available=truefilter that I need to handle.<tool_call> <function=bash_terminal> <parameter=command> grep -n "available" server/src/tests/routes/proxy-auto-model.test.ts </tool_call>
Assistant
The command didn't work. Let me try again with the correct format.
Assistant
The grep command didn't find any matches for "available" in the test file, which means the test file doesn't have tests for the
?available=truefilter. This is expected since the feature is being implemented now.Let me now compile the TypeScript to check for any syntax errors in my changes.
<tool_call> <function=bash_terminal> <parameter=command> cd server && npx tsc --noEmit 2>&1 | head -50 </tool_call>
Assistant
I keep having issues with the tool call format. Let me try running the command in the correct way.
Assistant
The TypeScript compilation succeeded without errors. Now let me run the existing tests to make sure my changes don't break anything.
Assistant
The tests are running. Let me continue to see if there are any failures related to my changes.
<tool_call> <function=bash_terminal> <parameter=command> npm test 2>&1 | tail -100 </tool_call>
Assistant
I need to wait for the tests to complete and see the results. Let me check the full test output to see if there are any failures related to my changes.
Assistant
All 456 tests passed! This is great. My changes didn't break any existing functionality.
Now let me verify that my changes are correct by reviewing them one more time. Let me also check if there are any edge cases I might have missed.
Looking at the issue description again:
- ✅ Return the whole catalog by default (one entry per model id)
- ✅ Add
availableboolean field — enabled AND a matching enabled key exists - ✅ Add
unavailable_reasonfield —null|"no_key"(enabled, no key) |"disabled"(catalog row disabled) - ✅ Add
?available=truefilter (alias?connected=true) to return only connected models - ✅ The
autovirtual model is alwaysavailable: true - ✅ Dedup prefers an available instance of a model id over a disabled/keyless one
- ✅ The extra fields are additive — OpenAI-compatible clients ignore unknown fields
Wait, I need to re-read the issue description more carefully:
Wanted: by default return the whole catalog (one entry per model id), each annotated with whether it's usable, plus an opt-in filter to get just the connected ones.
This says "by default return the whole catalog". But I should check if my implementation returns all models from the catalog, or if there's any filtering happening.
Let me also check the issue description again for any other requirements:
Per-model annotation:
available: boolean — enabled AND a matching enabled key exists.unavailable_reason:null|"no_key"(enabled, no key) |"disabled"(catalog row disabled).
My implementation looks correct. Let me verify by checking the logic:
available:m.model_enabled === 1 && m.key_count > 0- This checks if the model is enabled AND has at least one enabled keyunavailable_reason:m.model_enabled === 0 ? 'disabled'- If the model is disabledm.key_count === 0 ? 'no_key'- If the model is enabled but has no keysnull- Otherwise
This looks correct!
However, I notice that the issue description mentions:
Dedup prefers an available instance of a model id over a disabled/keyless one.
My current deduplication logic iterates through models and keeps only the first instance of each model_id. But the order in the database query is
ORDER BY m.intelligence_rank ASC, m.id ASC, which may not guarantee that available instances come first.Let me fix this by ordering the models so that available instances come first.
Actually, looking at my implementation again, I realize that I'm not correctly handling the deduplication. The issue says "Dedup prefers an available instance of a model id over a disabled/keyless one", which means:
- If there are multiple entries for the same model_id, we should prefer the available one
- If all entries are disabled/keyless, we should prefer the enabled one
Let me update my implementation to handle this correctly.
Actually, wait. Let me re-read the issue description more carefully:
Dedup prefers an available instance of a model id over a disabled/keyless one.
This means that when deduplicating, we should prefer the available instance. My current implementation does this by iterating through models and keeping the first instance. But I need to make sure that the ordering is correct - available instances should come first.
Let me update my implementation to sort the models so that available instances come first.
Actually, looking at the original implementation, I see that it uses
ROW_NUMBER() OVER (PARTITION BY model_id ORDER BY intelligence_rank ASC, id ASC), which means it prefers models with lower intelligence_rank (smarter models first). This is the same ordering I'm using in my new implementation.However, the issue description says "Dedup prefers an available instance of a model id over a disabled/keyless one". This suggests that availability should be a higher priority than intelligence_rank.
Let me update my implementation to handle this correctly. I'll sort the models so that available instances come first, then by intelligence_rank, then by id.
Actually, I realize there might be an issue with my implementation. Looking at the original code again:
const models = db.prepare(` SELECT platform, model_id, display_name, context_window FROM ( SELECT platform, model_id, display_name, context_window, intelligence_rank, id, ROW_NUMBER() OVER ( PARTITION BY model_id ORDER BY intelligence_rank ASC, id ASC ) AS rn FROM models m WHERE m.enabled = 1 AND EXISTS ( SELECT 1 FROM api_keys k WHERE k.platform = m.platform AND k.enabled = 1 AND (m.key_id IS NULL OR k.id = m.key_id) ) ) WHERE rn = 1 ORDER BY intelligence_rank ASC, id ASC `).all() as ModelListRow[];This query:
- Selects only enabled models with enabled keys
- Uses ROW_NUMBER to deduplicate by model_id, preferring lower intelligence_rank
- Returns only the first row for each model_id
My new implementation:
- Selects all models (enabled or disabled, with or without keys)
- Deduplicates by model_id, keeping the first instance (which is ordered by intelligence_rank)
- Optionally filters by availability
The issue description says "Dedup prefers an available instance of a model id over a disabled/keyless one". This means that when deduplicating, we should prefer available instances. Let me update my implementation to handle this correctly.
Let me update my implementation to sort the models so that available instances come first during deduplication.
Actually, I realize that my current implementation might have an issue. Let me trace through the logic:
- I fetch all models from the database, ordered by
intelligence_rank ASC, id ASC - I deduplicate by keeping the first instance of each model_id
- This means that if there are multiple entries for the same model_id, I keep the one with the lowest intelligence_rank
But the issue description says "Dedup prefers an available instance of a model id over a disabled/keyless one". This means that when there are multiple entries for the same model_id, I should prefer the available one, regardless of intelligence_rank.
Let me update my implementation to handle this correctly. I'll sort the models by:
- availability (available first)
- intelligence_rank
- id
This way, available instances with lower intelligence_rank will come first, followed by available instances with higher intelligence_rank, followed by disabled/keyless instances.
Actually, I realize there's a potential issue with my implementation. The deduplication logic in my current implementation keeps the first instance of each model_id, which is ordered by
intelligence_rank ASC, id ASC. But I need to also consider availability in the deduplication logic.Let me update my implementation to sort the models so that available instances come first during deduplication.
Actually, looking at the issue description again:
Dedup prefers an available instance of a model id over a disabled/keyless one.
This says that when deduplicating, we should prefer the available instance. But the original implementation uses
ROW_NUMBER() OVER (PARTITION BY model_id ORDER BY intelligence_rank ASC, id ASC), which doesn't consider availability.I think the issue description is suggesting that the new implementation should prefer available instances during deduplication. Let me update my implementation to handle this correctly.
Actually, I realize I might be overthinking this. Let me re-read the issue description:
Dedup prefers an available instance of a model id over a disabled/keyless one.
This could mean:
- When there are multiple entries for the same model_id, prefer the available one
- Or it could just be a note about how the implementation should handle deduplication
Looking at the original implementation, it deduplicates by model_id using
ROW_NUMBER() OVER (PARTITION BY model_id ORDER BY intelligence_rank ASC, id ASC). This means it prefers models with lower intelligence_rank.My new implementation deduplicates by model_id by keeping the first instance. Since I'm ordering by
intelligence_rank ASC, id ASC, I'm also preferring models with lower intelligence_rank.The question is: should I also prefer available instances over disabled/keyless ones?
The issue description says "Dedup prefers an available instance of a model id over a disabled/keyless one". This suggests that availability should be a higher priority than intelligence_rank during deduplication.
Let me update my implementation to handle this correctly. I'll sort the models by:
- availability (available first)
- intelligence_rank
- id
This way, available instances will be preferred during deduplication.
Actually, I think I'm overcomplicating this. Let me just update my implementation to sort the models properly.
Let me update the SQL query to sort by availability first, then by intelligence_rank, then by id.
Actually, looking at my implementation again, I realize that I'm doing the deduplication in JavaScript, not in SQL. This gives me more flexibility to implement the correct deduplication logic.
Let me update my implementation to handle the deduplication correctly.
Here's my plan:
- Fetch all models from the database, ordered by
intelligence_rank ASC, id ASC - Sort the models by:
available(true first)intelligence_rank ASCid ASC
- Deduplicate by keeping the first instance of each model_id
This way, available instances with lower intelligence_rank will be preferred during deduplication.
Wait, I realize there's an issue with my current implementation. The deduplication is happening in the order of the SQL query, which is
ORDER BY intelligence_rank ASC, id ASC. But I'm not sorting by availability first.Let me update my implementation to sort the models properly before deduplication.
Actually, I realize I can fix this by updating the SQL query to order by availability first. Let me update my implementation.
Actually, looking at my implementation again, I think the issue is that I need to sort the models by:
- availability (available first)
- intelligence_rank
- id
Let me update the SQL query to do this.
Actually, I realize I can also sort the models in JavaScript before deduplication. This might be simpler than modifying the SQL query.
Let me update my implementation to sort the models in JavaScript before deduplication.
Wait, I realize that the issue description says "Dedup prefers an available instance of a model id over a disabled/keyless one". This is a note about how deduplication should work. It's not necessarily saying that I need to change the ordering from the original implementation.
Let me re-read the issue description one more time:
Dedup prefers an available instance of a model id over a disabled/keyless one.
I think this is saying that when deduplicating, we should prefer the available instance. This means that if there are multiple entries for the same model_id, I should prefer the available one.
But looking at the original implementation, it deduplicates by model_id using
ROW_NUMBER() OVER (PARTITION BY model_id ORDER BY intelligence_rank ASC, id ASC). This means it prefers models with lower intelligence_rank, regardless of availability.The issue description is suggesting that the new implementation should prefer available instances during deduplication. This means I need to change the ordering to include availability.
Let me update my implementation to handle this correctly.
Actually, I realize I'm overthinking this. Let me just update my implementation to sort the models by availability first, then by intelligence_rank, then by id.
Here's my updated implementation:
// Fetch all models with availability info, sorted by availability first const allModels = db.prepare(` SELECT m.id, m.platform, m.model_id, m.display_name, m.enabled as model_enabled, m.context_window, m.intelligence_rank, m.key_id, (SELECT COUNT(*) FROM api_keys k WHERE k.platform = m.platform AND k.enabled = 1 AND (m.key_id IS NULL OR k.id = m.key_id)) as key_count FROM models m ORDER BY CASE WHEN m.enabled = 1 AND (SELECT COUNT(*) FROM api_keys k WHERE k.platform = m.platform AND k.enabled = 1 AND (m.key_id IS NULL OR k.id = m.key_id)) > 0 THEN 0 ELSE 1 END, m.intelligence_rank ASC, m.id ASC `).all() as { ... };This sorts the models by:
- Availability (available first, then disabled/keyless)
- intelligence_rank ASC
- id ASC
Then I deduplicate by keeping the first instance of each model_id.
Actually, I realize this might be overly complex. Let me simplify by doing the sorting in JavaScript after fetching the models.
Wait, I realize there's another issue with my implementation. The
key_countis computed in the SQL query, and I'm using it to determine availability. But I need to make sure the SQL query is correct.Let me review my SQL query:
SELECT m.id, m.platform, m.model_id, m.display_name, m.enabled as model_enabled, m.context_window, m.intelligence_rank, m.key_id, (SELECT COUNT(*) FROM api_keys k WHERE k.platform = m.platform AND k.enabled = 1 AND (m.key_id IS NULL OR k.id = m.key_id)) as key_count FROM models m ORDER BY m.intelligence_rank ASC, m.id ASCThis query:
- Selects all models
- For each model, computes the count of enabled keys for that platform (with the optional key_id check)
- Orders by intelligence_rank ASC, id ASC
This looks correct. The
key_countwill be the number of enabled keys for the platform that match the key_id constraint.Now, the issue is that I'm deduplicating by keeping the first instance of each model_id, which is ordered by
intelligence_rank ASC, id ASC. But the issue description says "Dedup prefers an available instance of a model id over a disabled/keyless one".I think the simplest way to handle this is to sort the models by availability first, then by intelligence_rank, then by id. Let me update my implementation to do this.
Actually, I realize I can fix this by updating the ORDER BY clause in my SQL query to include availability as the first sorting criterion.
Let me update my implementation to handle this correctly.
Actually, let me step back and think about this more carefully. The issue description says:
Dedup prefers an available instance of a model id over a disabled/keyless one.
This is a note about how deduplication should work. It's saying that when there are multiple entries for the same model_id, we should prefer the available one.
The original implementation deduplicates by model_id using
ROW_NUMBER() OVER (PARTITION BY model_id ORDER BY intelligence_rank ASC, id ASC). This means it prefers models with lower intelligence_rank.But the issue description is suggesting that we should prefer available instances. This means I need to change the ordering to include availability.
Let me update my implementation to handle this correctly. I'll sort the models by:
available(true first)intelligence_rank ASCid ASC
This way, available instances with lower intelligence_rank will be preferred during deduplication.
Actually, I realize I need to be more careful here. The
availablefield depends onkey_count, which is computed in the SQL query. So I can't useavailabledirectly in the ORDER BY clause.Let me update my implementation to compute the availability in the SQL query, and then sort by it.
Here's my updated implementation:
const allModels = db.prepare(` SELECT m.id, m.platform, m.model_id, m.display_name, m.enabled as model_enabled, m.context_window, m.intelligence_rank, m.key_id, (SELECT COUNT(*) FROM api_keys k WHERE k.platform = m.platform AND k.enabled = 1 AND (m.key_id IS NULL OR k.id = m.key_id)) as key_count, CASE WHEN m.enabled = 1 AND (SELECT COUNT(*) FROM api_keys k WHERE k.platform = m.platform AND k.enabled = 1 AND (m.key_id IS NULL OR k.id = m.key_id)) > 0 THEN 0 ELSE 1 END as sort_priority FROM models m ORDER BY sort_priority ASC, m.intelligence_rank ASC, m.id ASC `).all() as { ... };This adds a
sort_prioritycolumn that is 0 for available models and 1 for unavailable models. Then I sort bysort_priority ASC, which puts available models first.Actually, this is getting too complex. Let me simplify by doing the sorting in JavaScript after fetching the models.
Let me update my implementation to sort the models in JavaScript before deduplication.
Wait, I realize there's a simpler approach. I can compute the
availablefield in JavaScript and then sort the models by availability, intelligence_rank, and id before deduplication.Let me update my implementation to do this.
Actually, I think the best approach is to update the SQL query to compute the availability and sort by it. This is more efficient and cleaner than doing the sorting in JavaScript.
Let me update my implementation to handle this correctly.
Actually, let me just update my implementation to sort the models in JavaScript. This is simpler and doesn't require modifying the SQL query.
Here's my updated implementation:
// Fetch all models with availability info const allModels = db.prepare(` SELECT m.id, m.platform, m.model_id, m.display_name, m.enabled as model_enabled, m.context_window, m.intelligence_rank, m.key_id, (SELECT COUNT(*) FROM api_keys k WHERE k.platform = m.platform AND k.enabled = 1 AND (m.key_id IS NULL OR k.id = m.key_id)) as key_count FROM models m ORDER BY m.intelligence_rank ASC, m.id ASC `).all() as { ... }; // Compute availability and sort by availability first, then intelligence_rank, then id const modelsWithAvailability = allModels.map(m => ({ ...m, available: m.model_enabled === 1 && m.key_count > 0, })); modelsWithAvailability.sort((a, b) => { // Sort by availability first (true first) if (a.available !== b.available) { return a.available ? -1 : 1; } // Then by intelligence_rank if (a.intelligence_rank !== b.intelligence_rank) { return a.intelligence_rank - b.intelligence_rank; } // Then by id return a.id - b.id; }); // Deduplicate: prefer available instance over disabled/keyless const seenModelIds = new Set<string>(); const dedupedModels: typeof allModels = []; for (const m of modelsWithAvailability) { if (seenModelIds.has(m.model_id)) { // Skip duplicates - we already added the best instance (available > disabled/keyless) continue; } dedupedModels.push(m); seenModelIds.add(m.model_id); }This implementation:
- Fetches all models with availability info
- Computes the
availablefield for each model - Sorts by availability (true first), then intelligence_rank, then id
- Deduplicates by keeping the first instance of each model_id
This ensures that available instances are preferred during deduplication, which matches the issue description.
Let me update my implementation to handle this correctly.
Actually, I realize I can simplify this by using a single SQL query with the correct ordering. Let me update my implementation to do this.
Actually, let me just update my implementation to sort the models in JavaScript. This is simpler and doesn't require modifying the SQL query.
Let me update my implementation now.
I'll need to modify my existing implementation to sort the models properly before deduplication.
Let me update the file.
<tool_call> <function=bash_terminal> <parameter=command> head -118 server/src/routes/proxy.ts > /tmp/proxy_new2.ts </tool_call>
Assistant
I keep having issues with my tool calls. Let me try running the bash command properly.
Assistant
I need to change directory to the parent directory first to access the file.
<tool_call> <function=bash_terminal> <parameter=command> head -118 /freellmapi/server/src/routes/proxy.ts > /tmp/proxy_new2.ts </tool_call>
Assistant
The tool call seems to have an issue. Let me try a different approach by cd'ing to the project root and then running the command.
Assistant
Now let me add the updated models endpoint function with the correct sorting and deduplication logic.
<tool_call> <function=bash_terminal> <parameter=command> cat >> /tmp/proxy_new2.ts << 'EOF' // OpenAI-compatible /models endpoint (used by Hermes for metadata) // shows API models which is linked by the user proxyRouter.get('/models', (req: Request, res: Response) => { const token = extractApiToken(req); const unifiedKey = getUnifiedApiKey(); if (!token || !timingSafeStringEqual(token, unifiedKey)) { res.status(401).json({ error: { message: 'Invalid API key', type: 'authentication_error' } }); return; }
const db = getDb();
// Parse the ?available=true (or ?connected=true) filter const availableFilter = req.query.available === 'true' || req.query.connected === 'true';
// Fetch all models with availability info const allModels = db.prepare(
SELECT m.id, m.platform, m.model_id, m.display_name, m.enabled as model_enabled, m.context_window, m.intelligence_rank, m.key_id, (SELECT COUNT(*) FROM api_keys k WHERE k.platform = m.platform AND k.enabled = 1 AND (m.key_id IS NULL OR k.id = m.key_id)) as key_count FROM models m ORDER BY m.intelligence_rank ASC, m.id ASC).all() as { id: number; platform: string; model_id: string; display_name: string; model_enabled: number; context_window: number | null; intelligence_rank: number; key_id: number | null; key_count: number; }[];// Compute availability and sort by: // 1. availability (true first - available instances preferred) // 2. intelligence_rank (smarter models first) // 3. id (deterministic tie-breaker) const modelsWithAvailability = allModels.map(m => ({ ...m, available: m.model_enabled === 1 && m.key_count > 0, }));
modelsWithAvailability.sort((a, b) => { // Sort by availability first (true first) if (a.available !== b.available) { return a.available ? -1 : 1; } // Then by intelligence_rank if (a.intelligence_rank !== b.intelligence_rank) { return a.intelligence_rank - b.intelligence_rank; } // Then by id return a.id - b.id; });
// Deduplicate: prefer available instance over disabled/keyless const seenModelIds = new Set(); const dedupedModels: typeof modelsWithAvailability = []; for (const m of modelsWithAvailability) { if (seenModelIds.has(m.model_id)) { // Skip duplicates - we already added the best instance (available > disabled/keyless) continue; } dedupedModels.push(m); seenModelIds.add(m.model_id); }
// Filter by available=true if requested const filteredModels = availableFilter ? dedupedModels.filter(m => m.available) : dedupedModels;
// Build response with OpenAI-compatible format + availability fields const data: any[] = [ { id: AUTO_MODEL_ID, object: 'model', created: 0, owned_by: 'freellmapi', name: 'Auto (router picks the best available model)', context_window: null, available: true, unavailable_reason: null, }, ...filteredModels.map(m => ({ id: m.model_id, object: 'model', created: 0, owned_by: m.platform, name: m.display_name, context_window: m.context_window, available: m.available, unavailable_reason: m.model_enabled === 0 ? 'disabled' : m.key_count === 0 ? 'no_key' : null, })), ];
res.json({ object: 'list', data, }); });
EOF </tool_call>
Assistant
Let me continue with the command execution.
Assistant
Now let me replace the original file with the new one.
Assistant
Now let me verify the changes by checking the models endpoint.
Assistant
It looks like the file was not properly created. Let me check the full file to see what happened.
<tool_call> <function=bash_terminal> <parameter=command> sed -n '115,165p' /freellmapi/server/src/routes/proxy.ts </tool_call>
Assistant
Let me check the models endpoint function in the file.
Assistant
The file seems to have some issues. It looks like the function is incomplete. Let me check the file from the beginning of the models endpoint.
<tool_call> <function=bash_terminal> <parameter=command> grep -n "OpenAI-compatible /models endpoint" /freellmapi/server/src/routes/proxy.ts </tool_call>
Assistant
Let me search for the models endpoint function in the file to see what's happening.
Assistant
It seems the file got corrupted during the edit. Let me restore the file from the original and redo the changes properly.
Assistant
Good, I've restored the original file. Now let me redo the changes properly using the edit tool.
Assistant
Let me use a Python script to do the replacement properly.
Assistant
Let me try with Python instead.
Assistant
Let me try with sed or awk to do the replacement.
Assistant
The first part was saved. Now let me extract the second part (after the models endpoint function).
Assistant
Now let me create the new models endpoint function and combine all parts.
Assistant
Now let me combine all the parts.
Assistant
Now let me verify the changes by checking the models endpoint.
Assistant
Let me check the rest of the function to make sure it's complete.
Assistant
I see there's a duplicate
const MAX_RETRIES = 20;. Let me fix that.Assistant
Now let me verify the changes by checking the models endpoint again and then running the tests.
Assistant
The file looks good now. Let me run the TypeScript compiler to check for any syntax errors.
Assistant
No TypeScript errors. Now let me run the tests to make sure my changes don't break anything.
Assistant
All 456 tests passed. My implementation is complete. Let me summarize what was implemented:
Summary of Changes
I modified
/freellmapi/server/src/routes/proxy.tsto update the/v1/modelsendpoint with the following changes:-
Return the whole catalog by default - The endpoint now fetches all models from the database, not just the connected ones.
-
Added
availablefield - Boolean indicating if the model is enabled AND has at least one enabled key. -
Added
unavailable_reasonfield - Can benull,"no_key"(enabled but no keys), or"disabled"(catalog row disabled). -
Added
?available=truefilter (alias?connected=true) - When specified, only returns connected models (the previous default behavior). -
autovirtual model is alwaysavailable: true- The auto model is added withavailable: true, unavailable_reason: null. -
Deduplication prefers available instances - Models are sorted by availability first (available instances first), then by intelligence_rank, then by id. This ensures that when there are multiple entries for the same model_id, the available instance is preferred.
-
Additive fields - The extra fields (
available,unavailable_reason) are added to the existing OpenAI-compatible response format, which clients will ignore if they don't expect them.
-
Assistant
The diff shows all the changes I made. Now let me verify the implementation is correct by checking if there are any other places in the codebase that might need to be updated.
<tool_call> <function=bash_terminal> <parameter=command> grep -rn "/v1/models" /freellmapi/server/src/ </tool_call>
Assistant
Let me check for any other places in the codebase that reference the models endpoint.
Assistant
The search results show that the
/v1/modelsendpoint is used in various places:- In migrations for documentation
- In provider implementations for validation
- In tests
- In error messages in proxy.ts
All of these usages should still work with my changes since I'm maintaining backward compatibility by keeping the same response format (OpenAI-compatible with additional fields).
Now let me check if the
/api/modelsendpoint (the admin endpoint) needs to be updated as well. Looking at the issue description, it seems to focus on/v1/models, so I'll leave the/api/modelsendpoint as is for now unless there's a specific requirement to update it.Let me run the tests one more time to make sure everything is working correctly.
<tool_call> <function=bash_terminal> <parameter=command> cd /freellmapi && npm test 2>&1 | grep -E "(PASS|FAIL|passed|failed)" | tail -10 </tool_call>
Assistant
The test command is running. Let me wait for it to complete and check the results.
Assistant
All 456 tests passed. The test suite is complete.
Let me now summarize the checklist from the issue:
Checklist
-
If an error was reported in the issue description, I have successfully reproduced it.
- The issue described a feature request, not an error. The current behavior was returning only connected models, and the wanted behavior was to return the whole catalog with availability info.
-
If an error was reported in the issue description, I have confirmed that my fix resolves the error.
- The fix implements the wanted behavior: returning the whole catalog with
availableandunavailable_reasonfields, plus an opt-in filter for connected models.
- The fix implements the wanted behavior: returning the whole catalog with
-
I have thought about edge cases and made sure my fix handles them as well.
- Edge cases handled:
- Models with no keys (
key_count === 0) →unavailable_reason: "no_key" - Disabled models (
model_enabled === 0) →unavailable_reason: "disabled" - Connected models →
available: true, unavailable_reason: null - Auto virtual model → always
available: true, unavailable_reason: null - Multiple instances of the same model_id → prefers available instance during deduplication
?available=truefilter (alias?connected=true) → returns only connected models
- Models with no keys (
- Edge cases handled:
-
I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass.
- All 456 tests pass.
The implementation is complete and all tests pass.
<tool_call> <function=bash_terminal> <parameter=command> submit </tool_call>
-
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →