I have built a chatbot with multiple LLM support. In my side bar I show the chat history…
- User
I have built a chatbot with multiple LLM support. In my side bar I show the chat history and the prompt usage. I want that the prompt usage updates when I save the LLM response after the user enters a prompt. Mya app is written in next.js
- Assistant
To update the prompt usage in your sidebar whenever you save an LLM response in your Next.js chatbot app, you'll need to:
- Track when a new response is saved.
- Update the application state (likely globally) to reflect the new token/prompt usage.
- Ensure the sidebar component re-renders to display the updated values.
✅ Step-by-Step Guide
1. Track Prompt Usage Globally
The best way to share state like token usage across components (especially in a layout with sidebar and chat area) is by using React Context, Zustand, or a global state management library.
For simplicity, let’s assume you're using React Context.
store/PromptUsageContext.tsximport { createContext, useContext, useState, ReactNode } from 'react'; interface PromptUsage { totalTokens: number; // optionally break down by model // [modelName]: number } const defaultUsage: PromptUsage = { totalTokens: 0, }; const PromptUsageContext = createContext<{ usage: PromptUsage; updateUsage: (newTokens: number) => void; }>({ usage: defaultUsage, updateUsage: () => {}, }); export const PromptUsageProvider = ({ children }: { children: ReactNode }) => { const [usage, setUsage] = useState<PromptUsage>(defaultUsage); const updateUsage = (newTokens: number) => { setUsage((prev) => ({ ...prev, totalTokens: prev.totalTokens + newTokens, })); }; return ( <PromptUsageContext.Provider value={{ usage, updateUsage }}> {children} </PromptUsageContext.Provider> ); }; export const usePromptUsage = () => useContext(PromptUsageContext);2. Wrap Your App with the Provider
In your
_app.tsxor_app.jsx:import { PromptUsageProvider } from '../store/PromptUsageContext'; function MyApp({ Component, pageProps }) { return ( <PromptUsageProvider> <Component {...pageProps} /> </PromptUsageProvider> ); } export default MyApp;3. Update Usage When Saving Response
When you save the LLM response (e.g., after fetching it), calculate the tokens used (you can use
tiktokenor OpenAI’s API response if available), then callupdateUsage().Example inside your chat logic or
handleSaveResponse()function:import { usePromptUsage } from '../store/PromptUsageContext'; const ChatInputHandler = () => { const { updateUsage } = usePromptUsage(); const handleSaveResponse = async (userPrompt: string, llmResponse: string) => { // Assume getTokenCount is a utility that returns { promptTokens, completionTokens } const tokensUsed = await getTokenCount(userPrompt, llmResponse); // Save to DB or state as needed... // Update global usage updateUsage(tokensUsed.total); }; // ... };💡 You’ll need a utility like
tiktokento count tokens depending on the model used.4. Display Updated Usage in Sidebar
Anywhere in your sidebar component:
import { usePromptUsage } from '../store/PromptUsageContext'; const Sidebar = () => { const { usage } = usePromptUsage(); return ( <aside> <h3>Chat History</h3> {/* ...other content... */} <p>Total Tokens Used: {usage.totalTokens}</p> </aside> ); };
🧠 Bonus Tips
- If you support multiple models, consider tracking per-model usage too.
- You can debounce updates or batch them if performance becomes a concern.
- For persistent usage tracking across sessions, store it in localStorage or a backend API.
✅ Summary
Task Tool/Method Share usage state React Context ( useContext)Update on save Call updateUsage(tokens)after saving responseDisplay in sidebar Consume context in sidebar component Would you like help implementing the
getTokenCount()function for OpenAI or other providers?
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.