Skip to content
The Chat Archive
hunyuan-turbos-20250416 LMArena Human Preference 140K

I have built a chatbot with multiple LLM support. In my side bar I show the chat history…

  1. User

    I have built a chatbot with multiple LLM support. In my side bar I show the chat history and the prompt usage. I want that the prompt usage updates when I save the LLM response after the user enters a prompt. Mya app is written in next.js

  2. Assistant

    To update the prompt usage in your sidebar whenever you save an LLM response in your Next.js chatbot app, you'll need to:

    1. Track when a new response is saved.
    2. Update the application state (likely globally) to reflect the new token/prompt usage.
    3. Ensure the sidebar component re-renders to display the updated values.

    ✅ Step-by-Step Guide

    1. Track Prompt Usage Globally

    The best way to share state like token usage across components (especially in a layout with sidebar and chat area) is by using React Context, Zustand, or a global state management library.

    For simplicity, let’s assume you're using React Context.

    store/PromptUsageContext.tsx
    import { createContext, useContext, useState, ReactNode } from 'react';
    
    interface PromptUsage {
      totalTokens: number;
      // optionally break down by model
      // [modelName]: number
    }
    
    const defaultUsage: PromptUsage = {
      totalTokens: 0,
    };
    
    const PromptUsageContext = createContext<{
      usage: PromptUsage;
      updateUsage: (newTokens: number) => void;
    }>({
      usage: defaultUsage,
      updateUsage: () => {},
    });
    
    export const PromptUsageProvider = ({ children }: { children: ReactNode }) => {
      const [usage, setUsage] = useState<PromptUsage>(defaultUsage);
    
      const updateUsage = (newTokens: number) => {
        setUsage((prev) => ({
          ...prev,
          totalTokens: prev.totalTokens + newTokens,
        }));
      };
    
      return (
        <PromptUsageContext.Provider value={{ usage, updateUsage }}>
          {children}
        </PromptUsageContext.Provider>
      );
    };
    
    export const usePromptUsage = () => useContext(PromptUsageContext);
    

    2. Wrap Your App with the Provider

    In your _app.tsx or _app.jsx:

    import { PromptUsageProvider } from '../store/PromptUsageContext';
    
    function MyApp({ Component, pageProps }) {
      return (
        <PromptUsageProvider>
          <Component {...pageProps} />
        </PromptUsageProvider>
      );
    }
    
    export default MyApp;
    

    3. Update Usage When Saving Response

    When you save the LLM response (e.g., after fetching it), calculate the tokens used (you can use tiktoken or OpenAI’s API response if available), then call updateUsage().

    Example inside your chat logic or handleSaveResponse() function:

    import { usePromptUsage } from '../store/PromptUsageContext';
    
    const ChatInputHandler = () => {
      const { updateUsage } = usePromptUsage();
    
      const handleSaveResponse = async (userPrompt: string, llmResponse: string) => {
        // Assume getTokenCount is a utility that returns { promptTokens, completionTokens }
        const tokensUsed = await getTokenCount(userPrompt, llmResponse); 
    
        // Save to DB or state as needed...
    
        // Update global usage
        updateUsage(tokensUsed.total);
      };
    
      // ...
    };
    

    💡 You’ll need a utility like tiktoken to count tokens depending on the model used.

    4. Display Updated Usage in Sidebar

    Anywhere in your sidebar component:

    import { usePromptUsage } from '../store/PromptUsageContext';
    
    const Sidebar = () => {
      const { usage } = usePromptUsage();
    
      return (
        <aside>
          <h3>Chat History</h3>
          {/* ...other content... */}
          <p>Total Tokens Used: {usage.totalTokens}</p>
        </aside>
      );
    };
    

    🧠 Bonus Tips

    • If you support multiple models, consider tracking per-model usage too.
    • You can debounce updates or batch them if performance becomes a concern.
    • For persistent usage tracking across sessions, store it in localStorage or a backend API.

    ✅ Summary

    TaskTool/Method
    Share usage stateReact Context (useContext)
    Update on saveCall updateUsage(tokens) after saving response
    Display in sidebarConsume context in sidebar component

    Would you like help implementing the getTokenCount() function for OpenAI or other providers?

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.