Skip to content
TheChatArchiveThe Chat Archive
gpt-4o-mini-2024-07-18· OpenAIAi2 WildChat-4.8M (AI2)

My js script get uncompleted text, generated by ai. How to fix it? <Begin of…

  1. User

    My js script get uncompleted text, generated by ai. How to fix it? #!/usr/bin/env node

    const { chromium } = require('playwright-extra'); const stealth = require('puppeteer-extra-plugin-stealth')(); chromium.use(stealth);

    const url = 'https://www.perplexity.ai/search?focus=internet&copilot=false&q=' + process.argv[2]; const buttonCopy = 'svg[data-icon="clipboard"]'; // copy icon on answer const buttonMenu = 'button[data-testid="thread-dropdown-menu"]'; // "..." icon on top right const buttonDelete = 'div[data-testid="thread-delete"]'; // "Delete Thread" button const buttonConfirm = 'css=button:has-text("Confirm")'; const textMessage = 'div[dir="auto"]'; const totalLoopCount = 80; const printIntervalTime = 4000; function containsLetters(text) { return /[a-zA-Zа-яА-ЯЁё]/.test(text); } chromium.launch({ headless: true, timeout: 30000 }).then(async browser => { // start session const page = await browser.newPage(); await page.goto(url, { waitUntil: 'domcontentloaded' }); answer = ""; // get answer for (let i = 0; i < totalLoopCount; i++) { await new Promise((resolve) => setTimeout(resolve, printIntervalTime)); const result = await page.locator(textMessage).textContent(); if (containsLetters(result) && answer < result) { answer = result; } if (await page.locator(buttonCopy).isVisible() && i != (totalLoopCount - 1)){ i = (totalLoopCount - 1); } } console.log(answer); // delete thread await page.click(buttonMenu, {focus: true}); await page.waitForTimeout(300); await page.click(buttonDelete); await page.waitForTimeout(300); await page.click(buttonConfirm);

    // close browser await browser.close(); });

  2. Assistant

    It seems like you're trying to extract text generated by an AI from a web page, but you're facing an issue with getting incomplete text.

    Here are some suggestions to address the problem of incomplete text extraction:

    1. Increase Waiting Time: The current waiting time of printIntervalTime = 4000 milliseconds might not be sufficient for all responses. Consider increasing this delay to allow more time for the AI’s response to fully load.

    2. Check if Text is Complete: Instead of using a fixed number of iterations (totalLoopCount), implement a check for whether the text has changed or if it seems to be done loading. One way to achieve this is by looking for a specific phrase that indicates the text is complete or by checking if no new text appears after a certain period.

    3. Use waitUntil on Navigation: When you navigate to the page or interact with it, the waitUntil option might be modified to include 'networkidle' or similar, to ensure that all resources are fully loaded.

    4. Check for Text Changes: If you are continuously polling for new text, track the previous result and only update when the text actually changes. Additionally, implement a condition that detects when the new text is essentially the same as the previous text after some time has passed.

    5. Modify the Answer Accumulation Logic: Instead of simply checking if answer < result, consider modifying this logic to check whether the result is truly a new, more complete response compared to what you've previously captured.

    6. Improving the Selector for Text: Ensure that the CSS selector used to get the AI response is accurate. If more text is available but not captured, it could be due to the selector targeting an element that isn't fully representative of the complete text.

    Here’s an updated version of your script integrating some of these suggestions:

    #!/usr/bin/env node
    
    const { chromium } = require('playwright-extra');
    const stealth = require('puppeteer-extra-plugin-stealth')();
    chromium.use(stealth);
    
    const url = 'https://www.perplexity.ai/search?focus=internet&copilot=false&q=' + process.argv[2];
    const buttonCopy = 'svg[data-icon="clipboard"]';
    const buttonMenu = 'button[data-testid="thread-dropdown-menu"]';
    const buttonDelete = 'div[data-testid="thread-delete"]';
    const buttonConfirm = 'css=button:has-text("Confirm")';
    const textMessage = 'div[dir="auto"]';
    const printIntervalTime = 5000; // Increased interval
    const maxWaitCycles = 5;  // Max cycles of waiting for new text
    
    function containsLetters(text) {
        return /[a-zA-Zа-яА-ЯЁё]/.test(text);
    }
    
    chromium.launch({ headless: true, timeout: 30000 }).then(async browser => {
        const page = await browser.newPage();
        await page.goto(url, { waitUntil: 'domcontentloaded' });
    
        let answer = "";
        let lastResult = "";
        let waitCycles = 0;
    
        while (waitCycles < maxWaitCycles) {
            const result = await page.locator(textMessage).textContent();
            if (containsLetters(result) && lastResult !== result) {
                answer = result; // Update answer if new text is found
                lastResult = result;
                waitCycles = 0; // Reset wait cycles since we found new text
            } else if (lastResult === result) {
                waitCycles++; // Increment wait cycles since no new text was detected
            }
            await new Promise((resolve) => setTimeout(resolve, printIntervalTime));
    
            // Check if the clipboard button is visible to see if the answer is complete
            if (await page.locator(buttonCopy).isVisible()) {
                break; // Exit the loop if the clipboard button is visible
            }
        }
    
        console.log(answer);
    
        // Delete thread
        await page.click(buttonMenu, { focus: true });
        await page.waitForTimeout(300);
        await page.click(buttonDelete);
        await page.waitForTimeout(300);
        await page.click(buttonConfirm);
    
        // Close browser
        await browser.close();
    });
    

    In this updated version, I added a waiting mechanism that allows the script to stop when no new text appears for a few cycles, and I increased the print interval. Adjust the settings according to the behavior of the page you're working with to optimize for your use case.