Skip to content
TheChatArchiveThe Chat Archive
o1-mini-2024-09-12· OpenAIAi2 WildChat-4.8M (AI2)

I would like to create a script for Violentmonkey to scrap images from…

  1. User

    I would like to create a script for Violentmonkey to scrap images from http://d15.876515.xyz, http://98.987328.xyz, http://97.987327.xyz , let's call it WebURL . the @match should be set to WebURL/* .

    1. add a DOWNLOAD button in the element div.title h2 a span
    2. from the same element, capture the href , add WebURL to the front and call it PageURL
    3. When the DOWNLOAD button is pressed, go to PageURL, and from the content of div.jianjie capture the title which is embraced by, "[ "and ends with "P]" (quotation marks are excluded). Let's call this TITLE .
    4. from div.page, go through all the href (PageURL)inside and copy all images within div.content p from each PageURL.
    5. Rename the captured image files with serial number starting with 000 and add them to a zip file using TITLE as the filename.
  2. Assistant

    Certainly! Below is a comprehensive Violentmonkey userscript that accomplishes the tasks you've outlined. This script adds a DOWNLOAD button to specified elements on your target websites. When clicked, it navigates through related pages, collects images, renames them sequentially, packages them into a ZIP file named after the extracted title, and prompts the user to download the ZIP.

    Prerequisites:

    1. Violentmonkey Extension: Ensure you have the Violentmonkey extension installed in your browser.
    2. JSZip Library: This script utilizes the JSZip library to handle ZIP file creation. The library is included via the @require directive.

    Script Breakdown:

    1. @match Directive:

      • The script matches the specified URLs (http://d15.876515.xyz/*, http://98.987328.xyz/*, and http://97.987327.xyz/*).
    2. Button Injection:

      • A DOWNLOAD button is added to each div.title h2 a span element found on the page.
    3. Button Functionality:

      • Upon clicking the DOWNLOAD button:
        • The script captures the href attribute of the associated link and constructs the PageURL.
        • It navigates to PageURL to extract the TITLE from div.jianjie.
        • It then collects all related PageURLs from div.page.
        • For each PageURL, the script fetches the page and extracts all image sources within div.content p.
        • Images are fetched as blobs, renamed sequentially (starting from 000), and added to a ZIP archive named after the extracted TITLE.
        • Finally, the ZIP file is automatically downloaded to the user's device.
    4. Concurrency Handling:

      • The script utilizes asynchronous functions (async/await) to handle multiple concurrent network requests efficiently.
    5. Error Handling:

      • Basic error handling is implemented to alert the user in case of failures during the download process.

    Complete Userscript:

    // ==UserScript==
    // @name         WebURL Image Downloader
    // @namespace    http://tampermonkey.net/
    // @version      1.0
    // @description  Download images from specified WebURLs as a ZIP file.
    // @author       Your Name
    // @match        http://d15.876515.xyz/*
    // @match        http://98.987328.xyz/*
    // @match        http://97.987327.xyz/*
    // @grant        GM_download
    // @require      https://cdnjs.cloudflare.com/ajax/libs/jszip/3.10.0/jszip.min.js
    // @connect      *
    // ==/UserScript==
    
    (async function() {
        'use strict';
    
        // Utility function to fetch and parse HTML
        async function fetchHTML(url) {
            const response = await fetch(url);
            const text = await response.text();
            const parser = new DOMParser();
            return parser.parseFromString(text, 'text/html');
        }
    
        // Utility function to fetch image as blob
        async function fetchImage(url) {
            const response = await fetch(url);
            if (!response.ok) throw new Error(`Failed to fetch image: ${url}`);
            return await response.blob();
        }
    
        // Function to add DOWNLOAD button
        function addDownloadButton(spanElement, pageUrl) {
            // Create the button
            const button = document.createElement('button');
            button.innerText = 'DOWNLOAD';
            button.style.marginLeft = '10px';
            button.style.padding = '5px 10px';
            button.style.cursor = 'pointer';
    
            // Add click event listener
            button.addEventListener('click', async () => {
                button.disabled = true;
                button.innerText = 'Processing...';
                try {
                    await downloadImages(pageUrl);
                    button.innerText = 'DOWNLOAD';
                    button.disabled = false;
                    alert('Download ready.');
                } catch (error) {
                    console.error(error);
                    alert('An error occurred during the download process.');
                    button.innerText = 'DOWNLOAD';
                    button.disabled = false;
                }
            });
    
            // Append the button to the span element
            spanElement.appendChild(button);
        }
    
        // Main function to handle the download process
        async function downloadImages(initialPageUrl) {
            // Fetch the initial page
            const initialDoc = await fetchHTML(initialPageUrl);
    
            // Extract TITLE from div.jianjie
            const jianjieDiv = initialDoc.querySelector('div.jianjie');
            if (!jianjieDiv) throw new Error('div.jianjie not found.');
            const jianjieText = jianjieDiv.textContent;
            const titleMatch = jianjieText.match(/\[ (.*?)P\]/);
            if (!titleMatch || !titleMatch[1]) throw new Error('TITLE not found.');
            const TITLE = titleMatch[1].trim();
    
            // Initialize JSZip
            const zip = new JSZip();
    
            // Extract all related PageURLs from div.page
            const pageDiv = initialDoc.querySelector('div.page');
            if (!pageDiv) throw new Error('div.page not found.');
            const pageLinks = pageDiv.querySelectorAll('a[href]');
            const pageUrls = Array.from(pageLinks).map(link => {
                let href = link.getAttribute('href');
                // Ensure the href is absolute
                if (!href.startsWith('http')) {
                    // Assuming relative links, prepend the origin
                    href = new URL(href, initialPageUrl).href;
                }
                return href;
            });
    
            // Limit concurrent fetches to avoid overloading
            const CONCURRENT_LIMIT = 5;
            let index = 0;
    
            async function processNext() {
                if (index >= pageUrls.length) return;
                const currentIndex = index++;
                const pageUrl = pageUrls[currentIndex];
                try {
                    const pageDoc = await fetchHTML(pageUrl);
                    const contentDiv = pageDoc.querySelector('div.content');
                    if (contentDiv) {
                        const paragraphs = contentDiv.querySelectorAll('p');
                        paragraphs.forEach(p => {
                            const imgs = p.querySelectorAll('img[src]');
                            imgs.forEach(img => {
                                images.push(img.src);
                            });
                        });
                    }
                } catch (e) {
                    console.warn(`Failed to process ${pageUrl}: ${e.message}`);
                }
                await processNext();
            }
    
            // Collect all image URLs
            const images = [];
            const workers = [];
            for (let i = 0; i < CONCURRENT_LIMIT; i++) {
                workers.push(processNext());
            }
            await Promise.all(workers);
    
            if (images.length === 0) throw new Error('No images found.');
    
            // Fetch all images and add to zip
            let counter = 0;
            const total = images.length;
            for (const imgUrl of images) {
                try {
                    const blob = await fetchImage(imgUrl);
                    const extension = imgUrl.split('.').pop().split(/\#|\?/)[0] || 'jpg';
                    const filename = `${String(counter).padStart(3, '0')}.${extension}`;
                    zip.file(filename, blob);
                    counter++;
                    // Optional: Update progress here
                } catch (e) {
                    console.warn(`Failed to fetch image ${imgUrl}: ${e.message}`);
                }
            }
    
            if (counter === 0) throw new Error('No images were successfully downloaded.');
    
            // Generate the zip file
            const zipBlob = await zip.generateAsync({ type: 'blob' });
    
            // Trigger the download using a temporary link
            const downloadLink = document.createElement('a');
            downloadLink.href = URL.createObjectURL(zipBlob);
            downloadLink.download = `${TITLE}.zip`;
            document.body.appendChild(downloadLink);
            downloadLink.click();
            document.body.removeChild(downloadLink);
        }
    
        // Locate all target span elements and add DOWNLOAD buttons
        function initialize() {
            const spanElements = document.querySelectorAll('div.title h2 a span');
            spanElements.forEach(span => {
                // Avoid adding multiple buttons
                if (span.querySelector('button') === null) {
                    const aTag = span.parentElement;
                    if (aTag && aTag.href) {
                        // Construct absolute PageURL
                        const pageUrl = aTag.href.startsWith('http') ? aTag.href : new URL(aTag.getAttribute('href'), window.location.origin).href;
                        addDownloadButton(span, pageUrl);
                    }
                }
            });
        }
    
        // Wait for the DOM to load before initializing
        if (document.readyState === 'loading') {
            document.addEventListener('DOMContentLoaded', initialize);
        } else {
            initialize();
        }
    
    })();
    

    Explanation of Key Components:

    1. @grant GM_download:

      • Grants permission to use GM_download if you prefer using it for downloads. In this script, we handle downloads via temporary links, but you can modify the script to use GM_download if desired.
    2. Concurrency Limiting:

      • To prevent overwhelming the server or running into rate limits, the script limits concurrent page fetches to 5 (CONCURRENT_LIMIT). You can adjust this number based on the server's capabilities.
    3. Progress Feedback:

      • While not fully implemented, you can enhance the script by providing real-time progress updates to the user, especially if dealing with a large number of images.
    4. Error Handling:

      • The script includes basic error handling to notify users of issues during the download process. This can be further enhanced based on specific requirements.
    5. File Naming:

      • Images are renamed sequentially starting from 000 (e.g., 000.jpg, 001.png, etc.). The script preserves the original image file extensions.
    6. ZIP File Naming:

      • The ZIP archive is named after the extracted TITLE from div.jianjie and suffixed with .zip.

    Usage Instructions:

    1. Install Violentmonkey:

    2. Create a New Script:

      • Open the Violentmonkey dashboard.
      • Click on the + icon to create a new script.
      • Paste the above script into the editor.
    3. Save the Script:

      • Save the script and ensure it's enabled.
    4. Navigate to Target Websites:

      • Visit any of the specified URLs (e.g., http://d15.876515.xyz/somepage).
    5. Use the DOWNLOAD Button:

      • Locate the DOWNLOAD button added next to the relevant content.
      • Click the button to initiate the image download process.
    6. Download the ZIP:

      • After processing, a ZIP file named after the extracted TITLE will be automatically downloaded to your device.

    Notes:

    • CORS Considerations:

      • The script assumes that all image URLs are accessible and do not have restrictive CORS policies. If images are hosted on domains with strict CORS settings, fetching them might fail.
    • Performance:

      • Downloading a large number of images can be resource-intensive. Ensure your device has sufficient resources and allow some time for the process to complete.
    • Script Customization:

      • Depending on the actual structure of your target websites, you might need to adjust the selectors (e.g., 'div.jianjie', 'div.content') to match the exact HTML structure.
    • Security:

      • Always review and understand userscripts before installing them, especially those that interact with multiple web pages and handle file downloads.

    Conclusion

    This userscript should effectively automate the process of downloading and packaging images from your specified WebURLs into a single ZIP file. Feel free to customize and enhance the script further based on your specific needs and the structure of the target websites.