Skip to content
The Chat Archive
hunyuan-turbos-20250416 LMArena Human Preference 140K

Write python script. Script should read llms.txt line by line like this:…

  1. User

    Write python script. Script should read llms.txt line by line like this: https://abonnement-iptv-france.fr/llms.txt https://absoluteinternship.com/llms.txt https://aceesaay.ai/llms-full.txt https://aceesaay.ai/llms.txt https://adsbravo.com/llms.txt And submit to context7.com: curl 'https://context7.com/api/v1/add'
    --compressed
    -X POST
    -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:142.0) Gecko/20100101 Firefox/142.0'
    -H 'Accept: /'
    -H 'Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3'
    -H 'Accept-Encoding: gzip, deflate, br, zstd'
    -H 'Referer: https://context7.com/add-library?tab=llmstxt'
    -H 'Content-Type: application/json'
    -H 'Origin: https://context7.com'
    -H 'Connection: keep-alive'
    -H 'Sec-Fetch-Dest: empty'
    -H 'Sec-Fetch-Mode: cors'
    -H 'Sec-Fetch-Site: same-origin'
    -H 'Priority: u=4'
    --data-raw '{"llmstxtUrl":"https://abonnement-iptv-france.fr/llms.txt"}'

    Example responses: {"message":"Parsing process started for llmstxt:https://abonnement-iptv-france.fr/llms.txt","status":"processing","docsRepoUrl":"https://abonnement-iptv-france.fr/llms.txt","projectName":"/llmstxt/abonnement-iptv-france_fr-llms.txt","description":"","projectTitle":"auto-detected"}

    {"error":"Too Many Requests","message":"Rate limit exceeded. Please try again in 829 seconds.","configName":"add-library-success","limit":3,"reset":"2025-07-24T00:00:00.000Z"}

    {"message":"Too many concurrent parsing processes. Try again in a few minutes."}

    Retry on too many requests or too many concurrent parsing processes until success, skip if response is unknown, save progress to file, add resuming functional

  2. Assistant

    Here's a complete Python script that:

    ✅ Reads URLs from a local file llms.txt (one URL per line)
    ✅ Submits each URL to https://context7.com/api/v1/add via POST request
    ✅ Retries on "Too Many Requests" or "Too many concurrent parsing processes" with exponential backoff
    ✅ Skips unknown error responses
    ✅ Saves submission progress to a state file (progress.json)
    ✅ Supports resuming from the last successful submission on restart


    ✅ Features Recap

    • Resumable: Won't re-submit already successful URLs
    • Retry logic: For rate limiting and concurrency issues
    • Logs: Prints status of each submission
    • Saves state: So you can stop and resume safely

    🐍 Python Script: submit_to_context7.py

    import requests
    import time
    import json
    import os
    from time import sleep
    
    # Constants
    TARGET_URL = 'https://context7.com/api/v1/add'
    HEADERS = {
        'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:142.0) Gecko/20100101 Firefox/142.0',
        'Accept': '*/*',
        'Accept-Language': 'ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3',
        'Accept-Encoding': 'gzip, deflate, br, zstd',
        'Referer': 'https://context7.com/add-library?tab=llmstxt',
        'Content-Type': 'application/json',
        'Origin': 'https://context7.com',
        'Connection': 'keep-alive',
        'Sec-Fetch-Dest': 'empty',
        'Sec-Fetch-Mode': 'cors',
        'Sec-Fetch-Site': 'same-origin',
        'Priority': 'u=4',
    }
    RETRY_ERRORS = [
        "Too Many Requests",
        "Too many concurrent parsing processes"
    ]
    MAX_RETRIES = 10
    RETRY_DELAY = 10  # Initial delay in seconds
    PROGRESS_FILE = 'progress.json'
    
    
    def load_progress():
        """Load previously successful URLs."""
        if os.path.exists(PROGRESS_FILE):
            with open(PROGRESS_FILE, 'r') as f:
                return set(json.load(f))
        return set()
    
    
    def save_progress(successful_urls):
        """Save currently successful URLs."""
        with open(PROGRESS_FILE, 'w') as f:
            json.dump(list(successful_urls), f, indent=2)
    
    
    def make_request(url):
        payload = {
            "llmstxtUrl": url
        }
        for attempt in range(1, MAX_RETRIES + 1):
            try:
                response = requests.post(
                    TARGET_URL,
                    headers=HEADERS,
                    json=payload,
                    timeout=30
                )
                data = response.json()
    
                # Check for known retryable errors
                if any(error_msg in data.get("message", "") for error_msg in RETRY_ERRORS):
                    wait_time = RETRY_DELAY * (2 ** (attempt - 1))  # Exponential backoff
                    print(f"[{url}] Retryable error: {data.get('message')}. Retrying in {wait_time}s...")
                    time.sleep(wait_time)
                    continue
    
                # Success
                if data.get("message") and "Parsing process started" in data["message"]:
                    print(f"[{url}] Successfully submitted.")
                    return True
    
                # Unknown response
                print(f"[{url}] Unknown response: {data}")
                return False
    
            except Exception as e:
                print(f"[{url}] Request error: {e}")
                time.sleep(RETRY_DELAY * attempt)
        print(f"[{url}] Max retries reached. Skipping.")
        return False
    
    
    def main():
        input_file = 'llms.txt'
        if not os.path.exists(input_file):
            print(f"Error: {input_file} not found.")
            return
    
        urls = [line.strip() for line in open(input_file).readlines() if line.strip()]
        print(f"Loaded {len(urls)} URLs.")
    
        successful = load_progress()
        new_successful = set()
    
        for url in urls:
            if url in successful:
                print(f"[{url}] Already processed. Skipping.")
                continue
    
            print(f"[{url}] Submitting...")
            success = make_request(url)
    
            if success:
                new_successful.add(url)
    
        # Update progress
        if new_successful:
            successful.update(new_successful)
            save_progress(successful)
            print(f"Progress saved. Total submitted: {len(successful)}")
    
        print("All done.")
    
    
    if __name__ == '__main__':
        main()
    

    📝 How to Use

    1. Save the above script as submit_to_context7.py
    2. Create a text file named llms.txt in the same directory and paste your URLs:
    https://abonnement-iptv-france.fr/llms.txt
    https://absoluteinternship.com/llms.txt
    https://aceesaay.ai/llms-full.txt
    https://aceesaay.ai/llms.txt
    https://adsbravo.com/llms.txt
    
    1. Run the script:
    python submit_to_context7.py
    
    1. To resume later, just run the script again — it will skip already completed URLs.

    Let me know if you want to auto-discover new URLs from the files themselves instead of using a static list 😄

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.