Skip to content
TheChatArchiveThe Chat Archive
gpt-4-0125-preview· openaiHugging Face Suzhen/CodeChat

Hi how to install Apache AirFlow on windows 10 in VS Code without Docker

  1. User

    Hi how to install Apache AirFlow on windows 10 in VS Code without Docker

  2. Assistant

    Installing Apache AirFlow directly on Windows without Docker can be challenging because Airflow is designed to work on Unix/Linux systems. However, you can use the Windows Subsystem for Linux (WSL) as a workaround to create a Linux environment on your Windows 10 machine. Below are the steps to install Apache AirFlow in WSL on Windows 10 within VS Code.

    Step 1: Install Windows Subsystem for Linux (WSL)

    1. Open PowerShell as Administrator and run:

      dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
      
    2. Enable Virtual Machine feature (needed for WSL 2):

      dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
      
    3. Restart your PC.

    4. Download and install the Linux kernel update package from the official Microsoft website.

    5. Set WSL 2 as your default version:

      wsl --set-default-version 2
      
    6. Install a Linux distribution via the Microsoft Store (e.g., Ubuntu).

    7. Once installed, launch your Linux distribution from the Start menu and complete the initial setup (username and password).

    Step 2: Install Apache AirFlow in WSL

    1. Open your Linux distribution via the WSL.

    2. Install Python and other required packages. It’s recommended to use a virtual environment:

      sudo apt update && sudo apt upgrade
      sudo apt install python3-pip python3-venv
      python3 -m venv airflow-venv
      source airflow-venv/bin/activate
      
    3. Install Apache Airflow using pip. As of my last knowledge update, Airflow version 2.x is the latest, but you should install a version that meets your requirements:

      pip install apache-airflow==2.3.0 --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.3.0/constraints-3.7.txt"
      
    4. Initialize the Airflow database:

      airflow db init
      
    5. Create an admin user for Airflow:

      airflow users create \
          --username admin \
          --firstname FIRST_NAME \
          --lastname LAST_NAME \
          --role Admin \
          --email <PRESIDIO_ANONYMIZED_EMAIL_ADDRESS>
      

    Step 3: Accessing AirFlow through VS Code

    1. Ensure you have VS Code and the Remote - WSL extension installed.

    2. Open VS Code, then open the Command Palette (Ctrl+Shift+P) and select "Remote-WSL: New Window".

    3. In the WSL window, navigate to your Airflow project directory.

    4. From here, you can run Airflow commands in the VS Code terminal or through the WSL terminal directly.

    Step 4: Starting Airflow

    Within your virtual environment in WSL, start the Airflow web server and scheduler in separate terminals or as background processes:

    1. Start the web server:
      airflow webserver -p 8080
      
    2. Open another terminal/session, activate the virtual environment, and start the scheduler:
      airflow scheduler
      

    You can access the Airflow web interface by navigating to localhost:8080 in your web browser.

    Note: It's important to regularly check the official Apache AirFlow documentation and the WSL documentation for any updates or changes in the installation process.