Notes : Nb1 Now that you have gained a bit of practical experience, it is time to fully…
- User
Notes : Nb1
Now that you have gained a bit of practical experience, it is time to fully understand the syntax for workflows. Since workflows are written in YAML, it is important to fully understand YAML before writing workflows.
What is YAML
- YAML, which stands for "YAML Ain't Markup Language," is a data-serialization language optimized for human readability and writability. It's a strict superset of JSON but uses syntactically relevant newlines and indentation instead of braces.
YAML BASIC
To start, it’s essential to understand that YAML files have a
.ymlor.yamlextension, which makes them well-suited for version control systems like Git. This is because every change is tracked per line, allowing for efficient versioning.# A full-line comment in YAML key: value # An inline commentComments can appear anywhere in a line
Data Types :
Alright, here’s the same content in English:
In YAML, there are many data types, both primitive (simple scalars) and complex.
For example:
Scalar Types
In YAML, we can assign a value to a variable using this syntax:
key: valueHere, the
keyrepresents the variable name, and thevalueis its assigned type or value.To get a better understanding of the context in a project, let's look at this example:
name: String job: String age: int address: - zip_code - address_name - numero datetime: 1999-12-31T23:59:43.1ZNote that keys and values in YAML can contain spaces and do not require quotation marks! You can use single or double quotes for keys and values, but this is only necessary if they contain special characters or if YAML might misinterpret their data type.
- Double quotes (
" ") use the backslash (\) as the escape character. - Single quotes (
' ') use an additional single quote ('') for escaping.
There are cases when we want to write in multiple lines. In YAML syntax, this is done using the pipe operator and a four-space indentation, like in a literal block.
description: | This is a description that spans multiple lines. You can write as much as you want here, and each new line will be preserved in the output. - Lists can also be included within the block. - Each line will start with an indentation.This makes writing complex scripts in YAML workflows much easier than in other formats where you must quote variables
When defining complex data types, YAML provides two types of collections: maps and lists (also known as sequences).
Maps
Maps use two spaces of indentation with the same syntax as assigning variables:
parent: key1: value1 key2: value2 child: subkey1: subvalue1 subkey2: subvalue2Lists (Sequences)
Lists are created using hyphens:
fruits: - apple - banana - cherry nested_list: - item1 - item2: - subitem1 - subitem2Now that we've gotten a glimpse of what YAML can look like, let's return to some interesting topics.
Workflows
Now, we'll apply what we've learned so far in the context of GitHub Actions and build a foundation to tackle workflows.
The Workflow Syntax
As we know, the first element to start with when tackling a workflow in GitHub Action is the
nameelement. A simple but important piece of information is that the name of the workflow can be different from the name of the file. For example, the file can be calledmain.yml, while the workflow is defined as follows:name: My First WorkflowEvents and Triggers
Events and triggers play a central role in building a robust workflow. Identifying when, or based on which events, to trigger the pipeline is a difficult decision to make.
In this section, GitHub Actions provides us with three main categories of triggers.
- Webhook triggers
- Scheduled triggers
- Manual triggers
each triggers follow the key “on : “ in the workflow file
Webhook triggers
Here, webhook triggers work within the context of events in GitHub. For example:
Imagine you're working in a complex system with teammates from different team structures, and you want the workflow to start based on a webhook that triggers on a
git pushto the repository.To achieve this, you can simply add this line of code in your workflow:
on: pushit can also be a pull request in the repo
on:[push,pull_request]One of the nice features of webhook triggers is that they can be configured based on certain conditions. For example, you can start a workflow only when pushing to specific "branches" or when certain files in a path have been updated.
For example, in this scenario:
on: push: branches: - main # Trigger only when pushing to the 'main' branch paths: - 'src/**' # Trigger only if files in the 'src/' directory have been modified- We only trigger the pipeline when changes are pushed to the "main" branch.
- We also only trigger the workflow if files in the 'src/' directory have been modified.
These are just some examples of the most frequently used webhook triggers. You can also use many others, allowing you to customize your workflow to better fit your unique use case
Scheduled triggers
Here we have the schedule triggers. As the name suggests, these triggers start the workflow at a scheduled time—they use the same syntax as cron jobs. The syntax consists of five fields:
- Minute (0-59)
- Hour (0-23)
- Day of month (1-31)
- Month (1-12 or JAN-DEC)
- Day of week (0-6 or SUN-SAT)
an example of a schedule trigger in a GitHub Actions workflow:
on: schedule: # Runs every 16th minute - cron: '*/16 * * * *' # Runs every hour from 10 AM to 3 PM - cron: '0 10-15 * * *' # Runs every Friday at midnight - cron: '0 0 * * FRI' # Runs every quarter (at 00:00 on day 1 of every 3rd month) - cron: '0 0 1 */3 *'As you can see here, we can combine multiple scheduling options in the same workflow, which can be helpful for running the workflow in batch mode
Manual Triggers
Finally, the last type is Manual Triggers. These allow you to start a workflow manually.
The syntax for a manual trigger can look like this:
on: workflow_dispatchIn this case, the trigger has one input: the branch the workflow runs on. The default branch is usually the
mainbranch. You can use the GitHub UI to trigger the workflow manually, where you will be presented with a dialog to start the workflow."[image: image.png] open
You can also trigger the GitHub workflow using the GitHub CLI, either by name, ID, or relative file name to
.github/workflows:$ gh workflow run WORKFLOW_FILENAME ```"Workflow jobs and steps
Here, the logic of the workflow is defined in the
jobssection. As we know, each job is executed on a runner. The runner can either be self-hosted, or you can choose one from the cloud.Make sure to use the latest version of the runner, such as
ubuntu-latest,windows-latest, ormacos-latest, but always remember the cost factor.Operating System Price Linux $0.008 macOS $0.08 Windows $0.016 Workflow jobs
As we defined previously, jobs are the units of work here; they run in parallel by default.
You can chain them in a sequence by having a job depend on the successful output of one or multiple other jobs using the
needskeywordQ- given all the context here so far and my oun khnwolde
Q- i just create a github Repo like this
https://github.com/AmirUpSkill/GitHub-Action-HandsOn.git
This is my repository to test proof of concept GitHub Action workflows in different stacks (FastAPI, Node.js, Spring Boot, and more)
Q- as you can see we need to build a simple poc workflow okii
Q- with different stack the first one is fast api okii
Q- okii so help me to creat this tutor
Q- use step by step style form clone the repo
Q- to create the FastApi-Workflow
Q- initate the python venv venv
Q- install the packgest need it herre
Q- freeze them inside the requirements.txt
Q- create simple folder strutre
Q- create a simple rest api
Q- and create a simple unit test to this api
Q- then create main workflow
Q- and show my how this workflow diclant when just change the resturn string from github
form “Hello , This A Simple Prof Of Concept V1” to become
“Hello , This A Simple Prof Of Concept V1”
Q- do it at software level engineer oki
Comprehensive Guide: Setting Up a GitHub Actions Workflow for a FastAPI Project
This guide will walk you through cloning your repository, setting up the environment, creating a simple REST API, writing unit tests, and configuring a GitHub Actions workflow to automate your tasks.
Table of Contents
- Prerequisites
- Step 1: Clone the Repository
- Step 2: Set Up a Python Virtual Environment
- Step 3: Install Required Packages
- Step 4: Freeze Dependencies
- Step 5: Create Folder Structure
- Step 6: Develop a Simple REST API
- Step 7: Write Unit Tests for the API
- Step 8: Commit and Push Your Changes
- Step 9: Create the GitHub Actions Workflow
- Step 10: Trigger and Observe the Workflow
- Conclusion
Prerequisites
Before we begin, ensure you have the following installed on your local machine:
- Git: Download Git
- Python 3.7+: Download Python
- GitHub CLI (optional): Download GitHub CLI
Step 1: Clone the Repository
First, clone your GitHub repository to your local machine.
git clone https://github.com/AmirUpSkill/GitHub-Action-HandsOn.git cd GitHub-Action-HandsOn
Step 2: Set Up a Python Virtual Environment
Creating a virtual environment ensures that your project dependencies are isolated.
-
Create a virtual environment named
venv:python3 -m venv venv -
Activate the virtual environment:
-
On macOS/Linux:
source venv/bin/activate -
On Windows:
venv\Scripts\activate
You should see
(venv)prefixed in your terminal, indicating that the virtual environment is active. -
Step 3: Install Required Packages
For a basic FastAPI application, you'll need to install FastAPI and Uvicorn. Additionally, for testing, we'll use
pytest.pip install fastapi uvicorn pytest
Step 4: Freeze Dependencies
Freezing your dependencies ensures that anyone cloning your repository can install the exact same packages.
pip freeze > requirements.txtYour
requirements.txtshould look something like this:fastapi==0.95.2 uvicorn==0.22.0 pytest==7.3.1Note: Versions may vary based on the latest releases.
Step 5: Create Folder Structure
Organize your project with a clear folder structure. Here's a simple structure to get started:
GitHub-Action-HandsOn/ ├── app/ │ ├── main.py │ └── __init__.py ├── tests/ │ ├── test_main.py │ └── __init__.py ├── requirements.txt └── .github/ └── workflows/ └── main.ymlCreate the necessary directories:
mkdir app tests mkdir -p .github/workflowsCreate empty
__init__.pyfiles to make Python treat these directories as packages:touch app/__init__.py tests/__init__.pyQ- okii i just create what you have create but i get this error
venv) PS C:\Users\PCS\Desktop\Github\GitHub-Action-HandsOn\FastApi-Workflow> pytest
========================================================== test session starts ========================================================== platform win32 -- Python 3.11.5, pytest-8.3.3, pluggy-1.5.0 rootdir: C:\Users\PCS\Desktop\Github\GitHub-Action-HandsOn\FastApi-Workflow plugins: anyio-4.6.2.post1 collected 1 item
tests\test_main.py . [100%]
=========================================================== 1 passed in 0.32s ===========================================================
- : Le terme «*» n'est pas reconnu comme nom d'applet de commande, fonction, fichier de script ou programme exécutable. Vérifiez l'orthographe du nom, ou si un chemin d'accès existe, vérifiez que le chemin d'accès est correct et réessayez. Au caractère Ligne:2 : 1
-
- ~
- CategoryInfo : ObjectNotFound: (*:String) [], CommandNotFoundException
- FullyQualifiedErrorId : CommandNotFoundException
Step 6: Develop a Simple REST API
Let's create a simple FastAPI application with one endpoint.
-
Create
app/main.py:# app/main.py from fastapi import FastAPI app = FastAPI() @app.get("/") def read_root(): return {"message": "Hello, This is a Simple Proof Of Concept V1"} -
Run the application locally to test:
uvicorn app.main:app --reloadNavigate to http://127.0.0.1:8000 in your browser. You should see:
{ "message": "Hello, This is a Simple Proof Of Concept V1" }To stop the server, press
Ctrl+Cin the terminal.
Step 7: Write Unit Tests for the API
We'll use
pytestalong withhttpxfor making HTTP requests in tests.-
Install
httpx:pip install httpxUpdate
requirements.txt:pip freeze > requirements.txtYour
requirements.txtshould now include:fastapi==0.95.2 uvicorn==0.22.0 pytest==7.3.1 httpx==0.24.0 -
Create
tests/test_main.py:# tests/test_main.py from fastapi.testclient import TestClient from app.main import app client = TestClient(app) def test_read_root(): response = client.get("/") assert response.status_code == 200 assert response.json() == {"message": "Hello, This is a Simple Proof Of Concept V1"} -
Run the Tests Locally:
pytestYou should see output indicating that the test passed:
============================== test session starts ============================== platform darwin -- Python 3.9.7, pytest-7.3.1, pluggy-1.0.0 collected 1 item tests/test_main.py . [100%] ================================ 1 passed in 0.XXs ================================
tep 8: Commit and Push Your Changes
-
Initialize Git (if not already initialized):
git init -
Add Remote Repository (if not already added):
git remote add origin https://github.com/AmirUpSkill/GitHub-Action-HandsOn.git -
Add All Files and Commit:
git add . git commit -m "Initial commit: Set up FastAPI project with tests" -
Push to GitHub:
git branch -M main git push -u origin main
Step 9: Create the GitHub Actions Workflow
Now, let's set up a GitHub Actions workflow to automate testing every time you push changes to the repository.
-
Create Workflow File: Create a file at
.github/workflows/main.ymlwith the following content:# .github/workflows/main.yml name: FastAPI CI on: push: branches: [ main ] pull_request: branches: [ main ] jobs: build: runs-on: ubuntu-latest strategy: matrix: python-version: [3.9] steps: - name: Checkout repository uses: actions/checkout@v3 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v4 with: python-version: ${{ matrix.python-version }} - name: Install dependencies run: | python -m venv venv source venv/bin/activate pip install --upgrade pip pip install -r requirements.txt - name: Run Tests run: | source venv/bin/activate pytest - name: Lint with Flake8 run: | source venv/bin/activate pip install flake8 flake8 app/ tests/Explanation of the Workflow:
- Name: FastAPI CI – The name of the workflow.
- Triggers (on): Runs on every push and pull request to the main branch.
- Job: Named
buildrunning on the latest Ubuntu runner. - Strategy Matrix: Specifies Python version 3.9.
- Steps:
- Checkout repository: Uses GitHub's checkout action to clone your repository.
- Set up Python: Sets up Python 3.9.
- Install dependencies: Creates a virtual environment, activates it, upgrades pip, and installs dependencies from
requirements.txt. - Run Tests: Activates the virtual environment and runs
pytest. - Lint with Flake8: (Optional) Installs
flake8and lints the code for any style issues.
-
Commit and Push the Workflow:
git add .github/workflows/main.yml git commit -m "Add GitHub Actions workflow for CI" git push
Q- i get this error in my build
“
1sCheckout repository1sSet up Python 3.90sInstall dependencies4s1Run python -m venv venv 14Requirement already satisfied: pip in ./venv/lib/python3.9/site-packages (23.0.1) 15Collecting pip 16 Downloading pip-24.3.1-py3-none-any.whl (1.8 MB) 17 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 36.8 MB/s eta 0:00:00 18Installing collected packages: pip 19 Attempting uninstall: pip 20 Found existing installation: pip 23.0.1 21 Uninstalling pip-23.0.1: 22 Successfully uninstalled pip-23.0.1 23Successfully installed pip-24.3.1 24ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt' 25Error: Process completed with exit code 1.Run Tests0sLint with Flake80sPost Set up Python 3.90sPost Checkout repository0sComplete job0s
(venv) PS C:\Users\PCS\Desktop\Github\GitHub-Action-HandsOn> python --version
Python 3.11.5
- Assistant
It looks like you're encountering an issue where the GitHub Actions workflow cannot locate the
requirements.txtfile during the build process. This is a common issue related to the file's path within the repository and how the workflow is configured to access it.Let's walk through the steps to diagnose and resolve this problem.
1. Understanding the Issue
Error Message:
ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'Possible Causes:
- Location of
requirements.txt: The file might not be in the root directory where the workflow expects it. - Incorrect Workflow Configuration: The workflow might be executing commands in a different directory.
- Uncommitted
requirements.txt: The file hasn't been committed to the repository.
2. Step-by-Step Resolution
Step 1: Verify the Location of
requirements.txt-
Check Repository Structure:
Ensure that
requirements.txtis located in the root of your repository. Based on your description, your repository should look like this:GitHub-Action-HandsOn/ ├── app/ │ ├── main.py │ └── __init__.py ├── tests/ │ ├── test_main.py │ └── __init__.py ├── requirements.txt └── .github/ └── workflows/ └── main.yml -
If
requirements.txtis Inside a Subdirectory:For example, if it's inside
FastApi-Workflow/, the structure would be:GitHub-Action-HandsOn/ ├── FastApi-Workflow/ │ ├── app/ │ │ ├── main.py │ │ └── __init__.py │ ├── tests/ │ │ ├── test_main.py │ │ └── __init__.py │ └── requirements.txt └── .github/ └── workflows/ └── main.yml
Step 2: Ensure
requirements.txtis Committed and Pushed-
Check Git Status Locally:
git statusEnsure that
requirements.txtis listed as tracked and has been committed. -
If Not Committed:
git add requirements.txt git commit -m "Add requirements.txt" git push origin main -
Verify on GitHub:
Navigate to your repository on GitHub and confirm that
requirements.txtis present in the expected directory.
Step 3: Adjust the GitHub Actions Workflow
Depending on where
requirements.txtis located, you may need to adjust the workflow.Scenario A:
requirements.txtis in the Root DirectoryIf
requirements.txtis in the root, ensure your workflow is set up correctly.Workflow (
.github/workflows/main.yml):name: FastAPI CI on: push: branches: [ main ] pull_request: branches: [ main ] jobs: build: runs-on: ubuntu-latest strategy: matrix: python-version: [3.11] steps: - name: Checkout repository uses: actions/checkout@v3 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v4 with: python-version: ${{ matrix.python-version }} - name: Install dependencies run: | python -m venv venv source venv/bin/activate pip install --upgrade pip pip install -r requirements.txt - name: Run Tests run: | source venv/bin/activate pytest - name: Lint with Flake8 run: | source venv/bin/activate pip install flake8 flake8 app/ tests/Notes:
- Ensure that
requirements.txtis in the root where the workflow expects it. - Python Version: Updated to
3.11to match your local environment.
Scenario B:
requirements.txtis Inside a Subdirectory (FastApi-Workflow/)If your
requirements.txtis insideFastApi-Workflow/, you need to adjust the workflow to point to the correct directory.Updated Workflow (
.github/workflows/main.yml):name: FastAPI CI on: push: branches: [ main ] pull_request: branches: [ main ] jobs: build: runs-on: ubuntu-latest strategy: matrix: python-version: [3.11] steps: - name: Checkout repository uses: actions/checkout@v3 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v4 with: python-version: ${{ matrix.python-version }} - name: Change to FastApi-Workflow directory run: cd FastApi-Workflow - name: Install dependencies run: | cd FastApi-Workflow python -m venv venv source venv/bin/activate pip install --upgrade pip pip install -r requirements.txt - name: Run Tests run: | cd FastApi-Workflow source venv/bin/activate pytest - name: Lint with Flake8 run: | cd FastApi-Workflow source venv/bin/activate pip install flake8 flake8 app/ tests/Alternatively, Use
working-directory:You can simplify by specifying the
working-directoryfor each step.name: FastAPI CI on: push: branches: [ main ] pull_request: branches: [ main ] jobs: build: runs-on: ubuntu-latest strategy: matrix: python-version: [3.11] defaults: run: working-directory: FastApi-Workflow steps: - name: Checkout repository uses: actions/checkout@v3 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v4 with: python-version: ${{ matrix.python-version }} - name: Install dependencies run: | python -m venv venv source venv/bin/activate pip install --upgrade pip pip install -r requirements.txt - name: Run Tests run: | source venv/bin/activate pytest - name: Lint with Flake8 run: | source venv/bin/activate pip install flake8 flake8 app/ tests/Notes:
defaults.run.working-directorysets the default directory for allruncommands.- Ensure that all paths within the workflow are relative to
FastApi-Workflow/.
Step 4: Debugging the Workflow
To further diagnose issues, it's helpful to add debugging steps to your workflow.
Add a Step to List Directory Contents:
Place this step before the
Install dependenciesstep to verify the presence ofrequirements.txt.- name: List Repository Contents run: | pwd ls -laExample Placement:
- name: Checkout repository uses: actions/checkout@v3 - name: List Repository Contents run: | pwd ls -la - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v4 with: python-version: ${{ matrix.python-version }}What This Does:
pwd: Prints the current working directory.ls -la: Lists all files and directories, including hidden ones.
Review the Workflow Logs: After triggering the workflow, review the logs for the "List Repository Contents" step to confirm the presence and location of
requirements.txt.
3. Additional Recommendations
Use
actions/setup-pythonv4 FeaturesYou can leverage
actions/setup-pythonto handle virtual environments more efficiently.Example Updated Step:
- name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v4 with: python-version: ${{ matrix.python-version }} cache: 'pip' # Enables pip caching to speed up installsSimplify Virtual Environment Activation
Instead of manually activating the virtual environment, you can use the
python -m pipcommand to install packages without activation.Modified Install Dependencies Step:
- name: Install dependencies run: | python -m venv venv python -m pip install --upgrade pip python -m pip install -r requirements.txtRun Tests Without Activation:
- name: Run Tests run: | python -m venv venv python -m pip install --upgrade pip python -m pip install -r requirements.txt pytestAdvantage:
- Avoids the need to activate the virtual environment explicitly, which can be tricky in CI environments.
Final Optimized Workflow Example
Here’s a streamlined version of the workflow assuming
requirements.txtis in the root directory:name: FastAPI CI on: push: branches: [ main ] pull_request: branches: [ main ] jobs: build: runs-on: ubuntu-latest strategy: matrix: python-version: [3.11] steps: - name: Checkout repository uses: actions/checkout@v3 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v4 with: python-version: ${{ matrix.python-version }} cache: 'pip' - name: Install dependencies run: | python -m venv venv python -m pip install --upgrade pip python -m pip install -r requirements.txt - name: Run Tests run: | python -m venv venv source venv/bin/activate pytest - name: Lint with Flake8 run: | python -m venv venv source venv/bin/activate python -m pip install flake8 flake8 app/ tests/Notes:
- Caching: The
cache: 'pip'option speeds up dependency installation by cachingpippackages. - Virtual Environment: Still creates a virtual environment for isolation.
4. Triggering the Workflow
After making the necessary changes:
-
Commit and Push the Updated Workflow:
git add .github/workflows/main.yml git commit -m "Fix workflow: Ensure requirements.txt is found" git push origin main -
Observe the GitHub Actions Build:
- Navigate to your repository on GitHub.
- Click on the "Actions" tab.
- Select the latest workflow run to monitor progress.
-
Verify Successful Execution:
The build should now locate
requirements.txtsuccessfully and proceed with installing dependencies, running tests, and linting the code.
5. Confirming the Fix
Expected Successful Workflow Output:
========================================================== test session starts ========================================================== platform linux -- Python 3.11.5, pytest-8.3.3, pluggy-1.0.0 rootdir: /home/runner/work/GitHub-Action-HandsOn/GitHub-Action-HandsOn collected 1 item tests/test_main.py . [100%] =========================================================== 1 passed in 0.32s ===========================================================No Errors Related to
requirements.txtMissing.
6. Updating the Return String (Optional)
You mentioned wanting to see how the workflow reacts to changes in the return string of your API. Here's how you can test that:
-
Modify the API Response:
In
app/main.py, change the return message:@app.get("/") def read_root(): return {"message": "Hello, This is a Simple Proof Of Concept V2"} -
Commit and Push the Changes:
git add app/main.py git commit -m "Update API response message to V2" git push origin main -
Observe the Workflow:
The GitHub Actions workflow will trigger automatically (due to the push event) and run the updated tests.
-
Update the Unit Test (If Necessary):
If your test expects
"V1", update it to match the new message.In
tests/test_main.py:def test_read_root(): response = client.get("/") assert response.status_code == 200 assert response.json() == {"message": "Hello, This is a Simple Proof Of Concept V2"} -
Commit, Push, and Verify:
git add tests/test_main.py git commit -m "Update unit test for API response V2" git push origin mainEnsure that the workflow runs successfully, passing the updated tests.
7. Final Thoughts
By ensuring that your
requirements.txtis correctly placed and your workflow is appropriately configured to locate it, you've resolved the build issue. Additionally, implementing debugging steps and optimizing your workflow enhances reliability and maintainability.Feel free to reach out if you encounter further issues or have additional questions!
- Location of