Create a Python script to automate backups with version control.
- User
Create a Python script to automate backups with version control.
- Assistant
Designing version control options
Since the user mentioned "with version control," I'm planning to create a script that leverages Git for backups. The idea is to mirror the source into a backup directory and run Git commands for tracking changes. However, I wonder if it’s better to implement a custom versioning system using date-stamped directories instead. This approach would allow backups with changed files stored via hardlinks or diffs, but it adds complexity. It might be wise to offer both options: a simple timestamped backup and optional Git integration.Implementing incremental versioning
I’m considering an alternative where I implement incremental versioning using checksums and symlinks, but that might be lengthy. Instead, I’ll create a Python script called "backup_with_versions.py."
For each run, it will copy the source directory into a timestamped destination folder, like "2025-11-22_153000." I can add an option to limit the last N backups, which will give it a versioned feel, even if it's more about backup rotation. I might also integrate a Git repo inside the backup root to track versions, although that seems heavy for large folders. I’ll finalize with timestamped snapshots, optional compression, and scheduling tips using cron or Task Scheduler.Creating an OS-agnostic backup script
The user asked for a Python script to automate backups with version control, but they didn’t specify the environment or location. I’ll aim for something OS-agnostic. My plan is to create a single script with a configuration area at the top to define
SOURCE_DIRS,BACKUP_ROOT,MAX_BACKUPS, andCOMPRESS.I think using argparse for command-line options is a better approach, along with a potential YAML config for advanced users. I’ll design the script to include essential features like timestamping and compression for backups using
shutil.Implementing backup logicAlright, I’m thinking about how to set the destination directory with a timestamp using
shutil.copytree. For retention, I’ll list items matching a specific prefix, sort them, and delete those exceeding the maximum backups. If the user opts for--use-git, I need to ensure the destination is a Git repository and initialize it only if it’s not empty.Committing backups after each run makes sense for tracking, even if it just adds new tarballs. I’ll handle edge cases like permission errors and also implement minimal logging while keeping the script clean and straightforward.Crafting a backup automation script
I’m planning to create a backup script using only the standard library, including modules like os, shutil, logging, datetime, subprocess, and argparse. I'll provide cross-platform scheduling instructions for both cron and Windows Task Scheduler, even though the user didn't explicitly ask for that.
The script will have a brief introduction, the main code, and a summary of how to use it. Key functions will utilize shutil for copying and archiving, and I’ll implement retention logic to manage backup versions. Additionally, if the
--use-gitoption is included, I can ensure the destination is a Git repository and commit backups as needed. Finally, I'll ensure the script is organized and adheres to expected conventions, making it user-friendly and efficient. Below is a standalone Python script that:- Backs up a directory to a destination folder.
- Creates clearly versioned backups using timestamps.
- Optionally:
- Compresses each backup into a
.tar.gzarchive. - Uses Git in the backup directory to keep a version-controlled history.
- Limits the number of retained versions.
- Compresses each backup into a
#!/usr/bin/env python3 """ backup_with_versions.py Automate backups with simple versioning and optional Git-based version control. Features: - Timestamped backup versions. - Optional compression (.tar.gz). - Optional Git repository in the backup directory (each run = a commit). - Retention policy to keep only the latest N versions. Usage examples: python backup_with_versions.py /path/to/source /path/to/backups python backup_with_versions.py /src/dir /backups --compress --max-versions 10 python backup_with_versions.py /src/dir /backups --use-git --git-push origin main """ import argparse import datetime import os import shutil import subprocess import sys def parse_args(): parser = argparse.ArgumentParser( description="Automate backups with timestamped versions and optional Git." ) parser.add_argument("source", help="Source directory to back up") parser.add_argument("destination", help="Backup root directory") parser.add_argument( "--compress", action="store_true", help="Store each backup as a compressed .tar.gz archive", ) parser.add_argument( "--max-versions", type=int, default=0, help=( "Maximum number of backup versions to keep (0 or negative = keep all). " "Oldest versions are deleted when the limit is exceeded." ), ) parser.add_argument( "--use-git", action="store_true", help="Treat the destination as a Git repo and commit after each backup", ) parser.add_argument( "--git-push", nargs=2, metavar=("REMOTE", "BRANCH"), help="After committing, push to the given Git remote and branch " "(requires --use-git). Example: --git-push origin main", ) return parser.parse_args() def ensure_dir(path: str): os.makedirs(path, exist_ok=True) def make_timestamp() -> str: # YYYYMMDD-HHMMSS return datetime.datetime.now().strftime("%Y%m%d-%H%M%S") def backup_directory(source: str, destination: str, compress: bool, timestamp: str) -> str: """ Create one backup of `source` inside `destination`. Returns the path to the created backup (dir or archive). """ source = os.path.abspath(source) destination = os.path.abspath(destination) # Name prefix derived from source directory name src_name = os.path.basename(os.path.normpath(source)) if not src_name: src_name = "root" ensure_dir(destination) if compress: # Example: /backups/myproject_20251122-153000.tar.gz base_name = f"{src_name}_{timestamp}" archive_base = os.path.join(destination, base_name) print(f"[INFO] Creating compressed archive: {archive_base}.tar.gz") shutil.make_archive(archive_base, "gztar", root_dir=source) return archive_base + ".tar.gz" else: # Example: /backups/myproject_20251122-153000/ backup_dir = os.path.join(destination, f"{src_name}_{timestamp}") print(f"[INFO] Copying directory to: {backup_dir}") shutil.copytree(source, backup_dir) return backup_dir def enforce_retention(destination: str, source: str, compress: bool, max_versions: int): """ Keep only the latest `max_versions` backups for this source. If max_versions <= 0, do nothing. """ if max_versions <= 0: return destination = os.path.abspath(destination) src_name = os.path.basename(os.path.normpath(source)) or "root" prefix = f"{src_name}_" try: entries = os.listdir(destination) except FileNotFoundError: return # Filter backups for this source if compress: backups = [ name for name in entries if name.startswith(prefix) and name.endswith(".tar.gz") ] else: backups = [ name for name in entries if name.startswith(prefix) and os.path.isdir(os.path.join(destination, name)) ] if len(backups) <= max_versions: return backups.sort() # lexicographic sort works with the timestamp naming to_delete = backups[:-max_versions] for name in to_delete: path = os.path.join(destination, name) print(f"[INFO] Removing old backup: {path}") try: if os.path.isdir(path): shutil.rmtree(path) else: os.remove(path) except Exception as e: print(f"[WARN] Failed to remove {path}: {e}", file=sys.stderr) def ensure_git_repo(path: str): """ Ensure that `path` is a Git repository; initialize if needed. """ git_dir = os.path.join(path, ".git") if os.path.isdir(git_dir): return print(f"[INFO] Initializing Git repository in {path}") subprocess.run(["git", "init"], cwd=path, check=True) def git_commit_all(path: str, message: str): """ Stage all changes and commit. If there are no changes, do not fail. """ print("[INFO] Staging all changes for commit") subprocess.run(["git", "add", "-A"], cwd=path, check=True) print(f"[INFO] Committing with message: {message!r}") result = subprocess.run( ["git", "commit", "-m", message], cwd=path, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, ) if result.returncode != 0: # Usually means "nothing to commit, working tree clean" print("[INFO] No changes to commit:") print(result.stdout.strip()) else: print(result.stdout.strip()) def git_push(path: str, remote: str, branch: str): print(f"[INFO] Pushing to {remote} {branch}") subprocess.run(["git", "push", remote, branch], cwd=path, check=True) def main(): args = parse_args() source = os.path.abspath(os.path.expanduser(args.source)) destination = os.path.abspath(os.path.expanduser(args.destination)) if not os.path.isdir(source): print(f"[ERROR] Source directory does not exist: {source}", file=sys.stderr) sys.exit(1) timestamp = make_timestamp() # 1. Create backup try: backup_path = backup_directory( source=source, destination=destination, compress=args.compress, timestamp=timestamp, ) except Exception as e: print(f"[ERROR] Backup failed: {e}", file=sys.stderr) sys.exit(1) print(f"[INFO] Backup created at: {backup_path}") # 2. Enforce retention try: enforce_retention( destination=destination, source=source, compress=args.compress, max_versions=args.max_versions, ) except Exception as e: print(f"[WARN] Retention enforcement failed: {e}", file=sys.stderr) # 3. Optional Git version control in the destination directory if args.use_git: try: ensure_git_repo(destination) commit_msg = f"Auto backup of {source} at {timestamp}" git_commit_all(destination, commit_msg) if args.git_push: remote, branch = args.git_push git_push(destination, remote, branch) except FileNotFoundError as e: print( f"[ERROR] Git not found. Install Git or remove --use-git / --git-push.\n{e}", file=sys.stderr, ) sys.exit(1) except subprocess.CalledProcessError as e: print(f"[ERROR] Git command failed: {e}", file=sys.stderr) sys.exit(1) if __name__ == "__main__": main()
How to use it
- Save as
backup_with_versions.pyand make it executable (optional on Unix):
chmod +x backup_with_versions.py- Run a basic backup:
python backup_with_versions.py /path/to/source /path/to/backups- With compression and keep only last 7 versions:
python backup_with_versions.py /path/to/source /path/to/backups \ --compress --max-versions 7- With Git-based version control in the backup directory:
python backup_with_versions.py /path/to/source /path/to/backups \ --compress --max-versions 20 --use-gitAfter you configure a remote in
/path/to/backups, you can also auto-push:python backup_with_versions.py /path/to/source /path/to/backups \ --compress --max-versions 20 --use-git --git-push origin mainIf you tell me your OS and how often you want this to run, I can give you a ready-to-paste cron entry (Linux/macOS) or Task Scheduler command (Windows) to automate it.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.