Create a Python script to automate software testing workflows.
- User
Create a Python script to automate software testing workflows.
- Assistant
Designing a Python testing script
The user is asking for a Python script to automate software testing workflows, but it's very broad. They didn't specify the framework, environment, or CI/CD details. Since they want the script directly, I need to assume a typical setup—running tests and generating reports, likely triggered through the CLI. I think it should read a configuration file in YAML or JSON format while using something like pytest or unittest. Also, I can incorporate CLI options for commands like run, report, and watch to help automate common testing tasks.Creating test runner script
I need to provide a script named
test_runner.pythat usesargparseto manage subcommands likeall,unit,integration,lint,coverage, and possiblywatchorci. I'll utilizesubprocess.runto call tools likepytest,coverage, andflake8, ensuring they log outputs and return the correct exit codes. It could also leverage thepytestAPI internally, though calling command-line tools is often easier. I’ll define configuration at the top for directory paths, ensuring the script is reusable. I'll also implement logging functions usingpathlibfor file paths.Building the test runner scriptI’m drafting a script called
test_runner.py. It will useargparseto manage commands like running all tests, or specific ones for unit, integration, linting, and coverage. I’ll implement therun_cmdfunction to execute commands, and usecoverageto runpytestand generate HTML reports. For CI environments, the--cioption will allow for JSON/JUnit outputs. I should provide minimal explanations but include helpful comments throughout the script to enhance usability and clarity. Overall, the focus will be on making the script functional and efficient.Implementing the testing workflowI'm working on a script that includes a
--ciflag to toggle output formats to JUnit XML and coverage XML for CI environments. I’ll set up the main function withargparse, creating subcommands forall, which will run all tests. There’s anadd_commonfunction to include the--cioption. The individual commands will return Boolean values, and I'll ensure proper handling of return codes withsubprocess.run. I wonder if I should include something like watchers for incremental testing; keeping it simple is key, so I'll make that optional.Designing the testing automation scriptI'm setting out to design a testing automation script that includes a feature to run tests when files change. I'll start by implementing a shebang and ensuring a proper main method. I'll also leverage
argparse, keeping compatibility in mind sinceadd_subparsersbehavior is version-dependent.Next, I’ll add logging and a
--verboseflag. For CI environments, I'll use a common parent parser for options like--ci. Additionally, I'll create utility functions for running commands, ensuring they handle errors gracefully when certain tools aren't found.For linter options, I'll support both
ruffandflake8, choosing the one that's available. I'll also include type checking withmypyas an optional step. When it comes to tests, I'll ensure flexibility with both unit and integration testing strategies. Finally, the coverage function will include generating reports and handling situations where tools may not be installed. This should lead to a robust yet user-friendly workflow.Implementing the CI testing scriptI'm planning a CI testing script that allows early failure to save time while tracking the first non-zero exit code. I'll set up a function
run_allthat executes various testing steps, like linting and unit tests, logging their success or failure.I need to align function signatures and ensure integration tests are only run if the appropriate directory exists. For the CLI, I’ll allow users to pass additional
pytestarguments while keeping the usage straightforward.I'll utilize a helper function
run_stepto execute commands while handling errors gracefully. I’m keeping the script robust yet concise, focusing on user-friendliness!Building the CLI and Finalizing the ScriptI'm getting ready to create a command-line interface (CLI) and finalize the main function of my script. I’ll include a docstring at the top and add minimal comments to clarify each function. While I considered allowing
pytestto run inside a virtual environment, I've decided it's not necessary, which simplifies things. I aim to keep the script concise—about 200 lines total—with clear logic.Once everything is ready, I’ll provide a brief explanation followed by the code enclosed in triple backticks for clarity! Below is a standalone Python script that orchestrates common testing tasks (linting, unit tests, integration tests, coverage) and can be used locally or in CI.
Save it as
test_runner.pyin your project root and adjust the config section at the top as needed.#!/usr/bin/env python3 """ test_runner.py Simple automation for common software testing workflows: - Linting (ruff or flake8, if installed) - Unit tests (pytest) - Integration tests (pytest) - Coverage (coverage + pytest) - CI-friendly reports (JUnit XML, coverage XML/HTML) Usage examples: python test_runner.py all python test_runner.py lint python test_runner.py unit python test_runner.py integration python test_runner.py coverage Pass extra pytest args with "--", e.g.: python test_runner.py unit -- -k "my_test" -vv """ import argparse import logging import subprocess import sys import shutil from pathlib import Path # --------------------------------------------------------------------------- # Configuration – adjust these for your project structure # --------------------------------------------------------------------------- ROOT_DIR = Path(__file__).resolve().parent SRC_DIR = ROOT_DIR / "src" # Your application code lives here TEST_DIR = ROOT_DIR / "tests" # All tests live here UNIT_TEST_DIR = TEST_DIR / "unit" # Optional; falls back to TEST_DIR INTEGRATION_TEST_DIR = TEST_DIR / "integration" # Optional REPORTS_DIR = ROOT_DIR / "reports" # For JUnit / coverage output # --------------------------------------------------------------------------- # Utility helpers # --------------------------------------------------------------------------- def run_step(name, cmd, cwd=ROOT_DIR) -> int: """ Run a subprocess step, streaming output to the console. Returns the process's exit code (0 = success). """ logging.info("=== %s ===", name) logging.debug("Command: %s", " ".join(map(str, cmd))) try: result = subprocess.run(cmd, cwd=cwd) except FileNotFoundError: logging.error( "Command not found: %s. Is it installed and on your PATH?", cmd[0] ) return 127 if result.returncode == 0: logging.info("[OK] %s", name) else: logging.error("[FAIL] %s (exit code %s)", name, result.returncode) return result.returncode def choose_linter() -> str | None: """ Prefer ruff, then flake8, if installed. Returns tool name or None. """ for tool in ("ruff", "flake8"): if shutil.which(tool): return tool return None def run_pytest(label: str, targets: list[Path] | list[str], ci: bool = False, extra_args: list[str] | None = None) -> int: """ Run pytest with optional CI-friendly JUnit XML output. """ cmd: list[str] = ["pytest"] cmd.extend(str(t) for t in targets) if extra_args: cmd.extend(extra_args) if ci: REPORTS_DIR.mkdir(parents=True, exist_ok=True) junit_file = REPORTS_DIR / f"junit-{label}.xml" cmd.extend(["--junitxml", str(junit_file)]) return run_step(f"pytest ({label})", cmd) # --------------------------------------------------------------------------- # Workflow steps # --------------------------------------------------------------------------- def run_lint(ci: bool = False) -> int: """ Run a linter (ruff or flake8). 'ci' is unused here but kept for symmetry. """ linter = choose_linter() if not linter: logging.error("No linter found (ruff/flake8). Install one or skip 'lint'.") return 1 paths = [] if SRC_DIR.exists(): paths.append(SRC_DIR) if TEST_DIR.exists(): paths.append(TEST_DIR) if not paths: logging.warning("No src/tests directories found to lint.") return 0 cmd = [linter] + [str(p) for p in paths] return run_step(f"lint ({linter})", cmd) def run_unit_tests(ci: bool = False, pytest_args: list[str] | None = None) -> int: """ Run unit tests via pytest. Uses tests/unit if present, else tests/. """ if UNIT_TEST_DIR.exists(): target = UNIT_TEST_DIR elif TEST_DIR.exists(): target = TEST_DIR else: logging.error("No tests directory found at %s", TEST_DIR) return 1 return run_pytest("unit", [target], ci=ci, extra_args=pytest_args) def run_integration_tests(ci: bool = False, pytest_args: list[str] | None = None) -> int: """ Run integration tests via pytest, if the directory exists. """ if not INTEGRATION_TEST_DIR.exists(): logging.info( "Integration tests directory %s not found; skipping.", INTEGRATION_TEST_DIR, ) return 0 return run_pytest("integration", [INTEGRATION_TEST_DIR], ci=ci, extra_args=pytest_args) def run_coverage(ci: bool = False, pytest_args: list[str] | None = None) -> int: """ Run the full test suite under coverage.py and generate reports. """ if not TEST_DIR.exists(): logging.error("No tests directory found at %s", TEST_DIR) return 1 REPORTS_DIR.mkdir(parents=True, exist_ok=True) html_dir = REPORTS_DIR / "coverage_html" xml_file = REPORTS_DIR / "coverage.xml" # 1. Run tests under coverage cmd = ["coverage", "run", "-m", "pytest", str(TEST_DIR)] if pytest_args: cmd.extend(pytest_args) rc = run_step("coverage run", cmd) if rc != 0: return rc # 2. Console summary rc = run_step("coverage report", ["coverage", "report", "-m"]) if rc != 0: return rc # 3. HTML report rc = run_step("coverage html", ["coverage", "html", "-d", str(html_dir)]) if rc != 0: return rc # 4. XML for CI if ci: rc = run_step("coverage xml", ["coverage", "xml", "-o", str(xml_file)]) if rc != 0: return rc return 0 def run_all(ci: bool = False, pytest_args: list[str] | None = None) -> int: """ Run the full workflow: lint, unit, integration, coverage. Continues through all steps but returns first failing exit code. """ steps: list[tuple[str, callable]] = [ ("lint", lambda: run_lint(ci=ci)), ("unit", lambda: run_unit_tests(ci=ci, pytest_args=pytest_args)), ("integration", lambda: run_integration_tests(ci=ci, pytest_args=pytest_args)), ("coverage", lambda: run_coverage(ci=ci, pytest_args=pytest_args)), ] overall_rc = 0 for name, fn in steps: logging.info(">>> Running step: %s", name) rc = fn() if rc != 0 and overall_rc == 0: overall_rc = rc if overall_rc == 0: logging.info("All steps completed successfully.") else: logging.error("Workflow completed with failures (exit code %s).", overall_rc) return overall_rc # --------------------------------------------------------------------------- # CLI # --------------------------------------------------------------------------- def build_parser() -> argparse.ArgumentParser: parser = argparse.ArgumentParser( description="Automate software testing workflows (lint, tests, coverage)." ) parser.add_argument( "-v", "--verbose", action="store_true", help="Enable verbose logging (debug output)." ) subparsers = parser.add_subparsers(dest="command", help="Sub-commands") # Small helper to add common flags def add_common(sub): sub.add_argument( "--ci", action="store_true", help="Emit CI-friendly reports (JUnit XML, coverage XML)." ) sub.add_argument( "pytest_args", nargs=argparse.REMAINDER, help="Extra arguments to pass to pytest (prefix with '--')." ) # all p_all = subparsers.add_parser( "all", help="Run lint, unit tests, integration tests, and coverage." ) add_common(p_all) # lint p_lint = subparsers.add_parser( "lint", help="Run linting only." ) # lint doesn't need pytest args, but we keep CI flag for consistency p_lint.add_argument( "--ci", action="store_true", help="(Ignored) Kept only for interface consistency." ) # unit p_unit = subparsers.add_parser( "unit", help="Run unit tests only." ) add_common(p_unit) # integration p_integration = subparsers.add_parser( "integration", help="Run integration tests only." ) add_common(p_integration) # coverage p_cov = subparsers.add_parser( "coverage", help="Run tests under coverage and generate reports." ) add_common(p_cov) return parser def main(argv: list[str] | None = None) -> int: parser = build_parser() args = parser.parse_args(argv) if not args.command: parser.print_help() return 1 # Configure logging logging.basicConfig( level=logging.DEBUG if getattr(args, "verbose", False) else logging.INFO, format="%(message)s", ) # Normalize pytest_args (for commands that have it) pytest_args = getattr(args, "pytest_args", None) or [] if args.command == "all": return run_all(ci=args.ci, pytest_args=pytest_args) elif args.command == "lint": return run_lint(ci=args.ci) elif args.command == "unit": return run_unit_tests(ci=args.ci, pytest_args=pytest_args) elif args.command == "integration": return run_integration_tests(ci=args.ci, pytest_args=pytest_args) elif args.command == "coverage": return run_coverage(ci=args.ci, pytest_args=pytest_args) parser.print_help() return 1 if __name__ == "__main__": sys.exit(main())If you tell me your stack (e.g., only
unittest, or specific tooling liketox,nox,pytest-xdist,mypy, etc.), I can tailor this script more tightly to your setup.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.