Design a polyglot repo toolchain: shared config, linters, consistency checks, and CI.
- User
Design a polyglot repo toolchain: shared config, linters, consistency checks, and CI.
Assistant
The user wants a design for a "polyglot repo toolchain." Key components requested:
- Shared config
- Linters
- Consistency checks
- CI (Continuous Integration)
Context & Constraints:
- Polyglot: This means the repository contains code in multiple languages (e.g., Python, TypeScript, Go, Rust). The toolchain needs to handle this complexity without becoming a mess of conflicting dependencies or scripts.
- Toolchain: This implies a set of tools working together.
- Goal: Consistency, shared configuration, and automated checks.
Approach: I need to structure this response logically. A good structure would be:
- Philosophy/Architecture: High-level approach (e.g., "Root-centric" vs. "Tool-specific").
- Directory Structure: Visualizing where configs live.
- Tool Selection (The Stack): Choosing the right tools for a modern polyglot environment. (Mega-Linter vs. individual tools).
- Shared Configuration Strategy: How to avoid copy-pasting config files.
- Dependency Management: How to install the tools themselves.
- CI/CD Integration: Putting it all together in a pipeline.
Drafting the Content - Section by Section:
-
1. Architecture: The "Root Configuration" pattern is standard. Everything lives at the root, and sub-projects inherit or reference root configs. This prevents drift.
-
2. Tooling Strategy (The "How"):
- Option A: The "Manager" approach. Use a tool like
pre-commit(framework) ortrunk(tool) orMega-Linter(container). This is usually the best for polyglot because it abstracts language-specific installation. - Option B: The "Manual" approach. Install
eslint,black,golintindividually in CI. High maintenance. - Recommendation:
pre-commitis the industry standard for the local hook runner. For CI, we can usepre-commitor a dedicated linting action/container. Let's stick topre-commitas the unifying glue.
- Option A: The "Manager" approach. Use a tool like
-
3. Shared Config (The "Don't Repeat Yourself" part):
- Approach: "Presets" or "Base configs."
- Implementation: For ESLint, use
extends. For Python, usepyproject.toml. - The "Config Package" pattern: If this is a monorepo for a larger org, publishing a shared npm/pip package with the config is best. For a single repo, just relative paths.
-
4. Linters (The "Meat"):
- Need a generic formatter (Prettier covers YAML, JSON, Markdown, JS/TS).
- Need language specific: Python (Black/Ruff), Go (gofmt), Rust (rustfmt).
- Security:
gitleaksortrivy.
-
5. Consistency Checks (The "Glue"):
- Renovate/Dependabot: For keeping tools up to date.
- EditorConfig: For basic IDE consistency.
- License checks: Ensure dependencies are legal.
-
6. CI (The "Enforcement"):
- GitHub Actions is the standard assumption, but I should keep it generic enough.
- Use caching (vital for polyglot performance).
Refining the Solution - Specific Technologies:
- Tool Runner:
pre-commitis the clear winner for local + CI consistency. - Python:
ruffis the new standard (fast, replaces flake8/isort/black). - JS/TS:
eslint+prettier. - Generic:
prettier. - Make: Just a
Makefilefor convenience commands.
Structuring the Output:
- Architecture Overview: The "Universal Wrapper" pattern.
- Directory Layout: A visual tree.
- The Core Tool:
pre-commit: Why and how. - Language-Specific Configs:
- Python (Ruff)
- JS/TS (ESLint/Prettier)
- Go (Golangci-lint)
- Shared Config Strategy:
extendsandreuse. - CI Configuration: GitHub Actions example.
- Consistency & Security: EditorConfig, Secrets scanning.
Self-Correction/Review during drafting:
- Initial thought: Suggest Nx or Turborepo.
- Correction: Those are build systems. While they handle polyglot well, they are heavy. The prompt asks for a toolchain for linting/config/CI specifically.
pre-commitis lighter and more focused on linting polyglot code. I will mention Nx as an alternative for build orchestration but focus onpre-commitfor the "quality gate". - Formatting: Use code blocks. Keep it copy-paste friendly.
Final Polish: Make sure to emphasize why specific choices are made (e.g., Ruff for speed, Prettier for non-code files). Ensure the "Shared Config" actually looks shareable (e.g., using relative paths or local packages).
Let's write the response following this structure.
- Phase 1: The Manager (
pre-commit). It handles python, node, go, system hooks. - Phase 2: The Linters.
- Generic: Prettier.
- Python: Ruff (replaces Flake8/Black/Isort).
- JS: ESLint + Prettier.
- Go: golangci-lint.
- Phase 3: CI. Run
pre-commitin CI. - Phase 4: Config Sharing. Use
extendslogic.
This looks solid. I will create a concrete directory structure and file examples. Designing a toolchain for a polyglot repository (e.g., containing Python, TypeScript, Go, and Rust) requires a shift in mindset. You are no longer managing "a Python project"; you are managing a graph of dependencies and tools that must coexist peacefully.
Here is a comprehensive design for a Polyglot Toolchain that emphasizes shared configuration, strict linting, and CI consistency.
1. Architectural Philosophy: The "Universal Wrapper"
The core problem in polyglot repos is that every language has its own package manager and linter runner (npm, pip, cargo). To keep the repo maintainable, you need a Universal Wrapper that standardizes how tools are installed and run.
Recommended Stack:
- Task Runner:
task(ormake). ATaskfile.ymlabstracts the specific language commands (e.g.,npm run lintvsruff check) into standardized commands liketask lint. - Dependency Manager:
mise(formerly rtx) ordevbox. These tools replacenvm,pyenv, andrustup. They read a config file and ensure the exact versions of Node, Python, and Go are installed automatically when you enter the directory. - Lint Orchestration:
pre-commitortrunk. Do not write complex CI scripts to run linters manually. Use a tool that manages linter versions and execution order for you.
2. Directory Structure
Organize by Domain/Service, but keep toolchain config at the Root.
. ├── .github/ │ └── workflows/ │ └── ci.yml # Single source of truth for CI ├── .mise.toml # Universal tool version management (Node 20, Python 3.12, etc.) ├── Taskfile.yml # Universal command runner ├── .pre-commit-config.yaml # Linter version management and git hooks ├── services/ │ ├── backend-python/ │ │ ├── pyproject.toml │ │ └── src/ │ └── frontend-ts/ │ ├── package.json │ └── src/ └── libs/ └── shared-rust/ └── Cargo.toml
3. Shared Configuration Strategy
The goal is "Zero Copy-Paste." If you have two Python services, they should both inherit from a root config.
A. Tool Versioning (
.mise.toml)Ensure every developer and the CI runner use the exact same runtime versions.
# .mise.toml [tools] node = "20.12.1" python = "3.12.2" go = "1.22.1" rust = "1.77.0" usage = "a" # just an exampleB. Shared Configs via "Presets"
Instead of copying
.eslintrcorpyproject.tomlinto every folder, publish your own internal config packages.- For TypeScript: Create an npm package called
@my-org/eslint-config. - For Python: Create a pip package called
my-org-ruff-config.
Example
services/frontend-ts/.eslintrc.js:module.exports = { extends: ["@my-org/eslint-config"], // Consumes shared rules rules: { // Only local overrides go here } };If you cannot publish packages: Use symlinks or a "base config" file at the root that scripts reference.
4. The Linter Stack (Specific Recommendations)
Use modern, fast linters that support automatic fixing.
Language Tool Why? Config File Python Ruff Replaces Flake8, isort, and Black. Written in Rust (extremely fast). pyproject.tomlJS/TS ESLint + Prettier Industry standard. .eslintrc.jsGo Golangci-lint Aggregates go vet,staticcheck, etc..golangci.ymlRust Clippy The standard for Rust linting. Cargo.tomlAll Files Prettier Formats JSON, YAML, Markdown consistently. .prettierrcSecurity Trivy / Gitleaks Scans for secrets and container vulnerabilities. Root config The
pre-commitApproachInstead of running these manually, map them in
.pre-commit-config.yaml. This ensures they run locally on commit and identically in CI.# .pre-commit-config.yaml repos: - repo: https://github.com/astral-sh/ruff-pre-commit rev: v0.3.4 hooks: - id: ruff args: [ --fix ] - id: ruff-format - repo: https://github.com/pre-commit/mirrors-eslint rev: v8.57.0 hooks: - id: eslint files: \.[jt]sx?$ # *.js, *.jsx, *.ts and *.tsx additional_dependencies: - eslint@8.57.0 - typescript@5.4.3 - '@typescript-eslint/eslint-plugin' - '@typescript-eslint/parser' - repo: https://github.com/pre-commit/mirrors-prettier rev: v3.1.0 hooks: - id: prettier
5. Consistency Checks
Beyond linting code, you need to check the repo structure.
-
EditorConfig (
.editorconfig): Handles basic indentation and charset settings before linters even run. This ensures that if a Python dev opens a TS file, their IDE doesn't use 4 spaces if the project uses 2.# .editorconfig root = true [*] indent_style = space indent_size = 2 end_of_line = lf charset = utf-8 trim_trailing_whitespace = true [*.py] indent_size = 4 -
Renovate/Dependabot: In a polyglot repo, dependency drift is fatal. Configure Renovate to group updates by language (e.g., "Group all Python dependencies into one PR").
-
Syncing Config Files: Use a script (or the
pre-commithook) to ensure that ifpackage-lock.jsonchanges,node_modulesare updated, or ifrequirements.txtchanges, the virtualenv is refreshed.
6. CI/CD Pipeline (GitHub Actions Example)
The CI pipeline should strictly follow the "Universal Wrapper" logic. Do not put complex logic in the YAML; call the
Taskfileinstead.# .github/workflows/ci.yml name: Polyglot CI on: [push, pull_request] jobs: setup: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 # 1. Install the Universal Tool Manager - name: Install mise uses: jdx/mise-action@v2 with: install: true # 2. Cache everything (Languages + Package Managers) - name: Cache Dependencies uses: actions/cache@v4 with: path: | ~/.cache/pip node_modules target key: ${{ runner.os }}-${{ hashFiles('**/lockfiles') }} # 3. Install Deps via Taskfile - run: task install lint: needs: setup runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: jdx/mise-action@v2 - uses: pre-commit/action@v3.0.1 # Runs the exact same hooks as local with: extra_args: --all-files test: needs: setup strategy: matrix: # Run tests in parallel by language component: [backend-python, frontend-ts, shared-rust] runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: jdx/mise-action@v2 # Run tests for specific component using Taskfile - run: task test:${{ matrix.component }}Summary Checklist
- Versions: Use
miseto lock Node/Python/Go versions at the root. - Scripts: Use
Taskfileto create a CLI for your repo (task test,task lint). - Linting: Use
pre-commitframework to manage linter installation and execution. - Inheritance: Configure linters (ESLint/Ruff) to inherit from a shared preset.
- Formatting: Use
EditorConfigfor basic styles andPrettierfor code formatting.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.