I need help with structuring my cicd / deployment repo. Currently I have (among other…
- User
I need help with structuring my cicd / deployment repo. Currently I have (among other dirs) the following directories:
src (contains kcl files that are used to template everything that is rolled out) common (common source files that are imported by others) cluster-1 (template files for cluster-1) app (source for kubernetes manifests that are deployed) cluster (source for talos configuration (os layer)) infra (infrastructure that is provided via terraform) cluster-2 ... ... artifacts cluster-1 app (deployment files for cluster) cluster (controlplane.yaml and worker.yaml for talos) cluster-2 ... ... scripts (utility scripts)Perspectively all files that currently live under "artifacts" in the same repo could also be commited to a seperate gitops repo. But currently it is easier that way.
I am unsure about the following. Conceptually the flow is src -> artifacts. In src there are source files that generate artifacts. Only artifacts are then deployed in the end. But then I have this infa folder with terraform that lies somewhere in between. It somehow defines source that is "deployed". The terraform state sits there too. And there is no generated "terraform manifest". The terraform variables might be generated via templating by kcl. So that looks like a reason to put it into the src folder. But still I am unsure. What do you think or propose?
Assistant
The classic conundrum of organizing a CI/CD/deployment repository!
It sounds like you have a good grasp on the conceptual flow:
src→artifacts. I'll offer some thoughts to help you decide on the structure.Current structure:
src: templates for everything (KCL files)artifacts: generated deployment files (could be in a separate GitOps repo)scripts: utility scripts
The
infrafolder conundrum:You're right that the
infrafolder with Terraform files seems to be in betweensrcandartifacts. Here are some points to consider:- Terraform state: The Terraform state is a critical piece of the infrastructure configuration. Storing it in the same repository as the Terraform files might be convenient, but it also means that the state is tightly coupled with the template files.
- Generated Terraform variables: You're using KCL to generate Terraform variables, which suggests that the Terraform files are dependent on the template files.
Proposed options:
- Keep
infrainsrc: This approach makes sense if the Terraform files are tightly coupled with the template files and rely heavily on the generated variables. This way, thesrcfolder contains all the source files that generate artifacts. - Move
infrato a separate folder: If you want to keep the Terraform files separate from the template files, you could create a new folder, e.g.,infrastructure. This would allow you to manage the Terraform state and files independently of the template files. - Create a
terraformfolder undersrc: This approach would keep the Terraform files closely tied to the template files while still separating them from the rest of thesrcfolder.
Additional considerations:
- GitOps repository: If you plan to move the
artifactsfolder to a separate GitOps repository, you might want to reconsider the structure of your main repository. You could keep thesrcfolder as is and move theartifactsfolder to the GitOps repository. - Reproducibility and versioning: Ensure that your structure allows for reproducible builds and versioning of your artifacts. This might influence your decision on where to store the Terraform state and files.
Suggested structure:
Based on your current structure and the considerations above, here's a suggested structure:
. ├── src │ ├── common │ ├── cluster-1 │ │ ├── app │ │ ├── cluster │ │ ├── infra │ │ │ ├── main.tf │ │ │ ├── variables.tf │ │ │ └── ... │ │ └── ... │ └── ... ├── artifacts │ ├── cluster-1 │ │ ├── app │ │ ├── cluster │ │ └── ... │ └── ... ├── scripts │ └── ... └── infrastructure (optional) ├── cluster-1 │ ├── main.tf │ ├── terraform.tfstate │ └── ... └── ...Ultimately, the structure should reflect your team's workflow, tooling, and requirements. Take your time to experiment and adjust the structure as needed.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.