How to organize infrastructure as code projects?

Question

I have completed an Infrastructure-as-Code project using Terraform and Ansible. Currently, I organize my project as follows:

Project1
    Terraform
      ec2.tf
      rds.tf
      …
      variables.tf
      scripts\<some script files>
      templetes\<some tfl files as templates>
    Ansible
      Playbooks<some playbook yml files> 
         playbook1.yml
         playbook2.yml
         ……
      ansible_hosts
      vars\<some variable files used by vars_file>
      files\<files to be copied to remote hosts>
      templates\<jinja2 templated files>
      roles\<some roles such as webserver, dbserver, etc.>

Note I did not layout Ansible code using the following links yet:

e.g. I did not put vars/, templates/, files/ to individual role folder since Most of my roles did not have vars and templates. Besides, I want to manage vars, templates, files in one file instead of multiple roles, so I moved vars/, templates/ and files/ out of roles/.

For my next project, I may build an Infrastructure-as-code project using CloudFormation/Terraform and Ansible/Chef. Next, I may work on Jenkins, code pipeline, and so on.

In general, should I organize my DevOps projects based on projects, in this way:

Project1:
  Terraform
  Ansible
Project2:
  Terraform
  Ansible
  ......

Or should I organize my DevOps projects based on tools, in this way:

Terraform
  Project1
  Project2
Ansible
  Project1
  Project2
Cloudformation
  Project3
  Project4

I believe separate with projects makes more sense. but then how do we reuse code? for example, what if project1 and project2 use similar terraform code and some ansible roles. and I want to reuse the code of project1 to project2?

or is there any typical samples online I can refer?

score 4 · Accepted Answer · answered Jan 08 '18 at 09:54

The layout of your repository depends in many ways upon the context you are developing the automation in. If, for example, you are building out the infrastructure for a product as part of a product team, then it would make sense to tie the infrastructure to the product - i.e. keep the infrastructure in the same repository as the software source code.

If however, you are building out common infrastructure components to stand-up basic infrastructure for DNS, File Storage, Email, etc. then it probably makes more sense to have a single common-infrastructure repository. Some of this will be tied to the tool you are using, Terraform for example leads you down the path of having a single repository for all of your environments to support the terraform.tfstate construct.

Reusability

You quite rightly called out reusability and code sharing as a problem you are going to encounter sooner rather than later. This is an important principle in Software Development called Don't Repeat Yourse - or DRY for short. Most all DevOps tools allow you to modularize your code in such a way that you don't need to copy/paste code:

Conceptual Consistency

I advise that you don't use too many tools, try and standardise on a single tool rather than changing tools for each project, typically you see sets of tools being used across an organization:

Jenkins, Terraform and Ansible
TeamCity, Octopus Deploy and Desired State Configuration

This makes it easier to share code across projects, document what has been done so far and handover the code to other or future developers.

How to organize infrastructure as code projects?

1 Answers1

Reusability

Conceptual Consistency