0

We are using AWS CloudFormation JSON templates to manage our infrastructure. The templates are getting bigger by the day (5k Lines) because company policy is to write certain projects in one template.

I've been playing with YAML, JSON, and CloudFormation generators. I found some nice YAML generator (Spiff). Using it, I can reduce our templates by almost 90% in some cases.

Is it a good idea to generate CloudFormation Templates on the fly and work mainly with Spiff YAML generators? Please keep in mind that Spiff is a bit cryptic and not pure YAML, so there's a learning curve for people maintaining the templates.

Pat Myron
  • 105
Moataz Elmasry
  • 189

2 Answers2

2

I've been using the cfndsl RubyGem for about 9 months now in a couple of projects, and have been very happy with it for the following reasons:

  • Comments
    Invaluable as your scripts get more complex.
  • Related items are colocated in the source
    For example, the Output for a resource can be physically located in the source file next to that resource, rather than in another unrelated part of the file. The JSON generator ensures that all definitions go in their appropriate places.
  • Properties files
    I find that I have minimal need for stack parameters; they just add one more thing that isn't tracked in source control.
  • Ruby string substitutions
    Far easier to use than the Fn:Sub intrinsic function, although I still use the function when I want to include things like the account ID.
  • Configuration-driven stacks
    This is the big one: all of our application configs (instance, auto-scaling group, load balancer, DNS) are pretty similar, and can be represented in a few lines of configuration. Since the stack JSON is being generated by a full-featured programming language, I can loop over the config and use if-then statements to define relevant resources. A simple 100-line app template serves to generate the stacks for a dozen or more apps.

CFNDSL isn't the only option; it just happened to be something I was introduced to at the right time. I've also heard of a stack generator written in Python, and at a different time I might have preferred it.

Bottom line: generating stacks programmatically is far better than maintaining a purely declarative format, especially JSON.

kdgregory
  • 5,270
0

I might be a little bit late with my answer, but there is a cfpack tool that i have created to split huge CloudFormation templates into smaller ones. The idea is that when you split your main template, you create small templates that are fully compatible and actually are CF templates by itself. Then when you run cfpack bulid command it combines all templates and builds main template file that can be deployed to CloudFormation.

Another good thing is that you can create reusable drop-in templates and share it between different projects.

So, take a look at it, perhaps it can help you too. Here is a sample usage, check templates folder (that project isn't done yet, but demonstrates how to use cfpack).

Eugene Manuilov
  • 109