StackMaster 2.0 Proposal: Handling multiple, dependant, stacks

[patrobinson]

Hello one and all,

In October StackMaster will turn three and we've decided the next major goal should be a 2.0 release. This is an opportunity to put all options on the table, including those that break the API. This issue is a place to discuss a proposal to solve a problem born out of StackMaster's success. By making smaller, modular stacks we end up with some difficulty handling lots of dependant stacks.

Problem Statement

A system is generally composed of more than one stack. When we update one stack we sometimes have to update stacks that take an output from it. When we want to duplicate a stack in another environment or region, we have to know the order in which it needs to be created and destroyed.

Relevant Issues and PRs:
Offer to update dependent stacks #123
Add dependency graph #141

Current Solutions

None.

Proposed Solution

It would be beneficial to group these stacks into a larger construct so they can be easily deployed to multiple regions and multiple environments (i.e. stack_master apply us-east-1 my-bunch-of-stacks).
I propose a concept of "StackBundles" (working name, alternatively "SuperStacks") to facilitate this. These would live in the stack_master.yml file and list the stacks, they could either be created in the order they are defined or have a “depends_on” attribute for dependent stacks. At first we’d provide no smarts to detect incorrect or cyclical dependencies, but they could be added in later. Stacks would be deleted in the reverse order.

Implementing a Directed A-cyclical Graph would help produce a dot-graph and ensure there are no cyclical dependancies, but would likely be implemented later. The first iteration should simply allow the operator to define the dependancies.