Exploring Terraform Graphs With D3.js Part 1

TL;DR

As a newcomer to Terraform (and to AWS), I sometimes find it difficult to reason about the many available resource types, and the dependencies that can exist between them. Especially when coming to terms with larger configurations.

To address this difficulty, I want a tool to help me explore dependency graphs, and resource definitions, interactively. This is the first in a series (parts: one, two, three, four; code, documentation) of posts about building such a tool, using d3.js, starting with the simple example below¹, and building upon it.

Terraform and Dependency Graphs, Introduced

Terraform is a remarkable piece of software; it’s like Make for infrastructure. Rather than transforming source into libraries and executables, Terraform transforms resource definitions (such as vm instances, dns records, s3 buckets) into running infrastructure.

Like Make, Terraform walks a dependency graph to determine the order in which it should create resources, to identify what can be done in parallel, and to re-create resources affected by changes.

Consider the following example, a straightforward Terraform graph–the same as above–laid out by the graphviz package.²

This graph is easy to understand because it has only a handful of nodes, and an obvious structure. You can easily find the single instance, its provider (aws, in this case), and the few variables they depend on. Here is a slighly more complex example.³

This is graph remains fairly legible. But larger examples tend to sprawl, making resources harder to find, and dependencies harder to trace.⁴

Interactive Dependency Graphs

These visualizations could be improved in various ways. Adding color, and varying the shapes used, for instance, as well as collapsing less interesting parts of the graph. But an interactive visualization offers these possibilities and more.

As a reminder of how this might work, here is the first example, again. The root node is made larger than its dependencies, and nodes of different types are assigned colors, according to an arbitrary scheme.

And here is is the second. This example is worryingly dense, compared with the Graphviz version, but being able to call up resource definititions with the mouse is a striking advantage.

One possible improvement here is to use curved edges, so that their direction is more obvious. (Tracing an edge in the clockwise direction brings you to a dependency.)

That’s an encouraging result, but what about a much larger graph, like the sprawling third example?

This version is harder to make sense of than the Graphviz version! It contains so many types of resource, for example, that it exhausts the 20-color palette, used previously.

Additionally, many of the edges overlap, or are drawn so close together, that they become hard to distinguish.

Conclusion

So far this has been a fun exercise, and I’m satisfied with it as a proof of concept. However, larger configurations remain a problem. In the next post, I plan to take up this problem, and explore possible solutions to it.

Have a suggestion? patrick.mcmurchie@gmail.com