RackN Digital Rebar Infrastructure Pipelines helps you standardize code from any part of your infrastructure. But you don’t have to build Infrastructure Pipelines from scratch! You can use tools you already use, like Ansible and Terraform, to create Infrastructure Pipeline segments. In fact, we wrote about this in a previous blog post and showed even you how to do it at Cloud Field Day 13.
Today we’ll talk about using Digital Rebar Terraform IaC Modules with Infrastructure Brokers. If one team figures out how to use Terraform for a particular task, why shouldn’t it be readily available for all infrastructure you maintain?
Imagine a world where no one needs to make copies of Terraform plans
Digital Rebar does so much more than decompose IaC configuration into smaller segments. For one thing, you can make templates from those discrete units. Additionally, you can design those templates to include injectable behaviors. We talked about that in our post about Ansible.
You likely have a collection of Terraform plans that use the same resources but in slightly different ways. What if you could consolidate all of these plans into reusable and standardized templates? But why stop there? What if you had an API to manage and update the Terraform plans without worrying about requiring each team to make unique copies?
If you said give that to me now, you’re in luck! Because that’s exactly what Digital Rebar Resource Brokers do for you.
The Digital Rebar Broker design allows you to use the same IaC components for different platforms. This is true even if the brokers have different performance or implementation requirements. For example, you don’t have to rely on the Terraform DSL (domain specific language) for every operation. In fact, Digital Rebar can pre-render Terraform plans based on specific systems, brokers and cluster configuration.
But first, let’s do a little review of terms. The Terraform Plan Composition flow chart from our CFD13 presentation explains how Resource Brokers coordinate work.
In this post, we’ll start by injecting an IaC module into an existing Terraform plan. Then, we’ll make sure infrastructure consumers can use the module by putting it behind the Resource Broker API. Finally, we’ll discuss the benefits to using Digital Rebar Params and logic in template rendering.
Ready to learn how to do provisioning like it’s a dev process? Keep reading! Here’s a video to help as well:
Terraform Snippets via Golang Templates
Digital Rebar is written in Golang. We use Golang Templates (plus Sprig) to translate Digital Rebar information into other formats. In the Terraform Broker case, we dynamically build a Terraform Plan by evaluating a set of Digital Rebar Templates. Then we execute that plan.
Ultimately, operators extend or replace the built-in templates with ones that match their environment.
AWS Example
The AWS resource broker provides a clear example of how we designed this process to operate. Digital Rebar’s built-in AWS resource profile includes a list used for AWS plans in terraform/plan-templates:
- cloud-provision-aws-init.tf.tmpl
- cloud-provision-aws-security-group.tf.tmpl
- cloud-provision-machines.tf.tmpl
- cloud-provision-aws-instance.tf.tmpl
AWS security groups are highly customized. So Digital Rebar makes it easy for operators to create their own Terraform resource aws_security_group stanza, then replace the built-in terraform/plan-templates with their custom template. Terraform stanzas are sequential and cross-referenced. This gets messy.
In this case, we dynamically change the cross-referenced names with an AWS specific parameter (aws/security-groups) .
Linode Example
To further illustrate, let’s provision a firewall on the Linode cloud. Let’s walk through the cloud-provision-linode-firewall.tf.tmpl shown below.
In the file, Digital Rebar uses the .ComposeParam function to collect the network/firewall-port settings across multiple layers of the input configuration. Then it builds components of the Terraform resource using the Golang template Range function.
The generated Terraform template uses the network/firewall-port Params. As a result, these values are not hard-coded or limited to the Terraform scope. Additionally, Digital Rebar has a general way to define open ports using the network/firewall-port parameter. This method is now available consistently throughout the pipeline. That helps align critical information during provisioning, configuration and day N operations. Without Digital Rebar workflows, coordinating building systems and local firewalls by sharing config information is very difficult to achieve.
Digital Rebar templates are able to use Terraform variables. In the example below, the Golang template includes all the machine types. Once that list is built it pulls in the correct Terraform variable lists. As a result it is possible to build robust Terraform snippets that can be reused in a wide variety of scenarios.
You can review the full template here.
Extending your Resource Broker
The Digital Rebar Resource Broker is a general purpose abstraction. It requests machines from Digital Rebar-managed resources like Pools and Contexts. Additionally the Resource Broker can request machines from external resources like Clouds or Virtualization Platforms.
This post is focused on the Resource Brokers that use the terraform-apply task. You can find these brokers in the cloud-wrapper content pack. They are used by the Cloud Provision cluster pipeline.
The cluster and broker interaction standardize the way Digital Rebar calls Terraform. So it’s really important to make sure you’re building the appropriate Terraform plans.
Extending the Terraform Resource Broker
- Find your existing terraform/plan-templates information. Typically, the list includes
- An init template to set up the provider.
- An app template to create resources needed by the provider for machines.
- A machine template to map Digital Rebar information to the provider’s syntax.
RackN maintains Terraform IaC Modules as references. Find them in the “resource-*” profiles.
- Select which “resource-*” profile to use for your base profile. If you don’t want to create your own template, select Linode. Next, choose the Linode Firewall template that is included with Digital Rebar community content.
- Add an override for terraform/plan-templates. Use the reference list from the resource-[cloud] profile to add it to the new Resource Broker.
- Add your extra template(s) into the list of plan snippets used by your broker. These are used in the order provided to build Terraform plans for the broker. If you are following along with Linode, you should add cloud-provision-linode-firewall.tf.tmpl last in the list.
- Update the terraform/plan-templates list. Your broker is ready to use!
I’ve Injected a Terraform Resource! Now what?
Now that you’ve got the Terraform Resource added, the only thing left to do is crCreate a cluster using your new or updated Resource Broker. That’s all you need to do to use it!
Be prepared to debug! In general, it will take experimentation and testing to get a Terraform plan and cloud permission automation to be 100% repeatable. In fact, during the development process, you should add and set “terraform/debug-plan” to true. Setting that value tells the Resource Broker to store the raw Terraform Plan as a parameter on the calling cluster.
However, you should only use this feature during development and testing. This is because the raw Terraform Plan includes credentials and information that should not be retained outside of a secure environment.
Real Talk
Digital Rebar helps you reuse work you’ve already done, such as Terraform resource definitions to build clusters. In other words, Infrastructure as Code shouldn’t start from scratch! You know how your clusters need to be built, and you probably have already invested time and work to build components.
In conclusion, RackN Digital Rebar is the platform that can help you bring it all together. Why not try it now and see what it can do for you: https://rackn.com/trial
