Flux7 Blog

Allowing you to programmatically describe and provision your infrastructure, AWS  CloudFormation is a critical service in the AWS toolbox for anyone building infrastructure and applications in AWS. To further automate CloudFormation, at Flux7 we created dozens of custom scripts that generate CloudFormation templates and create Service Catalogs. Having worked with CloudFormation at all scales — from startups to Fortune organizations — we’ve encountered two CloudFormation challenges that we’re excited to see new CloudFormation capabilities address: Programmability and Reusability. These new features introduce streamlined workflows that allow us to abandon our custom tooling and in today’s article, we share three particular examples with you of how CloudFormation’s new capabilities can help reduce your customization efforts, too.

The new AWS CloudFormation functionality extends CloudFormation by enabling you to build and publish resource providers in CloudFormation. A resource provider has resource type specification and handlers that control API interactions with underlying AWS or third-party services. These interactions may include create, read, update, delete and list (CRUDL) operations for resources. Resource providers are used to model and provision resources using CloudFormation. For example,  AWS::EC2::Instance is a resource type by Amazon EC2 provider. You can use this type to model and provision an EC2 instance using CloudFormation. 

Now operators can build user-defined/custom resource providers to model and provision AWS or non-AWS resources using CloudFormation. And, APN Partners like ourselves can use the new capabilities to build resource providers to natively support AWS and non-AWS resources with CloudFormation. 

AWS CloudFormation now has two key components which are central to the examples we’ll share with you today:

CloudFormation Command Line Interface (CLI) toolkit, an open-source CLI toolkit for development and testing of resource providers. 

CloudFormation Registry, a managed service to publish resource providers for discovery and use.

Reusability

Reusability is a key principle of software engineering and similarly a key to scaling DevOps initiatives in organizations. Before, reusing modules was unwieldy. A common way to do it was to put CloudFormation templates in S3 and reuse them as nested stacks. But nested stacks can get clunky. The new CloudFormation functionality addresses both aspects of reusability; it creates a way to define a controlled interface to make sure we can blackbox the module and it provides a well-defined distribution mechanism.

A key component of Flux7’s Enterprise DevOps Framework for scaling DevOps is to create high-level, reusable modules to make patterns readily available to application teams. This allows them to focus on writing the business logic that adds value to the enterprise. With a good mechanism for reusability, we can create higher-level modules — e.g. for services.

Programmability

The other new, big feature is that CloudFormation now allows complete programmability. This opens the door to many possibilities, including smarter templates with more complete logic and the ability to create custom functions for external resources. 

Updates

While the above two use-cases unlock a lot of value, the biggest challenge for  large enterprises is the long-term maintenance of these modules. As needs change, high-level modules evolve and we need a strategy for upgrading existing deployed modules.  In line with the lean principle of reducing buffering, CloudFormation will now deploy the currently deployed version across the board on the deployed modules. This causes some upfront pain but reduces the long-term technical debt that plagues many enterprise systems.

Examples

Here are some example use-cases based on the hurdles we’ve seen in deploying Infrastructure as Code (IaC)  across our customers.

Creation Of EC2 Instances

When we create EC2 instances, we follow best practices around similar security and monitoring. So we also have to create security groups and instance recovery alarms. Using the new CloudFormation functions, we’d be able to create a module that can be shared across the organization that includes these best practices. In addition, without the new programmability features, it was difficult to create a security group set to the minimum permissions because the number of ports would need to be hard-coded. The new programmability features allow iterating over a list, thus increasing the reusability of the module.

Self-Sizing VPCs

One of the most tedious parts of creating a VPC is allocating space for each of the subnets. We’ve created tooling that we use with our advanced customers that will allow us to connect with an IPAM, get a CIDR block, partition that into the correct size subnets, and perform the CIDR math to get the subnet CIDRs, and use that to deploy the VPC and subnets. 

Service Deployment With External Resources

Taking further the concept of a high-level module that combines the building blocks AWS provides, we can launch complete services. For example, a microservice for deploying to ECS with a Jenkins pipeline for code and also external resources. Not only are we able to create the module, but we can also combine the external resources seamlessly. 

The new CloudFormation capabilities extend CloudFormation in important ways, allowing for greater programmability and reusability — key tenets of software engineering and enterprise-wide DevOps. Moreover, its streamlined workflows help reduce customization, allowing us to spend more time on other strategic initiatives. Interested in learning more about how AWS CloudFormation can be applied in your organization? Reach out today to learn how our AWS cloud configuration management services can help reduce maintenance and improve your security, consistency and developer self-service.

Written by Flux7 Labs

Flux7 is the only Sherpa on the DevOps journey that assesses, designs, and teaches while implementing a holistic solution for its enterprise customers, thus giving its clients the skills needed to manage and expand on the technology moving forward. Not a reseller or an MSP, Flux7 recommendations are 100% focused on customer requirements and creating the most efficient infrastructure possible that automates operations, streamlines and enhances development, and supports specific business goals.