How to Build Infrastructure as Code with AWS CloudFormation
Infrastructure as Code (IaC) has become a best practice for DevOps. Enterprise teams with a need for global infrastructure to support DevOps at Scale can build a strong foundation using IaC with Amazon Web Services (AWS).
AWS and IaC represent a journey for teams. Here we provide a roadmap to implement this very powerful combination of technologies.
What is Infrastructure as Code?
Infrastructure as Code is the provisioning and management of IT infrastructure. This includes servers, networking, load balancing, and security. Using software instead of physical hardware has major benefits.
Infrastructure as Code Benefits
IaC saves you money. Using automation takes less effort. The resulting infrastructure is better than if you spent hours configuring physical hardware and OS parameters.
IaC increases repeatability. This results in fewer deployment problems. You don’t have to worry about someone configuring a machine incorrectly (or even just differently), because they missed a step.
IaC scales to meet your needs. Now you can add people, ship software, add servers, and expand storage without waiting for equipment delivery and “racking and stacking.”
IaC enhances compliance. It brings a repeatable process that self-documents. And it delivers better monitoring for security and auditing purposes.
Infrastructure as Code Tools
If you have begun to apply DevOps practices in your organization, you will find that many of the same principles and practices, and even some tools, can also be applied to your infrastructure.
There are a number of tools available that provide IaC functionality. These include:
- AWS CloudFormation
- Microsoft Azure Resource Manager
What Is AWS CloudFormation?
CloudFormation is the AWS tool for infrastructure as code. It is just one of many reasons AWS has become so popular. You can use CloudFormation to describe a complete environment using software instead of physically configuring hardware and software environments.
With AWS on-demand computing power and storage, implementation is a click away with IaC. You can deploy new production servers. You can also quickly simulate new workflows and test upgrades. Previously this would have been difficult (or impossible) to accomplish because expensive hardware would have been exclusively dedicated to production workloads.
CloudFormation vs. Scripting
When it comes to OS and software configurations, CloudFormation is better than scripting. It can orchestrate the provisioning of resources, and it can understand AWS services natively. Common situations –– like waiting, retries, and acting on signals from other servers and applications –– are easy handled without having to write additional code.
Using CloudFormation over scripting streamlines DevOps. You can more easily deploy/swap out a server, install software, and test. All of this is accomplished automatically right from your keyboard and browser. This technology makes managing a large topology simpler.
Cloud Deployment Options
IaC tools can be used to implement hybrid deployments with both on-premises and cloud servers. Some companies even use multiple clouds. But it is important to note that an exclusive topology deployed with single provider (over a multi-cloud strategy) has some advantages.
First, the service offerings and management tools for each of the cloud providers are different. If you use a single provider, engineers only need to understand one platform.
Second, a single provider gives you global management capabilities — such as the ability to control all of your servers from one dashboard. It can simplify identity and access management, and improve your ability to monitor and collect data on operations.
Helix Core –– version control from Perforce –– works with any cloud provider. You can deploy it on AWS, Microsoft Azure, or Google Cloud Platform (GCP). Here’s an example of AWS deployment using CloudFormation as the infrastructure as code tool.
Why Choose Helix Core and AWS?
As many applications are moving to the cloud, so are Helix Core users. Multiple-server configurations of Helix Core are popular for organizations that depend on hardware and/or software products for revenue.
Global multiple server configurations are made infinitely better by using Helix Core, AWS, and IaC to:
1. Facilitate Collaboration
Companies look to Helix Core to improve developer productivity and scale globally. It enables collaboration between teams across geographically diverse facilities through advanced replication capabilities. Many customers that use Helix Core have installations with tens or hundreds of servers. These are configured to serve users and enable automation around the globe.
AWS makes this diversity immediately accessible with a global infrastructure footprint that is comprised of:
- Regions represent different geographic areas.
- Availability Zones are data centers within regions.
- Virtual Private Clouds (VPCs) are your “private” data centers within the AWS data centers.
2. Scale Without Limits
High performance and scalability are critical to large product development organizations. Both Helix Core and AWS share the ability to scale to meet your needs. These systems can handle:
- Lots of developers (around the globe).
- Lots of (big) files.
- Need for CI/CD speed.
Using Helix Core and AWS lets you support many topologies. These range from a single virtual machine to complex, multi-server, multi-geography, and multiple LAN/WAN architectures.
3. Enhance Security
When it comes to security and compliance, AWS and Helix Core are very compatible. Helix Core permissions and AWS services provide options to build your strategy.
Using CloudFormation for infrastructure as code makes it possible to automatically deploy and manage:
Type of Helix Core Server
Master (Commit) Servers
Define and deploy a master Helix Core server. You can secure and configure the server for user and replica access.
Backup and Disaster Recovery
Backup replicas can reside in the same AWS region, a different region, or in the same region but in a different availability zone. This supports appropriate level of diversity for your HA/DR needs. AWS can help automate failover operations, including switching certs and DNS changes, across a topology.
Replicas support teams of potentially 100s of contributors in remote facilities. You can deploy replicas in an AWS VPC close to a facility (i.e., in a different AWS region). These servers connect to the master using AWS built-in networking.
Build servers –– using Jenkins or another build runner –– can reside in the same availability zone as a replica that is serving a remote developer team. This gives them high-performance CI/CD in a remote office. Or, build servers can be deployed in a AWS placement zone next to the Helix Core commit server, taking advantage of AWS high-speed networking.
Backup and Storage
On AWS, the process of protecting backups of checkpoint files, journal files and versioned files is simplified. You can “snapshot” server drives, and utilize AWS services –– such as object-based storage (S3) or Glacier –– to efficiently store archives.
Just like any machine, VMs can encounter issues periodically. When you need a replacement, AWS lets you deploy a replacement easier and more quickly than using physical resources.
When a server needs to be faster, the lift is far lighter with AWS. You can upgrade to faster vCPU and memory configurations on-demand when you add people, or when projects increase in size and velocity.
The benefits of CloudFormation extend beyond the provisioning of resources. It also helps with:
- Configuration management
- Monitoring and performance optimization
- Governance and compliance
- Lifecycle optimization
How to Get Started on AWS
For those new to AWS, we’ll outline the basics of selecting the OS, making virtual hardware choices, and setting up your security. Getting started is easy. Once you make all your selections, your Stack can be replicated around the globe.
Select Your OS
Preparing a virtual machine is similar to installing Helix Core on a physical machine. Using the browser-based AWS Management Console, select the EC2 (Elastic Compute Cloud) Dashboard, launch your instance.
First, you choose your OS platform. All of the Perforce-supported Linux distributions –– Amazon Linux, RHEL, Ubuntu from AWS, and Centos from 3rd parties –– and Windows server platforms are available.
Configure Your AMI
Your Amazon Machine Image (AMI) dictates the vCPU, memory, and networking (among other details) of your virtual hardware. You will want to select options to meet performance and scalability needs. Helix Core can be deployed on a wide variety of vCPU, memory, and storage configurations.
These choices range. Non-production options could include a t2.micro instance with 1 vCPU, 1 GiB of Memory, and low to moderate networking performance. The range can go up to instances with many vCPUs, lots of memory, and ultra-high performance networking.
We suggest using instance types C (Computer Optimized) or M (Memory Optimized), and available options of high-speed networking.
Set Up a Security Group
Security groups on AWS provide most of the functionality of a firewall. Setting up security groups is easy. Once you establish and name the group, you can apply it to any of your machines. You simply input the protocol and ports you want to open for inbound and outbound traffic. Then you can limit traffic to your machine(s) by IP addresses using CIDR notation.
For our example, we used our “default” port 1666 (use 1667 for TLS connections). It is open to the entire world (CIDR notation used by AWS, 0.0.0.0/0). Once the machine is ready and the networking and basic security are in place, you can install software and utilities by logging in from your work with Secure Shell (SSH).
Using a CloudFormation Template For IaC
You can use CloudFormation to describe a complete environment using either JSON or YAML. A collection of AWS resources described in code is called a Stack. Using a template, you can repeat the creation of the same Stack. For example, you could deploy cookie cutter Stacks around the globe.
When you go to deploy the Stack, the template provides choices and then CloudFormation executes the deployment. There are many sample templates available on the web to download. Ensure you select one that addresses your performance and storage needs.
The template is then edited to reflect:
- The region (US East 2, Ohio).
- Instance types and AMIs that you want to make available.
- AWS key pair for encrypted, secure access with a private key.
- The operating system (Linux or Windows).
In addition, you could assign an Identity and Access Management (IAM) role to give the instance access to defined AWS resources. Defining metadata would also make easier to find and reference instances once you have many servers deployed.
Deploying Your Helix Core Server
If you want to start using a Helix Core on AWS, there are several ways of getting software installed. For this example, you can use the AWS EC2 feature called user data.
User data provides the ability to execute preset commands when an instance is created, and optionally every time that instance starts up.
To Set Up Helix Core:
- Download the appropriate . This can be done on a workstation.
- Zip the executables into an archive and name it. For this example, “MyHelixCore.zip.”
- From the AWS Dashboard, navigate and create a S3 bucket.
- Upload the zip file with Helix Core executables into the bucket.
- Give the zipped file (MyHelixCore.zip) public access.
The user data is located in the CloudFormation template. In the example, the user data installs utilities, gets the server zip file from an S3 bucket, sets a path, and makes the p4d file executable. Finally, it runs p4d as a daemon.
Once CloudFormation completes the tasks described in the template, the Stack is available. You now have a new, running instance of a Helix Core server.
Using Infrastructure as Code for Complex Topologies
Using CloudFormation to deploy a single server is not taking advantage of what IaC can truly offer. Using the powerful combination of AWS and Helix Core, you can build bigger.
AWS can deploy more complex topologies, in different geographic regions quickly. And with Helix Core, you gain the ability to support your large teams and files no matter where they are located. We are here to help you accelerate development and enhances security with unlimited scale.