Edit This Site

HashiCorp who

By: Andreas Bigger
Posted: June 25, 2020
––– views
What is HashiCorp? Where did it start? And most interestingly, what does it do?

The Cloud

A necessity nowadays, cloud computing began as a vague, questionable phrase on Compaq's business plan describing the paradigm shift that practically powers the internet today [1]. In this new world, businesses could be relieved of complex and costly infrastructure by paying a cloud provider such as Amazon's AWS or Google's GCP to maintain and optimize their hardware. As we can experience with on-demand streaming services such as Netflix, the cloud is extremely reliable. But, like most important pieces of software, developing on the cloud was not quick, and this is what HashiCorp took on to solve.

Repeated History

Undergraduates Of the University of Washington in Seattle, Mitchell Hashimoto and Armon Dadgar were diligently working on a research project named Seattle - a project that gave them access to program against early versions of cloud computing provided by tech giants Amazon, Google, and Microsoft. The year was 2008 and AWS was just beginning to scale, while Google and Microsoft hadn’t even officially announced anything. Recently, Mitchell described his experience [2]:

And maybe we had two servers here and two servers over here, and we wanted to be able to deploy all of them. So that was my job. And I like to say I failed pretty spectacularly. I couldn’t get it to work.

But these rough beginnings would soon prove to be invaluable experience and the foundations of their empire. Mitchell continues on in his interview about how they discovered a bunch of issues in the process which ultimately would become the groundwork for HashiCorp years later.
After a brief stint at a mobile ad company, Mitchell and Armon found themselves in a carbon-copied situation a whole four years later. As it first arose, their cloud issues began as a discussion of their familiar pains of provisioning, securing, and connecting cloud infrastructure. After Mitchell and Armon revisited their time tested list, captivated by its prophetic encompassment of the cloud, the two decided to take a stab at solving cloud computing. And with their clever hindsight, HashiCorp was born.

A Wiki Summary

As always, the cornerstone of an essay is it's ability to interpret and contextualize the wikipedia summary. So, HashiCorp is a completely software-based company in San Fransisco, California (surprise) [3]. The software is ultimately intended to provide a simple interface for users to manage their cloud services, while still providing the functionality and scalability of direct cloud implementation. This is labeled the HashiCorp Stack and defined, verbatim [4]:

Our tools provide a control plane for each layer of the cloud, enabling enterprises to make the shift to a cloud operating model. Each product addresses specific technical and organizational challenges of cloud infrastructure automation.

The main tools in their stack are Terraform, Vault, Consul, and Nomad - each with unique and necessary functionality, but all interconnected. Let's dive more into each of these.

Terraform

A HashiCorp introduction [5]:

Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions.

What this essentially means is Terraform, at a high level, is a tool that allows you to specify cloud providers such as AWS or Google Cloud or even an in-house data center. The reason versioning is mentioned is to suggest its intent to label and map out how your cloud works in a simple way. Next, lets dive into the technicals.

Configuration files describe to Terraform the components needed to run a single application or your entire datacenter. Terraform generates an execution plan describing what it will do to reach the desired state, and then executes it to build the described infrastructure.

These configuration files are the building blocks of Terraform and the foundations of Infrastructure as Code. When stitched together, all managed by Terraform, a unique blueprint of your datacenter is created that will incrementally change as versions are updated.
In between the generation of a blueprint and the creation or recreation of a data center, Terraform creates an execution plan that acts as a step of verification and acts to abstract certain technicalities from the higher level blueprint. Another interesting approach Terraform takes is to parallelize resources that don't have interdependencies, allowing the deployment to be much quicker and offer visibility into the resource dependency structure.
While these steps appear complex, independent processes, the entire orchestration is managed by running Terraform CLI (command line interface) commands against a *.tf file. For further detail, Armon Dadgar gives a walk through below and the documents provide a building tutorial complete with videos: https://learn.hashicorp.com/terraform/getting-started/intro

Vault

There are two primary use cases for HashiCorp Vault - Secret Management and Data Protection [6].
Vault Secret Management provides secure storage and management of tokens, passwords, certificate, encryption keys and more through central storage across on-prem infrastructure and the cloud. Basically this means wide distribution, yet tightly secured storage, of important information.
As one would expect, wide distribution of sensitive information becomes dangerous as transmitting secrets is risky. Data Protection service to the rescue! As a solution, Vault offers encryption as a service connecting Secret Management through Data Protection. Encryption and decryption are managed through Vault using HTTP (TLS) api calls, providing a barrier to unintentional decryption.
Implementation is handled by grouping roles specifically verified against Vault.
Below, Armon Dadgar shares his introduction to Vault:

Consul

Now, Terraform and Vault are applicable off the ground, but Consul becomes important when scaling and connecting multiple resources. As an individual, this is difficult to replicate, but large infrastructure requires Consuls heavy lifting for devops. Here is HashiCorp's description [7]:

A multi-cloud service networking platform to connect and secure services across any runtime platform and public or private cloud.

As infrastructure expands and contracts dynamically, various services may create interdependencies inside an infrastructures network. Consul extends the service to dynamically connect clouds and datacenters with dynamic IPs, ephemeral containers, dominated by east-west traffic, no clear network parameters [7]. While Consul manages the connections between these resources, it reveals a centralized registry with the convenient ability to locate any service.
Other key metrics that can be gathered from this service include real-time health statuses across various environments and reduction of manual ticketed network tasks. This includes dynamic load balancing and governance to adjust dependency outage nightmares. And, this is all easily accessible through a central control plane to limit human interaction with the network.

Nomad

While scaling infrastructure broadly, it is crucial to scale individual applications in a flexible, yet simple, approach. So, HashiCorp appropriately build Nomad as a workload orchestration tool for deploying, managing, and scaling any legacy or containerized application across multiple regions on both public and private clouds [8].
Some dynamic features that Nomad offers include pooling resources between on-prem and clouds, automated mixed workflows for development, and optimizing resource utilization with bin-packing [8]
Another important feature is the extensibility to orchestrate both legacy and containerized applications without rewrite using the same automation. This allows cheap upgrades, greatly simplifying incremental build processes by eliminating deployment orchestration tooling.
These deployments are handled through "declarative infrastructure-as-code HCL" [8], meaning deployments are inexpensively configurable. This also implies zero downtime as migrations can occur efficiently with multi-cloud environments.
HashiCorp offers successful case studies for various infrastructure sizes:
HashiCorp Nomad Enterprise

HashiCorp Nomad Enterprise

Nomad is a single binary, both for clients and servers, and requires no external services. It is designed to natively handle multi-datacenter and multi-region deployments and is cloud agnostic. Developers define application deployment requirements, constraints, and preferences using Nomad's job specification.

HashiCorp 2020++

As a final note, let's look at the future of HashiCorp. On June 22, 2020, HashiCorp, or more specifically Mitchell Hashimoto himself, announced the HashiCorp Cloud Platform. This is a "fully managed cloud offering to automate deployment of HashiCorp products on any cloud provider." [9]
The goal of this service is to further simply the deployment of cloud resources for smaller business and entities with minimal infrastructure demands. Multi-cloud infrastructure can be costly and impractical to navigate for a small organization; and while HashiCorp services provide great flexibility with streamlined tools, small business do not even need the configurations that this can provide. Ultimately, I believe this will shift toward a one-click-deployment process, more similar to https://vercel.com, but with the sophisticated insights and dynamic services that HashiCorp tooling provides.
For more information, Mitchell Hashimoto goes into much greater depth in his blog post: https://www.hashicorp.com/blog/announcing-cloud-platform

Thanks!

I hope you were able to gain insight into HashiCorp and the incredible journey of Mitchell and Armon, so far, breaking into cloud computing - a field dominated by tech giants with massive infrastructure. As a software engineering student myself, I strive to understand the landscape as broadly as possible and identify the pervasive trends. One of which, I believe is the accessibility and connectivity service HashiCorp provides for the cloud connecting.
Thank you for reading and be sure to subscribe and reach out @abigger87 (Twitter)

Works Cited

1. Regalado, A. (2020, February 11). Who Coined 'Cloud Computing'? Retrieved June 21, 2020, from https://www.technologyreview.com/2011/10/31/257406/who-coined-cloud-computing/
2. HashiCorp. (2019, January 10). What is the HashiCorp origin story? Retrieved June 21, 2020, from https://www.hashicorp.com/resources/what-is-the-hashicorp-origin-story/
3. HashiCorp. (2020, April 30). Retrieved June 21, 2020, from https://en.wikipedia.org/wiki/HashiCorp
4. HashiCorp. (2020, May 14). Infrastructure enables innovation. Retrieved June 21, 2020, from https://www.hashicorp.com/
6. HashiCorp. “HashiCorp Vault - Manage Secrets & Protect Sensitive Data.” HashiCorp, 22 June 2020, www.hashicorp.com/products/vault/.
7. HashiCorp. “HashiCorp Consul - Connect and Secure Any Service.” HashiCorp, 22 June 2020, www.hashicorp.com/products/consul/.
8. HashiCorp. “HashiCorp Nomad Enterprise.” HashiCorp, 5 May 2020, www.hashicorp.com/products/nomad/.
9. HashiCorp. Announcing the HashiCorp Cloud Platform. 21 June 2020, www.hashicorp.com/blog/announcing-cloud-platform.