DevOps has seen a lot of success stories, on the contrary DevSecOps has been a real struggle. DevOps teams got busy in welcoming customers with new and continuous releases, unknowingly, they welcomed application vulnerabilities. DevOps enabled developers with speed and agility to roll-out features faster and goals to meet the deadlines to do so, they started taking security shortcuts leading to the DevSecOps failure stories. But with new processes like Infrastructure as Code it is easy to create immutable servers helping DevOps team to include security in their CI-CD pipelines. In this article, we will talk about how mutable servers create a security threat and how immutable infrastructure lead by immutable servers can be of help for DevSecOps teams. 
Before moving further, let us define Mutable Infrastructure for common understanding throughout the article –
Mutable Infrastructure is the IT infrastructure subjected to change as per the developer, application or deployment requirements. Traditionally, IT teams changed same structure with the changing requirements. With DevOps, even developers have the access rights to change the infrastructure to Staging or Pre-production environment, if not to the Production environment.
These jumbled decisions of developers to change the environment or Infrastructure, whenever required was necessary to speed up releases. But Mutable Infrastructure gave rise to security threats such as not keeping up to documentation process making version tracking difficult for everyone. Consequences trickle down to everyone including security, testing, QA, product managers to keep up with the changes leaving the loopholes for security. Hence the concept of Immutable Infrastructure came into picture.
Infrastructure Security
As we are talking about Immutable servers, it is essential to highlight challenges with infrastructure security. When it comes to securing infrastructure, systems or server access is the most common practice. Next is, patching and vulnerability management to update software timely. Companies not doing patching regularly are highly susceptible to attack. Organizations are trying to automate and standardize patching processes to avoid these attacks but automation, also requires, considerable time and continuous monitoring. API management is another challenge in Infrastructure.
With DevOps and Automation practices, Configuration management has become very tricky. In mutable infrastructure, tools such as Chef, Puppet, Ansible automatically updates software on the server creating long history of changes very hard to reproduce and difficult to diagnose for any problems occurring. Often, in mutable infrastructure DevOps teams face the situation where someone modified server leading to poor production server, leading to downtime or unknown consequences.
Sounds like a lot is going on in mutable infrastructure–let’s see how Immutable servers can reduce or eliminate the chances of configuration drift.
Immutable Servers are here for rescue!
Immutable servers are answers to a lot of security threats by applying unalterable Application Specific Network-as-code enables directly into applications, containers or VMs. Immutable servers strengthen the security posture of DevOps teams as there are no patching or upgrading old servers leaving them to pipeline vulnerabilities. Developers rebuild the server and archive old server. As they do not change the underlying infrastructure, there are fewer chances of open areas for attackers. But with agile development practices, rolling out new immutable servers every time can slow down the release pipeline. So, what’s the way ahead?
Implement Immutable Servers with infrastructure as code
The basic premise of immutability–Servers cannot be patched or modified, and every server is rebuilt. This entails the complete rebuilt of the entire OS and application stack for every fix or upgrade. DevOps teams implement immutable servers by baking entire server and software configuration into one reference server and use it to rebuild servers, whenever it is required.
Infrastructure as code (IaC) doesn't quite trip off the tongue, and its meaning isn't always clear. Infrastructure as code is the process of provisioning and managing resources in public clouds such as AWS, GCP, and Azure via a set of editable text files that describe how and where infrastructure resource configurations are deployed. These input scripts are ingested by Infrastructure as Code tools which then automatically create, update, or delete resources to ensure that the cloud infrastructure complies with the state defined in the IaC file. Infrastructure as code tools are designed to manage the implicit and explicit dependencies between cloud infrastructure resources so that the creation, maintenance, and removal of resources is performed logically and takes into account these resource dependencies.
The high-level steps of creating immutable servers are very similar to infrastructure as code. Just like in mutable servers, it starts from creating a reference server image–including baselines OS and other installed packages. The infrastructure as code tools such as Chef or Jenkins are used to configure the server with the right security codes to bring it to a state required for hosting code. For incremental fix and upgrades, create a new server image from the reference server using infrastructure as code tools and run prescribed, automated tests to test the new server image. Once the new server image passes all the tests, use it to deploy code to the production. Once the new production server is live, destroy the previous server. Using infrastructure as code the benefits of security as code, automation and repeatability are already passed on to the Immutable Servers for better DevSecOps. Packer is one such tool used as a compilation too to launch a server, run Chef, and turn it into an image.
Easy to monitor and automate–The codification of security practices in infrastructure as code gives discipline to the entire process of rolling out new environments in immutable infrastructure. Once all the security practices are coded and put to automation mode, there’s no room for manual error and false interpretations. As the Gene Kim Thought Experiment explains–What is easier to secure? 10 disparate systems or 10,000 identical ones? Immutable servers simplify it further as there’s only one server that teams have to look after, make changes, update and then destroy.
Ease of testing–Immutable Servers are easy to test and provide accurate test results because with a pre-built server everything is already been baked into it and there are smaller chances of immediate deviation. But later down the months or even days, developers have to change the server and rebuilt it. As Mitchell Hashimoto – CEO of HashiCorp, said in one of his interview, Immutable Infrastructure is to configuration management what a compiled application is to source code.
Immutable Servers Strategy for DevSecOps
Successful DevOps earns revenues, but successful DevSecOps continues it. Feeling that your application is secure can create a positive impact on the frequent software deliveries. But, checking infrastructure security manually every time before the release can contribute to fatigue, especially for Security and Developers, for whom the number of security checks can be endless.
DevSecOps is a logical and natural extension of DevOps. The way configuration management tools (such as Puppet, Chef and CFEngine) create mutable servers, DevSecOps can use them for creating immutable servers. Instead of using configuration synchronization for changing server configurations, new server is rebuilt with the base image. This prevents the state of configuration drift as the reference base image is taken as default. With immutable servers, it becomes easy for teams to update base images rather than managing thousands of servers and maintaining cumbersome documentation for their configuration management.
Initially, it can be a little challenging for organizations and teams to adopt it. Without a mature DevOps practice, it is tough to implement immutable infrastructure. Organizations have to invest in architects having skills to create immutable infrastructure and manage it. Developers need to understand the new infrastructure where application is operating. Application changes will be deeply connected to infrastructure changes and there are no provisions to make changes in the environment. Similarly, IT administrators have to be prepared for unavailable instances for few seconds while the new servers are rebuilt and deployed.
As cyberattacks continue to grow, immutable infrastructure becomes even more important for DevSecOps teams. Immutable servers provide security protection against malware injections in servers, compromised code, patching and upgrading vulnerabilities. Immutable servers block lot of ways cybercriminals can access your infrastructure and strengthen DevSecOps. A reliable DevOps solution provider could help you in figuring out a way to DevSecOps.
