2021

Terrascan Expands Beyond Policy as Code for IaC

Terrascan emerged from the need for a scalable way to ensure that cloud infrastructure configuration adheres to evolving security best practices. It helps identify issues such as missing or misconfigured encryption on resources and communication, and inadvertent exposure of cloud services. Terrascan fundamentally enhances the value of Infrastructure as Code (IaC) used by organizations to define and manage cloud infrastructure, and improves security by enabling teams to eliminate risk before infrastructure is deployed.

Policy as Code has proven a great fit for enforcing policies in automated workflows, and Terrascan’s focus on Infrastructure as Code (IaC) such as Terraform and Kubernetes has filled an important role. As we integrated Terrascan into DevOps tooling, pipelines and software supply chains, we realized that IaC is not the only source of risk we can help mitigate before deployment.

Over the past few months, we’ve been busy expanding Terrascan’s ability to enforce policies across multiple important dimensions of risk.

Addressing Risks in IaC and Container Source Files

We added support for scanning Dockerfiles, to help identify security misconfigurations in container-based architectures including Kubernetes. We also added support for Azure Resource Manager (ARM) templates, an IaC technology that’s native to Azure, and for scanning CloudFormation templates including nested stacks.

Adding support for new input formats includes creating policies for relevant security risks in those formats. For example, we added numerous Dockerfile policies including one to ensure that Dockerfiles don’t RUN commands as root.

Managing Vulnerability Risks in Containers

In addition to identifying security misconfigurations in IaC, Terrascan can now identify security vulnerabilities in container images referenced from Elastic Container Registry (ECR), Azure Container Registry, Google Container Registry, Google Artifact Registry, and Harbor. This is especially useful for ensuring the absence of vulnerabilities in custom and third-party containers used and built in your development pipelines.

Enforcing Policy During Deployment to Kubernetes Clusters

Terrascan can now be used as an admission controller for Kubernetes, enforcing policies as changes are applied to the cluster and preventing security issues from making it into the runtime environment. Gone are the days of managing different tools and policies for pipelines and runtime. Terrascan enables you to enforce the same policies in development and runtime!

Improving Relevance with Prioritization and Filtering

Finding violations is important, but sometimes less is more. Terrascan now offers the ability to prioritize or de-prioritize issues based on specific levels of severity for policies in each IaC file. This helps tailor policies within Terrascan for the particular use case or architecture being tested, so you can focus specifically on the issues you care most about.

Deeper Integrations, Better Automation

Automating policy enforcement into existing workflows ensures that controls are applied consistently without increasing friction in the development process. The Terrascan GitHub action can be used within GitHub workflows to detect and prevent security issues from being merged into your code repository.

For those that need earlier enforcement, evaluating IaC even before code is committed into version control, Terrascan provides a pre-commit hook which runs on the developer workstation before code changes are committed into the git repository.

Better Collaboration with Reporting and Compliance Processes

Enforcing policies in development processes is important, but it’s sometimes just as important to be able to demonstrate that development processes are continuously compliant with the security policy. Terrascan has added support for SARIF output to facilitate integration with common compliance and security reporting tools, such as GitHub security reports.

Easier Custom Policy Development for Advanced Needs

Terrascan includes an extensive library of policies and best practices, but sometimes you need to implement policies that address your unique requirements or circumstances. The newly published Terrascan Rego Editor, a VS Code extension, makes it easier to create and test policies with a seamless workflow driven in the IDE.

Automating Terraform Security with pre-commit-terraform and Terrascan

One of the best things about using Terraform to manage your systems is that best practices can be defined and applied to your templates in a manner similar to what is done with application code. This means that linting and testing the infrastructure as code (IaC) templates for quality, operational, and security issues can be accomplished easily, using some of the same workflows as application code. The most common place to enforce coding best practices is in continuous integration (CI) pipelines.

You can configure your CI pipelines to help enforce compliance with your coding policies by automatically executing validation scripts and tools and failing the CI job whenever there’s a violation. These can be configured to prevent any merges to your main branch if there’s a violation that could be considered a blocker to your system.

One of the drawbacks to relying on pipelines for enforcement is that they are bottlenecks in the development process. Every code change goes through them, and one bad commit can break the build for everybody. This can be extremely disruptive when teams are iterating quickly. One way to avoid this problem is to use a pre-commit hook to enforce standards locally on a developer’s system before pushing code to the repository. By adding a pre-commit configuration to your repository, you can help those contributing to your repository test locally to ensure their commit will not break the build once it is pushed and CI pipelines are completed.

For Terraform, one of the best projects out there to configure your repository’s pre-commit scripts is pre-commit-terraform. The pre-commit-terraform project includes multiple git hooks specifically for Terraform that can help with linting, documentation, operational, and security compliance. As part of its security toolbox pre-commit-terraform includes Terrascan, the static code analyzer for IaC maintained by Accurics.

Terraform security with pre-commit-terraform

Let’s take a look at how Terrascan can be leveraged to find Terraform security issues using pre-commit-terraform in an example.

The first step is to install pre-commit. Alternate ways of installing are available here. ~ ➜ pip install pre-commit

Once pre-commit is installed we can create the repository that will contain our Terraform templates.

~ ➜  mkdir pre-commit-tf-example
~ ➜  git init pre-commit-tf-example
~ ➜  cd pre-commit-tf-example

We’ll add a .pre-commit-config.yaml file in the root of our repository configured to use the terrascan git hook within the pre-commit-terraform repository. We’ll reference release v1.50.0 which is the latest release as of this writing.

pre-commit-tf-example git:(main) ✗ ➜  cat .pre-commit-config.yaml
repos:
-   repo: https://github.com/antonbabenko/pre-commit-terraform
	rev: v1.50.0
	hooks:
	-   id: terrascan
pre-commit-tf-example git:(main) ✗ ➜

Now, we’ll run the pre-commit install, which will configure pre-commit for this particular repository using the information obtained from the .pre-commit-config.yaml file.

pre-commit-tf-example git:(master) ✗ ➜  pre-commit install
pre-commit installed at .git/hooks/pre-commit
pre-commit-tf-example git:(master) ✗ ➜  

Let’s add a Terraform template with a known violation. In this case we’re going to add an AWS S3 object resource that’s missing the encryption at rest configuration.

pre-commit-tf-example git:(master) ✗ ➜  cat main.tf
variable "bucket" {}

resource "aws_s3_bucket_object" "html" {
  bucket   	= var.bucket
  key      	= "index.html"
  source   	= "index.html"
  acl      	= "public-read"
  content_type  = "text/html"
  etag     	= filemd5("index.html")
}
pre-commit-tf-example git:(master) ✗ ➜  

Once we try to commit the Terraform template, pre-commit will execute Terrascan. Since we have not set encryption for this object, Terrascan will fail and provide feedback on what it found.

pre-commit-tf-example git:(master) ✗ ➜  git add . && git commit -m 'add terraform'
[INFO] Initializing environment for https://github.com/antonbabenko/pre-commit-terraform.
terrascan................................................................Failed
- hook id: terrascan
- exit code: 3

Violation Details -
    
    Description	:    Ensure S3 object is Encrypted
    File       	:    main.tf
    Line       	:    3
    Severity   	:    MEDIUM
    -----------------------------------------------------------------------
    

Scan Summary -

    File/Folder     	  :    /Users/therasec/pre-commit-tf-example
    IaC Type        	  :    terraform
    Scanned At      	  :    2021-05-10 03:32:22.117445 +0000 UTC
    Policies Validated  :    607
    Violated Policies   :    1
    Low             	  :    0
    Medium          	  :    1
    High             	  :    0
pre-commit-tf-example git:(master) ✗ ➜  

Let’s fix the issue by configuring server side encryption for the object and try to commit again.

pre-commit-tf-example git:(master) ✗ ➜  cat main.tf

variable "bucket" {}

resource "aws_kms_alias" "a" {
  name = "alias/my-key-alias"
}

resource "aws_s3_bucket_object" "html" {
  bucket             	= var.bucket
  key                	= "index.html"
  source             	= "index.html"
  acl                	= "public-read"
  content_type       	= "text/html"
  etag               	= filemd5("index.html")
  server_side_encryption = "AES256"
  kms_key_id         	= data.aws_kms_alias.a.target_key_id
}
pre-commit-tf-example git:(master) ✗ ➜  

Since we added the server_side_encryption and kms_key_id parameters, the issue should be solved and we should be able to perform git commit.

pre-commit-tf-example git:(master) ✗ ➜  git add .
pre-commit-tf-example git:(master) ✗ ➜  git commit -m 'add terraform'
terrascan................................................................Passed
[master (root-commit) 41138a6] adds terraform
 2 files changed, 22 insertions(+)
 create mode 100644 .pre-commit-config.yaml
 create mode 100644 main.tf
pre-commit-tf-example git:(master) ✗ ➜  

By using a pre-commit hook we’re able to get quick feedback into security issues affecting the code we’re trying to commit. This allows us to solve issues quickly and prevent any misconfigurations from being merged into our baseline code. Terrascan can be leveraged as a pre-commit hook and as part of your CI/CD pipelines to find Terraform security weaknesses and is included as part of the pre-commit-terraform project.

Terraform Security: Terrascan in Atlantis Workflows

Atlantis is a popular open source automation platform for Terraform that leverages an organization’s code repository, such as Git, to streamline and automate Terraform workflows. At its most basic:

  • New Terraform code is introduced as a pull request
  • Atlantis automatically creates a Terraform plan with the new changes.
  • A colleague reviews the pull request, a common development practice.
  • Atlantis will apply the new plan when it receives the command, deploying new cloud infrastructure.

This is a very powerful way to manage complex cloud infrastructure across multiple teams. Moreover, the version control system naturally provides useful features like version control, permission management and audit logs.

At Tenable, we are most excited about how the entire workflow is managed as a single pull request thread. Moreover, at the end of the process, the Terraform code repository should reflect the actual deployed infrastructure. While not explicitly a GitOps approach for Terraform automation, a lot of the core tenets are present. We work well in GitOps workflows, so we decided to explore whether we can seamlessly integrate Terrascan into the Atlantis workflow.

Terraform Security with Atlantis and Terrascan

Terrascan is an infrastructure as Code (IaC) security scanner. Powered by hundreds of OPA’s Rego security policies, it scans Terraform source code and detects security vulnerabilities embedded within them. It allows users to fix vulnerable infrastructure before it is ever deployed. Scanning can be done at different stages of the deployment lifecycle. Ideally, it would be done in multiple stages.

By integrating Terrascan directly into the Atlantis workflow, scan results are reported as part of the same pull request workflow, which will inform the pull request reviewer before approving. In the spirit of automation, Terrascan can fail the automated plan build if a vulnerability of a certain severity is found.

Terraform security - Terrascan finds policy violations in Atlantis workflows

Of course, that is not the only place that Terrascan can fit in the development lifecycle. A developer can scan their code as they are writing it. Terrascan’s Github Action can automatically scan new code as it is pushed, which could actually reduce the turnaround time for a fix. To enforce policy at the organizational level, just before code is applied as infrastructure, integrating Terrascan into the Atlantis flow provides a welcome new level of security.

Getting Started with Terrascan in Atlantis

Terrascan supports being run as a server on a separate environment, so only network connectivity is required to integrate Terrascan into existing Atlantis environments. For even more flexibility, we will soon publish an Atlantis container image that includes the Terrascan binary.

To integrate Terrascan into your Atlantis deployment, please follow the steps in our documentation.

2020

Kustomize gets Policy as Code with Terrascan

Most organizations – in fact, over 78% – leverage Kubernetes in their move to cloud-native applications. This powerful and flexible platform enables teams to deploy and manage sophisticated systems while delivering innovation to market faster than ever. Unfortunately, that power and flexibility often requires significant care and effort, especially as apps migrate through their lifecycle. Each new environment may require configuration and connectivity changes which need to be tracked and managed.

Kustomize helps teams address these challenges by providing a way to tweak configurations based on declarative overrides. It works great for building environment-specific configurations into automated processes without adding brittle pre-processing steps or managing external properties and templates.

Given the popularity of Kustomize and Kubernetes – they are regularly downloaded millions of times each month – we’re excited to include support for both Kustomize and Kubernetes in release 1.2.0 of Terrascan.

Terrascan is an open, extensible architecture that enables teams to enforce policies and compliance in numerous types of IaC, and has supported Kubernetes for some time now. With the addition of Kustomize support, it now has a better understanding of those configurations and can deliver more accurate and relevant results. Policy as Code guardrails are a best practice for cloud native development, and this is just one more example of Terrascan’s commitment to delivering that capability for the most popular technologies.

Kustomize and Terrascan: Getting Started

Getting started with Terrascan is easy, regardless of whether you use the portable Go binary, a Docker container, or you build from source. The command line interface works well in many contexts, regardless of whether you want to run from a shell, a script, or in a pipeline.

Start in the directory where your Kustomize project is stored and launch Terrascan:

terrrascan scan -i kustomize

Kustomize support is implemented as a new IaC provider, leveraging Terrascan’s extensible architecture. The “scan” command will scan the infrastructure, and the “-i” option enables the Kustomize IaC provider.

Terrascan defaults to scanning from the current directory, and you can add the -d option one or more times if you would like to scan other directories. The output will be sent to the terminal in YAML format by default, and includes a summary of the results as well as the details needed to prioritize and fix the findings. The structured output works well for programmatic processing and is easy for humans to read.

Please tell us what you’d like to see next! Feedback is always welcome in our Community Discord or via Terrascan GitHub Issues.

Terrascan extends Policy as Code to Kubernetes

Accurics is excited to announce Terrascan v1.1.0, with Kubernetes (k8s) support! Cloud native apps and infrastructure are notoriously complex and difficult to secure with traditional tools, and kubernetes adds automation and orchestration that es##calate those problems to another level. Practically speaking, security automation is mandatory because it’s not realistic to expect humans to comprehend such complex, dynamic environments.

Terrascan is an extensible, open source tool that enables teams to detect compliance and security violations across Infrastructure as Code to mitigate risk before provisioning cloud native infrastructure. By adding k8s support to Terrascan, we’re ensuring that all teams, regardless of budget, have access to the tools they need to secure their cloud native apps and infrastructure well before they are ever deployed in the cloud.

Release 1.1.0 works with k8s YAML and JSON configurations, and includes policies for security risks present in those files. Future releases will add support for k8s infrastructure managed through other IaC providers such as Terraform.

Using Terrascan with Kubernetes

Terrascan is usually run as a portable Go binary or a Docker container. Its command line interface can easily be adapted to run it from a terminal, a script, from within a pipeline, and numerous other contexts. To use it, simply run terrascan from a directory where your kubernetes project lives.

terrascan scan -t k8s

Terrascan defaults to scanning YAML and JSON files in the current directory and subdirectories. If your project spans multiple directories, you can use the -d option one or more times to specify which directories to scan.

By default, output is sent to the terminal in YAML format.

The structured output includes a summary of the results as well as the details needed to prioritize and fix the findings. It’s suitable for humans to read, and for programmatic processing.

We’re just getting started, and we’re excited about the opportunity to help teams secure their cloud native apps and infrastructure. Join us in the Community Discord for more Terrascan tips and tricks, and stay tuned for more exciting announcements about new technologies and policies that cover even more of the cloud native landscape.

Terrascan Leverages OPA to Make Policy as Code Extensible

I’m really excited about our release of Terrascan v1.0!

Brief history of Terrascan and Accurics

When I created Terrascan, I was working on a big cloud migration project, doing assessments on my employer’s cloud security posture. One of the most tedious parts of the job was manually reviewing Terraform to ensure it adhered to security best practices. My second child was born, and I was having sleepless nights during my paternity leave.

I kept thinking about the problem we had where developers relied on security experts to help them secure their Infrastructure as Code (IaC). I thought that there should be a way to automatically scan IaC similar to what we were doing for application code (e.g. Java, Python, C#, etc.), where we had static code analysis tools to give developers immediate feedback on security risks.

At the time, I couldn’t find any existing tools that could scan Terraform and meet the requirements I had. So I put something together with some Python packages that worked with Terraform HCL, some regexes, and that’s how Terrascan was born. The project was really useful and I decided that the best way to maintain a project like this would be to open source it.

A couple of months ago, I joined Accurics to start the next phase in Terrascan’s journey. We have a lot of exciting ideas for Terrascan, and perhaps a few surprises in store. The release of v1.0 is the first step on that path.

New features, Extensibility

Terrascan was initially designed to serve a specific need: static analysis of Terraform templates for the security risks my applications faced. A primary goal of Terrascan v1.0 is extensibility. We’ve introduced a pluggable architecture that can use the same approach to scan Terraform, AWS CloudFormation, Kubernetes and any other type of IaC tooling. We will be adding new configuration languages in the future, and it’s easy to extend to your needs.

In addition to allowing multiple input formats, we wanted to have a standardized way to create and apply policies across these IaC tools. So we replaced the regular expression based rules with the Open Policy Agent (OPA) engine from the CNCF. Using the Rego policy language it’s easy to create, modify, or extend the policies that apply to your specific needs.

The new architecture uses a common intermediate language for IaC with the idea being to be able to leverage the 500+ policies we’re initially releasing with v1.0 across AWS, GCP, and Azure in a provider agnostic way. This means, for example, that the policy we wrote to detect public S3 buckets can be leveraged when scanning Terraform, CloudFormation, or any other IaC language that provisions AWS resources.

With v1.0 we’re also introducing server mode. This allows you to run Terrascan as a server, where an API will give you the ability to scan any IaC being sent with the policies configured. With this capability, Terrascan can be used as the central hub of policy enforcement across your organization.

Given this new extensibility, we wanted to ensure that users have a place to share ideas and experience. So the release of Terrascan v1.0 is accompanied by the launch of the Accurics Community, a place for users of Terrascan and other Accurics solutions to discuss and collaborate. Please check it out, and remember that it’s only as useful as what you put into it.

Dedicated to open source, vision for future

Security is an important, foundational concern of any cloud project, and open source tools like Terrascan help to standardize and democratize security in a way that anyone can contribute to. It benefits all organizations, and the community itself, to have security policies exposed for everyone to look at so we can quickly identify the best practices, and then apply those consistently across all applications.

Be safe.

Securing Infrastructure as Code Using Terrascan

I remember one of my first public cloud projects. We created a cross functional team that included representatives from the business, developers, architects, security, and operations. The goal was to have a minimum viable product for an important customer facing system as our first cloud native deployment in 12 weeks.

At first the task seemed daunting. There was a lot to learn and implement in a short period of time, and as the representative of the security team, I wanted to make sure security was embedded into every decision we made.

That meant having a scalable way to review and provision network security settings and configuration, identity and access management policies, and ensuring that any cloud resource was configured following security best practices.

IaC Benefits

Around that time I discovered a tool called Terraform and the concept of Infrastructure as Code (IaC). Using Terraform we were able to quickly provision our infrastructure in a consistent manner where the code to provision our infrastructure lived side by side to our application code.

At the same time, using Terraform to provision and manage the security of our cloud environment meant that development teams had greater visibility into how the security of the infrastructure was configured, how it affected our application, and they were empowered to submit pull requests if there were any changes needed.

This was a huge benefit compared to the way things were done in our on-premises data center. Where we used ticketing systems to engage the security team and from the perspective of developers security was a black box that a siloed team handled.

The Challenge

Although IaC empowered our development teams to take ownership of their infrastructure and devops lifecycle, it also presented some challenges. Our architecture was increasingly complex due to being in a hybrid environment and the pace of change was increasing as we increased our cloud adoption.

Security defects in our IaC could be augmented and replicated through the environment if we didn’t have a way to review or control changes to prevent security defects. Issues like exposing your private network to the public internet, not encrypting any sensitive data at rest, or missing access logs could put the environment and business at risk.

What’s Terrascan?

As I thought about these issues I realized that the same techniques we were using for our application’s code like static code analysis could be used to identify security weaknesses in our IaC. This would ensure security best practices were embedded as early as possible into the development lifecycle.

To solve this I developed Terrascan. Terrascan is an open source static code analyzer for Terraform. It helps you test your Terraform code to find security weaknesses including:

  • Server side encryption misconfigurations
  • Using AWS Key Management Service (KMS) with Customer Managed Keys (CMS)
  • Encryption in-transit SSL/TLS is not enabled and configured properly
  • Security Groups open to the public internet
  • Inadvertent public exposure of cloud services
  • Access logs not enabled on resources that support them

Using Terrascan

To install Terrascan, you’ll need Python 3.6 or later installed in your system.

$ pip install terrascan
Collecting terrascan
  Downloading terrascan-0.2.1-py2.py3-none-any.whl (28 kB)
Requirement already satisfied: pyhcl>=0.4.4 in ./.pyenv/versions/3.7.5/envs/terrascan/lib/python3.7/site-packages (from terrascan) (0.4.4)
Installing collected packages: terrascan
Successfully installed terrascan-0.2.1

Now that you have Terrascan installed, lets scan some code. Here’s an example s3_bucket resource that’s missing encryption.

resource "aws_s3_bucket" "my_insecure_s3_bucket" {
  bucket = "my-insecure-s3-bucket"

  logging {
	target_bucket = "logging_bucket"
	target_prefix = "log/"
  }

  tags = {
	Name    	= "my-insecure-s3-bucket"
	Environment = "production"
  }
}

Here are the results of running Terrascan against that resource.

$ terrascan -l .
Logging level set to error.
........................................................................
----------------------------------------------------------------------
Ran 72 tests in 0.005s

OK

Processed 1 files in /Users/therasec/test/.

Processed on 06/20/2020 at 22:45
Results (took 0.26 seconds):

Failures: (1)
[high] [aws_s3_bucket.my_s3_bucket] should have property: 'server_side_encryption_configuration' in module ., file /Users/therasec/test/./s3.tf

Errors: (0)

As you can see, Terrascan detected that the s3 bucket resource is missing the “server_side_encrytpion_configuration”.

Terrascan can be installed as a pre-commit hook to help detect issues before code is pushed into your repository. It can also be integrated into your CI/CD pipeline. You can learn more about Terrascan or contribute at github.com/accurics/terrascan.

Accurics Commitment to Open Source

I am thrilled to join the Accurics team and continue to support Terrascan together. We’re also committed to increasing our contributions to the open source community - you can expect to see more projects from us in the future