ISE Blog

Continuous Integration and Compliance Enforcement

The ever-increasing torrent of reports of “misconfigured” S3 buckets contributing to egregious breaches of customer data is an epidemic, unfairly placing a black mark on the name of Amazon Web Services’ (AWS) outstanding object store. Worse, these breaches are completely avoidable through the application of simple automated compliance enforcement. Tying my last two posts together, let’s take a quick look at how applying a DevOps mentality could save these companies the public embarrassment and expense of remediation that results from human carelessness.


We want automation to serve two specific operational goals. The first is reduction of human error. Second is reduction of work by improving consistency. This is accomplished by using configuration tools such as Terraform and Ansible. Configuration as code allows error mitigation to become part of the development process through peer review and static code analysis. It also has the benefit of being repeatable: both on a specific resource, and across a set of resources which serve the same purpose or have the same associated policies or regulatory requirements.

Adding these configuration tools to an integration server such as Jenkins allows static analysis to be triggered by a change to the configuration code, and periodically to enforce that configuration, confirming that the settings in production match the expectations set by the code.

In Practice

The most common misconfiguration of an S3 bucket in AWS is to set that bucket “public” when you really want it private. Here’s a sample terraform template defining the bucket:

resource "aws_s3_bucket" "b" {

  bucket = "secret-customer-data"
  acl    = "private"
  tags {  
    Name        = "My secret customer data"
    ContainsPII = "true" 


Now if we were really doing this right, we would also add a policy to force each object to be private, and to add object encryption. For now, we just want to make sure that this bucket isn’t publicly browsable. Note the line acl = “private”. That is enough to achieve our objective. We’ll put that file in a private git repository.

Now for the automation. On Jenkins, install the terraform plugin. Create a terraform installation from the “global tools” section of “Manage Jenkins.”

Jenkins Terraform Plugin

Configure a new freestyle job. In the build environment add your private git repository. Set the build triggers to “build periodically” with the desired schedule. As shown here, the enforcement will be checked daily at midnight.

Build Triggers

Under “environment” select Terraform, then add the credentials necessary for the target environment, generated from the IAM control panel. If you want to be a little safer, you can use STS switching to grant temporary access, or an EC2 instance profile if your Jenkins is in your AWS environment.


When this job is saved, it will run daily at midnight to validate that the bucket meets the defined properties in the terraform script. This can be modularized to be repeatable across many buckets and environments. The last step would be to add some failure notification, so that if for some reason it cannot enforce the desired policy, an administrator is notified. 

This type of automation is not adequate to protect against bad actors on the inside. It can however provide some peace of mind that carelessness will not lead to a breach. Other infrastructure can have similar push-based enforcement, including security groups and network ACLs, as well as user and group access policies. The types of enforcements you need will depend on your environment.

What do you think? Questions about the Cloud or Continuous Integration? Comment below and join the conversation!

Samuel Thurston, Software Engineer

Samuel Thurston, Software Engineer

Samuel Thurston is a Software Engineer and Cloud Practice Lead for ISE, architecting and implementing cloud solutions for enterprise clients. He enjoys running, yoga, and cooking, and is frequently found on the disc golf course.

Samuel Thurston, Software Engineer

Latest posts by Samuel Thurston, Software Engineer (see all)

What’s Wrong with Monoliths? Oct 25, 2018

FaaS: Function as a Service Sep 28, 2018

Networking is Hard Aug 23, 2018

Observability in Distributed Systems Jul 19, 2018

Hands-On With Amazon AWS DeepLens Jun 21, 2018