Introduction to AWS CloudFormation Drift Detection

On Nov 13, 2018, AWS announced that AWS CloudFormation now allows users to detect any changes made to the deployed AWS stack. It is an excellent way for users to detect configuration changes made to the AWS stack resources made outside of CloudFormation via the AWS Management Console, CLI, and SDKs. Think of drift as the difference between the expected configuration values of stack resources defined in CloudFormation templates and the actual configuration values of these resources in the corresponding CloudFormation stacks. This allows for better management of the AWS stacks managed through CloudFormation and ensures consistency in resource configurations. In this blog, we will see some worked examples when using drift detection in action.

Before we witness drift detection in action it is important to understand what drift is and what do we mean by drift detection. Drift detection enables all users to detect if the actual configuration of stack resources differs, or have drifted, from the expected configuration. With the help of this feature, the user can detect drift on the entire stack, or on selected resources from within a stack. If the user has enabled drift detection on the whole stack, then it will be considered as drifted if one or more than one of the resources have drifted and CloudFormation will generate detailed information on each resource in the stack that has drifted. If the drift detection is enabled on a selected resource, then the resource is considered as drifted if its current property values differ from the expected values.

There are several examples of how a stack or resource can drift away from its expected configuration. Users can knowingly or unknowingly edit resources directly through the service that created the resource when the stack was deployed. For example, users can use the Amazon EC2 console to update a server instance that was created as part of a CloudFormation stack. After a while, such changes are impossible to track and this issue becomes a huge concern when there are multiple users are working on the same stack or its resources. All such changes which are made outside of CloudFormation can complicate stack update or deletion operations. In order to solve this issue, users can use drift detection to identify stack resources to which configuration changes have been made outside of CloudFormation. When the drift detection has identified a drift on a stack or selected resources you can then make appropriate changes to the stack resources so that the stack is in sync with their original definitions in the stack template. Resolving drifts helps to ensure configuration consistency and successful stack operations.

CloudFormation can only detect drift on the resources that support drift detection. Resources that do not support drift detection are assigned a drift status of NOT_CHECKED which simply means that AWS CloudFormation has not checked if the stack differs from its expected template configuration. Users can perform drift detection on stacks with the following statuses: CREATE_COMPLETE, UPDATE_COMPLETE, UPDATE_ROLLBACK_COMPLETE, and UPDATE_ROLLBACK_FAILED.

One thing worth mentioning here is that CloudFormation does not detect drift on any nested stacks. If users wish to detect drifts on nested stacks, then initiate drift detection operation directly on the nested stacks.It is important that we go over the important status codes that CloudFormation assigns to stack drift detection operations, Stack Drift Status, Resource Drift Status, and Property Difference Types. Below table gives a brief overview of CloudFormation status codes:

Picture1.png

Hopefully with this background we now have enough understanding on drift detection to proceed with the step by step tutorial and using drift detection in action:

 

Pre-Requisites:

  1. Amazon AWS Account.

  2. Default VPC with default subnet created.

 

Step-by-Step Instructions:

Here are the steps involved in drift detection tutorial:

1. Login to your AWS account and head over to the AWS CloudFormation console.

Step-1.png

2. For the sake of this tutorial we will launch a new stack which will launch a simple EC2 instance. Click on the Create Stack button.

Step-2.png
Step-3.png

4. Click Next and enter the stack name and specify parameter values.

5. Click Next and configure tags.

6. Set the monitoring time to 0.

Step-6.png

7. Expand Advanced section and check Enabled against Termination Protection.

8. Click next and review the stack.

Step-8.png

9. Click Create to launch the stack.

10. When the stack has been created successfully you will see the stack status as CREATE_COMPLETE.

11. Now we will simulate a scenario where a stack configuration has been changed outside of the CloudFormation stack. To accomplish this, we will head over to the EC2 console and delete the SSH inbound rule.

12. Delete the rule SSH rule by clicking on Edit and then clicking on the cross symbol at the end of the rule.

13. Now we will head back to the CloudFormation console, select our stack and click on Detect Drift.

Step-13.png

14. Click on Yes, Detect on the screen.

15. Once the drift detection process has been completed observe the Drift Status.

Step-15.png

16. We can see that the stack has been DRIFTED. Now let’s try to find out what exact configuration has changed from the CloudFormation console. Click on View Details next to DRIFTED.

17. From this screen we find out that our EC2 instance is IN_SYNC and nothing has changed regarding that. However, the instance security group shows the resource status of MODIFIED. In order to find out what exact configuration has changed we will expand the InstanceSecurityGroup.

18. Here we can clearly see that the SSH ingress rule from the security group has been removed. Now, we will run another drift test after adding the SSH rule into the security group.

19. Re-run the drift test again by clicking on Detect drift and click on Yes, Detect on the following screen.

Step-19a.png

20.After the drift detection process is complete we will see that stack and resource drift statuses are IN_SYNC.

Conclusion:

In this blog, we witnessed a very simple use case of drift detection. This new feature has proved to be of much more value in complex environments and stack which spins up hundreds or even thousands of resources in AWS. We hope that this blog will help you understand the basics of drift detection and get started with drift detection on your production environments.

 

Note: Please note that drift detection is available in the US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Canada (Central), Asia Pacific (Mumbai), Asia Pacific (Seoul), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Paris), and South America (São Paulo) regions.


Moiz Arif
Bluestack IT Solutions