CIS AWS Foundations Benchmark Monitoring with Sumo Logic

The Center for Internet Security (CIS) released version one of the CIS AWS Foundations Benchmark in February this year. It’s a fantastic first draft, and represents the minimum security controls that should be implemented in AWS.

4 Sections of the CIS AWS Foundations Benchmark

1. Identity and Access Management
2. Logging
3. Monitoring
4. Networking

This post focuses on Monitoring. IMO, it should actually be called Monitoring and Alerting. CIS implemented the Monitoring controls based on CloudWatch Logs (CWL) integration with CloudTrail and CWL Alarms via the Simple Notification Service (SNS). This is fantastic if you already use these services liberally or cannot get funding for third-party solutions, but they aren’t needed if you already use appropriate third-party solutions. And of course, although I really dig AWS, there’s something to be said for avoiding cloud lock-in, too.

While we do use the required services, and have the pre-requisites configured already, we are shipping AWS logs to Sumo Logic (Sumo). Thus, I thought,“can’t we just use Sumo to satisfy the Monitoring requirements”? The answer is yes and no.

There are sixteen (16) Monitoring controls total. Fourteen (14) of them can be monitored using Sumo’s CloudTrail integration. Let’s have a look at the controls:

3.1 Ensure a log metric filter and alarm exist for unauthorized API calls
3.2 Ensure a log metric filter and alarm exist for Management Console sign-in without MFA
3.3 Ensure a log metric filter and alarm exist for usage of “root” account
3.4 Ensure a log metric filter and alarm exist for IAM policy changes
3.5 Ensure a log metric filter and alarm exist for CloudTrail configuration changes
3.6 Ensure a log metric filter and alarm exist for AWS Management Console authorization failures
3.7 Ensure a log metric filter and alarm exist for disabling or scheduled deletion of customer created CMKs
3.8 Ensure a log metric filter and alarm exist for S3 bucket policy changes
3.9 Ensure a log metric filter and alarm exist for AWS Config configuration changes
3.1 Ensure a log metric filter and alarm exist for security group changes
3.11 Ensure a log metric filter and alarm exist for changes to Network Access Control Lists (NACL)
3.12 Ensure a log metric filter and alarm exist for changes to network gateways
3.13 Ensure a log metric filter and alarm exist for route table changes
3.14 Ensure a log metric filter and alarm exist for VPC changes
3.15 Ensure security contact information is registered
3.16 Ensure appropriate subscribers to each SNS topic

Security contact information (3.15) has to be audited via the management console, and SNS subscribers (3.16) are not applicable for our configuration. Once we have the monitoring configured in Sumo Logic, we’ll use its Slack and PagerDuty integrations for alerting. Thus, the Monitoring section of the benchmark is really monitoring and alerting. We will cover Alerting as Phase Two of our Benchmark project. But first, monitoring 3.1-3.14.

CIS AWS: The finished product.

Although I’m a Sumo Logic novice, this was very simple to accomplish, albeit by standing on the shoulders of giants. The majority of the searches that power the dashboards are derivatives of those used in Sumo’s out-of-the-box dashboards (dashboards are not available in Sumo Free). Next are the searches you’ll need to configure.

3.1 Detect unauthorized API calls

_sourceCategory<span class="token operator">=</span><span class="token punctuation">[</span>YOUR SOURCE CATEGORY<span class="token punctuation">]</span> 
<span class="token operator">|</span> parse <span class="token string">"\"errorCode\":\"*\""</span> <span class="token keyword">as</span> error
<span class="token operator">|</span> where error<span class="token operator">=</span><span class="token string">"AccessDenied"</span> or error<span class="token operator">=</span><span class="token string">"UnauthorizedOperation"</span>
<span class="token operator">|</span> count by error

3.2 Detect console login without MFA

_sourceCategory<span class="token operator">=</span><span class="token punctuation">[</span>YOUR SOURCE CATEGORY<span class="token punctuation">]</span> 
<span class="token operator">|</span> parse <span class="token string">"\"sourceIPAddress\":\"*\""</span> <span class="token keyword">as</span> src_ip nodrop
<span class="token operator">|</span> parse <span class="token string">"\"eventName\":\"*\""</span> <span class="token keyword">as</span> eventName nodrop
<span class="token operator">|</span> parse <span class="token string">"\"userName\":\"*\""</span> <span class="token keyword">as</span> userName nodrop
<span class="token operator">|</span> parse <span class="token string">"\"responseElements\":{\"ConsoleLogin\":\"*\"}"</span> <span class="token keyword">as</span> loginResult nodrop
<span class="token operator">|</span> parse <span class="token string">"\"MFAUsed\":\"*\""</span> <span class="token keyword">as</span> mfaUsed nodrop
<span class="token operator">|</span> where eventName<span class="token operator">=</span><span class="token string">"ConsoleLogin"</span>
<span class="token operator">|</span> where mfaUsed<span class="token operator"><</span><span class="token operator">></span><span class="token string">"Yes"</span>
<span class="token operator">|</span> count by username<span class="token punctuation">,</span> src_ip

3.3 Detect Root Account Usage

_sourceCategory<span class="token operator">=</span><span class="token punctuation">[</span>YOUR SOURCE CATEGORY<span class="token punctuation">]</span> 
<span class="token operator">|</span> parse <span class="token string">"\"userIdentity\":{\"type\":\"*\"}"</span> <span class="token keyword">as</span> authData nodrop
<span class="token operator">|</span> parse <span class="token string">"\"type\":\"*\""</span> <span class="token keyword">as</span> loginType nodrop
<span class="token operator">|</span> where loginType<span class="token operator">=</span><span class="token string">"Root"</span>
<span class="token operator">|</span> count by loginType

3.4 Detect IAM Policy Changes

_sourceCategory<span class="token operator">=</span><span class="token punctuation">[</span>YOUR SOURCE CATEGORY<span class="token punctuation">]</span> 
<span class="token operator">|</span> parse <span class="token string">"\"eventName\":\"*\""</span> <span class="token keyword">as</span> event nodrop
<span class="token operator">|</span> where event matches <span class="token string">"Put*Policy"</span> or event matches <span class="token string">"Delete*Policy*"</span> or event matches <span class="token string">"Attach*Policy"</span> or event matches <span class="token string">"Detach*Policy"</span> or event matches <span class="token string">"CreatePolicy*"</span>
<span class="token operator">|</span> count by event

3.5 Detect CloudTrail config changes

_sourceCategory<span class="token operator">=</span><span class="token punctuation">[</span>YOUR SOURCE CATEGORY<span class="token punctuation">]</span> 
<span class="token operator">|</span> parse <span class="token string">"\"eventName\":\"*\""</span> <span class="token keyword">as</span> event nodrop
<span class="token operator">|</span> where event matches <span class="token string">"*Trail"</span> or event matches <span class="token string">"StartLogging"</span> or event matches <span class="token string">"StopLogging"</span>
<span class="token operator">|</span> count by event

3.6 Detect AWS Mgmt Console authorization failures

_sourceCategory<span class="token operator">=</span><span class="token punctuation">[</span>YOUR SOURCE CATEGORY<span class="token punctuation">]</span> 
<span class="token operator">|</span> parse <span class="token string">"\"responseElements\":{\"ConsoleLogin\":\"*\"}"</span> <span class="token keyword">as</span> loginResult nodrop
<span class="token operator">|</span> where eventName<span class="token operator">=</span><span class="token string">"ConsoleLogin"</span>
<span class="token operator">|</span> where errorMessage<span class="token operator">=</span><span class="token string">"Failed authentication"</span>
<span class="token operator">|</span> count by errorMessage

3.7 Detect disabling or scheduled deletion of CMK

_sourceCategory<span class="token operator">=</span><span class="token punctuation">[</span>YOUR SOURCE CATEGORY<span class="token punctuation">]</span> 
<span class="token operator">|</span> parse <span class="token string">"\"eventName\":\"*\""</span> <span class="token keyword">as</span> event nodrop
<span class="token operator">|</span> where event matches <span class="token string">"DisableKey"</span> or event matches <span class="token string">"ScheduleKeyDeletion"</span>
<span class="token operator">|</span> count by event

3.8 Detect S3 bucket policy changes

_sourceCategory<span class="token operator">=</span><span class="token punctuation">[</span>YOUR SOURCE CATEGORY<span class="token punctuation">]</span> 
<span class="token operator">|</span> parse <span class="token string">"\"eventName\":\"*\""</span> <span class="token keyword">as</span> event nodrop
<span class="token operator">|</span> where event matches <span class="token string">"*BucketAcl"</span> or event matches <span class="token string">"*BucketPolicy"</span> or event matches <span class="token string">"*BucketCors"</span> or event matches <span class="token string">"*BucketLifecycle"</span>
<span class="token operator">|</span> count by event

3.9 Detect AWS Config config changes

_sourceCategory<span class="token operator">=</span><span class="token punctuation">[</span>YOUR SOURCE CATEGORY<span class="token punctuation">]</span> 
<span class="token operator">|</span> parse <span class="token string">"\"eventName\":\"*\""</span> <span class="token keyword">as</span> event nodrop
<span class="token operator">|</span> where event matches <span class="token string">"StopConfigurationRecorder"</span> or event matches <span class="token string">"DeleteDeliveryChannel"</span> or event matches <span class="token string">"PutDeliveryChannel"</span> or event matches <span class="token string">"PutConfigurationRecorder"</span>
<span class="token operator">|</span> count by event

3.10 Detect Security Group changes

_sourceCategory<span class="token operator">=</span><span class="token punctuation">[</span>YOUR SOURCE CATEGORY<span class="token punctuation">]</span> 
<span class="token operator">|</span> parse <span class="token string">"\"eventName\":\"*\""</span> <span class="token keyword">as</span> event nodrop
<span class="token operator">|</span> where event matches <span class="token string">"CreateSecurityGroup"</span> or event matches <span class="token string">"DeleteSecurityGroup"</span> or event matches <span class="token string">"RevokeSecurityGroupEgress"</span> or event matches <span class="token string">"RevokeSecurityGroupIngress"</span>
<span class="token operator">|</span> count by event

3.11 Detect Network ACL changes

_sourceCategory<span class="token operator">=</span><span class="token punctuation">[</span>YOUR SOURCE CATEGORY<span class="token punctuation">]</span> 
<span class="token operator">|</span> parse <span class="token string">"\"eventName\":\"*\""</span> <span class="token keyword">as</span> event nodrop
<span class="token operator">|</span> where event matches <span class="token string">"CreateNetworkAcl"</span> or event matches <span class="token string">"CreateNetworkAclEntry"</span> or event matches <span class="token string">"DeleteNetworkAcl"</span> or event matches <span class="token string">"DeleteNetworkAclEntry"</span> or event matches <span class="token string">"ReplaceNetworkAclEntry"</span> or event matches <span class="token string">"ReplaceNetworkAclAssociation"</span>
<span class="token operator">|</span> count by event

3.12 Detect Network Gateway changes

_sourceCategory<span class="token operator">=</span><span class="token punctuation">[</span>YOUR SOURCE CATEGORY<span class="token punctuation">]</span> 
<span class="token operator">|</span> parse <span class="token string">"\"eventName\":\"*\""</span> <span class="token keyword">as</span> event nodrop
<span class="token operator">|</span> where event matches <span class="token string">"CreateCustomerGateway"</span> or event matches <span class="token string">"DeleteCustomerGateway"</span> or event matches <span class="token string">"AttachInternetGateway"</span> or event matches <span class="token string">"CreateInternetGateway"</span> or event matches <span class="token string">"DeleteInternetGateway"</span> or event matches <span class="token string">"DetachInternetGateway"</span>
<span class="token operator">|</span> count by event

3.13 Detect Route Table changes

_sourceCategory<span class="token operator">=</span><span class="token punctuation">[</span>YOUR SOURCE CATEGORY<span class="token punctuation">]</span> 
<span class="token operator">|</span> parse <span class="token string">"\"eventName\":\"*\""</span> <span class="token keyword">as</span> event nodrop
<span class="token operator">|</span> where event matches <span class="token string">"CreateRoute"</span> or event matches <span class="token string">"CreateRouteTable"</span> or event matches <span class="token string">"ReplaceRoute"</span> or event matches <span class="token string">"ReplaceRouteTableAssociation"</span> or event matches <span class="token string">"DeleteRouteTable"</span> or event matches <span class="token string">"DeleteRoute"</span> or event matches <span class="token string">"DisassociateRouteTable"</span>
<span class="token operator">|</span> count by event

3.14 Detect VPC changes

_sourceCategory<span class="token operator">=</span><span class="token punctuation">[</span>YOUR SOURCE CATEGORY<span class="token punctuation">]</span> 
<span class="token operator">|</span> parse <span class="token string">"\"eventName\":\"*\""</span> <span class="token keyword">as</span> event nodrop
<span class="token operator">|</span> where event matches <span class="token string">"CreateVpc"</span> or event matches <span class="token string">"DeleteVpc"</span> or event matches <span class="token string">"ModifyVpcAttribute"</span> or event matches <span class="token string">"*VpcPeeringConnection"</span> or event matches <span class="token string">"*tachClassicLink"</span> or event matches <span class="token string">"*ableVpcClassic"</span>
<span class="token operator">|</span> count by event

As mentioned previously, I’m a Sumo Logic novice—there is no doubt these searches can be improved. The searches looking for more than a few events, like S3 bucket policy changes, can take a longer time to run depending on the date/time range chosen. The initial 7-day search we ran took over an hour to provide results, but we haven’t done any tuning or partitioning yet so YMMV.

This CIS AWS Foundations Benchmark Monitoring blog was written by expert Joey Peloquin who can be reached on Twitter @jdpeloquin.