How to Read, Search, and Analyze AWS CloudTrail Logs

In a recent post, we talked about AWS CloudTrail and saw how CloudTrail can capture histories of every API call made to any resource or service in an AWS account. These event logs can be invaluable for auditing, compliance, and governance. We also saw where CloudTrail logs are saved and how they are structured.

Enabling a CloudTrail in your AWS account is only half the task. Its real value is gained by analyzing the logs and making sense of any unusual pattern of events or finding root cause of an event.

In this post, we will talk about a few ways you can read, search and analyze data from AWS CloudTrail logs.

Understanding Cloudtrail log structure

CloudTrail logs are nothing but JSON-formatted, compressed files. If you download a CloudTrail log file and open it in a text editor, you will see something like this:

{
 "Records": [{
 "eventVersion": "1.05",
 "userIdentity": {
 "type": "IAMUser",
 "principalId": "AIDAIPFJYALOFOP2DK2SY",
 "arn": "arn:aws:iam::xxxxxxx:user/Administrator",
 "accountId": "xxxxxxx",
 "accessKeyId": "xxxxxxx",
 "userName": "Administrator",
 "sessionContext": {
 "attributes": {
 "mfaAuthenticated": "false",
 "creationDate": "2018-10-21T09:17:01Z"
 }
 },
 "invokedBy": "signin.amazonaws.com"
 },
 "eventTime": "2018-10-21T12:59:14Z",
 "eventSource": "s3.amazonaws.com",
 "eventName": "GetBucketLocation",
 "awsRegion": "us-east-1",
 "sourceIPAddress": "12.34.56.78",
 "userAgent": "signin.amazonaws.com",
 "requestParameters": {
 "bucketName": "mytest-cloudtrails",
 "location": [""]
 },
 "responseElements": null,
 "additionalEventData": {
 "vpcEndpointId": "vpce-xxxxxxxxxx"
 },
 "requestID": "544A415A398A169C",
 "eventID": "5989cc55-f752-468f-b669-f8abbeb008ba",
 "eventType": "AwsApiCall",
 "recipientAccountId": "xxxxxxxxxx",
 "vpcEndpointId": "vpce-xxxxxxxxxx"
 }
 ]
}

If you look closely, this particular JSON document is giving us enough information about an S3 event: GetBucketLocation. We can see an IAM user called Administrator had logged in through the console, without any multi-factor authentication, from an IP address: 12.34.56.78, and invoked this event.

There can be dozens or even hundreds of events like this in one CloudTrail file. The JSON snippet we are seeing here had been part of a larger document, the other entries were deliberately removed.

As you can see, an event consists of multiple fields, and each field describes a specific attribute of the event. When you are looking at a CloudTrail file, here are the important fields you need to be aware of:

Searching Cloudtrail logs from event history

Reading through hundreds of thousands of lines from hundreds of CloudTrail logs is not a practical solution. It would be ideal if there was a search facility that allowed users to specify one or more search criteria while the tool searched through all log files to find the matching events.

Fortunately, there is one such facility in the CloudTrail event history console. The event history console allows searching through events that occurred in the past 90 days. The search console has a default search condition to filter out read-only events like ListKeys or DescribeInstance. The image below shows how we are searching for events related to “DBInstance” type resource:

You can expand any event from the resultset to reveal further information about the event.

Final words

So now you have seen a few ways to make sense of AWS CloudTrail logs. There are other sophisticated AWS monitoring tools, like Sumo Logic's Cloud SIEM solution, that can go even further than what we have seen here. Such SIEM tools can create better visualizations from CloudTrail logs and are able to show anomalies and trend graphs in those visualizations. They can also send automated alerts based on error conditions that you define. One such tool is Sumo Logic. Sumo Logic’s CloudTrail app can be a valuable addition to any AWS administrator’s toolset.