Extend AWS observability beyond CloudWatch

It’s essential to choose the right tool for the job. I have an old, sturdy screwdriver that I use for lots of odd DIY jobs around my house, like cleaning gutters, opening paint cans, and general maintenance on my lawnmower. However, when I’m performing an upgrade on my computer, a large, rusty screwdriver isn’t the best tool to remove the screws anchoring my motherboard.

Managing and maintaining your Amazon Web Services (AWS) environment is similar. Some tasks require an all-purpose tool that’s robust and straightforward to use, while others require well-tuned, precision tools. In this article, we will examine various tools that you have at your disposal, including CloudWatch. We’ll explain what CloudWatch does well, and also why it can be extended with the help of additional solutions. Then, we’ll discuss why these circumstances matter and show you how to choose the right tool to monitor and tune your environment for optimal performance.

CloudWatch - The Right Tool for Some Jobs

AWS implements Amazon CloudWatch as the monitoring and management tool for all of its services. With CloudWatch, you can collect logs and metrics, and you can gain visibility into each system in the AWS environment. You can view this data on your CloudWatch dashboard.

CloudWatch is a portal through which you can view the state of all of your AWS infrastructure in one place. You can view current and past data as well as set up custom alarms with predefined limits and the ability to alert when those limits have been breached for a specific amount of time. CloudWatch alarms can trigger notifications to support personnel or invoke other AWS services to automatically address needs within your environment.

CloudWatch may appear to be the ultimate command center for all of your performance metrics and infrastructure monitoring, but it has some limitations. We’ll explore those next.

With Great Power Comes Some Downsides

A Steep Learning Curve

The volume of data and variety of services that report it through CloudWatch is both a blessing and a curse. There is a relatively steep learning curve with CloudWatch, both in terms of understanding what data is available and how to access it, and also in learning how to compare and contrast metrics in a meaningful way.

Due to the range of services that CloudWatch reports, it has to be very generic and flexible. Unfortunately, this generic ability means that it doesn’t include any built-in dashboards or analytical tools. You can create custom dashboards, but these can be tedious to develop and maintain.

Comparing and Contrasting Data from Different Sources

CloudWatch allows you to view metrics from a single instance or service. You can also group different services into log groups to compare them. Unfortunately, comparing metrics from instances or services in other log groups gets increasingly complicated, and in most cases, it is beyond the capabilities of CloudWatch.

Unexpected Costs

Whether you access CloudWatch data through the AWS Console, AWS CLI, or programmatically, all of your requests go through the CloudWatch API, which comes at a cost. The more data you request, the more you have to pay, and it can be challenging to accurately predict the final cost ahead of time.

AWS also charges fees for each dashboard, with additional fees for more detailed logs gathered in shorter periods of time. If you need AWS to help you with custom dashboards or metrics, you might find yourself incurring unexpected expenses.

Alarm Rigidity

As mentioned above, AWS offers the ability to set alarms, which can be a handy feature. However, if you have a large account, you might find that you need to regularly update the alarm thresholds on different services of your infrastructure.

The Benefits of a Third-Party Solution

AWS is so popular because it’s incredibly versatile, it’s scalable, and it offers an increasing range of services for its customers. While the platform is continuously being improved, its monitoring and observability capabilities have not kept pace with many of their customers’ needs.

Fortunately, third-party providers have identified this as an opportunity to develop platforms and tools that integrate with AWS and provide such capabilities for customers who need advanced monitoring.

Sumo Logic provides monitoring, analytics, and visualization services for AWS as well as for Microsoft Azure, Google Cloud, and many other cloud-based platforms. By focusing on their customers’ specific needs in these areas, they’ve become an experienced partner with demonstrated expertise.

Addressing the BlindSpots

The Learning Curve

Sumo Logic ingests CloudWatch logs and metrics and CloudTrail logs and displays it in predefined dashboards with very little (if any) configuration required. The interface is intuitive and ensures that it is easy to update queries, add additional fields, and customize dashboards as needed. Dashboards are designed and tuned with industry best practices in mind to make it easy to see the most relevant data for each service.

Comparing and Contrasting Data

Sumo Logic is an analytics platform, and the analysis of your AWS metrics won’t be constrained by log groups or accounts. You can organize metrics using tags, build custom filters to include/exclude different services, and quickly compare relevant data.

Sumo Logic Search includes functions like LogCompare and LogReduce as well as a host of other advanced and well-documented analytics functions that empower you to organize and analyze data based on your needs.

Fixed Costs

Your Sumo Logic subscription includes access to your data, your dashboards, and all of the analytics tools. You can even sign up for a free trial to see if it works for you, and you can experiment with it for free and the pricing is based on the amount of data you ingest. There is also an active Sumo Logic community that can answer many of your questions. If you need additional assistance, your subscription includes help from the world-class Sumo Logic support team.

Alarms, Threat Detection, and Other Perks

Sumo Logic also provides out of the box alerts for all of the services supported through AWS Observability, so customers don't have to spend time determining and fine tuning thresholds. These are also based on AWS docs and best practice guidelines.

Sumo Logic Alerts can make use of variables for greater extensibility. In addition to this robust monitoring service, you can take advantage of integrated and proactive threat-detection from CloudStrike.

With Sumo Logic, you ingest CloudWatch and CloudTrail data to build custom and pre-built dashboards. While AWS reduces, simplifies, and eliminates CloudWatch data over time, you manage the retention period when you use Sumo Logic. You can even configure periodic migrations of long-term datasets to AWS S3 for a more cost-effective retention strategy.

Getting Setup and Connected

Once you’ve set up a Sumo Logic account and configured the collection of CloudWatch and CloudTrail data into Sumo Logic via AWS Cloud Formation, you’ll have instant visibility into your AWS applications. The preferred way to grant Sumo Logic access to your AWS account is with an AWS IAM Role, although it is possible (but not advisable) to use an AWS IAM User instead. Sumo Logic even provides CloudFormation Templates to assist you in correctly configuring it.

After granting your Sumo Logic account access to your AWS account, you need to select the appropriate application from the App Catalog. Then, you’ll have instant access to a collection of predefined dashboards. Learn more here.