How to dynamically auto-steer your traffic to multi-CDN or multiple data-centers

Whether it’s Cyber Monday deals or streaming the Super Bowl to millions of users, website’s traffic management & SRE teams often struggle to deliver a reliable user experience.

Multi-CDN strategy is critical & integral to this process and is often determined by the following factors:

• CDN Cost
• End User Geography
• Securing Revenue from Outages & Incidents
• Uptime SLA, Performance & Reliability of CDN
• and CDN’s capacity to handle traffic

Delivering a consistent experience to end users has a higher success rate. For example, Amazon.com routes 30-60% of their traffic to Akamai CDN on regular days and routes 80-100% of traffic to Akamai on Prime Day Sale. At the same time, CDNs such as Limelight, BitGravity & StackPath have their own competitive edge in the market due to their cost. Though they might not be able to match the CDN footprint and performance to CloudFlare, Fastly & the Premium CDNs like Akamai.

The Legacy Way:

Traditionally, when websites have multiple CDN in their tech-stack, they either tier the CDN one on top of the other, or they primarily use a Global Traffic Manager to route the traffic at the DNS level.

The Traffic Split can be done many ways, including:

• Active/Passive Failover
• Weighted Load Balancing or by Percentage
• Geographical Mapping
• or Health of the Data-Center or CDN, by measuring Availability, Performance

In Modern Day Traffic routing, SRE teams & Architects of eCommerce, Online Retailers & Streaming Platform, couple weighted Load balancing with a metric of their preference such as CDN health checks, Error Rate, or CDN Cost.

Mostly these multi-CDN strategies involve only one metric to set the weights. Also, they are statically set three ways: either by human intervention via the Traffic manager’s UI, or by a Script, or through a scheduled cron job via Traffic manager’s API.

Few Traffic managers have the ability to steer the traffic automatically to different destinations by measuring the availability & performance of a health-check object served by the CDN.

Need of the Hour:

This brings us to the need of the hour for dynamically steering traffic to multiple CDN or data centers. You must consider multiple metrics including CDN Error Rate, Cache HIT/MISS Ratio, CDN Cost, Health Checks, Network Error Logging (NEL) aggregates, Traffic Prediction, Synthetic & Real User Monitoring, and CDN Vendor Reliability/Uptime SLA,along with a fail-safe strategy to route traffic during loss of intelligence feed of these metrics.

Sumo Logic’s Data Lake Agility & Power coupled with its Continuous Intelligence Platform provides us the ability to take into consideration all these data sources and make an actively informed decision to route traffic to multiple destinations.

Project S.M.A.R.T Route:

Project S.M.A.R.T Route is a Sumo Logic Customer Success Engineering initiative to support dynamic auto-steering of Traffic to Multi-CDN or multiple data centers. It sends Real-time Score feeds either directly to the traffic manager or via a middleware.

Using the Real-time score feeds received from Sumo Logic, the middleware will have the ability to create a Scoring Table or a Routing Table for Steering Traffic to multiple destinations.

Scoring Table (or) Routing Table:

In the scoring table, each data source (or scoring parameter) is assigned with a weighted priority. Meanwhile, each CDN or data-center target will receive live-feed scores based on the data collected in Sumo Logic on the scoring parameter. The Traffic weight for Each CDN or Data-center target is calculated by the summation of multiplying the weighted priority of data source and live-feed weighted score of the CDN target.

In the above example for Akamai CDN, the traffic weight is calculated as

= (3x10) + (3x10) + (3x0) + (3x5) + (5x7) + (5x5) + (2x1) + (10x10) + (10x10) + (4x9) + (2x9) + (3x10) + (10x4) + (1x5) = 30 + 30 + 0 + 15 + 35 + 25 + 2 + 100 + 100 + 36 + 18 + 30 + 40 + 5

= 466

The higher the weight, the more the percentage of traffic gets assigned to that destination.

Auto-Steering Algorithm & Preferential Presets for Strategy:

Based on preferences in various combinations of data-source consideration, there can be numerous presets for auto-steering the traffic.

For Example, e-Commerce & Online retailers provide preference to reliability and CDN Capacity during Sale days, where High Traffic & High Revenue is expected during that time frame. While websites with budget cuts prefer a low cost CDN and failover to moderate cost CDN during outages and incidents.

Based on the Business & SRE needs, there can be customizable, pre-approved presets to manage operations, reliability, efficiency, cost, and performance.

Once the Scores are calculated, the middleware can contact the Traffic Manager via API in real-time and set the weights to the traffic destination for auto-steering the traffic. In the case of any failure in receiving the score-feeds, the middleware can switch to static fail-safe values for the weights of the traffic destinations.