Rethinking Cloud Proxies

Editor's Notes

• #12: Our gateway strategy will change the way you think about resiliency, debugging, continuous delivery, service operations, and insights.
• #19: Devices slow to update Need emergency policies Fast action
• #20: Limited range of functionality Hard to program Authentication Authorization Static responses / Origin specific headers Why? Federation of logic across systems creates complexity Minimize gateway dependencies to maximize availability
• #24: Origin services run many clusters Route to service clusters based on dynamic routing rules Shape or reject traffic based on service, regional health, or attack React fast in emergencies Realtime analytics and insights Ensures request delivery from internet to services running in the cloud Dynamically changes routing behaviors Routes to services Services have multiple clusters Clusters have dynamically changing nodes Bridges multiple cloud regions and data centers Provides system Insights
• #25: Same service: Subclusters for many purposes Set up by filters in Zuul Self serviceable by cluster owners Automated Quality assurance / Test Automation Targeted debugging Test Automation Canary / Baseline A/B testing of service behavior per build Squeeze Testing Service capacity testing Trickle traffic Instrumented builds Sticky Canary A/B testing of client behavior per origin build
• #28: Trickling traffic into clusters High Overhead profiling tools “Coalmine” verbose logging
• #29: Server capacity testing Gateway gradually increases traffic until performance degradation is detected Automated or manual
• #30: Isolate requests by customer, route, type of device, or any routing rule Debug node(s) are often instrumented to give verbose logging Custom Request Routing
• #31: Compare server behavior and metrics Equal traffic rates hit both clusters Automated part of production push process Error rates CPU for equivalent work Automated metrics analysis returns a score of how well the canary cluster performed A poor score stops the push process
• #32: Servers may be healthy data may be bad API changes that affect devices Data changes certain devices can’t interpret Protocol and transport changes that some devices can’t accept Testing 1000’s of types of devices would be a time consuming, tedious process. Sticky Canary idea - Stick all requests for a small subset of customers for a limited time to a “sticky canary” or “sticky baseline” If servers are equivalent, there should be no behavioral differences. Insights can help find these anomalies Limited scope of impact - a very small subset of customers could be affected but only for a short period of time
• #37: Reroute to the closer region to the client - DNS accuracy issues, etc Reroute due to region failure.
• #40: Speedbump Dynamic DDOS prevention