Measuring latency from your Azure Application Gateway is often a great early indicator of application issues; latency often increases as applications get overwhelmed or experience errors. Blue Matador watches the BackendFirstByteResponseTime metric for the amount of time it takes for your application gateway to begin responding to a request and creates events when this metric is anomalous.
Despite its nature as a good canary, latency is more useful for correlation with other events in your infrastructure. When you see increased latency from your application gateway, check for other events that may be contributing to latency.
If there aren’t any other events and latency persists, enable and check access logs for your application gateway. Look for the endpoint or endpoints that are causing high average latency. Be aware that if your API doesn’t get much traffic a single slow response can really affect the average latency. Once you’ve found the slow endpoint, scan the logs it emits (with the log management solution your organization uses) for exceptions or other errors that could be resulting in slow response times.
