In general, HTTP 5xx responses indicate server issues. However, when creating a distributed web service with Azure Application Gateway, some number of 5xx responses are expected. Blue Matador watches the Response Status metric with the HttpStatusGroup=5xx dimension for the number of 5xx responses returned by your application gateway, and creates events when this metric is anomalous. A sudden spike in the number of 5xx responses can be the result of the following:
- Deploying new code that contained a bug
- Errors returned by upstream services that your application depends on
- Your application is overwhelmed by the amount of traffic it is receiving
HTTP 5xx responses usually result from problems specific to your application, but the general process for troubleshooting the problem is the same. First, enable and check access logs for your application gateway. Look for the endpoint or endpoints that are returning the 5xx status codes. Next, scan the logs emitted by that endpoint (with the log management solution your organization uses) for exceptions or other errors that could be resulting in 5xx responses. Finally, fix the issue and deploy the solution.
