AWS SQS Consumer

A queue consumer receives messages from an SQS queue. Blue Matador detects when a queue has messages that are not being processed by monitoring the NumberOfMessagesDeleted and ApproximateNumberOfMessagesVisible metrics. If there are no deletes, but messages are still in the queue, then there is likely an issue with the consumers. Issues with SQS consumers tend to have cascading effects on other parts of your system and should be addressed ASAP.

Troubleshooting

Always be on the lookout for changes to your application or infrastructure when you have an issue with SQS. It is always more likely that a change you made is causing issues than that the issue is on Amazon's side.  Here are some examples of changes that could cause your SQS consumers to break:

• The application code for the consumers is not calling the ReceiveMessage API at all, and must be debugged using application logs
• The application code uses ReceiveMessages to read SQS messages but does not call DeleteMessages to indicate that a message has been processed
• A permissions issue with the consumer and the affected queue was introduced. Check IAM permissions for both the consumers and the affected SQS queue
• Messages are being sent with a delay higher than the queue retention. This means the messages are automatically deleted by SQS before any consumer has a chance to process them

Resources

• SQS Permissions (Amazon)
• SQS Visibility Timeout (Amazon)
• SQS Cloudwatch Metrics (Amazon)

Note: Our system uses SQS metrics in Amazon CloudWatch to detect possible issues with consumers of your queue. Due to the API limitations of CloudWatch, there can be a delay of as many as 20 minutes before our system can detect these issues.