Docs

    SQS Message Retention

    Messages in Amazon SQS have a limited amount of time to be received and processed before they are automatically deleted by SQS. If a message is not processed before the message retention period has passed, it will be permanently deleted. Blue Matador monitors the ApproximateAgeOfOldestMessage metric in CloudWatch for each SQS queue in your account to detect if messages are not being processed fast enough.

     

    Retention Limit

    When the oldest message in a queue is halfway towards being expired, Blue Matador will create an alert. For example, an event will be created if the oldest message was sent 15 minutes ago, and the visibility timeout is 30 minutes.

     

    Old Messages

    When Blue Matador detects that the oldest message age is anomalous for the queue, a warning is created. This condition is usually an indication that consumers are not processing messages fast enough, messages are not being fully processed, or messages are reaching the visibility timeout.

     

    Troubleshooting

    CloudWatch does not expose the average age of messages, so only the maximum age is known. It is either the case that messages in general are not being processed quickly, or a subset of messages are not being processed. If there are no consumers in the queue, then some consumers must be added to avoid data loss. If there are consumers and the maximum age remains high, then it is possible that some messages are not being processed correctly. Check application logs to see if any errors are preventing the DeleteMessage operation from being called when messages are finished processing. Keep in mind that SQS messages must be deleted before the visibility timeout expires or they will be requeued. Ensure that your application is calling DeleteMessage as soon as the message is no longer needed, and using the ChangeMessageVisibility API call when more processing time is needed.

     

    Resources