AWS SQS Dead Letter Queue

SQS allows users to specify a redrive policy for a particular queue. This policy describes what to do when a message fails to be consumed from a queue, including how many times to try to consume the message. It also allows you to specify a Dead Letter Queue, a queue that the message is sent to after it fails too many times. Specifying a redrive policy allows you to avoid “poison pills” (messages that cannot be handled and will clog up your consumer) and provides a place for failed messages to be stored for further inspection. Blue Matador alerts you when your dead letter queue is nonempty, as this queue is often configured and then neglected, resulting in failed messages that simply disappear without being addressed.

Effects

• Messages crucial to your application’s pipeline may be discarded, causing loss of data.
• Bugs in code that produce failing messages may remain undetected.

• Application code that produces messages that are sent to your SQS queue are malformed. Check that the code that sends to and receives from your queue agree on the message protocol.
• Your application is retrying the message, but it is unable to delete it. This can happen for many reasons, including an error in logic that aborts the operation that handles the message or network issues that prevent your application from receiving or deleting the message. In this case, it is especially important to check the dead letter queue, as it will contain properly formatted messages that will be lost if not handled.

Resources

• Dead Letter Queues (AWS Documentation)