Effects
Exceeding SQS limits can result in:
- Errors returned from the API
- Messages being lost without errors returned from the API
1. Inflight Messages
Inflight messages are messages in SQS that have been received by a consumer but not yet deleted. Each SQS queue is limited to 120,000 inflight messages, or 20,000 if it is a FIFO queue. When sending a message to a queue with too many inflight messages, SQS returns the "OverLimit" error message. To avoid hitting this limit, consider the following actions:
- Delete messages as soon as they are processed so that they are no longer inflight.
- Split up your messages between multiple queues. You can send messages to separate queues round-robin, and consume from multiple queues in code.
2. Message Size
The maximum message size attribute on a queue controls the size of messages entering the queue. Our system detects when the average "SentMessageSize" metric is approaching this value. Since messages over the limit are rejected, it is actually impossible to hit the limit, so our recommendation is to leave at least 10% overhead when configuring your maximum message size.
To avoid SendMessage errors, ensure the maximum message size is larger than the messages you send. If your messages are larger than the SQS limit of 256 KB, you can use the Amazon SQS Extended Library for Java to store messages in S3 and send references to them.
3. FIFO Queue Throughput
FIFO queues are a type of SQS queue that offer exactly-once processing, message deduplication, and in-order processing. The drawback is that FIFO queues can only support 300 send, receive, or delete operations per second. When using message batching at 10 messages, this effectively is increased to 3,000 operations per second. Our system will detect when your FIFO queues are near the 3,000 operations per second limit.
To get around this limit you can either use multiple queues to split up the workload, or request a limit increase from AWS support.
Resources
Note: Our system uses SQS metrics in Amazon CloudWatch to detect possible issues with limits for your queue. Due to the API limitations of CloudWatch, there can be a delay of as many as 20 minutes before our system can detect these issues.
