amazon sqs - What are the drawbacks of SQS poller which AWS Lambda removes?

Question

I have an architecture which looks like as follows:-

Multiple SNS -> (AWS Lambda or SQS with Poller)??? -> Dynamo Db

So, basically multiple SNS have subscribed to AWS Lambda or SQS with Poller and that thing pushes data to Dynamo Db.

But this ? thing do lot of transformation of message in between. So, now for such case, I can either use AWS Lambda or SQS with Poller. With AWS Lambda, I can do transformation in Lambda function and with SQS with Poller, I can do transformation in Poller. With AWS Lambda, I see one problem that code would become quite large as transformation is quite complex(has lot of rules), so I am thinking to use SQS. But before finalising on SQS, I wanted to know of the drawbacks of SQS which AWS Lambda removes?

Please help. Let me know if you need further information.

Answer 1

Your question does not contain much detail, so I shall attempt to interpret your needs.

Option 1: SQS Polling

• Information is sent to an Amazon SNS topic
• An SQS queue is subscribed to the SNS topic
• An application running on Amazon EC2 instance(s) regularly poll the SQS queue to ask for a message
• If a message is available, the data in the message is transformed and saved to an Amazon DynamoDB table

This approach is good if the transformation takes a long time to process. The number of EC2 instances can be scaled based upon the amount of work in the queue. Multiple messages can be received at the same time. It is a traditional message-based approach.

Option 2: Using Lambda

• Information is sent to an Amazon SNS topic
• An AWS Lambda function is subscribed to the SNS topic
• A Lambda function is invoked when a message is sent to the SNS topic
• The Lambda function transforms the data in the message and saves it to an Amazon DynamoDB table

AWS Lambda functions are limited to five minutes of execution time, so this approach will only work if the transformation process can be completed within that timeframe.

No servers are required because Lambda will automatically run multiple functions in parallel. When no work is to be performed, no Lambda functions execute and there is no compute charge.

Between the two options, using AWS Lambda is much more efficient and scalable but it might vary depending upon your specific workload.