Amazon SQS & MQ

What is Amazon SQS?

📌Amazon Simple Queue Service (SQS) is a managed message queue service offered by Amazon Web Services (AWS).

📌It provides an HTTP API over which applications can submit items into and read items out of a queue.

📌The queue itself is fully managed by AWS, which makes SQS an easy solution for passing messages between different parts of software systems that run in the cloud.

Which concept does it work?

📌SQS provides an API endpoint to submit messages and another endpoint to read messages from a queue. Each message can only be retrieved once, and you can have many clients submitting messages to and reading messages from a queue at the same time i.e. Message Queue(MQ)

What is Amazon MQ?

📌Amazon MQ is a managed message broker service for Apache ActiveMQ and RabbitMQ that makes it easy to set up and operate message brokers in the cloud, so you can migrate your messaging and applications without rewriting code.

Why is SQS an essential part of the Serverless ecosystem?

📌In an optimal Serverless architecture, when using multiple microservices, you want your services to be independent of each other. This means that they use separate databases and even run in different network segments but can still communicate with each other.

📌The distributed systems term for this kind of independence is a decoupled system. When services are decoupled, you can’t pass messages, for example, by having them both write to a shared database.

📌The available options under this model are: implement an API in each service or use message queues to pass information between services.

How does SQS integrate with other AWS services?

📌Most interesting for Serverless developers is SQS‘s integration with Amazon Lambda: SQS can act as an AWS Lambda event source.

📌When configured, every SQS message triggers a Lambda function run that processes a batch of SQS messages.

📌SQS also provides standard integrations for monitoring and debugging SQS queues using Amazon CloudWatch and AWS X-Ray.

📌Serverless developers can manually integrate an SQS queue with any other AWS service (or a third-party service) by writing code that uses the AWS SDK to submit messages to SQS and read them from there, or by using the SQS API directly.

How is Amazon SQS different from Amazon MQ?

📌If you’re using messaging with existing applications, and want to move your message to the cloud quickly and easily, recommended is Amazon MQ.

📌It supports industry-standard APIs and protocols so you can switch from any standards-based message broker to Amazon MQ without rewriting the messaging code in your applications.

📌If you are building brand new applications in the cloud, we recommend you consider Amazon SQS and Amazon SNS. Amazon SQS and SNS are lightweight, fully managed message queue and topic services that scale almost infinitely and provide simple, easy-to-use APIs.

The benefits of using SQS

📖Scalability: Your SQS queues scale to the volume of messages you’re writing and reading. You don’t need to scale the queues; all the scaling and performance-at-scale aspects are taken care of by AWS.

📖Pay for what you use: When using SQS, you only get charged for the messages you read and write (see the details in the Pricing section). There aren’t any recurring or base fees.

📖Ease of setup Since SQS is a managed service, so you don’t need to set up any infrastructure to start using SQS. You can simply use the API to read and write messages or use the SQS <-> Lambda integration.

📖Options for Standard and FIFO queues: When creating an SQS queue, you can choose between a standard queue and a FIFO queue out of the box. Both of these queue types can be useful for different purposes.

📖Automatic deduplication for FIFO queues: Deduplication is important when using queues, and for FIFO queues SQS will do the work to remove any duplicate messages for you. This makes FIFO queues on SQS suitable for tasks where it’s critical to have each task done exactly once.

📖A separate queue for unprocessed messages: This feature of SQS is useful for debugging. All messages that couldn’t be processed are sent into a “dead-letter” queue where you can inspect them. This queue has all the usual integrations enabled, so you can subscribe to it using an AWS Lambda event, for example, to send a notification when an item can’t be processed.

Disadvantages of using SQS

📖High cost at scale With pay-per-use pricing, if the number of messages you send is very high, your SQS bill can be quite significant. Part of SQS pricing is data transfer charges, and those can add up if you send larger messages, or if you process messages from outside the main AWS region in which the queue is located. In some cases, when running at scale with millions of messages processed every day, the cost of using SQS might be higher than the cost of operating your own queue system, even including the overhead to manage your own solution.

📖Lack of support for broadcast messages With its “exactly once” delivery, SQS doesn’t support a way for multiple entities to retrieve the same message, making SQS not so useful for one-to-many broadcasts.

📖Reduced control over performance When running a message queue system at scale, something you may well end up wanting to do is to fine-tune its performance to suit your needs. With SQS this isn’t an option: the service is fully managed, and you don’t get to look under the hood.

𝕽𝖊𝖉 𝕭𝖚𝖘 𝕮𝖆𝖘𝖊 𝕾𝖙𝖚𝖉𝖞

RedBus is an Indian travel agency that specializes in bus travel throughout India by selling bus tickets throughout the country. Tickets are purchased through the company’s Website or the Web services of its agents and partners. The company also offers software, on a Software as a Service (SaaS) basis, which gives bus operators the option of handling their own ticketing and managing their own inventories. To date, the company says they have sold over 30 million bus tickets and have more than 1750 bus operators using the software to manage their operations.

The Challenge

📌The company previously ran its operations from a traditional data center by purchasing and renting its systems and infrastructure. In addition to the expense, several logistical problems evolved from this arrangement.

📌The biggest problem was that the infrastructure could not effectively handle processing fluctuations, which hurt productivity. Additionally, the procurement of servers or upgrading the server configuration was an extremely time-consuming endeavor.

📌Over time, redBus realized that a better solution was imperative — a solution that offered scalability to handle the company’s processing fluctuations. redBus looked to Amazon Web Services (AWS) for a solution.

Why Amazon Web Services?

📌After testing the AWS solution on a small application for several months, the travel agency determined that it was very workable and convenient. Although redBus was quite enthusiastic about the on-demand instances and variety of instance types, several other features cemented the company’s decision to migrate completely to AWS.

📌These features included the ability to easily manage access to servers through security groups, the easy-to-use, self-service management console, the concept of Elastic IPs, and superior support.

📌The company has incorporated many of the AWS products into its solution, including Amazon Elastic Compute Cloud (Amazon EC2), Elastic Load Balancing, Amazon Relational Database Service (Amazon RDS), Amazon Simple Storage Service (Amazon S3), Amazon Elastic Block Store (Amazon EBS), and Amazon CloudWatch.

The Benefits

📌Since migrating to AWS, redBus has seen measurable improvements in the bottom line. Chef Padmaraju says, “By scaling up and down dynamically based on the load, we maintain performance as well as minimize cost. With the time savings that the IT and development staffs obtain from the AWS solution, AWS gives us an overall cost-benefit of about 30–40%.” He adds, “By hosting at [the AWS Asia Pacific (Singapore) region], redBus.in gained significantly in terms of website performance by way of reduced latency (about 4x). This is a great advantage when the customers are from India.”

📌Of the many excellent characteristics of AWS, perhaps the most significant to redBus is the ability to “instantly replicate the whole setup on-demand for testing by creating and destroying instances on demand for experimentation, thereby reducing the time to market.” Less time to market translates to increased profitability and success.

📌The travel agency anticipates expanding the AWS solution to include Amazon Simple Notification Service (Amazon SNS) and Amazon Simple Queue Service (Amazon SQS) for monitoring, alerts, and intercommunication.

“Amazon SQS is an especially good solution for enabling messaging between external applications and our applications,” says Padmaraju.

📌Since joining forces with AWS, redBus has gained the freedom to experiment on new solutions and applications at minimal cost, increased the efficiency of its operations, and improved its profitability.

Conclusion

🔗SQS is a great way to decouple services, especially when there is a lot of heavy-duty, batch-oriented processing required.

🔗Amazon MQ is a managed message broker service for Apache ActiveMQ that makes it easy to set up and operate message brokers in the cloud.

🔗AWS Lambda (serverless computing platform) functions perfectly well with it.

Thank you for reading my article🎀

Keep Learning🎊 Keep Sharing🤗

Good Day!