RabbitMQ: Patterns For Applications By Derick Bailey
At InVision, we're currently in the process of decomposing our monolithic application down into an ecosystem of microservices. These microservices will coordinate through a set of remote API calls and a series of asynchronous messages that get published on a Kafka event stream. I'm super excited to learn about Kafka; but, I have basically no experience with message-based architectures. Two years ago, I played around with Amazon's SQS (Simple Queue Service); but, I've done nothing since. As such, I must start building out my mental model for messaging. And, RabbitMQ: Patterns for Applications by Derick Bailey seemed like a solid introduction and "pocket guide" to the core patterns of queue-based communication.
| |
|
|
||
| |
 |
|
||
| |
|
|
I call RabbitMQ: Patterns for Applications a "pocket guide" because it feels like a "cheat sheet" for message queues. It's a quick read - about 55 pages; and, it repurposes the content of Bailey's email-based course. To be fair, he is completely upfront about this, both on the LeanPub page and in the introduction to the book:
The original content of this book was created for a 9-day email course title RabbitMQ: Patterns for Applications. This book is a copy & paste of the same content, delivered in the same order, to give you a collected works edition... As you read this book, keep in mind that each of these short chapters was originally an email with each email sent 1 day apart. As such, you'll see references to "tomorrow's email" and "coming up tomorrow", etc.
The one point of criticism that I have for the book is that this copy & paste approach leaves the book feeling like it could use some polish. It reads well, given the aforementioned caveats. But, it lacks diagrams and other visual aids that could have brought additional clarity to the relationship between brokers, exchanges, queues, messages, producers, consumers, and the request-response life-cycle. This also makes the price of the book feel a bit high; but, I try not begrudge price because I know how much work goes into comprehending a subject and then distilling it down to its most critical parts.
That said, I rather enjoyed the book. Of course, at work we're going to be using Kafka, not RabbitMQ. But, I think there is overlapping functionality and philosophy; and, I do feel like I walked away from this book with a better understanding of how message queues can be used in a distributed application, regardless of the underlying choice of technology.
Message based architecture has brought me out of the dark ages of hard coded and deeply nested structured with a new world of asynchronous communication and decoupled applications. (Page 8)
With Kafka, we're discussed using two different types of messages: events and commands. But, Bailey keyed me into the fact that there are actually three distinct message patterns, (Page 23):
- Event Message: a document that describes something that already happened - including any data associated with the event.
- Command Message: a document that contains data for a command to execute, or work to be done.
- Request Message: similar to a command, this document also contains information on how to reply to the request.
For each of these three message archetypes, Bailey goes on to talk about how exchanges and queues should be configured. For example, Bailey recommends creating a dedicated exchange for each "event" message type; but, concedes that for "command" message types, a single exchange is capable of handling an entire subset of messages:
Rather than sending an "Account.Closed" or "Account.Updated" event through a single exchange, create an "Account.Closed exchange and a separate "Account.Updated" exchange. Any subscriber that knows how to handle the closed event will create a non-durable, exclusive queue with a binding to the "Account.Closed" exchange. The same applies to the updated event and exchange, or any other event / exchange that you need. When a message is published to this exchange and is routed to the queue, you know the code can handle it because the subscriber would not be there if it couldn't. (Page 29)
.... Unlike event messages where a single exchange should be dedicated to a single event, command messages can easily be grouped into a single exchange. A single "vehicle" exchange can handle a multitude of commands, using routing keys to ensure the command message is delivered to the correct queue. (Page 33)
Bailey also talks about what kind of data should be included in each type of message payload. For example, passing just an ID can be great in situations where security and message authenticity is a top priority:
I often use this pattern - send just an ID across the queue, and then load the message from the database. This is a great way to go when you have a database available on both ends (the message producer and message consumer)... You can also use this to protect yourself against bad messages - messages with "fake" or malicious data. (Page 20)
Of the three message types, the Request Message is still the most fuzzy in my head. The best I can gather is that the producer of the Request Message will (or can?) actually block and hold-open the parent request while the message is being shuttled through its asynchronous, queue-based life-cycle. So, essentially, the Request Message is a replacement for a direct service-to-service API call. If this is the case, then it certainly makes the "Need Pattern" discussed by Fred George in "Challenges in Implementing MicroServices" a bit more clear.
After reading RabbitMQ: Patterns For Applications by Derick Bailey, I feel like I have a much better sense of how message queues can be organized and consumed in a distributed system. And, I'll certainly be able to have more educated conversations about our decoupling effort at work. But now, I need to start learning about applied queue-based architectural patterns. Having just read Patterns of Enterprise Application Architecture by Martin Fowler, I was planning to look at "Enterprise Integration Patterns" next. And, as it turns out, Bailey actually recommends "Enterprise Integration Patterns" at the end of his book, referring to it as *the book* on messaging patterns. So, I guess I'm on the right track!
Reader Comments