Every day in the world of modern technology, high availability has become the key requirement of any layer in technology. Message broker software has become a significant component of most stacks. In this article, we will present a RabbitMQ tutorial: how to create highly available message queues using RabbitMQ.
RabbitMQ is an open-source message broker software (also called message-oriented middleware) that implements the Advanced Message Queuing Protocol (AMQP). RabbitMQ server is written in the Erlang programming language.
The RabbitMQ Cluster
Clustering connects multiple nodes to form a single logical broker. Virtual hosts, exchanges, users and permissions are mirrored across all nodes in a cluster. A client connecting to any node can see all the queues in a cluster.
Clustering enables high availability of queues and increases the throughput.
A node can be a Disc node or RAM node. RAM node keeps the message state in memory with the exception of queue contents which can reside on a disk if the queue is persistent or too big to fit into memory.
RAM nodes perform better than Disc nodes because they don’t have to write to a disk as much as disk nodes. But, it is always recommended to have disk nodes for persistent queues.
We’ll discuss how to create and convert RAM and Disk nodes later in the post.
Prerequisites:
Network connection between nodes must be reliable.
All nodes must run the same version of Erlang and RabbitMQ.
All TCP ports should be open between nodes.
We have used CentOS for the demo. Installation steps may vary for Ubuntu and OpenSuse. In this demo, we have launched two m1.small servers in AWS for master and slave nodes.
1. Install Rabbitmq
Install Rabbitmq in master and slave nodes.
$ yum install rabbitmq-server.noarch
2. Start Rabbitmq
/etc/init.d/rabbitmq-server start
3. Create the Cluster
Stop RabbitMQ in Master and slave nodes. Ensure service is stopped properly.
/etc/init.d/rabbitmq-server stop
Copy the file below to all nodes from the master. This cookie file needs to be the same across all nodes.
$ sudo cat /var/lib/rabbitmq/.erlang.cookie
Make sure you start all nodes after copying the cookie file from the master.
Start RabbitMQ in master and all nodes.
$ /etc/init.d/rabbitmq-server start
Then run the following commands in all the nodes, except the master node:
$ rabbitmqctl stop_app
$ rabbitmqctl reset
$ rabbitmqctl start_app
Now, run the following commands in the master node:
$ rabbitmqctl stop_app
$ rabbitmqctl reset
Do not start the app yet.
The following command is executed to join the slaves to the cluster:
$ rabbitmqctl join_cluster rabbit@slave1 rabbit@slave2
Update slave1 and slave2 with the hostnames/IP address of the slave nodes. You can add as many slave nodes as needed in the cluster.
Check the cluster status from any node in the cluster:
$ rabbitmqctl cluster_status
By default, the cluster stores messages on the disk. You can also choose to store Queues in Memory.
You can have a node as a RAM node while attaching it to the cluster:
$ rabbitmqctl stop_app
$ rabbitmqctl join_cluster --ram rabbit@slave1
It is recommended to have at least one disk node in the cluster so that messages are stored on a persistent disk and can avoid any loss of messages in case of a disaster.
The performance of RAM nodes are a little better than disk nodes, and gives you better throughput.
4. Set the HA Policy
The following command will sync all the queues across all nodes:
$ rabbitmqctl set_policy ha-all "" '{"ha-mode":"all","ha-sync-mode":"automatic"}'
Ex: Policy where queues whose names begin with “ha.” are mirrored to all nodes in the cluster: $ rabbitmqctl set_policy ha-all “^ha.” ‘{“ha-mode”:”all”}‘
5. Test the Queue mirror
We are going to run a sample python program to create a sample queue. You need the below packages installed from where you want to run the program.
Install python-pip
$ yum install python-pip.noarch
Install Pika
$ sudo pip install pika==0.9.8
Create send.py file and copy the content below. You need to update the “localhost” with name/ip of master/slave node.
#!/usr/bin/env python
import pika
connection = pika.BlockingConnection(pika.ConnectionParameters(
host='localhost'))
channel = connection.channel()
channel.queue_declare(queue='hello')
channel.basic_publish(exchange='',
routing_key='hello',
body='Hello World!')
print " [x] Sent 'Hello World!'"
connection.close()
Run the python script using the command:
$ python send.py
This will create a Queue (hello) with a message on the RabbitMQ cluster.
Check if the message is available across all nodes.
$ sudo rabbitmqctl list_queues
Now, create a file named receive.py and copy the content below.
#!/usr/bin/env python
import pika
connection = pika.BlockingConnection(pika.ConnectionParameters(
'localhost'))
channel = connection.channel()
channel.queue_declare(queue='hello')
def callback(ch, method, properties, body):
print " [x] Received %r" % (body,)
channel.basic_consume(callback,
queue='hello',
no_ack=True)
print ' [*] Waiting for messages. To exit press CTRL+C'
channel.start_consuming()
Run the script and check the Queue in either the slave or master:
$ sudo rabbitmqctl list_queues
6. Setup Load Balancer
Now, we have multiple MQs running in a cluster. All are in sync and have the same queues.
How do we point our application to the cluster? We can’t point our application to a single node. If a node fails, we need a mechanism to auto-failover to other nodes in the cluster. There are multiple ways to achieve it. But, we prefer to use the load balancer.
There are two advantages in using the load balancer:
-
High availability
-
Better network throughput because the load is evenly distributed across nodes.
Create a load balancer in front of it and map the backend MQ instance. You can choose either HAProxy or Apache or Nginx or any hardware load balancer you use in your organization.
If the servers are running in an AWS VPC, then choose internal load balancer. Update the application to point to the load balancer endpoint.
Get Started with AWS
Receive AWS tips, DevOps best practices, news analysis, commentary and more. Sign up for our blog here and set your topic and frequency preferences. Or, download our guide on getting started with AWS, establishing a secure AWS enterprise architecture with Flux7 Landing Zones.
For more RabbitMQ knowledge, read: Comparing RabbitMQ for AWS with AWS SQS FIFO queues
