RabbitMQ on EC2 - MQ implement in Erlang

RabbitMQ Amazon EC2 Images
Using RabbitMQ Amazon EC2 Images, RabbitMQ can be easilydeployed on the Amazon cloud. Images containing Ubuntu 9.10(Karmic Koala) with pre-installed RabbitMQ can be found below.Both 32 and 64 bit architectures are available.
32-bit RabbitMQ Images
Can be run on the smallest and cheapestm1.smallcloud computer instances. Fine for developmentpurposes, but due to small 32-bit virtual memory spacenot recommended for production.
64-bit RabbitMQ Images
Run on more demanding 64-bit instances. Recommended for productionusage.
Amazon EC2 Published Images
Following Ubuntu images with preinstalled RabbitMQ 1.7.0 have beenpublished:
Availability zoneArchAmiEc2 command
eu-west-1 x86_64 ami-c9002bbd ec2-run-instances ami-c9002bbd --key ${EC2_KEYPAIR} --instance-type m1.large --region eu-west-1
eu-west-1 i386 ami-b5002bc1 ec2-run-instances ami-b5002bc1 --key ${EC2_KEYPAIR} --instance-type m1.small --region eu-west-1
us-east-1 x86_64 ami-36cc2e5f ec2-run-instances ami-36cc2e5f --key ${EC2_KEYPAIR} --instance-type m1.large
us-east-1 i386 ami-3ecc2e57 ec2-run-instances ami-3ecc2e57 --key ${EC2_KEYPAIR} --instance-type m1.small
To start RabbitMQ instance just run the command from the fourth column.The string ${EC2_KEYPAIR} should bereplaced with the public ssh key name that is going to be used tolog in to the instance. You can find more informationabout ssh keys inGetting Started Guide. To run the instance, you should runa command like one of the above:
localhost:~$ ec2-run-instances ami-XXX --key XXXX --instance-type m1.large --region eu-west-1
Integration with Amazon EBS
RabbitMQ stores some data on disk. It may be worth storing thispersistent data on an Amazon EBS (Elastic Block Store) Volume.Decoupling data from the instance using EBS brings some benefits:
Data is less likely to be lost
Migration data to different instance is straightforward
EBS has a proper backup strategy
RabbitMQ images provide features to simplify integrationwith Amazon EBS block storage.Using EBS volumes with RabbitMQ instances is straightforward.All you need to do is:
Create an EBS volume based on our snapshot
Attach it under /dev/sdh
Finally, to apply the changes,reboot the instance
Setting up an EBS volume
Public snapshots of pre-configured RabbitMQ EBS volumes are listed below.
Availability zonePublic snapshot idEc2 command
eu-west-1 snap-321df35b ec2-create-volume --snapshot snap-321df35b --size 8 \
--region eu-west-1 --availability-zone eu-west-1b
us-east-1 snap-3640df5f ec2-create-volume --snapshot snap-3640df5f --size 8 \
--region us-east-1 --availability-zone us-east-1b
Create an EBS volume based on our pre-configured shared EBS snapshot.Our snapshot was created from 8GB EBS, so your volume musthave at least that size.
To create an EBS volume run the command from third column. Pleasetweak the --availability-zone parameter to matchthe availability zone of your instance. You can also specify sizeto be more than 8GB. The command might look like:
localhost:~$ ec2-create-volume --snapshot snap-XXX --size 8 --region us-east-1 --availability-zone us-east-XX
Attach a volume to an instance
To attach a volume to the instance, use the ec2-attach-volumecommand. Please remember to attach the volume under /dev/sdh.
localhost:~$ ec2-attach-volume --region eu-west-1 vol-xxxxxxxx -i i-xxxxxxxx -d /dev/sdhAttaching volumes is sticky, so EBS will remain attached until youkill the instance or unattach it manually. Volumes attached under/dev/sdh are automatically mounted as /vol.
Configure the RabbitMQ instance to use EBS
First, log into the instance as root:
localhost:~$ ssh -i KEYPAIR.pem root@ec2-XXXX.compute.amazonaws.comTo persuade RabbitMQ to use disk space from /volactual paths must be specified in its configuration. A scriptcalled rabbit-copy-config should be run on start-up. Itsets RabbitMQ's configuration to be read from an EBS volume.It links the configuration options prepared for RabbitMQ under /vol/etcto the standard /etc location. Now the default RabbitMQ installation will be affected.It will store data in the correct path automatically after a reboot.
rabbitmq:~$ rebootIf, however, you want to avoid reboots, you can run the scriptsmanually:
rabbitmq:~$ mount /volrabbitmq:~$ /etc/init.d/rabbit-copy-configrabbitmq:~$ /etc/init.d/rabbitmq-server restartFrom now on, RabbitMQ will store persistent data on EBS in/vol.
Resizing EBS
If your volume is bigger than 8GB, you need to expand the sizethe filesystem to use all the available space.
rabbitmq:~$ xfs_growfs /vol
EBS migration strategy
Moving RabbitMQ storage from one RabbitMQ instance to another istrivial.
Detach EBS volume from old RabbitMQ EC2 instance
Attach it to the new RabbitMQ instance
Reboot the instance (orfollow alternative steps)
EBS backup strategy
Amazon tools can create a snapshot of EBS volume as a file on S3.This approach is often used to backup EBS data.
localhost:~$ ec2-create-snapshot --region eu-west-1 vol-xxxxxxxx -d "Backup"
Running custom scripts on start-up
By using the ec2-run-instances --user-data-file option, you can specifya script to be executed during virtual machine start-up. This scriptwill be run before RabbitMQ starts.
localhost:~$ ec2-run-instances --user-data-file init-script.shFor example:
#!/bin/sh# This scrit will be executed before RabbitMQ has startedexport DEBIAN_FRONTEND=noninteractiveapt-get updateapt-get -y upgradeapt-get -y install man screen links htop
RabbitMQ status plugin
RabbitMQ EC2 Images have the status plugin preinstalled by default.To use it you must be able to access port 55672 of the instance.You can achieve that by opening that port on Amazon firewall. Be aware that thestatus plugin is secured only using RabbitMQ AMQP credentials.Unless you change default guest password, the status pagewill be open to everyone that can connect to the port.Tochange guest passwordrun rabbitmqctl change_password guest inside the instance.
Alternatively, you can forward a TCP port on your local machine.
localhost:~$ ssh -L 55672:localhost:55672 root@ec2-xx-xx-xx-xx.eu-west-1.compute.amazonaws.comYou should be able to see RabbitMQ status page under the forwarded porton your local machine, for example:http://guest:guest@localhost:55672/.
Testing RabbitMQ instance
Once the RabbitMQ instance has been started and configured RabbitMQ instance,check if it actually works.You can do this by installing the RabbitMQ java client,and running the test scripts that come with it. Use the following instructionsto install the required packages on the instance:
apt-get updateapt-get install sun-java6-jre wgetwget http://www.rabbitmq.com/releases/rabbitmq-java-client/v1.7.0/rabbitmq-java-client-bin-1.7.0.tar.gztar xzf rabbitmq-java-client-bin-1.7.0.tar.gzcd rabbitmq-java-client-bin-1.7.0When the dependencies are available, you can run the test:
rabbitmq:~/rabbitmq-java-client-1.7.0$ sh ./runjava.sh com.rabbitmq.examples.MulticastMainOn the screen you should see output similar to this:
starting consumer #0starting producer #0recving rate: 2920 msg/s, min/avg/max latency: 13431/137515/349586 microsecondssending rate: 7182 msg/srecving rate: 4513 msg/s, min/avg/max latency: 352244/535973/715864 microsecondssending rate: 4897 msg/srecving rate: 4119 msg/s, min/avg/max latency: 654393/829167/1021911 microsecondssending rate: 4989 msg/sIf you followed thesteps above, about the status plugin,you should be able to see the load on the status page:http://guest:guest@localhost:55672/.
Please remember, this is all quite new and its status is experimental. If you run intoany difficulties pleaselet us know.