RabbitMq之WorkQueues

Round-robin dispatching

默认情况下，RabbitMQ将会发送的每一条消息给下一个消费者，在序列中。平均每个消费者都会得到相同数量的信息 

实例说明:模拟存在多个rabbitmq consumer端的时候,每个Consumer消息的接受情况.

首先在cmd中执行rabbitmq-server.bat命令,将rabbittmq server服务器启动起来,在浏览器中查看一下channel情况:

编写rabbitmq-send端代码:

                ConnectionFactory factory = new ConnectionFactory();factory.setHost("localhost");Connection connection = factory.newConnection();Channel channel = connection.createChannel();//it will only be created if it doesn't exist alreadychannel.queueDeclare(QUEUE_NAME, true, false, false, null);String[] messages = new String[]{"11 message.","21 message..","31 message...","41 message....","51 message.....","61 message.....","71 message.....","81 message.....","91 message....."};for(String msg : messages){channel.basicPublish("", QUEUE_NAME, null, msg.getBytes());            //channel.basicPublish("", QUEUE_NAME, MessageProperties.PERSISTENT_TEXT_PLAIN, msg.getBytes());System.out.println(" [x] Sent '" + msg + "'");}channel.close();connection.close();

编写两个rabbitmq-consumer端代码:(两个consumer是一样的即可)

package yzr.main;import java.io.IOException;import com.rabbitmq.client.AMQP;import com.rabbitmq.client.Channel;import com.rabbitmq.client.Connection;import com.rabbitmq.client.ConnectionFactory;import com.rabbitmq.client.Consumer;import com.rabbitmq.client.DefaultConsumer;import com.rabbitmq.client.Envelope;public class Worker {private final static String QUEUE_NAME = "hello";public static void main(String[] args) throws Exception {ConnectionFactory factory = new ConnectionFactory();    factory.setHost("localhost");    Connection connection = factory.newConnection();    Channel channel = connection.createChannel();    boolean durable = true;    channel.queueDeclare(QUEUE_NAME, durable, false, false, null);    System.out.println(" [*] Waiting for messages. To exit press CTRL+C");    //channel.basicQos(30);    Consumer consumer = new DefaultConsumer(channel) {      @Override      public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body)          throws IOException {        String message = new String(body, "UTF-8");        System.out.println(" [x] Received '" + message + "'");        try {            doWork(message);          } catch (Exception e) {e.printStackTrace();} finally {            System.out.println(" [x] Done");          }      }    };    boolean autoAck = true;    channel.basicConsume(QUEUE_NAME, autoAck, consumer);}private static void doWork(String task) throws InterruptedException {    for (char ch: task.toCharArray()) {        if (ch == '.') Thread.sleep(1000);    }} }

将两个consumer运行起来,在localhost:15672中的channel中可以看到:

然后来测试一下当send发送消息时,每个consumer接受到的消息是如何的.:

send端发送消息:

 [x] Sent '11 message.'
 [x] Sent '21 message..'
 [x] Sent '31 message...'
 [x] Sent '41 message....'
 [x] Sent '51 message.....'
 [x] Sent '61 message.....'
 [x] Sent '71 message.....'
 [x] Sent '81 message.....'
 [x] Sent '91 message.....'

consumer1:

[x] Received '11 message.'
 [x] Done
 [x] Received '31 message...'
 [x] Done
 [x] Received '51 message.....'
 [x] Done
 [x] Received '71 message.....'
 [x] Done
 [x] Received '91 message.....'
 [x] Done

consumer2:

 [*] Waiting for messages. To exit press CTRL+C
 [x] Received '21 message..'
 [x] Done
 [x] Received '41 message....'
 [x] Done
 [x] Received '61 message.....'
 [x] Done
 [x] Received '81 message.....'
 [x] Done

结果:rabbitmq将消息按平均分到了每个consumer端.序列为奇数的消息发送到consumer1,序列为偶数消息的发送到了consumer2.每个consumer端接受到的消息数量基本是一样多的.这就是round-robin.

By default, RabbitMQ will send each message to the next consumer, in sequence. On average every consumer will get the same number of messages. This way of distributing messages is called round-robin. 

Message acknowledgment

为了确保消息不会丢失，RabbitMQ支持message acknowledgments.。ACK（nowledgement）是从消费后发送到一个特定的消息告诉RabbitMQ已经收到、处理和RabbitMQ删除它。 

为了在consumer挂掉之后消息不丢失,rabbitmq提供了一种方式,可以自动识别挂掉的consumer,将发往这个挂掉的consumer的消息发往下一个正常的consumer,这样就保证了消息不会因为某一个consumer挂掉之后丢失.

将consumer端的代码改动一下,如下图:

boolean autoAck = false;channel.basicConsume(QUEUE_NAME, autoAck, consumer);

将此处的autoAck的值设为false之后,我们发现当send端发送消息之后,当接受消息的consumer端挂掉之后,rabbitmq会将消息发往到正常的consumer,(消息不会丢失)

案例说明:我们使用send发送9条消息到两个consumer,默认情况下,假设奇数序列的消息会发送到consumer1,偶数序列的消息会发送到consumer2.并在此时,我们关闭consumer模拟挂掉的情况,我们看一下消息的发送情况:

开启两个consumer,然后使用send发送消息:

 [x] Sent '11 message.'
 [x] Sent '21 message..'
 [x] Sent '31 message...'
 [x] Sent '41 message....'
 [x] Sent '51 message.....'
 [x] Sent '61 message.....'
 [x] Sent '71 message.....'
 [x] Sent '81 message.....'
 [x] Sent '91 message.....'

切换到consumer2中,我们会看到已经接收到了一两条消息,然后我们将consumer2关闭掉,模拟挂掉的情况:

再切换到consumer1中,会看到如下图所示:

 使用以下命令行可以看到message ack的消息:

rabbitmqctl.bat list_queues name messages_ready messages_unacknowledged

If a consumer dies (its channel is closed, connection is closed, or TCP connection is lost) without sending an ack, RabbitMQ will understand that a message wasn't processed fully and will re-queue it. If there are other consumers online at the same time, it will then quickly redeliver it to another consumer. That way you can be sure that no message is lost, even if the workers occasionally die.

对于ack延迟接收的数据,在视图中显示如下图:

Ready:保存在rabbitmq服务器中且未被消费的消息数量

Unacked:未确认消费的消息数量.

在上面的代码中只在handleDelivery函数将消息打印了出来,为了确认ack消息,还需要添加一句代码,通知rabbitmq服务器ack消息已经被消费:

 //确认消息消费 channel.basicAck(envelope.getDeliveryTag(), true);

完整的代码:

package yzr.main;import java.io.IOException;import com.rabbitmq.client.AMQP;import com.rabbitmq.client.Channel;import com.rabbitmq.client.Connection;import com.rabbitmq.client.ConnectionFactory;import com.rabbitmq.client.Consumer;import com.rabbitmq.client.DefaultConsumer;import com.rabbitmq.client.Envelope;public class Worker2 {private final static String QUEUE_NAME = "hello";public static void main(String[] args) throws Exception {ConnectionFactory factory = new ConnectionFactory();    factory.setHost("localhost");    Connection connection = factory.newConnection();    Channel channel = connection.createChannel();    boolean durable = false;    channel.queueDeclare(QUEUE_NAME, durable, false, false, null);    System.out.println(" [*] Waiting for messages. To exit press CTRL+C");    //int prefetchCount = 100;    //channel.basicQos(prefetchCount);    Consumer consumer = new DefaultConsumer(channel) {      @Override      public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body)          throws IOException {        String message = new String(body, "UTF-8");        System.out.println(" [x] Received '" + message + "'");        try {            doWork(message);            //确认消息消费            channel.basicAck(envelope.getDeliveryTag(), true);          } catch (Exception e) {e.printStackTrace();} finally {            System.out.println(" [x] Done");                      }      }      @Override    public void handleConsumeOk(String consumerTag) {    System.out.println(consumerTag+" reisgter");    }    };    boolean autoAck = false;    channel.basicConsume(QUEUE_NAME, autoAck, consumer);}private static void doWork(String task) throws InterruptedException {    for (char ch: task.toCharArray()) {        if (ch == '.') Thread.sleep(1000);    }} }

Message durability

消息持久化,比如当我们重启rabbitmqserver服务器的时候,send端发送的消息(已经保存在rabbitmq中的消息)将会全部丢失,为了保证服务器不正常(包括重启)的情况下,保存在服务器中的消息不会丢失,我们可以将发送的消息设置为持久化,即使重启服务器,只要服务器能正常工作的情况下,消息就可以正常获取到.

我们需要更改一下代码如下图所示:

 boolean durable = true; channel.queueDeclare(QUEUE_NAME, durable, false, false, null); .... channel.basicPublish("", QUEUE_NAME, MessageProperties.PERSISTENT_TEXT_PLAIN, msg.getBytes());

启动send端发送消息,注意此时先不要开启consumer.我们此时只需要确保将消息发送到rabbitmq服务端即可:

 [x] Sent '111 message.'
 [x] Sent '211 message..'
 [x] Sent '311 message...'
 [x] Sent '411 message....'
 [x] Sent '511 message.....'
 [x] Sent '611 message.....'
 [x] Sent '711 message.....'
 [x] Sent '811 message.....'
 [x] Sent '911 message.....'

然后,现在直接关闭rabbitmq-server服务器:Ctrl+C

然后再执行rabbitmq-server.bat,开启rabbitmq服务器,然后开启consumer,查看一下刚才发送的消息能否收到:

消息还是能够正常收到.

Marking messages as persistent doesn't fully guarantee that a message won't be lost. Although it tells RabbitMQ to save the message to disk, there is still a short time window when RabbitMQ has accepted a message and hasn't saved it yet. Also, RabbitMQ doesn't do fsync(2) for every message -- it may be just saved to cache and not really written to the disk. The persistence guarantees aren't strong, but it's more than enough for our simple task queue.

Fair dispatch

设置consumer接收消息的数量最大值:

    int prefetchCount = 100;    channel.basicQos(prefetchCount);

consumer只会最多接收指定的条数的消息,比如这里指定了100,那么即使send端发送了101条消息到这里,此consumer只会接收100条.超出的消息会发送到另外一个不忙的consumer中.

In order to defeat that we can use the basicQos method with the prefetchCount = 1 setting. This tells RabbitMQ not to give more than one message to a worker at a time. Or, in other words, don't dispatch a new message to a worker until it has processed and acknowledged the previous one. Instead, it will dispatch it to the next worker that is not still busy.

因为rabbitmq平均分配消息的这种默认机制,无法确定每个consumer是否能够负载均衡,因为简单的以序列的顺序分配给每个consumer,比如上面例子两个consumer的奇偶数分配,这样并不能保证每个consumer工作压力能够正常负载,比如有可能处理奇数序列的消息压力很重,而处理偶数序列的消息很轻松.所以在这里可以设置fair dispatch的方式,形同告诉服务器我很忙,把消息发送到下一个不忙的consumer处理.以这种形式负载压力.

If all the workers are busy, your queue can fill up. You will want to keep an eye on that, and maybe add more workers, or have some other strategy.