Consumer Brokers: MySQL to RabbitMQ.

RabbitMQ vs MySQL

At the time of this writing, the default broker for consumers in Magento 2.3+ is MySQL, with the exception of the async.operations.all consumer which utilizes RabbitMQ. The Magento Dev docs explain how this can be changed in the env.php file. The RabbitMQ broker provides much better performance than MySQL. We recommend that high traffic sites utilize RabbitMQ for the most often used consumers.

Example: Single consumer

The following example is for exportProcessor which is a consumer used for exporting data. By default it uses MySQL as a broker.

1. First confirm that you have RabbitMQ enabled and configured.

2. Find the XML files for the module in question. For exportprocessor, this the Magento 2 core module named ImportExport located in the app/code/Magento/ImportExport/ directory. Each of these files will have a piece of information that we will need in our configuration.

   • queue_consumer.xml
   • queue_publisher.xml
   • queue_topology.xml

   queue_consumer.xml contains the consumer name

   queue_publisher.xml contains the publisher topic

   queue_topology.xml contains the binding topic

3. Once you have these pieces of information you can add them to the env.php as follows to switch the RabbitMQ broker for exports:

'queue' => [
        'amqp' => [
            'host' => 'rabbitmq',
            'port' => '5672',
            'user' => 'username',
            'password' => 'yb2sovjbaa4atsbbn0s0ilovemagemojo',
            'virtualhost' => '/',
            'ssl' => 'false',
            'ssl_options' => []
        ],
        'consumers_wait_for_messages' => 1,
        'topics' => [
            'import_export.export' => [
                'publisher' => 'amqp-magento'
            ]
        ],
        'config' => [
            'publishers' => [
                'import_export.export' => [
                    'connections' => [
                        'amqp' => [
                            'name' => 'amqp',
                            'exchange' => 'magento',
                            'disabled' => false
                        ],
                        'db' => [
                            'name' => 'db',
                            'disabled' => true
                        ]
                    ]
                ]
            ]
        ],
        'consumers' => [
            'exportProcessor' => [
                'connection' => 'amqp'
            ]
        ]
    ],

Note

Afterwards you must run php bin/magento setup:upgrade command so Magento 2 can generate a new queue named "export". You will see the following output: 2020-11-03 23:05:34.999 [info] <0.8343.0> closing AMQP connection <0.8343.0> (127.0.0.1:38132 -> 127.0.0.1:5672, vhost: '/', user: 'guest')

Once setup:upgrade has been run, you can view the new queue that has been created with the rabbitmqadmin command:

rabbitmqadmin --host rabbitmq -u username -p ENTER-PASSWORD-HERE list queues name

Example: Multiple consumers

This Example is for multple consumers switched to use RabbitMQ.

'queue' => [
        'amqp' => [
            'host' => 'rabbitmq',
            'port' => '5672',
            'user' => 'username',
            'password' => 'yb2sovjbaa4atsbbn0s0ilovemagemojo',
            'virtualhost' => '/',
            'ssl' => 'false',
            'ssl_options' => []
        ],
        'consumers_wait_for_messages' => 1,
        'topics' => [
            'product_action_attribute.update' => [
                'publisher' => 'amqp-magento'
            ],
            'import_export.export' => [
                'publisher' => 'amqp-magento'
            ]
        ],
        'config' => [
            'publishers' => [
                'product_action_attribute.update' => [
                    'connections' => [
                        'amqp' => [
                            'name' => 'amqp',
                            'exchange' => 'magento',
                            'disabled' => false
                        ],
                        'db' => [
                            'name' => 'db',
                            'disabled' => true
                        ]
                    ]
                ],
                'import_export.export' => [
                    'connections' => [
                        'amqp' => [
                            'name' => 'amqp',
                            'exchange' => 'magento',
                            'disabled' => false
                        ],
                        'db' => [
                            'name' => 'db',
                            'disabled' => true
                        ]
                    ]
                ]
            ]
        ],
        'consumers' => [
            'product_action_attribute.update' => [
                'connection' => 'amqp'
            ],
            'exportProcessor' => [
                'connection' => 'amqp'
            ]
        ]
    ],