Let me start saying that this article will be quick and simple, but it was a headache to find out the issues we encountered with running Magento crons with queue consumers on Kubernetes (or even Docker).
Configuring the cron is pretty simple and Magento has a nice documentation here: https://devdocs.magento.com/guides/v2.3/config-guide/cli/config-cli-subcommands-cron.html
But the problem is not just running the crons, it’s if you run crons while you have queues enabled using RabbitMQ, which will require to use the consumers. You can read more about the queues and consumers here: https://devdocs.magento.com/guides/v2.3/config-guide/mq/rabbitmq-overview.html
Let’s say you added a new module that will consume the queues or you just added to your app/etc/env.php:
'queue' =>
array (
'amqp' =>
array (
'host' => 'rabbitmq.example.com',
'port' => '11213',
'user' => 'magento',
'password' => 'magento',
'virtualhost' => '/',
'ssl' => true,
'ssl_options' => [
'cafile' => '/etc/pki/tls/certs/DigiCertCA.crt',
'certfile' => '/path/to/magento/app/etc/ssl/test-rabbit.crt',
'keyfile' => '/path/to/magento/app/etc/ssl/test-rabbit.key'
],
),
),Or you added the piece of code below during installation:
--amqp-host="<hostname>" --amqp-port="5672" --amqp-user="<user_name>" --amqp-password="<password>" --amqp-virtualhost="/"Good, now Magento will use consumers to work the queues. Now we encounter two problems. The first one is that every time a cron runs, it will create background processes with the consumers, for example:
$ kubectl exec -n my-store magento-75788af261-adwdc -c magento ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 18048 1932 ? Ss 14:13 0:00 /bin/bash /setup.sh
root 16 0.0 0.0 4284 588 ? S 14:13 0:00 /bin/sh /usr/sbin/apache2ctl -D FOREGROUND
root 18 0.0 0.0 274652 19168 ? S 14:13 0:00 /usr/sbin/apache2 -D FOREGROUND
www-data 20 0.9 0.2 344228 90216 ? S 14:13 1:50 /usr/sbin/apache2 -D FOREGROUND
www-data 24 0.9 0.2 344232 86344 ? S 14:13 1:41 /usr/sbin/apache2 -D FOREGROUND
www-data 59 0.0 0.2 291524 94128 ? S 14:15 0:03 /usr/local/bin/php /var/www/html/bin/magento queue:consumers:start commonErrorManagement --pid-file-path=commonErrorManagement-magento76799dc865rglqw.pid --max-messages=10000
www-data 61 0.0 0.2 291524 94024 ? S 14:15 0:03 /usr/local/bin/php /var/www/html/bin/magento queue:consumers:start ping --pid-file-path=ping-magento76799dc865rglqw.pid --max-messages=10000
www-data 63 0.0 0.3 317288 118348 ? S 14:15 0:05 /usr/local/bin/php /var/www/html/bin/magento queue:consumers:start productConsumer --pid-file-path=productConsumer-magento76799dc865rglqw.pid --max-messages=10000
www-data 65 0.0 0.2 293740 95408 ? S 14:15 0:03 /usr/local/bin/php /var/www/html/bin/magento queue:consumers:start stockConsumer --pid-file-path=stockConsumer-magento76799dc865rglqw.pid --max-messages=10000
www-data 67 0.0 0.2 293752 95652 ? S 14:15 0:03 /usr/local/bin/php /var/www/html/bin/magento queue:consumers:start productReturnUpdatedConsumer --pid-file-path=productReturnUpdatedConsumer-magento76799dc865rglqw.pid --max-messages=10000
www-data 69 0.0 0.3 293704 97224 ? S 14:15 0:03 /usr/local/bin/php /var/www/html/bin/magento queue:consumers:start refundUpdatedConsumer --pid-file-path=refundUpdatedConsumer-magento76799dc865rglqw.pid --max-messages=10000
www-data 71 0.0 0.3 317288 118240 ? S 14:15 0:04 /usr/local/bin/php /var/www/html/bin/magento queue:consumers:start orderCancellation --pid-file-path=orderCancellation-magento76799dc865rglqw.pid --max-messages=10000
www-data 73 0.0 0.3 317288 118116 ? S 14:15 0:04 /usr/local/bin/php /var/www/html/bin/magento queue:consumers:start orderGetNotifications --pid-file-path=orderGetNotifications-magento76799dc865rglqw.pid --max-messages=10000
www-data 75 0.0 0.3 317288 118440 ? S 14:15 0:04 /usr/local/bin/php /var/www/html/bin/magento queue:consumers:start orderResendNotification --pid-file-path=orderResendNotification-magento76799dc865rglqw.pid --max-messages=10000
www-data 77 0.0 0.3 317292 118508 ? S 14:15 0:04 /usr/local/bin/php /var/www/html/bin/magento queue:consumers:start orderConsumer --pid-file-path=orderConsumer-magento76799dc865rglqw.pid --max-messages=10000
www-data 79 0.0 0.2 291524 93944 ? S 14:15 0:03 /usr/local/bin/php /var/www/html/bin/magento queue:consumers:start orderUpdatedConsumer --pid-file-path=orderUpdatedConsumer-magento76799dc865rglqw.pid --max-messages=10000
www-data 82 0.0 0.3 295756 97992 ? S 14:15 0:03 /usr/local/bin/php /var/www/html/bin/magento queue:consumers:start customerShipmentDoneConsumer --pid-file-path=customerShipmentDoneConsumer-magento76799dc865rglqw.pid --max-messages=10000
www-data 84 0.0 0.2 291656 93784 ? S 14:15 0:03 /usr/local/bin/php /var/www/html/bin/magento queue:consumers:start orderLineCancelledConsumer --pid-file-path=orderLineCancelledConsumer-magento76799dc865rglqw.pid --max-messages=10000
www-data 86 0.0 0.2 291656 94968 ? S 14:15 0:03 /usr/local/bin/php /var/www/html/bin/magento queue:consumers:start shippingAddressUpdated --pid-file-path=shippingAddressUpdated-magento76799dc865rglqw.pid --max-messages=10000
www-data 92 0.0 0.3 317292 119656 ? S 14:15 0:04 /usr/local/bin/php /var/www/html/bin/magento queue:consumers:start serviceBusDiscover --pid-file-path=serviceBusDiscover-magento76799dc865rglqw.pid --max-messages=10000
www-data 94 0.0 0.3 317292 118524 ? S 14:15 0:04 /usr/local/bin/php /var/www/html/bin/magento queue:consumers:start async.operations.all --pid-file-path=async.operations.all-magento76799dc865rglqw.pid --max-messages=10000
www-data 178 1.6 0.2 336144 81796 ? S 16:24 0:53 /usr/sbin/apache2 -D FOREGROUND
www-data 179 1.2 0.2 336220 83776 ? S 16:24 0:42 /usr/sbin/apache2 -D FOREGROUND
www-data 195 1.3 0.2 336160 81964 ? S 16:27 0:42 /usr/sbin/apache2 -D FOREGROUND
www-data 198 1.0 0.2 333196 74692 ? S 16:30 0:31 /usr/sbin/apache2 -D FOREGROUND
www-data 199 1.0 0.2 333196 74696 ? S 16:30 0:31 /usr/sbin/apache2 -D FOREGROUND
www-data 200 1.0 0.2 333212 74708 ? S 16:30 0:29 /usr/sbin/apache2 -D FOREGROUND
www-data 201 1.3 0.2 333140 74632 ? S 16:30 0:40 /usr/sbin/apache2 -D FOREGROUND
www-data 210 1.1 0.2 333140 74636 ? S 16:30 0:34 /usr/sbin/apache2 -D FOREGROUND
root 440 0.0 0.0 36640 2712 ? Rs 17:19 0:00 ps auxAs you can see from above there are several consumers that started, they are all from Magento. If you have any custom queue, it will add the consumer to the list as well. That’s when the second issue comes up.
Magento has on its documentation that each consumer processes 1000 messages and then terminates: https://devdocs.magento.com/guides/v2.3/config-guide/mq/manage-message-queues.html#configuration
max_messages - the maximum number of messages for each consumer that must be processed before consumer terminate, by default is 1000. If it is 0, then the consumer never stops working.But as you can see above it is set to 10000 and we can confirm on their repo: https://github.com/magento/magento2/blob/2.3-develop/app/code/Magento/MessageQueue/Model/Cron/ConsumersRunner.php#L114
All right, no big deal if you are running the cron on a single or multi servers. But once you start the cron, it has these background processes that will only end if the queue has completed 10000 messages (by default). The cron will run until Kubernetes timesout (and the default is one hour).
We could then set –max-messages to 1, but the way Magento works it will ALWAYS wait for that one, instead of just checking if queue is empty or not and finish if it’s empty. There is an issue opened right now on their repository, but no sing that they will ever fix it: https://github.com/magento/magento2/issues/17951
So what to do?
Well, for now we just decided to run Supervisord on its own pod and the consumers inside Supervisord. And don’t remember to disable the consumers from running during cron on env.php file:
...
'cron_consumers_runner' => array(
'cron_run' => false
),
...I would just hope that Magento would consider building its application DevOps friendly.
If you managed to get this working a different way, don’t forget to add your comment below.
Powered by WPeMatico