Issues when deploying static content on EFS

And this article is another article about how annoying Magento 2 can be and how it wasn’t built to be a scalable application.

Anyway, we have been getting several errors when running a static content deploy with the –jobs option:

magento setup:static-content:deploy --jobs 5

Right now each store only has one theme being used, but they are dependent on a base custom theme and also Magento blank theme. So once you run the command without –jobs it will deploy the blank theme, then the base theme and finally your theme.

But if you have several themes or even one on EFS, you will have some bottleneck as EFS is not very fast and it’s all done over a NFS connection, so it won’t be as fast as deploying on a local SSD and that’s when you add the –jobs option to the static content deploy. But on EFS if you have several files to be deployed on each theme, it might take several minutes and that’s when you get this nice error:

In Queue.php line 366:

Error while waiting for package deployed: 42; Status: 0


setup:static-content:deploy [-f|--force] [-s|--strategy [STRATEGY]] [-a|--area [AREA]] [--exclude-area [EXCLUDE-AREA]] [-t|--theme [THEME]] [--exclude-theme [EXCLUDE-THEME]] [-l|--language [LANGUAGE]] [--exclude-language [EXCLUDE-LANGUAGE]] [-j|--jobs [JOBS]] [--symlink-locale] [--content-version CONTENT-VERSION] [--refresh-content-version-only] [--no-javascript] [--no-css] [--no-less] [--no-images] [--no-fonts] [--no-html] [--no-misc] [--no-html-minify] [--] [<languages>...]

There is an issue open on Magento 2 repository and I even added a comment there: https://github.com/magento/magento2/issues/21852

This issue might be caused by this: https://github.com/magento/magento2/blob/2.3-develop/app/code/Magento/Deploy/Process/Queue.php#L391
And the function for the timeout: https://github.com/magento/magento2/blob/2.3-develop/app/code/Magento/Deploy/Process/Queue.php#L372
All points to const DEFAULT_MAX_EXEC_TIME = 400;
https://github.com/magento/magento2/blob/2.3-develop/app/code/Magento/Deploy/Process/Queue.php#L32

So it looks like if the static code deploy runs for over 400 seconds, it will fail. But Magento has fixed their ECE tool to add an option to increase the timeout: https://github.com/magento/ece-tools/pull/418
But for now this is not fixed on the Magento Open Source or Commerce edition.

So what can you do? Well, for now what we are going to do is to store the theme information for a store somewhere and when a code is deployed we get the theme and deploy only the theme assigned to a store. So for example if MageBR/mytheme is assigned to my store, I will run:

magento setup:static-content:deploy --theme MageBR/mytheme

It will speed things up and not fail as we can see below:

Deploy using quick strategy
frontend/Magento/blank/en_US 2456/2456 ============================ 100% % 1 min
frontend/MageBR/base/en_US 2765/2765 ============================ 100% % 2 mins
frontend/MageBR/mytheme/en_US 2765/2765 ============================ 100% % 2 mins

Execution time: 483.77401208878

But it would still be great to run that command with jobs to speed up the process even more. We should be safe to run the static content deploy with the theme and jobs option if it doesn’t take too long, but it would be nice to have an option to increase the timeout directly in the command line.

I hope you liked this article and please leave your comment and don’t forget to share!