You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We're running delayed_job on a beta environment and a production environment simultaneously. Both access the same database.
When we push a change to beta that e.g. deletes a job type, beta will continue to pull those job types from the queues (put there by production) and try to process them, failing every time, until we promote beta to production. Same in reverse if we add a job type on beta, production starts failing to process jobs.
The natural solution seems to be
# ensure the environment doesn't pull jobs it doesn't know aboutexport QUEUES=job1,job2,job3
However, manually keeping QUEUES in sync with the various jobs we have seems subject to forgetfulness and poor knowledge transfer when jobs are added or deleted.
Additionally as the number of jobs increases, the environment variable gets more and more unwieldy to manage.
I would love a DRY way to achieve this same result, like matching against the existing queue prefix:
moduleMyAppclassApplication < Rails::Applicationconfig.active_job.queue_name_prefix=ENV['QUEUE_PREFIX']# beta or productionendend
such that any job that starts with the prefix is processed, and others are not. This would enforce that an environment can only process jobs it knows how to without needing to keep QUEUES in sync manually.
The text was updated successfully, but these errors were encountered:
glacials
added a commit
to glacials/splits-io
that referenced
this issue
Jul 1, 2019
Previous commit didn't work because delayed_job takes from all queues
indiscriminately unless you manually specify the QUEUES env var. There
is no way to do this based on prefix, but I've opened
collectiveidea/delayed_job#1095.
For now we'll have to keep QUEUES in sync with the jobs we have.
glacials
added a commit
to glacials/splits-io
that referenced
this issue
Jul 1, 2019
* Fix poisoned queues
Fixes beta and production trying to process jobs enqueued by the other,
which can make a box error indefinitely as it tries to run a job it
doesn't know how to run.
Along with this I'll set QUEUE_PREFIX to `production` and `beta` on the
respective boxes.
* Prevent delayed_job from taking from every queue
Previous commit didn't work because delayed_job takes from all queues
indiscriminately unless you manually specify the QUEUES env var. There
is no way to do this based on prefix, but I've opened
collectiveidea/delayed_job#1095.
For now we'll have to keep QUEUES in sync with the jobs we have.
* Fix a comment and remove an unused config option
We're running delayed_job on a beta environment and a production environment simultaneously. Both access the same database.
When we push a change to beta that e.g. deletes a job type, beta will continue to pull those job types from the queues (put there by production) and try to process them, failing every time, until we promote beta to production. Same in reverse if we add a job type on beta, production starts failing to process jobs.
The natural solution seems to be
However, manually keeping
QUEUES
in sync with the various jobs we have seems subject to forgetfulness and poor knowledge transfer when jobs are added or deleted.Additionally as the number of jobs increases, the environment variable gets more and more unwieldy to manage.
I would love a DRY way to achieve this same result, like matching against the existing queue prefix:
such that any job that starts with the prefix is processed, and others are not. This would enforce that an environment can only process jobs it knows how to without needing to keep
QUEUES
in sync manually.The text was updated successfully, but these errors were encountered: