How to create a repeatable job, thar repeat after complete?

[adinvadim]

And for example if I create 5 of such repeatable tasks each one repeat only after completion of it job.

Could I do it by API of worker and job?

[manast]

I am sorry but I find the question a bit confusing, we would need a bit more context and explanation in order to understand what you are asking.

[nullndr]

Hey @manast, this is something I also found a bit annoying. I'll try to explain the situation.

I have a repetable job that repeats every 10 seconds, but this job is pretty fragile and should be handled by only 1 worker at time to avoid race conditions.

The problem is that if it takes more than 10 seconds, then a second worker could pick up the second job, and that would cause a race condition.

This happens because currently bull adds a delayed job when the repetable job is picked up, and not when it has finished.

I would like to have a setting that does this, it adds the new repetable job when the previous has finished.

This would be great to avoid race conditions, since at any time there would be only 1 worker in every node processes that is running this job.

I don't know if this is possible, since bull should handle job failures before adding the new job.

[wernermorgenstern]

I actually want to follow this. Right now, for that part of the repeatable job, I have concurrency = 1, so only one worker can run the repeatable job. That works for me for now. However it would be great if there is a setting, where you can specify to create the next job when it is picked up (current functionality), or where the job gets created at end of the worker being run (on fail or success - it needs to be both, I think)

[vsamofal]

I need it as well, sometimes there are jobs like sending emails every minute, that for any reason may take longer, and running it twice will be super strange.

@manast do you have any concerns about the current bull architecture to reschedule after the job is completed? or any other hidden issues? as for me it should be ok, If the server is restarted in the middle of a job run, then this job will eventually become stalled and will be executed again and reschedule itself only when done. So should be ok?

I can help with implementing it for nodejs

[vsamofal]

I checked the source code, and seems like it's done when the job is completed and not when the job started, but the docs say that it will process job on each worker, if there is a time overlap, how is it possible if the next one won't be scheduled

strange ...

[manast]

The issue is that since it is not guaranteed that a job will be completed or failed in a reasonable time you could end with repeatable jobs stopping their iterations. Thats why they are added before, so that we can give the guarantee that the job is really going to be scheduled next time. A possible solution could be to check if a given repeatable job is still being processed, and then you could use this check to exit from the next iteration without doing anything.

[vsamofal]

but if the job won't be completed or fails in a reasonable timeframe it will go to the stalled job, and will repeat, so we can just set a stalled count to some big number and do that reschedule on complete or fail