Big amount of repeatable jobs

[bmestanov]

Hi community, for a project of mine I need to run tasks periodically. Currently, I am doing this manually via node-cron, but I'm thinking about delegating this to bullmq with the repeat option.

I would have ~16k jobs to run every 14 days, ideally spread evenly. This number will go up in time.

Do you have any experience with a bigger number of repeating jobs, have you faced any issues or pitfalls using bullmq?

Thanks!

[manast]

Is it one repeatable job that is going to repeat 16k per 14 days or is it 16k independent repeatable jobs?

[bmestanov]

It's 16k independent repeatable jobs, each with its own data.

[manast]

I think that it is a bit overkill. Repeatable jobs are designed for having few but that can repeat very often if you want.