Replies: 1 comment
-
@dSecret rate limiting is implemented by limiting the number of jobs that are being picked by the processors. Although your example is valid, I do not see what change would make it better. You seem to suggest that the limit should be based on "completed" jobs, but then you will get the same problem in reverse, if your jobs are slow to process then you process less per second, so your workers will pick faster and thus it could also happen they call the third party api faster than 10 per second... |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I'm using BullMQ in one of my projects & I'm required to limit the no. of requests hitting the 3rd party API per second.
As per the official doc, I'm using the limiter with Worker & it seems to be great for me. I'm just a little confused with the behaviour of rate limiter though here -
By processed - Does it mean workers will take a maximum (e.g.)10 jobs/sec from the queue or they will complete a maximum of 10 jobs per second ? -
What I mean by these -
Let's say, I have 20 rate limiter workers (with job concurrency equal to 2) are running & time of job completion is 0.5s to 2 sec. For each job, worker is only sending data to 3rd party API. Once the data is sent, job is complete.
At nth second, 10 jobs(batch1) are picked by the 5 workers & for (n+1)th second again 10 jobs(batch2) would be picked. Here there is a possibility that some of the job from batch1 would be getting completed(3rd party API call) during (n+1)th second for some reason.
Consider a situation that all 10 API calls from batch2 are getting done during (n+1)th second & 3 API calls from batch1 are also being done in (n+1)the second. So total API cals in (n+1)th second would be 13, crossing the rate limit of 10.
Here the no. of job getting picked by worker is fixed but per second how many jobs are getting completed can vary.
No. of API requests hitting the 3rd party per second will depend on how much time the workers are taking to complete the task(send it to 3rd party with 200 http status). While in the second scenario, I'm sure that at most 10 requests would be hitting the 3rd party per second.
I have gone through the documentation & issues raised in the official repo. I have found a similar issue but couldn't get any clarity there.
Also, In above scenario whether 10 jobs would be picked by 5 workers or 10 jobs would be picked by 10 workers(as there are idle workers available) ? Basically, How round robin distribution of jobs works in case of concurrency set to more than 1?
Please guide me here & do correct me if my understanding related to BullMQ is wrong anywhere.
Beta Was this translation helpful? Give feedback.
All reactions