You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Example case: Application receives 3k events from API per second. We want to store them in the database, but processing every event as single insert would generate 3k/s inserts and heavily loads the db. The solution is to use bulk insert, but to use that we need a way to read jobs in a bulk way inside the processor. This is a common practice with other queue systems.
Describe the solution you'd like
Allow the processor to receive many jobs in once instead of only one.
Describe alternatives you've considered
Additional context
The text was updated successfully, but these errors were encountered:
mariuszbeltowski
changed the title
Bulk Processor - process many jobs at once
Bulk / Batch Processor worker - process many jobs at once
Apr 9, 2024
Thank you @roggervalf . How it will work if we set 10 items per batch, but on the queue we will get just 8? Is there any timeout for fulfilling the worker for items.length < batch? Can we control that timeout? I couldn't find that in the docs.
Is your feature request related to a problem? Please describe.
Example case: Application receives 3k events from API per second. We want to store them in the database, but processing every event as single insert would generate 3k/s inserts and heavily loads the db. The solution is to use bulk insert, but to use that we need a way to read jobs in a bulk way inside the processor. This is a common practice with other queue systems.
Describe the solution you'd like
Allow the processor to receive many jobs in once instead of only one.
Describe alternatives you've considered
Additional context
The text was updated successfully, but these errors were encountered: