Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for using inside Worker (worker_threads module) #151

Open
tarquas opened this issue Apr 8, 2019 · 6 comments · May be fixed by #182
Open

add support for using inside Worker (worker_threads module) #151

tarquas opened this issue Apr 8, 2019 · 6 comments · May be fixed by #182

Comments

@tarquas
Copy link

tarquas commented Apr 8, 2019

Please add support of Worker Threads to native addon.

Currently, trying to use it inside Worker, it produces error:

Error: Module did not self-register.
-----
 > Object.Module._extensions..node --- internal/modules/cjs/loader.js:846:18
 > Module.load --- internal/modules/cjs/loader.js:672:32
 > tryModuleLoad --- internal/modules/cjs/loader.js:612:12
 > Function.Module._load --- internal/modules/cjs/loader.js:604:3
 > Module.require --- internal/modules/cjs/loader.js:711:19
 > require --- internal/modules/cjs/helpers.js:14:16
 > bindings --- /aufs/devsave/dev/projects/arfc/node_modules/bindings/bindings.js:112:48
 > Object.<anonymous> --- /aufs/devsave/dev/projects/arfc/node_modules/gl/webgl.js:4:35
 > Module._compile --- internal/modules/cjs/loader.js:805:30
 > Object.Module._extensions..js --- internal/modules/cjs/loader.js:816:10

Related docs: https://github.com/nodejs/node/blob/master/doc/api/addons.md#worker-support
IMHO This is vital since the processing is synchronous and is most likely to be run inside separate thread.

Thanks

@dhritzkiv
Copy link
Member

What is your use-case for needing worker threads, out of curiosity? Is your workload CPU intensive?

We currently have no plan for adding this functionality to headless-gl, but if we get more requests for this feature, and when the node API stabilizes, we'll revisit it.

You're more than welcome to fork this project, add the functionality yourself, and submit a PR!

@robertleeplummerjr
Copy link
Collaborator

https://stackoverflow.com/questions/28486891/uncaught-error-module-did-not-self-register was an interesting read. Specifically https://stackoverflow.com/a/28859693/1324039. It may be as simple as rebuilding via:
npm rebuild or rm -r node_modules then npm install

Can you try this and see if that works?

@tarquas
Copy link
Author

tarquas commented Apr 10, 2019

Hello! Thanks for the replies.

What is your use-case for needing worker threads, out of curiosity?

I develop web service for my scientific research dealing with time series autoregressive forecasting. I use gpu.js to scale on GPU of some heavy operations like matrix inversion and matrix multiplication (height of matrix is the size of given historical data to analyze!). Since the computation is synchronous, it's designed to perform in separate thread, powered up by worker pool, and delivering the results using messaging (.postMessage()).

You're more than welcome to fork this project, add the functionality yourself, and submit a PR!

Thanks. I was thinking about it and I can do it if I get some spare time coupled with inspiration.
For now I rewrote the worker pool to use cluster and that was successful. In prospective future i gonna get back to threads as they're more optimal.

npm rebuild or rm -r node_modules then npm install

That didn't work, as the native addon is written on nan, which doesn't provide proper context awareness support (related issue comment). NAN_MODULE_WORKER_ENALBED didn't solve. As I understood, making the support mainly consists of isolating the globals.

@felicemarra
Copy link

Any updates about this? I'm also trying to gl context in worker_threads and using cluster and messaging it to slow for 30 fps

@dhritzkiv
Copy link
Member

dhritzkiv commented Aug 19, 2020

@felicemarra initial work had begun in one of the PRs, but has been held up by a few blockers, and the substantial amount of work to refactor the library to be "thread safe". Because of this, it's unlikely this will be completed any time soon.

Further, there's no guarantee that utilizing worker threads will in fact bring dependable, performant rendering that you're expecting. It'd definitely be faster than cluster + messaging, though.

@prideout
Copy link

prideout commented Mar 17, 2023

The headless-gl module is super useful in jest and vitest unit tests. One motivation for supporting worker threads is that vitest enables threads by default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants