Runtime environment for libui-node

[mimecorg]

I think that one of the biggest problems with libui-node is that it lacks a proper runtime environment. Right now the only way to run a libui-node application is to create a wrapper script which basically runs "node path/to/script.js". This is problematic especially on Windows, because you get a console window.

I've been experimenting with the following solution:

• compile Node.js as a shared library, node.dll
• compile libui as libui.dll
• create a wrapper executable, let's call it shell.exe, which links to both node.dll and libui.dll

The differences between shell.exe and regular node.exe are as follows:

• it's a Windows application, so there's no console
• it can be run without any arguments, and it will automatically pass the path to the application script (for example, <exe_path>/app/main.js) to node::Start(), so no wrapper script is necessary
• it can report fatal errors using uiMsgBox() instead of printing them to the console (similar to how Electron handles fatal errors)

In that configuration, the nbind.node library from libui-node will not work "out of the box", because it tries to link to node.exe, not node.dll. But it can be rebuilt by running auto-gyp with the --nodedir option. This is pretty much how native dependencies are recompiled by electron-builder to work with Electron which also uses a node.dll.

I think that this is the bare minimum to have a runtime environment for libui-node, especially on Windows, but this probably applies to other platforms as well. I was able to create and successfully run a proof of concept, but I would like to discuss and hear your opinion before I continue working on this.

The next step would be to integrate libui-node into shell.exe as a built-in module. Then, at least in theory, it would be possible to integrate the libui main loop with Node.js main loop, and possibly eliminate the async hooks and other hacks.

[parro-it]

Hi @mimecorg, thank you to open the issue, I think it can start an interesting discussion.

I don't agree with you, and I actively try, from the start of the project, to make libui-node works with standard Node.js distribution out of the box.

According to me, this will greatly simplify development, because you can use all the libraries that npm offers to you, without the need to recompile it.

I understand what you're saying about running the app with a console script, but this problem will be solved by packaging the app using tools such as https://github.com/albe-rosado/create-proton-app/.

I haven't tried it yet, but I strongly think that is the way to go.

What you are describing will need a huge amount of work to be maintained, and I'm writing libui-node in my spare time. @mischnic is doing really a great job here, too, but (I think) he's in the same position as me.

Some other specific points:

it's a Windows application, so there's no console

If the platform has no console, a huge part of existing modules from npm cannot works, because they rely on console existence. How you can replace the console for them to work? Even internal part of node.js relay on the console.

it can report fatal errors using uiMsgBox() instead of printing them to the console (similar to how Electron handles fatal errors)

uiMsgBox is a blocking call, it cannot replace the console, because that is supposed to be non-blocking.

Then, at least in theory, it would be possible to integrate the libui main loop with Node.js main loop, and possibly eliminate the async hooks and other hacks.

Last time I evaluate the idea of of using a separate node.js runtime, was to improve the integration between node.js event-loop and the GUI native platform one. But then I found I can solve the same problem using async hooks, so I go that way.

Some other disadvantage:

• If you fork the Node.js environment, then you'll have to reapply your patches every time you want to upgrade to a new Node.js release. So you'll probably always run on an older Node.js version.

• Think an npm package declares it runs on node engine > 11.0. On what version of libui-node will that works? You'll need to maintain a mapping somehow...

It will probably simplify the integration between the event loops, but I don't think too much, it's just a difficult problem in itself... Electron event-loop use a hack they insert into libuv to do a proper integration with the node.js one, but the async hooks replace this hack (I think with elegance).
I don't think async hooks was available at the time they start to write atom (but I'm just guessing)

Having said this, the event lop integration is actually still not perfect (you can see the last incarnation, still WIP in https://github.com/parro-it/libui-napi repo) BTW, libui-napi is a new version ,written almost from scratch, me and @mischnic are writing to use the new N-API framework for native modules. It will allow to ship prebuilt binaries without user having to recompile the library every time, and it will probably constitute the 0.4.0 release of libui-node

[parro-it]

🙏🏻 Sorry for my english, I know it's sub-perfect, and I need to improve on this 🤪
I will be really interested in giving a look at your proof of concept. Is it publicly available somewhere?

[parro-it]

I said https://github.com/albe-rosado/create-proton-app/ but I really meant
https://github.com/electron-userland/electron-builder.

[mimecorg]

I don't agree with you, and I actively try, from the start of the project, to make libui-node works with standard Node.js distribution out of the box.

Absolutely, libui-node should work with standardard Node.js. My goal here isn't to make libui-node incompatible with Node.js, but rather to create a very simple wrapper for Node.js which would make distributing and running GUI apps written using Node.js easier.

If the platform has no console, a huge part of existing modules from npm cannot works, because they rely on console existence. How you can replace the console for them to work? Even internal part of node.js relay on the console.

The problem is that when you run the application on Windows, an ugly black console window appears in addition to the normal window. My goal is simply to hide that console window. It doesn't interfere with any modules, they can still use console.log() and similar functions, but the result will be invisible.

In order to do that, it's enough to set the /SUBSYSTEM:WINDOW option in Visual Studio and to implement WinMain() instead on main(). That's all my proof of concept basically does. The entire code is essentially this:

int APIENTRY wWinMain( HINSTANCE hInstance, HINSTANCE hPrevInstance, LPWSTR lpCmdLine, int nCmdShow )
{
	WCHAR exePath[ MAX_PATH + 1 ];
	WCHAR dirPath[ MAX_PATH + 1 ];
	WCHAR scriptPath[ MAX_PATH + 1 ];

	GetModuleFileName( NULL, exePath, MAX_PATH + 1 );
	wcscpy( dirPath, exePath );
	PathRemoveFileSpec( dirPath );
	PathCombine( scriptPath, dirPath, L"app\\main.js" );

	char exePathUtf8[ MAX_PATH + 1 ];
	char scriptPathUtf8[ MAX_PATH + 1 ];

	wcstombs( exePathUtf8, exePath, MAX_PATH + 1 );
	wcstombs( scriptPathUtf8, scriptPath, MAX_PATH + 1 );

	char* node_argv[ 3 ] = { exePathUtf8, scriptPathUtf8, NULL };

	return node::Start( 2, node_argv );
}

I don't think that electron-builder will solve that. All it does is basically download the original Node.js executable and wrap it in a script which runs it with the appropriate arguments. The problem is that if you give a package containing node.exe and start.bat to the end user, they will probably click node.exe and that won't work. On the other hand, if you have start.exe and node.dll, there is no doubt how to actually run the application.

If you fork the Node.js environment, then you'll have to reapply your patches every time you want to upgrade to a new Node.js release.

Forking Node.js isn't really necessary. The unmodified Node.js source code can already be compiled as a shared library. The wrapper application can use all functions from Node.js, so there is no need to patch anything.

[parro-it]

Absolutely, libui-node should work with standardard Node.js. My goal here isn't to make libui-node incompatible with Node.js, but rather to create a very simple wrapper for Node.js which would make distributing and running GUI apps written using Node.js easier.

Ok, so I completely misunderstand you, sorry.

My goal is simply to hide that console window. It doesn't interfere with any modules, they can still use console.log() and similar functions, but the result will be invisible.

That's should be fine!

So, what you're thinking is basically a tool that ship a shared library recompiled version of node.js, the standard version of libui.dll, a (maybe) recompiled version of libui-node, plus a window subsystem executable that start the user app? I love the idea...

• Are you sure there aren't similar solutions already done in the wild?
• If you'll continue to work on your POC, can you try it with libui-napi? Being it based on N-API, maybe we can avoid recompilation of libui-node, and it anyway will be the future version of libui-node

[mimecorg]

Yes, exactly. I haven't seen any similar solutions, all tools that package Node.js apps that I've seen assume that they are console apps. I will definitely take a look at libui-napi, the recompilation is the biggest drawback of this idea.

[mimecorg]

I played with libui-napi, it looks very good, I think it's a step in the right direction. I managed to run the Vuido example with the latest version of libui-napi. I just had to fix two small problems:

• the devDependency on the debug package is missing (it's needed by libui-download.js)
• napi_fatal_exception() requires Node v9.10 and I'm using v8, so I replaced it with napi_throw

Unfortunately recompilation cannot be avoided, because the ui.node DLL still links to node.exe, not to node.dll.

But I think that I solved that problem. During development, the application will be run using regular node, so there is no need to recompile anything. I'm currently working on a tool which will be used for packaging the application. I called it "launchui". It will automatically download the compiled binaries for selected platforms, including launchui.exe, node.dll, libui.dll and the recompiled version of libui-node. It will also copy the application scripts to the package. So it will be a bit similar to electron-packager. This can be later integrated with electron-builder to automatically create installers, etc. I'm planning to publish the first version of launchui soon.

[mischnic]

napi_fatal_exception() requires Node v9.10 and I'm using v8, so I replaced it with napi_throw

Just for reference, this already came up here: parro-it/libui-napi#21 (comment)