fix: do not assume that /usr/bin/env exists on macOS

[tie]

This changes fixes build error when /usr/bin/env is not available (for example, when building in Nix sandbox). It makes gyp-generated build configurations invoke Python directly on the gyp tools (similar to gyp-win-tool that uses sys.executable on Windows).

See also NixOS/nixpkgs#261820 and NixOS/nixpkgs#248708

[tie]

Force-pushed to fix issues reported by ruff.

[cclauss]

Not a fan of this. Why not just create or symlink to /usr/bin/env ?

[tie]

Why not just create or symlink to /usr/bin/env?

In general, only build directory is writable in sandbox, and a limited set of host directories is readable.

While it should be possible to pass /usr/bin/env as an impure derivation dependency, as far as I know, this is usually used only for bootstrapping stdenv, and not in the packages themselves. Note that for Node.js, /usr/bin/env would also need to be a propagated impure dependency since some packages (e.g. iosevka) depend on vendored node-gyp (in Node.js, gyp exists in deps/npm/node_modules/node-gyp/gyp and tools/gyp, the latter is used to build Node.js itself).

Alternatively, Nixpkgs can patch shebang lines in pylib/*_tool.py to absolute Python executable path (rewrite #!/usr/bin/env python3 → #!/nix/store/g5cm6iik6p4k39cj9k7a6sg2p09hl7wf-python3-3.10.12/bin/python3), e.g. NixOS/nixpkgs#261820 (comment) and NixOS/nixpkgs#248708, but I don’t really like doing that since technically it is a change to the upstream source and not necessary with this PR.

[tie]

@cclauss, hi, are there any updates on this issue? I’d like to move forward with NixOS/nixpkgs#261820 and NixOS/nixpkgs#262124, and ideally I don’t want to apply this patch without getting it merged upstream (see also my previous comment).

[legendecas]

Why not set up /usr/bin/env in the sandbox macOS environment? The default macOS distribution comes with it.

[wegank]

Sandbox in the Nix term means that only packages built by Nix (which are in /nix/store) are available in the build environment. Access to /usr/bin/env is hence prohibited.

[legendecas]

If a sandbox is going to provide a macOS-like environment, it should provide /usr/bin/env as a default macOS does.

[wegank]

It depends on what you think a build environment should look like.

• In a Homebrew build environment, for example, there are a lot of Apple stuffs, including binaries in /usr/bin and pre-installed toolchains in /Library/Developer/CommandLineTools.
• In a Nix sandbox, you won't have access to /usr/bin or /Library/Developer/CommandLineTools, but to LLVM Clang, GNU Core Utilities, and other open-source components, all built from source and isolated in /nix/store.

Both models are valid, tested in practice (165,000 PRs in Homebrew Core, 270,000 PRs in Nixpkgs), and can be used to build a multitude of things.

[tie]

@cclauss @legendecas, thanks for taking a look at this, can you please explain what concerns do you have with this change? I understand that we can disable/relax sandbox to a certain extent, but that is really the last resort unless there is some technical reason why this change is not sound.

Gyp already uses sys.executable on Windows hosts, so it’s not like this change introduces some new behavior, but rather makes it consistent across platforms. Am I missing something here?

E.g. (%(python)s gyp-win-tool)

gyp-next/pylib/gyp/generator/ninja.py

Lines 2147 to 2157 in 4a8e328

	def _AddWinLinkRules(master_ninja, embed_manifest):
	"""Adds link rules for Windows platform to |master_ninja|."""
	def FullLinkCommand(ldcmd, out, binary_type):
	resource_name = {"exe": "1", "dll": "2"}[binary_type]
	return (
	"%(python)s gyp-win-tool link-with-manifests $arch %(embed)s "
	'%(out)s "%(ldcmd)s" %(resname)s $mt $rc "$intermediatemanifest" '
	"$manifests"
	% {
	"python": sys.executable,

[aduh95]

Wouldn’t this change be problematic if the python executable path contains some shell special char (e.g. ; or some white space)?

[tie]

Yes, but I don't think the existing code base handles that well too. Some of the formatting arguments should be using shlex.quote, but they are not. I think it would be rather confusing to correctly escape in this case but not in others.

[cclauss]

I hesitate because the assumption is correct for the vast majority of our Mac users and the others can use the --python option. NixOS should provide a pervasive solution instead of requiring all tools to make Nix-specific patches.

[aduh95]

IMO as long as NixOS community keeps sending patches and having support for their use case is not something that adds maintenance burden for us, there are no downsides for us. AFAICT this patch does not break the common use-case we actively support, it does not make the code less maintainable, so I think we should land it.