feat: v6 - Environment API

[patak-dev]

Description

Note

Check out the Environment API Docs

We're starting a new PR for the Environment API work in progress proposal for v6 to have a clean slate for reviews and discussions now that the implementation and docs are more stable.

This PR merges the work done in:

• feat: environment api #16129
• docs: environment API proposal #16089

Refer to the discussions in these PRs for context. If you have general feedback about the APIs, let's use this discussion so we can have proper threads:

• Environment API #16358

If you have comments about the implementation or would like to collaborate a feature, fix, test, or docs, please comment in this PR as an usual review or send a PR against the v6/environment-api branch.

Ecosystem Compatibility

These projects are failing in CI because of known reasons.

Framework	Reason	PR
VitePress	using internal server._importGlobMap, which is moved	vuejs/vitepress#3706

[stackblitz]

Run & review this pull request in StackBlitz Codeflow.

[patak-dev]

/ecosystem-ci run

[vite-ecosystem-ci]

📝 Ran ecosystem CI on e91e269: Open

suite	result	latest scheduled
astro	❌ failure	✅ success
histoire	❌ failure	✅ success
marko	❌ failure	✅ success
previewjs	❌ failure	❌ failure
qwik	❌ failure	✅ success
rakkas	❌ failure	✅ success
remix	❌ failure	✅ success
sveltekit	❌ failure	❌ failure
vike	❌ failure	❌ failure
vitepress	❌ failure	✅ success
vitest	❌ failure	❌ failure

✅ analogjs, ladle, laravel, nuxt, quasar, unocss, vite-plugin-pwa, vite-plugin-react, vite-plugin-react-pages, vite-plugin-react-swc, vite-plugin-svelte, vite-plugin-vue, vite-setup-catalogue

[dario-piotrowicz]

are these options correct?

this fetchModule is what the DevEnvironment class uses right?

these options seem to be specifically targeting node? (given the .cjs extension and the main field usage)

should this be make more generic, or actually could this be made configurable?
(so that DevEnvironment instances could tweak these options as needed?)

[dario-piotrowicz]

Or actually, shouldn't this inherit from the environment's options, some fields are indeed inherited:
https://github.com/vitejs/vite/pull/16471/files/2b3329d4aa0b91a5920527a9712b82ae59becba3#diff-fde6fc19368b69a4726cb38039db6dee6abf54bc69e1484c4f6e6c9d952acd9dR40-R42

[sheremet-va]

These are the same options that were used for ssrLoadModule before. It might be a good idea to provide a way to customize it, but you can also intercept the fetchModule call at any time

[dario-piotrowicz]

what do you mean by intercepting the call? 🙏

Right now what I am doing is creating a new DevEnvironment instance (source):

const devEnv = new ViteDevEnvironment(name, config, {
  hot,
}) as DevEnvironment;

and then using the instance's fetchModule in order to let vite apply its module resolution (source):

const result: any = await devEnv.fetchModule(...(args as [any, any]));

So that I can feed this result back to the module runner (source):

transport: {
  fetchModule: async (...args) => {
    const response = await env.__viteFetchModule.fetch(
      new Request('http://localhost', {
        method: 'POST',
        body: JSON.stringify(args),
      }),
    );
    const result = response.json();
    return result as any;
  },
},

is there a different way which would allow me to fetch the modules with my own resolve options?

[dario-piotrowicz]

@sheremet-va me and Igor have been looking more into this and we agree that the default node environment should preserve the ssrLoadModule behavior.

The fetchModule implementation in this class is however generally useful for custom environments, and in fact it's already exposed as part of the Environment API, so custom environments get to benefit from it.

For the custom environments to however truly benefit from this implementation, the environment config needs to be honored as I've pointed out above.

In order to preserve the behavior for the default node environment, we just need to ensure that it's default node environment is configured in a way that preserves the original ssrLoadModule implementation.

Would this work for you or are we missing anything else that complicates the matter? Thank you

[patak-dev]

You can see what createIdResolver does here:

vite/packages/vite/src/node/idResolver.ts

Line 33 in bbca0b4

async function resolve(

. It is properly using the environment configuration (things like resolve.conditions, resolve.alias still needs to be worked out but that will also work). It is the alias+resolve plugins, instead of the whole plugin pipeline for the environment so custom resolving added by users won't apply (optimized deps ids also don't play a role here). It is currently implemented using these two plugins to reuse the implementation, but I think it would be better at some point to have the resolve logic as free functions we can call instead of needing a plugins container for two plugins.
Internally, we are using this resolver to implement custom resolution. See for example resolving of CSS preprocessor files, that have their own rules:

vite/packages/vite/src/node/plugins/css.ts

Line 1067 in bbca0b4

return (sassResolve ??= createIdResolver(config, {

. We don't let users hook into these.

We also use it to make esbuild follow Vite alias and resolve rules during dependency optimization, so it matches what Rollup will do during build. See here

vite/packages/vite/src/node/optimizer/esbuildDepPlugin.ts

Line 68 in bbca0b4

// default resolver which prefers ESM

. I think checking this file should be useful for you. We are also dealing with externals, CJS and esm, etc.

[dario-piotrowicz]

Perfect! I'll need to thoroughly look into those! thanks so much 🙏

[dario-piotrowicz]

PS: I am not sure why but I can't resolve this thread, if you want please feel free to resolve this thread since the original issue has been resolved and this might just add unnecessary noise to this PR at this point (and I can ping you on discord for any further discussion on the resolver situation 🙂)

[sheremet-va]

Anyways he made a great point in the fact that we really need to have the module resolution applied here be the exact same as the one that will be used for the production build (so that this behaves as close as possible to what gets actually deployed), so:

But Vite leaves external modules as is (so, vue will be left as vue, not /node_modules/vue) in production, there is no Vite resolution during build for SSR 🤔 (if noExternal is used for build)

The workerd does the resolution. You said that it works in production somehow.

[dario-piotrowicz]

The workerd does the resolution. You said that it works in production somehow.

@sheremet-va not really 😅 I said that workerd doesn't do the resolution and that in production the modules are bundled in and not external. The issue is that they need to be external during development because they are cjs (which the Vite dev server can't handle).

So during build they are resolved by the Vite resolution and during development they unfortunately need to be external, but they should be resolved to, as close as possible, the same modules that would be used during the build time resolution

[IgorMinar]

Finally posting my review of the api-vite-environment.md file. Many of the comments were written a few weeks ago, but I forgot to post them, I hope they didn't bit rot too much.

I'm super excited about this work! I think the docs will need one more thorough pass once the code has settled down more.

[IgorMinar]

This is confusing a bit to me.... the plugin here only "provides a configuration" for the environment and doesn't "add" or "create" an environment.

It's unclear from the text/API what runtime will actually power this newly configured environment. Will it be createNodeEnvironment? Or something else? What do I need to do if I want to create an environment to power a web worker in by browser app, how do I create such environment in my plugin?

This coupling of transform/build configuration to the runtime and module runner is what I struggle with the most with the current design.

I think it's important to enable plugins to create new environments and for users to adjust the build/transform config for all (or specific) environments, but I think it is just as important for the plugin's environment not to be tied to a particular js runtime. I see signs of that desire in the current design, but I can't really see how it would work end to end.

[patak-dev]

I think this comment may be outdated. Let me know if there is still doubts about how plugins are created by plugins or configured (for dev and build).

[IgorMinar]

is this the only way you can expose the environment? Exposing APIs vai this is hard to document/discover and it's especially confusing when this state is mutable as it is the case of environments. How about including it in the HotContext?

[patak-dev]

The alternative to this.environment is having an environment param in every hook. We're using this because this is how vanilla Rollup already exposes the plugin context to hooks, and what we provide at the environment instance are Vite specific extensions to that context. See:

• feat: environment in hooks, context vs param #16261

[IgorMinar]

I've thought about the "client" name quite a bit and I wonder why not just call it a "browser" environment? Client feels simply too generic, as any environment connecting to the vite dev server is its "client".

[IgorMinar]

We discussed this in person with @patak-dev and he made an argument that browser might need multiple environments (e.g. for web worker) and there are also other non-browser clients, like electron.

My 2c is that we should name environments by the purpose they serve rather than technology that powers them or generic category like "client" that can be easily misunderstood. So following this logic, the default browser environment could be called browserMain or browserUI to represent the browser main thread or ui envionment.

[patak-dev]

Calling browserMain to tauri doesn't seems to me to convey the purpose. I think the purpose of the browser, tauri, electron, web extension, etc is to be the client of the app. There is another reason for the client and ssr naming in that it has been the way we have always called these two environments (there are options prefixed by ssr and by client in the config currently)

[IgorMinar]

I think what is confusing here is that we use fetchModule to get hold of external modules. I believe usually fetchModule is meant to be called from runInlineModule only, because previously Vite made the assumption that once an import enters an external module the rest of the module resolution and evaluation will happen outside of Vite.

This assumption is valid for node which has access the the fs, but not valid for other runtimes that might need to rely on Vite and fetchModule to get hold of the script content for a given resolved module id.

In the latter case, we do want to honor the resolution configuration of the current environment, so that's why in Dario's remix PoC we patched Vite to do that.

Now we could add externalMainFields, etc but that raises three questions which I can't answer right now:

1. Why do we need to have a different set of resolution config attributes for external modules? Doesn't mainFields option only apply to resolving external modules nowadays anyway?

2. What should be the value of these external* options? And who determines that? Are the values just a copy of the original mainFields and friends?

3. What would happen if we didn't introduce external* options and instead honored the env specific mainFields and friends? What would break?