Allow rust-project.json to include arbitrary shell commands for runnables #16840

Wilfred · 2024-03-15T00:00:02Z

This is a follow-up on #16135, resolving the feedback raised :)

Allow rust-project.json to include shell runnables, of the form:

{
  "build_info": {
    "label": "//project/foo:my-crate",
    "target_kind": "bin",
    "shell_runnables": [
      {
        "kind": "run",
        "program": "buck2",
        "args": ["run", "//project/foo:my-crate"]
      },
      {
        "kind": "test_one",
        "program": "test_runner",
        "args": ["--name=$$TEST_NAME$$"]
      }
    ]
  }
}

If these runnable configs are present for the current crate in rust-project.json, offer them as runnables in VS Code.

This PR required some boring changes to APIs that previously only handled cargo situations. I've split out these changes as commits labelled 'refactor', so it's easy to see the interesting changes.

bors · 2024-03-15T13:30:08Z

☔ The latest upstream changes (presumably #16845) made this pull request unmergeable. Please resolve the merge conflicts.

bors · 2024-03-18T09:18:43Z

☔ The latest upstream changes (presumably #16868) made this pull request unmergeable. Please resolve the merge conflicts.

bors · 2024-03-21T11:50:30Z

☔ The latest upstream changes (presumably #16895) made this pull request unmergeable. Please resolve the merge conflicts.

davidbarsky · 2024-03-19T01:47:27Z

crates/project-model/src/project_json.rs

@@ -87,6 +87,21 @@ pub struct Crate {
    pub(crate) exclude: Vec<AbsPathBuf>,
    pub(crate) is_proc_macro: bool,
    pub(crate) repository: Option<String>,
+    pub build_info: Option<BuildInfo>,


i meant to change this in the original PR, but build_info's runnables should be under ProjectJson, not the Crate. label/target_kind should stay under Crate, however.

(that assumes a Bazel/Buck-style labeling system, but even I think for non-Cargo build systems, that's a good enough bar to set.)

@davidbarsky If I'm reading this right, you want the runnables to be stored at the root of the project.json. That will require:

a substitution mechanism like $label so that the "runnable templates" can be turned into real commands for a specific crate

probably RA using the build_info.kind field to figure out which runnables to offer for a given target

I also think we also want custom runnables in build_info for specific crates anyway. Right? Because the build system might simply know better. Substitution is nice because it saves a few MB of JSON. But ultimately you want the ability to add more to specific crates.

Because I believe you're going to want runnables per-crate anyway, and they are completely capable of handling anything substitution can, I think we can merge this PR pretty much as-is. I think the substitution mechanism is more of an optimisation than anything else.

I am happy to put together a PR that allows flycheck to use these.

@cormacrelf

a substitution mechanism like $label so that the "runnable templates" can be turned into real commands for a specific crate

Yup: the substitution mechanism is thankfully not too complex: I think it boils down to just a str::replace.

(If you squint, I'd argue that the substitution already exists in rust-analyzer today via Cargo, but the substitution knowledge is pretty implicit...)

probably RA using the build_info.kind field to figure out which runnables to offer for a given target

Will need to repage some context in, but I'm pretty sure the answer is also "yup".

I also think we also want custom runnables in build_info for specific crates anyway. Right? Because the build system might simply know better. Substitution is nice because it saves a few MB of JSON. But ultimately you want the ability to add more to specific crates.

I don't know if I agree: while I'm assuming that while there is a smart build system, I don't think there isn't much surface area for more specialized commands for individual crates. There's also the fact that rust-analyzer is pretty darn fast even compared to buck2, which is the fastest build system I'm aware of—single digit milliseconds versus ~300 milliseconds. There's also the matter of the duplication being somewhat concerning for the Fuschia folks, as their rust-project.json is already pretty large and I'm not really inlined to make things more complicated for them, however slightly.

Because I believe you're going to want runnables per-crate anyway, and they are completely capable of handling anything substitution can, I think we can merge this PR pretty much as-is. I think the substitution mechanism is more of an optimisation than anything else.

You are right it's an optimization, but it's a pretty cheap one. Wilfred and I need to get this over the finish line, but it's genuinely not too much work to implement it.

OK. A good metric for the design is whether rust-project.json can encode all the cargo-related logic. If you set build_info.label to a cargo crate name, and a root level fly check template of cargo check -p $label --message-format=json, then we almost have that mission in the bag. The main missing piece is flycheck for a crate vs for the whole workspace, which is configurable through check.workspace in RA. Having two more runnable kinds, flycheck and flycheckWorkspace, should do it.

OK. A good metric for the design is whether rust-project.json can encode all the cargo-related logic. If you set build_info.label to a cargo crate name, and a root level fly check template of cargo check -p $label --message-format=json, then we almost have that mission in the bag.

Yeah, that's generally the stance I've pursuing.

The main missing piece is flycheck for a crate vs for the whole workspace, which is configurable through check.workspace in RA. Having two more runnable kinds, flycheck and flycheckWorkspace, should do it.

hmm, probably. i don't know if i'd want to set a flycheckWorkspace for rust-project, but I guess it can be a buck2 build 'rdeps(:target, 1)'. at least, i don't know if i'd put it on by default.

(A blessing and a curse of Buck is that there's no clearly delineated "workspace" concept like there is in Cargo.)

I wouldn't set it either.

nemethf · 2024-03-27T16:37:41Z

docs/dev/lsp-extensions.md

+
+```typescript
+{
+    kind: string;


How should the LSP client use the kind field? Does it describe what the runnable does? Then how does it differ then from label field of the parent Runnable?

Also should the "shell" kind have a expectTest field just like the "cargo" kind?

BTW, how should the LSP client interpret the workspaceRoot, expectTest, and overrideCargo fields?

Additionally, I think it would make sense to copy the example from the Pull Request into the lsp-extension.md file, or just reference a user-level documentation describing this feature.

Thanks

Also, shouldn't this PR explain the new build_info field of rust-project.json in manual.adoc?

When expectTest is set the client should set the UPDATE_EXPECT env var to 1 when invoking the runnable. ideally there should be a field for arbitrary env var overrides in the payload, then this field could be replaced by the server instructing the client to just set the env var with that general mechanism as we'll need this for other things as well.

overrideCargo I think should just replace the cargo command that is being used by the client.

bors · 2024-03-29T02:21:10Z

☔ The latest upstream changes (presumably #16971) made this pull request unmergeable. Please resolve the merge conflicts.

Veykril · 2024-04-18T12:13:13Z

This is a follow-up on #16135, resolving the feedback raised :)

So this means #16135 can be closed in favor of this one?

davidbarsky · 2024-04-18T12:41:27Z

This is a follow-up on #16135, resolving the feedback raised :)

So this means #16135 can be closed in favor of this one?

Yup! Just closed. Wilfred or I will push to this branch as needed.

Veykril

Generally looks good to me (aside from what has been raised already by others)

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 15, 2024

Wilfred mentioned this pull request Mar 15, 2024

Refactor extension to support arbitrary shell command runnables #16839

Merged

Wilfred force-pushed the shell_runnable branch 5 times, most recently from 65e9be3 to c7bdd91 Compare March 15, 2024 23:14

Wilfred marked this pull request as ready for review March 15, 2024 23:14

Wilfred force-pushed the shell_runnable branch from c7bdd91 to f88aff3 Compare March 19, 2024 19:14

davidbarsky suggested changes Mar 21, 2024

View reviewed changes

nemethf mentioned this pull request Mar 27, 2024

Announcements: Client Changes #4604

Open

nemethf reviewed Mar 27, 2024

View reviewed changes

Wilfred force-pushed the shell_runnable branch 2 times, most recently from 125b58b to b9662e6 Compare March 28, 2024 00:04

Veykril approved these changes Apr 18, 2024

View reviewed changes

Veykril added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 18, 2024

davidbarsky mentioned this pull request Apr 23, 2024

WIP: Add support for detecting tests in source files, and implement it for Rust zed-industries/zed#10873

Closed

4 tasks

Veykril mentioned this pull request May 23, 2024

Fix inconsistent cwd of run and debug command in client #17275

Merged

davidbarsky force-pushed the shell_runnable branch 2 times, most recently from e41b15b to 7faa78d Compare May 30, 2024 20:31

feature: add build system info; runnables to rust-project.json

e454839

davidbarsky force-pushed the shell_runnable branch from 7faa78d to e454839 Compare May 30, 2024 23:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow rust-project.json to include arbitrary shell commands for runnables #16840

Allow rust-project.json to include arbitrary shell commands for runnables #16840

Wilfred commented Mar 15, 2024 •

edited

bors commented Mar 15, 2024

bors commented Mar 18, 2024

bors commented Mar 21, 2024

davidbarsky Mar 19, 2024

cormacrelf May 24, 2024

davidbarsky May 24, 2024

cormacrelf May 24, 2024

davidbarsky May 24, 2024

cormacrelf May 24, 2024

nemethf Mar 27, 2024

nemethf Mar 30, 2024

Veykril Apr 18, 2024

bors commented Mar 29, 2024

Veykril commented Apr 18, 2024

davidbarsky commented Apr 18, 2024

Veykril left a comment

Allow rust-project.json to include arbitrary shell commands for runnables #16840

Are you sure you want to change the base?

Allow rust-project.json to include arbitrary shell commands for runnables #16840

Conversation

Wilfred commented Mar 15, 2024 • edited

bors commented Mar 15, 2024

bors commented Mar 18, 2024

bors commented Mar 21, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bors commented Mar 29, 2024

Veykril commented Apr 18, 2024

davidbarsky commented Apr 18, 2024

Veykril left a comment

Choose a reason for hiding this comment

Wilfred commented Mar 15, 2024 •

edited