Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The splatting operator '@' cannot be used to reference variables in an expression. '@trigger' can be used only as an argument to a command. To reference variables in an expression use '$trigger' #21291

Closed
5 tasks done
deadcoder0904 opened this issue Mar 2, 2024 · 16 comments
Labels
Resolution-By Design The reported behavior is by design. WG-Engine core PowerShell engine, interpreter, and runtime WG-Triaged The issue has been triaged by the designated WG.

Comments

@deadcoder0904
Copy link

Prerequisites

Steps to reproduce

  1. Open Powershell 7 in Windows Terminal
  2. Type pnpm dlx @trigger.dev/cli@latest init -k tr_dev_1213241234zxcz -t https://cloud.trigger.dev in Terminal & hit Enter

Expected behavior

It should create a folder that is created by https://trigger.dev when successful like others like Git Bash do.

Just sign up for Trigger Dev & the first command if you select through Next.js in Onboarding would be above. Also, select `pnpm`.

Actual behavior

ParserError:
Line |
   1 |  pnpm dlx @trigger.dev/cli@latest init -k tr_dev_1213241234zxcz  …
     |           ~~~~~~~~
     | The splatting operator '@' cannot be used to reference variables in an expression. '@trigger' can be used only
     | as an argument to a command. To reference variables in an expression use '$trigger'.

Error details

$ Get-Error

Type        : System.Management.Automation.ParseException
Errors      :
    Extent  : @trigger
    ErrorId : SplattingNotPermitted
    Message : The splatting operator '@' cannot be used to reference variables in an expression. '@trigger' can be
used only as an argument to a command. To reference variables in an expression use '$trigger'.
Message     : At line:1 char:10
              + pnpm dlx @trigger.dev/cli@latest init -k tr_dev_1213241234zxcz  …
              +          ~~~~~~~~
              The splatting operator '@' cannot be used to reference variables in an expression. '@trigger' can be
used only as an argument to a command. To reference variables in an expression use '$trigger'.
ErrorRecord :
    Exception             :
        Type    : System.Management.Automation.ParentContainsErrorRecordException
        Message : At line:1 char:10
                  + pnpm dlx @trigger.dev/cli@latest init -k tr_dev_1213241234zxcz  …
                  +          ~~~~~~~~
                  The splatting operator '@' cannot be used to reference variables in an expression. '@trigger' can be
used only as an argument to a command. To reference variables in an expression use '$trigger'.
        HResult : -2146233087
    CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
    FullyQualifiedErrorId : SplattingNotPermitted
    InvocationInfo        :
        ScriptLineNumber : 1
        OffsetInLine     : 10
        HistoryId        : 12
        Line             : pnpm dlx @trigger.dev/cli@latest init -k tr_dev_1213241234zxcz -t
https://cloud.trigger.dev
        Statement        : @trigger
        PositionMessage  : At line:1 char:10
                           + pnpm dlx @trigger.dev/cli@latest init -k tr_dev_1213241234zxcz  …
                           +          ~~~~~~~~
        CommandOrigin    : Internal
TargetSite  :
    Name          : Invoke
    DeclaringType : System.Management.Automation.Runspaces.PipelineBase, System.Management.Automation,
Version=7.4.1.500, Culture=neutral, PublicKeyToken=31bf3856ad364e35
    MemberType    : Method
    Module        : System.Management.Automation.dll
Source      : System.Management.Automation
HResult     : -2146233087
StackTrace  :
   at System.Management.Automation.Runspaces.PipelineBase.Invoke(IEnumerable input)
   at Microsoft.PowerShell.Executor.ExecuteCommandHelper(Pipeline tempPipeline, Exception& exceptionThrown,
ExecutionOptions options)

Environment data

Name                           Value
----                           -----
PSVersion                      7.4.1
PSEdition                      Core
GitCommitId                    7.4.1
OS                             Microsoft Windows 10.0.22631
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Visuals

No response

@deadcoder0904 deadcoder0904 added the Needs-Triage The issue is new and needs to be triaged by a work group. label Mar 2, 2024
@237dmitry
Copy link

Try to escape command line arguments of pnpm:

pnpm --% dlx @trigger.dev/cli@latest init -k tr_dev_1213241234zxcz -t https://cloud.trigger.dev

@mklement0
Copy link
Contributor

mklement0 commented Mar 2, 2024

To add to @237dmitry's comment:

  • Indeed, use of --%, the stop-parsing token is an option, but note that it comes with many limitations, notably the inability to reference PowerShell variables.

  • The problem is that @ - unlike in other shells - is a metacharacter at the start of arguments in PowerShell, used for splatting - that's what the error message is referencing.

    • Additional characters that you may not expect to be metacharacters are { } , ; ` (see full list below).
  • Therefore a @ (or any other metacharacter) that is to be used verbatim at the start of an argument must either be inside a quoted string or must be individually escaped with `, the so-called backtick, PowerShell's escape character.

Applying the latter technique to your command (note how only the initial @ need be escaped, though escaping both would work too):

# Note the ` before the first @
pnpm dlx `@trigger.dev/cli@latest init -k tr_dev_1213241234zxcz -t https://cloud.trigger.dev

# Alternative with a quoted string
# Note the ` before the first @
pnpm dlx '@trigger.dev/cli@latest' init -k tr_dev_1213241234zxcz -t https://cloud.trigger.dev

The list of PowerShell metacharacters in argument-parsing mode is:

<space> ' " ` , ; ( ) { } | & < > @ #

Note:

  • Of these, < > @ # are only special at the start of an argument.
  • Situationally, . (dot) requires escaping too, namely if the argument can syntactically be interpreted as accessing a variable's property (e.g., Write-Output $env:computername.csv outputs nothing - see this answer).
  • If you want a $ to be treated verbatim rather than refer to a variable or subexpression, you must escape it too.
  • For a complete overview of how unquoted command arguments are parsed in PowerShell, see this answer.

@deadcoder0904
Copy link
Author

I think this should work by default as countless npm modules will start from @ as more packages are used.

I'd rather want @ to work than remember to escape it.

Otherwise, I have to open another terminal with Git Bash which is not convenient.

@MartinGC94
Copy link
Contributor

The parsing seems to already handle other special characters like: ls @Test\ or ls @Test* so it doesn't seem unreasonable to think it should handle . as well, considering that it isn't possible to access a member from a splatted variable anyway.

@mklement0
Copy link
Contributor

mklement0 commented Mar 4, 2024

Good point, @MartinGC94.

There's several awkward behaviors around argument mode - both with native PowerShell commands and native (external) programs, and it may be worth revisiting all of them in the context of a focused feature request, so they can be addressed with a single change, which will at least technically amount to breaking changes.

Using the case at hand as an example:

Here, the issue is that the .dev part of @trigger.dev is considered a property access, which triggers the error message:

  • Without splatting, such a property access does work, and is useful in isolation:

    • E.g., Write-Output $HOME.Length works and reports the value of $HOME.Length, i.e. the count of characters in the value of the $HOME variable.
  • In a compound argument, it technically still "works", but exhibits behavior that is unlikely to be helpful in practice:

    • E.g., Write-Output $HOME.Length/foo is the same as Write-Output $HOME.Length /foo, i.e. whatever follows the property reference becomes a separate argument.

Given that @trigger.dev never makes sense as a property access, it is reasonable to fall back to treating it as a verbatim string, especially in native program calls.

Arguably, also treating $HOME.Length/foo as "$($HOME).Length/foo" makes more sense than the current behavior.

However, even in native program calls treating an isolated argument such as @trigger verbatim is not an option, because splatting does work with external programs (often seen with @args), albeit only meaningfully with arrays rather than hashtables.

Currently, something like cmd /c echo @foo effectively passes no arguments to the external program, if no $foo variable exists.

Perhaps @foo is a rare use case and having subsequent characters, such as a filename extension (e.g, @foo.txt) is the typical use case, in which case falling back to verbatim use is safe - and perhaps, pragmatically speaking, that is therefore sufficient.

But the point is that all these behaviors and edge cases need to spelled out and agreed upon in detail before potentially making changes.


See also:

@SteveL-MSFT
Copy link
Member

There seems to be two possible options here and both might be bucket 3 in terms of a breaking change.

  • since we have a specific error here, we could simply treat the "splatting" as literal instead of returning an error
  • if the variable being splatting doesn't exist, then it becomes literal to avoid accidental splatting

@SteveL-MSFT SteveL-MSFT added WG-Engine core PowerShell engine, interpreter, and runtime WG-NeedsReview Needs a review by the labeled Working Group labels Mar 4, 2024
@MartinGC94
Copy link
Contributor

MartinGC94 commented Mar 7, 2024

A reddit thread indicates that there's also been some parsing regression between 5.1 and 7 when = is used in the string. Where npm config set @scope:registry=https://www.myregistry.com/ works in 5.1 but not 7 see: https://www.reddit.com/r/PowerShell/comments/1b85las/npm_works_different_in_51_vs_7/ for more details.

@mklement0
Copy link
Contributor

mklement0 commented Mar 7, 2024

Yes, it looks like WinPS - sensibly - falls back to treating the whole argument like an expandable string ("@scope:registry=https://www.myregistry.com/"), whereas PS Core expands @scope:registry to the empty string, and therefore passes only =https://www.myregistry.com:

@scope:registry is technically the splatted form of $scope:registry, which is technically a valid variable reference, using namespace variable notation, and "splatting" @scope:registry alone results in no argument getting passed, in both editions, because $scope:registry happens not to exist.

In both editions, however, referencing existing variables via namespace variable notation does work, so that something like @env:PATH does splat, and - because a [string] is enumerably and therefore triggers array-based splatting - passes the individual characters in $env:PATH as separate arguments (this behavior is fundamentally questionable).

When you follow a "splatted" namespace variable reference with a non-variable-name character other than = (e.g, #, % or !), the whole argument is (sensibly) interpreted like an expandable string in both editions.

However - in PS Core only - a subsequent = still triggers the splatting, and passes the = and whatever follows it as a separate argument. In the case at hand, since $scope:registry doesn't exist, the @scope:registry part effectively expands to no argument, and =https://www.myregistry.com/ is passed as the only argument resulting from the original token.

@JamesWTruher
Copy link
Member

The WG discussed this and believe that this is operating as it was designed. You will need to either quote or escape in order to avoid the variable parsing behavior.

@JamesWTruher JamesWTruher added Resolution-By Design The reported behavior is by design. and removed WG-NeedsReview Needs a review by the labeled Working Group Needs-Triage The issue is new and needs to be triaged by a work group. labels Mar 18, 2024
@daxian-dbw daxian-dbw added the WG-Triaged The issue has been triaged by the designated WG. label Mar 18, 2024
Copy link
Contributor

This issue has been marked as by-design and has not had any activity for 1 day. It has been closed for housekeeping purposes.

Copy link
Contributor

microsoft-github-policy-service bot commented Mar 20, 2024

📣 Hey @deadcoder0904, how did we do? We would love to hear your feedback with the link below! 🗣️

🔗 https://aka.ms/PSRepoFeedback

@mklement0
Copy link
Contributor

@deadcoder0904, perhaps the following translation of the WG decision helps:

The WG discussed this and believe that this is operating as it was designed

This means (do tell me if I got things wrong):

  • "Even though we've been told that there are at least two versions of 'as designed' that changed over time, we chose not to address that in our decision."

  • "Even though we've been pointed to a better way forward, we chose to ignore that in our decision."

This is very much in the spirit of #18502 (comment)

@MartinGC94
Copy link
Contributor

Yeah, I'm a bit confused about the WG decision as well. One of the recent goals in PowerShell was to make native program arguments "just work" to make it a nicer shell to use, hence all the PSNative* experimental features we've seen over the last couple of releases.
I'm not a web guy, but npm seems quite popular in that world and if those users are having problems using pwsh then I think it's worth looking into how we can improve that situation (but of course without making it worse for others).
If we think Command @Identifier1.Identifier2 can be a meaningful syntax in the future then of course we should keep the existing behavior so we can reserve that syntax for later but I don't read that WG conclusion as if that was a consideration at all.

@kilasuit
Copy link
Collaborator

I agree that this is working as currently designed like the WG have said but do wonder if it could it be possible in future for in scenarios like this to have the engine ignore specific symbols like @ when used in a line like the one referenced here for specific executables, instead of how we have it now & have this be an easily configurable list of executables via a env var or other method

@mklement0
Copy link
Contributor

Let's start with the basics, @kilasuit:

working as currently designed

So which of the following behaviors is as designed, and what motivated the change in behavior between WinPS and PS Core, and where is that change documented?

# Run on Windows
cmd /c echo @foo=bar

WinPS output:

@foo=bar

PS Core output:

 =bar

Also note that the following works in both editions, and is behavior that must therefore be preserved:

& { cmd /c echo @args } foo bar # -> 'foo bar' in both editions

@mklement0
Copy link
Contributor

configurable list of executables

Frankly, this will lead to ever-lasting confusion as to which executable exhibits which behavior in which specific version.

It is highly regrettable that a similar dance has already been started with respect to the ill-fated Windows mode of $PSNativeCommandArgumentPassing - see #18554 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Resolution-By Design The reported behavior is by design. WG-Engine core PowerShell engine, interpreter, and runtime WG-Triaged The issue has been triaged by the designated WG.
Projects
None yet
Development

No branches or pull requests

8 participants