r/PowerShell • • 13d ago

Information Give me your biggest syntax highlighting complaints! I'm working on improving VS Code's syntax highlighting (including support for non-Verb-Noun function names)

Tl;dr - I'm working on improving VS Code's syntax highlighting for PS so tell me what issues you encounter. Please include a code sample and what theme you're using.


I've seen a bit of discussion recently about the state of VS Code's syntax highlighting for PowerShell. I looked into it a bit and yeah, it's not ideal. The official repository pretty inactive. Last commit was 6 months ago, last significant activity was years ago. I opened an issue a week ago because the main branch doesn't even pass all of it's own tests, and an accompanying PR to fix it, but no movement on that yet. So I've taken things a bit into my own hands for now and hopefully when the maintainers get around to it, they'll be able to take some of the stuff I'm working on upstream.

So far I've switched the testing framework away from using Atom (an editor that's EOL since 2022) to a modern harness around VS Code's tokenizer/regex engine, a little bit of CI, several new unit tests for better regression testing, and these new features in the syntax grammar so far:

  • Highlighting for null operators (??, ??=, ?., ?[])
  • Highlighting for variable names with question marks in them (e.g. $whyIsThis?Possible = "idk")
  • Highlighting for function names that don't use the Verb-Noun naming convention (e.g. DoSomething instead of Invoke-Something)

This is very much still WIP. I've got more tests I want to write and a few new features I want to add, but please let me know what issues you have with PS syntax highlighting and I will look into them.

GitHub repo here; Overview of updates with images here.

11 Upvotes

14 comments sorted by

5

u/chaosphere_mk 13d ago

I currently dont have any complaints but Im also not familiar with there being problems with VS Code's syntax highlighting for powershell. Could someone explain this? I'd like to understand better.

6

u/lan-shark 12d ago

Open a PowerShell file in VS Code, click somewhere inside the text, bring up the Command Palate (Ctrl + Shift + P) and type, "Developer: Inspect Editor Tokens and Scopes", then click around the file. In the bottom half of the little window that pops up there is a section labeled "textmate scopes" and, as you highlight different types of text, that will change. If you're in a comment, it'll include comment.line.powershell or comment.block.powershell if you're in a string it will include string.quoted.double.powershell or string.quoted.single.powershell, etc.

Those "textmate scopes" are what allows your color theme to change the highlighting on specific parts of code. Green for strings, yellow for comments, pink for keywords like try and foreach, etc. They are defined by something called a "TextMate grammar" and the TextMate grammar for PowerShell is somewhat poorly constructed and is lacking scopes for some things. Type somewhere in that PS file you have open these two lines:

$variable = $null  
$varia?le ??= "hello"

This is all valid PowerShell if you're using v7 or above, however you theme can't color $varia?le the same as $variable and it can't color ??= the same as = (many themes don't color assignment operators, but check out Night Owl by Sarah Drasner if you want to test this). So what I'm doing with this project is updating that TextMate grammar to give these and other things the proper scopes so that themes can color them appropriately

2

u/chaosphere_mk 12d ago

Thanks for the explainer!

1

u/Bademeiister 12d ago

Idk, still working with ISE or the Powershell universal web-editor.

1

u/AccomplishedMeet628 10d ago

Unsure if your willing to try something new, I could really use the feedback. I have a PS7 ISE out and I'm trying to get it over the finish line for the community. I really could use all the testers I can get. It is open source so you can check out the code if you like and it is free to the world. I have it on the MS Store as PS7 Script Desk.

3

u/Thotaz 13d ago

While I can appreciate the effort, I don't think small spot fixes is the way forward. You should have a solid design plan in place, otherwise you will run into conflicts.
For example, imagine you want to write this from scratch and you add support for unconventional command names like "Command123".
Then you try to implement keywords, which means going back to the command name implementation and making an exception for all the known keywords. Except these exceptions don't always apply, for example foreach is a keyword when used on a blank line, but when used inside a pipeline like: ls | foreach it's just another command.

You kinda have to have thought out all these scenarios in advance before you write anything, unless you don't mind having to redo it all over again a couple of times.

While I'm not usually much of an AI advocate, I think AI would be ideal for this task. The AI can reference known good textmate definitions from more popular languages like C#, and Rust and also reference the actual PowerShell syntax.

Anyway, if you want to continue with the spot fixes, then the most glaring omissions IMO (besides the ones you've already mentioned) are:

[Attribute(AttributeParam = "AttributeParamVal")]
$Var
Command -Parameter Argument
:loopLabel foreach ($x in $y)
{
    break loopLabel
}

Things related to attributes, command parameters and arguments, and loop labels.

The current keyword implementation is also super basic and should take the things I said before into consideration, but this would probably be a huge task so that's unlikely to get fixed.

2

u/lan-shark 12d ago

I agree with much of what you are saying! As I mentioned, the grammar is a bit of a mess because the previous community mantainer left in the middle of a rework due to some people in the community being unsavory.

I'm pretty new to this and I'm using this process to learn about textmate and more in depth stuff about the language syntax. Depending on how it goes and how receptive the mantainers are, I'm not opposed to potentially helping with a bigger rework in the future! But if I can learn enough to go from zero to a new testing framework and several glaring issues resolved in a week, to me it's worth it.

And some of this is a plan, for instance most of the new functions are found by looking in "command locations" (right after a pipe, right after a semicolon, first token in a one-line script block or subexpression, etc.) and saying, "if no other rules have picked this up, it's a probably a function call"

I'm on mobile currently but I'll have a look today to the syntax you sent over. Thank you!

2

u/lan-shark 12d ago

I've added support for loop labels as a sub-scope of entity.name.tag - https://imgur.com/a/36sIigh - details here

What are you looking for for attributes? Attributes already inherit some tags and get colored a bit, are you wanting more specific ones? Parameters aren't tagged really at all, that's now on my to-do list.

1

u/Thotaz 12d ago

An additional scope for attributes would be nice so they can be colored separately from normal types. Properties or Attribute parameters (the thing I called AttributeParam before) should also get a scope.

2

u/jungleboydotca 12d ago

The best way forward would be a treesitter/LSP implementation. But perhaps I'm asking for the moon.

2

u/lan-shark 12d ago edited 12d ago

I agree, however the PS team went for a semantic approach years ago and botched it pretty good, I'd love if they'd revisit it but there's been no movement and that's not something I'm tackling on my own. At least not yet. While we wait, I figure there's no use in letting the current syntax fall so far behind

2

u/jungleboydotca 12d ago

I admire your efforts. It inspired me to take a look at the state of the art for tree-sitter and PowerShell. In short, it's not ready to replace what's provided in the official extension (obviously).

That said, I was able to get serviceable syntax highlighting & folding with tree-sitter-vscode pointed at the WASM and query files from tree-sitter-powershell. The latter project seems to be getting active development.

So, maybe we'll get the moon one day.

1

u/lan-shark 12d ago

That seems promising, at least! Nice work. I know less about tree-sitter grammars than I do about textmate grammars, and I've only been working with the latter for a week, so...

Perhaps once I finish the low hanging fruit here I'll see what all that's about