-
Notifications
You must be signed in to change notification settings - Fork 557
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove shim #7367
Remove shim #7367
Conversation
cda5f44
to
9a055e1
Compare
Tests are passing and double checked everything I want to manually double check, so this should be good to start reviewing cc @jedevc @vito. There's a couple TODOs left about some simplifications made possible by commits later in the series and areas in need of more commenting, but not expecting any significant changes left before feedback. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ahhh this is neat 🎉 Really nice to have the shim
gone - even just from an architectural perspective, the practical performance implications are super great as well. Agreed on the potential containerd-worker stuff as well - since containerd already has it's own shim layer for calling runc, having another layer there was horrible.
Despite the large diff, it's pretty easy to read - most of the code is just moved, and the big refactorings are mostly fairly easy to understand. One big thing to call out though - we definitely need to make sure to test the new Filesyncer
on Windows, there's a lot of fragile stuff there (ahhh we need tests for this #3462).
I'm not as familiar with the services/networking here, so not really able to comment on how that's changed - I like the RunInNetNS
function though, that's neat.
I will of course also cleanup the commit history here with more thorough explanations of the changes.
This would definitely be appreciated ❤️ If you do this, merging without a squash would also be great, that way we can have a bit more insight with blame 🎉
engine/client/filesync.go
Outdated
if rootDir == "" { | ||
rootDir = "/" | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We probably want a different default on windows 🪟 See my previous comment as well about this, there's some complexity here.
Specifically, I think we probably want to have this be empty, and not a drive letter. But then this also means that the abs path returned might not begin with a /
, bleh (again, see above).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, did you end up with a good way of testing dagger on windows? I'll just fallback to running a VM on my laptop, but double checking there's not some easy way to run tests on a GHA windows runner manually or something.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(I just installed it in a VM, it was less painful than I remember other than the fact that I can't run nested docker in there and have to connect to a remote docker on linux over tailscale 🤪)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay fixed this here: 48e774a
Tested on windows manually and could still call modules loaded from local sources, so I think we are good there.
Let me know if you see anything off still, obviously manual testing is prone to missing cases.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
end up with a good way of testing dagger on windows
Nope :( I just ended up using my personal windows install that I have around for gaming.
Will try and find another moment to give it a go 🎉
engine/buildkit/executorSpec.go
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like this approach, it makes the executor significantly easier to understand, good refactoring 🎉
Of course, now I'm imagining how these callbacks could operate on a [T any]
type, and then potentially upstream into buildkit? There's not a lot of dagger-specific logic left in executor.go
.
Maybe again I'm misreading it, but suddenly it looks a lot more doable than when we discussed it previously in #7228 (comment), now the hook mechanism here contains almost all of the complexity - and is so, so, much more readable.
For context - I promise I'm not trying to be super nitpicky here 😅 The reason I'm keen on not having a hard-fork of these files as much is because we've definitely had some horrible context-lifetime bugs, etc with runc in callWithIO
. These were really nasty to track down and understand, and I'm just trying to come up with ways to avoid divergence for some of those code paths.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Totally, it's much more doable than before, though I think it might require refactoring upstream to use the same sort of approach. Honestly the simplest thing we could do upstream might just be to capitalize some of the structs/funcs it uses so we can import them and use them?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it might require refactoring upstream to use the same sort of approach
Yup, I think that's what I was imagining - the upstream implementation as is is quite complex, I think this approach is much easier to read and understand, so it definitely has my vote :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A mic drop of a PR if I've ever seen one 🎉
core/c2h.go
Outdated
|
||
for _, port := range d.tunnelServicePorts { | ||
var frontend int | ||
getFrontend := func(port PortForward) int { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
super nit: might make sense as a method, if there's a clear name. (FrontendOrBackend
? FrontOrBackEnd
? End
? lol)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I went to do this and just saw FrontendOrBackend
already exists haha: https://github.com/sipsma/dagger/blob/48e774a1e499482d67ae8c05760663b951751c95/core/net.go#L81-L81
Updating to that 😄
return c.worker.newNetNS(ctx, hostname) | ||
} | ||
|
||
func RunInNetNS[T any]( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤯
// we don't need a full container, just a CNI provisioned network namespace to listen in | ||
netNS, err := bk.NewNetworkNamespace(ctx, fullHost) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice nice, much more svelte
for _, port := range ports { | ||
retry := backoff.NewExponentialBackOff(backoff.WithInitialInterval(100 * time.Millisecond)) | ||
endpoint, err := backoff.RetryWithData(func() (string, error) { | ||
return buildkit.RunInNetNS(ctx, d.bk, d.ns, func() (string, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
😍
container.go is understandably a pretty big file but the exec related code is the most complicated and interesting part, which makes it annoying to always have to find amongst everything else. Hence, it feels a bit nicer to have in its own dedicated file. Signed-off-by: Erik Sipsma <erik@sipsma.dev>
we only sometimes need the server ID as an env var in order to invalidate cache, so it's a lot cleaner+nicer to just pass it through the exec metadata than rely solely on an env var to pass it around. Also makes it easier for other code (i.e. service + dockerfile frontend) to set it without jumping through the hoops of stringly-typed env vars. Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Not much of note here, just update the OCI spec from the executor rather than the shim. Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
I was unable to replicate any problems with linebreaks not displaying correctly once this was removed from the shim, even though I verified that stty showed `-onlcr` in `dagger call ... terminal` containers. I *suspect* that the more recent move in the CLI to just use the raw terminal rather than an emulator results in the ONLCR of *my* terminal (as a client) mattering rather than that of the container. Worst case and this is needed after all, we can use OCI hooks for it instead of the shim. Signed-off-by: Erik Sipsma <erik@sipsma.dev>
This is mostly just a transplant of existing code for writing to the meta mount from the shim to the executor. Things worth noting: 1. Secret scrubbing code is a bit simplified by just using filepaths rather than trying to use an fs.FS to support the unit tests. There's less cases to handle and thus less tests needed. The unit tests just rely on t.Tmpdir now, which is just as good for this use case. 2. The `cleanups` util here exists because `defer` doesn't work as expected when setup is spread across multiple functions. It operates the same sort of way but lets you "defer the defer" (if you will) so it only runs when the outermost caller decides it should run. Signed-off-by: Erik Sipsma <erik@sipsma.dev>
The healthchecks and tunneling that used to require a full-blown container that short-circuited to special shim commands can now just be done in the engine directly. That is done here by adding a util for executing go funcs inside the network namespace of existing containers (or creating a new raw network namespace in the case of the tunelling, since we just need an ip and listener there. The util hides the tricky details of needing to lock goroutines so callers can just run the func they want. It works by maintaining a pool of goroutines that will Setns into the container's network namespace when a job comes in over a channel. It also will stay locked in the namespace up to a timeout (currently just a second). * The reasoning here is that we don't want to consume an entire OS thread for too long but also want to avoid rapidly entering and leaving the namespace for e.g. health checks. The OTEL related code is grouped in here since the bulk of the work is setting up the TCP proxying of the otel unix sock, which relies on the above mechanisms. Signed-off-by: Erik Sipsma <erik@sipsma.dev>
This is the last step needed to remove the shim. Things worth noting: * Serving the session is just a matter of getting a listener in the container's network namespace and then using the existing `DAGGER_SESSION_PORT` mechanism (supported by the CLI and all SDKs) to tell the nested client where to make requests to. * There's a slight adjustment to the BuildkitController.Session methods to support directly serving a net.Conn since nested clients no longer need to tunnel through grpc and thus don't need to hit the .Session method directly. * The filesync session attachable was theoretically the trickiest part, but managed to avoid the need to do weird things with trying to use the container's /proc/<pid>/root magic symlink (which is vulnerable to pid-reuse attacks and finicky in that any code that calls filepath.EvalSymlinks on it will break). * Instead, we just do all the bind+overlays mounts to the rootfs rather than relying on runc to do it. We were already doing this for the / mount; we just do it for the rest of the non-special (i.e. not proc, sys, etc.) mounts now too. This gives us a view of the container's filesystem that's read-writable both outside and inside the container. * The other session attachables involving sockets and network tunnelling were more straightforward since they could just re-use the support for running callbacks in network namespaces added in previous commits. Signed-off-by: Erik Sipsma <erik@sipsma.dev>
The removal of the shim was actually a breaking change due to subtle but important differences in how Linux treats pid 1 processes. Namely, pid 1 will not get any default signal handlers and instead must explicitly register them. For example, without the shim our integ tests that ran python -m http.server stopped responding to SIGTERM. Not having an init also made it easier for zombie processes to lurk while the container was still running. To avoid this, we now build dumb-init and package it into the engine container. It's extremely lightweight (20kb statically linked and extremely minimal in what it does) and is also possible make a configuration option on the Container API for users who don't want it (though it should remain the default). This is as opposed to our shim which was much heavier and also was not optional until its functionality migrated to the executor. Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Buildkit frontend implementations sometimes end up creating containers with no network enabled at all, which wasn't handled well previously. Signed-off-by: Erik Sipsma <erik@sipsma.dev>
The executor does not Setns into the mount namespace of containers, only the network namespace, so the Go resolver doesn't have access to the /etc/resolv.conf with search domains specific to the container, which resulted in service hostnames failing to be resolved. While adding support for mount namespace entering is feasible, it would require we do a dance to pre-create a mount namespace and set it in the spec. It's a lot simpler to just check the search domain directly in our code. If any future use cases call for entering mount namespaces we can revisit and just use that instead. Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Buildkit puts these emulators under /dev, but we don't bind mount /dev when setting up the rootfs, so we were ending up binding the emulator to the rootfs but then hiding it when runc mounted /dev. We don't really need to bind mount these ahead of time since we never need to read/write them outside the container, so we can just leave them as bind mounts in the oci spec. Signed-off-by: Erik Sipsma <erik@sipsma.dev>
The previous code assumed that / was universal, which is obviously not the case on Windows. The new code separates out handling a bit cleaner so that Windows clients (i.e. running in the CLI) are handled correctly. Signed-off-by: Erik Sipsma <erik@sipsma.dev>
A few misc cleanups that were a pain to squash into the commit history: * Use consistent file names * Remove outdated comment about meta mount shim permissions (the meta mount doesn't actually get mounted into the final container anymore, it's just an LLB mount so its cache key is tied to the exec op). * Reuse existing method for getting port rather than duplicating. Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Done! Did some consolidation to cut the number of commits in half and added more context to the messages as needed. Hopefully I didn't fat finger anything while resolving conflicts during it. I'll double check with fresh eyes on Monday before pressing merge (will avoid the squash in this case too like you said). |
Double-checked the diff after doing commit cleanup, everything looks correct, so I'm gonna do rebase+merge now while it's still up to date with main to get the big diff out of the way for any other pending PRs w/ conflicts. cc @jedevc if you find anything off with Windows still let me know and will fixup in a follow-up! |
[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [aquaproj/aqua-registry](https://togithub.com/aquaproj/aqua-registry) | minor | `v4.185.1` -> `v4.188.0` | | [casey/just](https://togithub.com/casey/just) | minor | `1.26.0` -> `1.27.0` | | [charmbracelet/gum](https://togithub.com/charmbracelet/gum) | patch | `v0.14.0` -> `v0.14.1` | | [cli/cli](https://togithub.com/cli/cli) | minor | `v2.49.2` -> `v2.50.0` | | [dagger/dagger](https://togithub.com/dagger/dagger) | patch | `v0.11.4` -> `v0.11.6` | | [dprint/dprint](https://togithub.com/dprint/dprint) | minor | `0.45.1` -> `0.46.1` | | [fujiwara/awslim](https://togithub.com/fujiwara/awslim) | patch | `v0.1.1` -> `v0.1.2` | | [golangci/golangci-lint](https://togithub.com/golangci/golangci-lint) | minor | `v1.58.2` -> `v1.59.0` | | [simulot/immich-go](https://togithub.com/simulot/immich-go) | minor | `0.14.1` -> `0.15.0` | | [snyk/cli](https://togithub.com/snyk/cli) | patch | `v1.1291.0` -> `v1.1291.1` | | [twpayne/chezmoi](https://togithub.com/twpayne/chezmoi) | patch | `v2.48.1` -> `v2.48.2` | | [weaveworks/eksctl](https://togithub.com/weaveworks/eksctl) | minor | `v0.179.0` -> `v0.180.0` | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>aquaproj/aqua-registry (aquaproj/aqua-registry)</summary> ### [`v4.188.0`](https://togithub.com/aquaproj/aqua-registry/releases/tag/v4.188.0) [Compare Source](https://togithub.com/aquaproj/aqua-registry/compare/v4.187.1...v4.188.0) [Issues](https://togithub.com/aquaproj/aqua-registry/issues?q=is%3Aissue+milestone%3Av4.188.0) | [Pull Requests](https://togithub.com/aquaproj/aqua-registry/pulls?q=is%3Apr+milestone%3Av4.188.0) | aquaproj/aqua-registry@v4.187.1...v4.188.0 #### 🎉 New Packages [#​23408](https://togithub.com/aquaproj/aqua-registry/issues/23408) [getsavvyinc/savvy-cli](https://togithub.com/getsavvyinc/savvy-cli): Create, share, and run runbooks from your terminal [#​23384](https://togithub.com/aquaproj/aqua-registry/issues/23384) [joshmedeski/sesh](https://togithub.com/joshmedeski/sesh): Smart session manager for the terminal [@​CrystalMethod](https://togithub.com/CrystalMethod) ### [`v4.187.1`](https://togithub.com/aquaproj/aqua-registry/releases/tag/v4.187.1) [Compare Source](https://togithub.com/aquaproj/aqua-registry/compare/v4.187.0...v4.187.1) [Issues](https://togithub.com/aquaproj/aqua-registry/issues?q=is%3Aissue+milestone%3Av4.187.1) | [Pull Requests](https://togithub.com/aquaproj/aqua-registry/pulls?q=is%3Apr+milestone%3Av4.187.1) | aquaproj/aqua-registry@v4.187.0...v4.187.1 #### Fixes [#​23378](https://togithub.com/aquaproj/aqua-registry/issues/23378) authzed/zed: Use gnu binary instead of musl binary [@​sapphi-red](https://togithub.com/sapphi-red) musl version exists for linux, but it is dynamically linked to musl libc and therefore doesn't work on glibc systems. ### [`v4.187.0`](https://togithub.com/aquaproj/aqua-registry/releases/tag/v4.187.0) [Compare Source](https://togithub.com/aquaproj/aqua-registry/compare/v4.186.0...v4.187.0) [Issues](https://togithub.com/aquaproj/aqua-registry/issues?q=is%3Aissue+milestone%3Av4.187.0) | [Pull Requests](https://togithub.com/aquaproj/aqua-registry/pulls?q=is%3Apr+milestone%3Av4.187.0) | aquaproj/aqua-registry@v4.186.0...v4.187.0 #### 🎉 New Packages [#​23375](https://togithub.com/aquaproj/aqua-registry/issues/23375) [authzed/zed](https://togithub.com/authzed/zed): Official command-line tool for managing SpiceDB [@​sapphi-red](https://togithub.com/sapphi-red) [#​23277](https://togithub.com/aquaproj/aqua-registry/issues/23277) [nucleuscloud/neosync](https://togithub.com/nucleuscloud/neosync): Open source data anonymization and synthetic data orchestration for developers. Create high fidelity synthetic data and sync it across your environments ### [`v4.186.0`](https://togithub.com/aquaproj/aqua-registry/releases/tag/v4.186.0) [Compare Source](https://togithub.com/aquaproj/aqua-registry/compare/v4.185.1...v4.186.0) [Issues](https://togithub.com/aquaproj/aqua-registry/issues?q=is%3Aissue+milestone%3Av4.186.0) | [Pull Requests](https://togithub.com/aquaproj/aqua-registry/pulls?q=is%3Apr+milestone%3Av4.186.0) | aquaproj/aqua-registry@v4.185.1...v4.186.0 #### 🎉 New Packages [#​23253](https://togithub.com/aquaproj/aqua-registry/issues/23253) [stripe/stripe-cli](https://togithub.com/stripe/stripe-cli): A command-line tool for Stripe #### Fix [#​23254](https://togithub.com/aquaproj/aqua-registry/issues/23254) dlvhdr/gh-dash: Follow up changes of gh-dash v4.0.0 - [dlvhdr/gh-dash#336 </details> <details> <summary>casey/just (casey/just)</summary> ### [`v1.27.0`](https://togithub.com/casey/just/blob/HEAD/CHANGELOG.md#1270---2024-05-25) [Compare Source](https://togithub.com/casey/just/compare/1.26.0...1.27.0) ##### Changed - Use cache dir for temporary files ([#​2067](https://togithub.com/casey/just/pull/2067)) ##### Added - Add `[doc]` attribute to set and suppress documentation comments ([#​2050](https://togithub.com/casey/just/pull/2050) by [neunenak](https://togithub.com/neunenak)) - Add source_file() and source_directory() functions ([#​2088](https://togithub.com/casey/just/pull/2088)) - Add recipe groups ([#​1842](https://togithub.com/casey/just/pull/1842) by [neunenak](https://togithub.com/neunenak)) - Add shell() function for running external commands ([#​2047](https://togithub.com/casey/just/pull/2047) by [gyreas](https://togithub.com/gyreas)) - Add `--global-justfile` flag ([#​1846](https://togithub.com/casey/just/pull/1846) by [neunenak](https://togithub.com/neunenak)) - Add shell-expanded strings ([#​2055](https://togithub.com/casey/just/pull/2055)) - Add `encode_uri_component` function ([#​2052](https://togithub.com/casey/just/pull/2052) by [laniakea64](https://togithub.com/laniakea64)) - Add `choose` function for generating random strings ([#​2049](https://togithub.com/casey/just/pull/2049) by [laniakea64](https://togithub.com/laniakea64)) - Add predefined constants ([#​2054](https://togithub.com/casey/just/pull/2054)) - Allow setting some command-line options with environment variables ([#​2044](https://togithub.com/casey/just/pull/2044) by [neunenak](https://togithub.com/neunenak)) - Add prepend() function ([#​2045](https://togithub.com/casey/just/pull/2045) by [gyreas](https://togithub.com/gyreas)) - Add append() function ([#​2046](https://togithub.com/casey/just/pull/2046) by [gyreas](https://togithub.com/gyreas)) - Add --man subcommand ([#​2041](https://togithub.com/casey/just/pull/2041)) - Make `dotenv-path` relative to working directory ([#​2040](https://togithub.com/casey/just/pull/2040)) - Add `assert` expression ([#​1845](https://togithub.com/casey/just/pull/1845) by [de1iza](https://togithub.com/de1iza)) - Add 'allow-duplicate-variables' setting ([#​1922](https://togithub.com/casey/just/pull/1922) by [Mijago](https://togithub.com/Mijago)) ##### Fixed - List modules in source order with `--unsorted` ([#​2085](https://togithub.com/casey/just/pull/2085)) - Show submodule recipes in --choose ([#​2069](https://togithub.com/casey/just/pull/2069)) - Allow multiple imports of the same file in different modules ([#​2065](https://togithub.com/casey/just/pull/2065)) - Fix submodule recipe listing indentation ([#​2063](https://togithub.com/casey/just/pull/2063)) - Pass command as first argument to `shell` ([#​2061](https://togithub.com/casey/just/pull/2061)) - Allow shell expanded strings in mod and import paths ([#​2059](https://togithub.com/casey/just/pull/2059)) - Run imported recipes in root justfile with correct working directory ([#​2056](https://togithub.com/casey/just/pull/2056)) - Fix output `\r\n` stripping ([#​2035](https://togithub.com/casey/just/pull/2035)) ##### Misc - Forbid whitespace in shell-expanded string prefixes ([#​2083](https://togithub.com/casey/just/pull/2083)) - Add Debian and Ubuntu install instructions to readme ([#​2072](https://togithub.com/casey/just/pull/2072)) - Remove snap installation instructions from readme ([#​2070](https://togithub.com/casey/just/pull/2070)) - Fallback to wget in install script if curl isn't available([#​1913](https://togithub.com/casey/just/pull/1913) by [tgross35](https://togithub.com/tgross35)) - Use std::io::IsTerminal instead of atty crate ([#​2066](https://togithub.com/casey/just/pull/2066)) - Improve `shell()` documentation ([#​2060](https://togithub.com/casey/just/pull/2060) by [laniakea64](https://togithub.com/laniakea64)) - Add bash completion for snap ([#​2058](https://togithub.com/casey/just/pull/2058) by [albertodonato](https://togithub.com/albertodonato)) - Refactor list subcommand ([#​2062](https://togithub.com/casey/just/pull/2062)) - Document working directory ([#​2053](https://togithub.com/casey/just/pull/2053)) - Replace FunctionContext with Evaluator ([#​2048](https://togithub.com/casey/just/pull/2048)) - Update clap to version 4 ([#​1924](https://togithub.com/casey/just/pull/1924) by [poliorcetics](https://togithub.com/poliorcetics)) - Cleanup ([#​2026](https://togithub.com/casey/just/pull/2026) by [adamnemecek](https://togithub.com/adamnemecek)) - Increase --list maximum alignable width from 30 to 50 ([#​2039](https://togithub.com/casey/just/pull/2039)) - Document using `env -S` ([#​2038](https://togithub.com/casey/just/pull/2038)) - Update line continuation documentation ([#​1998](https://togithub.com/casey/just/pull/1998) by [laniakea64](https://togithub.com/laniakea64)) - Add example using GNU parallel to run tasks in concurrently ([#​1915](https://togithub.com/casey/just/pull/1915) by [amarao](https://togithub.com/amarao)) - Placate clippy: use `clone_into` ([#​2037](https://togithub.com/casey/just/pull/2037)) - Use --command-color when printing shebang recipe commands ([#​1911](https://togithub.com/casey/just/pull/1911) by [avi-cenna](https://togithub.com/avi-cenna)) - Document how to use watchexec to re-run recipes when files change ([#​2036](https://togithub.com/casey/just/pull/2036)) - Update VS Code extensions in readme ([#​2034](https://togithub.com/casey/just/pull/2034)) - Add rust:just repology package table to readme ([#​2032](https://togithub.com/casey/just/pull/2032)) </details> <details> <summary>charmbracelet/gum (charmbracelet/gum)</summary> ### [`v0.14.1`](https://togithub.com/charmbracelet/gum/releases/tag/v0.14.1) [Compare Source](https://togithub.com/charmbracelet/gum/compare/v0.14.0...v0.14.1) #### What's Changed - Show help with Huh? by [@​maaslalani](https://togithub.com/maaslalani) in [charmbracelet/gum#587 - Support using the Home/End keys in pager by [@​lzm0](https://togithub.com/lzm0) in [charmbracelet/gum#548 ##### Fixes - Handle `huh?` user aborted error by [@​MikaelFangel](https://togithub.com/MikaelFangel) in [charmbracelet/gum#578 - Cursor styling by [@​MikaelFangel](https://togithub.com/MikaelFangel) in [charmbracelet/gum#592 - Re-introduce <kbd>ESC</kbd> for `gum write` by [@​MikaelFangel](https://togithub.com/MikaelFangel) in [charmbracelet/gum#579 #### New Contributors - [@​CodeZea1ot](https://togithub.com/CodeZea1ot) made their first contribution in [charmbracelet/gum#542 - [@​camcui](https://togithub.com/camcui) made their first contribution in [charmbracelet/gum#537 - [@​lzm0](https://togithub.com/lzm0) made their first contribution in [charmbracelet/gum#548 **Full Changelog**: charmbracelet/gum@v0.14.0...v0.14.1 *** <a href="https://charm.sh/"><img alt="The Charm logo" src="https://stuff.charm.sh/charm-badge.jpg" width="400"></a> Thoughts? Questions? We love hearing from you. Feel free to reach out on [Twitter](https://twitter.com/charmcli), [The Fediverse](https://mastodon.technology/@​charm), or [Slack](https://charm.sh/slack). </details> <details> <summary>cli/cli (cli/cli)</summary> ### [`v2.50.0`](https://togithub.com/cli/cli/releases/tag/v2.50.0): GitHub CLI 2.50.0 [Compare Source](https://togithub.com/cli/cli/compare/v2.49.2...v2.50.0) #### What's Changed - Refactor git credential flow code by [@​williammartin](https://togithub.com/williammartin) in [cli/cli#9089 - feat: add json output for `gh pr checks` by [@​nobe4](https://togithub.com/nobe4) in [cli/cli#9079 - Rework first auth tests with new gitcredential abstractions by [@​williammartin](https://togithub.com/williammartin) in [cli/cli#9095 - list the various alias permutations for the command and subcommands, via '--help' and 'gh reference' by [@​gabemontero](https://togithub.com/gabemontero) in [cli/cli#8824 - Removed tty message when checking for extension upgrades by [@​leevic31](https://togithub.com/leevic31) in [cli/cli#9088 - Fix doc bug for gh run watch by [@​jasonodonnell](https://togithub.com/jasonodonnell) in [cli/cli#9052 - feat: add support for stateReason in `gh pr view` by [@​nobe4](https://togithub.com/nobe4) in [cli/cli#9080 - fix: rename the `Attempts` field to `Attempt`; expose in `gh run view` and `gh run ls` by [@​cawfeecake](https://togithub.com/cawfeecake) in [cli/cli#8905 - Update regex in changedFilesNames to handle quoted paths by [@​anda3](https://togithub.com/anda3) in [cli/cli#9115 - Add a `gh variable get FOO` command by [@​arnested](https://togithub.com/arnested) in [cli/cli#9106 - Add macOS pkg installer to deployment ([#​7554](https://togithub.com/cli/cli/issues/7554)) by [@​paulober](https://togithub.com/paulober) in [cli/cli#7555 - Add integration tests for `gh attestation verify` shared workflow use case by [@​malancas](https://togithub.com/malancas) in [cli/cli#9107 - Add build provenance for gh CLI releases by [@​malancas](https://togithub.com/malancas) in [cli/cli#9087 - build(deps): bump github.com/gabriel-vasile/mimetype from 1.4.3 to 1.4.4 by [@​dependabot](https://togithub.com/dependabot) in [cli/cli#9124 - Build completions during release on macos by [@​williammartin](https://togithub.com/williammartin) in [cli/cli#9136 - Clarify Mac OS Installer packages are unsigned by [@​andyfeller](https://togithub.com/andyfeller) in [cli/cli#9140 #### New Contributors - [@​gabemontero](https://togithub.com/gabemontero) made their first contribution in [cli/cli#8824 - [@​jasonodonnell](https://togithub.com/jasonodonnell) made their first contribution in [cli/cli#9052 - [@​anda3](https://togithub.com/anda3) made their first contribution in [cli/cli#9115 - [@​arnested](https://togithub.com/arnested) made their first contribution in [cli/cli#9106 - [@​paulober](https://togithub.com/paulober) made their first contribution in [cli/cli#7555 **Full Changelog**: cli/cli@v2.49.2...v2.50.0 </details> <details> <summary>dagger/dagger (dagger/dagger)</summary> ### [`v0.11.6`](https://togithub.com/dagger/dagger/blob/HEAD/CHANGELOG.md#v0116---2024-05-30) [Compare Source](https://togithub.com/dagger/dagger/compare/v0.11.5...v0.11.6) ##### Added - Add `withName` method to File by [@​TomChv](https://togithub.com/TomChv) in [dagger/dagger#7491 ##### Fixed - cli: don't validate flags when requesting `--help` by [@​helderco](https://togithub.com/helderco) in [dagger/dagger#7417 - fix container init being wrong platform in arm image by [@​sipsma](https://togithub.com/sipsma) in [dagger/dagger#7497 - fix container DNS resolution when host has no search domains by [@​sipsma](https://togithub.com/sipsma) in [dagger/dagger#7501 - honor system proxy config during git operations by [@​sipsma](https://togithub.com/sipsma) in [dagger/dagger#7504 - fix windows-style paths used as file+directory arg values in dagger call by [@​sipsma](https://togithub.com/sipsma) in [dagger/dagger#7506 ##### What to do next? - Read the [documentation](https://docs.dagger.io) - Join our [Discord server](https://discord.gg/dagger-io) - Follow us on [Twitter](https://twitter.com/dagger_io) ### [`v0.11.5`](https://togithub.com/dagger/dagger/blob/HEAD/CHANGELOG.md#v0115---2024-05-27) [Compare Source](https://togithub.com/dagger/dagger/compare/v0.11.4...v0.11.5) ##### Added - cli: `dagger login` cloud traces support by [@​aluzzardi](https://togithub.com/aluzzardi) in [dagger/dagger#7125 - cli: improved `--progress=plain` implementation for better visibility by [@​jedevc](https://togithub.com/jedevc) in [dagger/dagger#7272 ##### Changed - cli: cleaner tty progress view by [@​jedevc](https://togithub.com/jedevc) in [dagger/dagger#7347 [dagger/dagger#7371 [dagger/dagger#7386 - cli: don't show functions that can't be called by [@​helderco](https://togithub.com/helderco) in [dagger/dagger#7418 - cli: don't show inherited flags in function commands by [@​helderco](https://togithub.com/helderco) in [dagger/dagger#7419 - core: remove shim and switch to [dumb-init](https://togithub.com/Yelp/dumb-init) by [@​sipsma](https://togithub.com/sipsma) in [dagger/dagger#7367 ##### Fixed - core: fixed custom CA certs in modules by [@​sipsma](https://togithub.com/sipsma) in [dagger/dagger#7356 - cli: don't validate flags when requesting `--help` by [@​helderco](https://togithub.com/helderco) in [dagger/dagger#7417 ##### What to do next? - Read the [documentation](https://docs.dagger.io) - Join our [Discord server](https://discord.gg/dagger-io) - Follow us on [Twitter](https://twitter.com/dagger_io) </details> <details> <summary>dprint/dprint (dprint/dprint)</summary> ### [`v0.46.1`](https://togithub.com/dprint/dprint/releases/tag/0.46.1) [Compare Source](https://togithub.com/dprint/dprint/compare/0.46.0...0.46.1) #### Changes - fix: dprint 0.46 release for `cargo install` without `--locked` flag ([#​852](https://togithub.com/dprint/dprint/issues/852)) - fix: upgrade to rustls 0.23.8 ([#​853](https://togithub.com/dprint/dprint/issues/853)) Please run `dprint config update` after upgrading as some BOM handling happens in the plugins now and some may fail to parse files with a BOM now. See [#​854](https://togithub.com/dprint/dprint/issues/854) and sorry for any headaches (I'm unsure of the impact of this change, so let me know if this is a huge hassle for you). #### Install Run `dprint upgrade` or see https://dprint.dev/install/ #### Checksums |Artifact|SHA-256 Checksum| |:--|:--| |dprint-x86\_64-apple-darwin.zip|cdea84bce1d84c26e8eced2265d246b79a849ec2e7d1377d98dd7bdb21c7ce83| |dprint-aarch64-apple-darwin.zip|f3ff4faef83d14c3b4ae262e79a40d4e0fc3fa1903d0b6e9b82f0b25b00e9499| |dprint-x86\_64-pc-windows-msvc.zip|74e5ab38c744d5903862c2b5174d0fef9759b5506da775e1fb93b6a68c63101d| |dprint-x86\_64-pc-windows-msvc-installer.exe|107786c41be76b49463a50d7d9d788397bba723e107e723347f8e8dde65339dc| |dprint-x86\_64-unknown-linux-gnu.zip|cb72fa6b474e2847a3cf5705b43ee2cbfdafddd7c69ff162309fd1f4f43c872a| |dprint-x86\_64-unknown-linux-musl.zip|4a7d6fa6b920ab150f580965556086cdd7992e07078e627ab9a9d1c3bd30ba85| |dprint-aarch64-unknown-linux-gnu.zip|c4e892d5d237a57ede7900255e5ce669b56160e61c89798c118fbd4c36d48ff2| |dprint-aarch64-unknown-linux-musl.zip|e2b6d87167d21f1f01571790e79526ef9caff3b8b75f5cac348c4f06f60a8c16| ### [`v0.46.0`](https://togithub.com/dprint/dprint/releases/tag/0.46.0) [Compare Source](https://togithub.com/dprint/dprint/compare/0.45.1...0.46.0) ##### Changes - feat: gitignore support ([#​832](https://togithub.com/dprint/dprint/issues/832)) - feat: `DPRINT_TLS_CA_STORE` and `DPRINT_CERT` ([#​850](https://togithub.com/dprint/dprint/issues/850)) - fix: remove BOM handling from the CLI ([#​844](https://togithub.com/dprint/dprint/issues/844)) Please run `dprint config update` after upgrading as some BOM handling happens in the plugins now and some may fail to parse files with a BOM now. See [#​854](https://togithub.com/dprint/dprint/issues/854) and sorry for any headaches (I'm unsure of the impact of this change, so let me know if this is a huge hassle for you). ##### Install Run `dprint upgrade` or see https://dprint.dev/install/ ##### Checksums |Artifact|SHA-256 Checksum| |:--|:--| |dprint-x86\_64-apple-darwin.zip|e339f1f891c60087676d72f70ba5f80dcaedde4bdc58730b9cb68a5483b3abfd| |dprint-aarch64-apple-darwin.zip|4b608b3676f10e04328c3d8be396bded96328ebca9b95b70bf5baf67bed7b135| |dprint-x86\_64-pc-windows-msvc.zip|786201545938f6f7c6d407e6404b31ae9bbf9e5a4abc4c88dc9bd73da369a906| |dprint-x86\_64-pc-windows-msvc-installer.exe|e445b37af124e5d8ef691685632509d2bfc701962c58db89eebc8a8de7352ab4| |dprint-x86\_64-unknown-linux-gnu.zip|8274ea44d2ab4d10b8bdfcc824d946a6d051594aede49c9db8c5e810887abd67| |dprint-x86\_64-unknown-linux-musl.zip|7a2c12edc868259be890174c4ec3bd51c81ec8773aa294e12fac0634f36d15f5| |dprint-aarch64-unknown-linux-gnu.zip|6617465acba53c9b939e73f20538a8027e45593342c34f7ac4826c9f4e6cf53b| |dprint-aarch64-unknown-linux-musl.zip|e52c0a3398e34e88ffe560e719bf8361ba3f35b4e0927ab9ba0761796884ce24| </details> <details> <summary>fujiwara/awslim (fujiwara/awslim)</summary> ### [`v0.1.2`](https://togithub.com/fujiwara/awslim/releases/tag/v0.1.2) [Compare Source](https://togithub.com/fujiwara/awslim/compare/v0.1.1...v0.1.2) #### What's Changed - feature: Add suport for building with specified OS/Arch by [@​ToshihitoKon](https://togithub.com/ToshihitoKon) in [fujiwara/awslim#20 - merge [#​20](https://togithub.com/fujiwara/awslim/issues/20) into main by [@​fujiwara](https://togithub.com/fujiwara) in [fujiwara/awslim#21 #### New Contributors - [@​ToshihitoKon](https://togithub.com/ToshihitoKon) made their first contribution in [fujiwara/awslim#20 **Full Changelog**: fujiwara/awslim@v0.1.1...v0.1.2 </details> <details> <summary>golangci/golangci-lint (golangci/golangci-lint)</summary> ### [`v1.59.0`](https://togithub.com/golangci/golangci-lint/compare/v1.58.2...v1.59.0) [Compare Source](https://togithub.com/golangci/golangci-lint/compare/v1.58.2...v1.59.0) </details> <details> <summary>simulot/immich-go (simulot/immich-go)</summary> ### [`v0.15.0`](https://togithub.com/simulot/immich-go/releases/tag/0.15.0) [Compare Source](https://togithub.com/simulot/immich-go/compare/0.14.1...0.15.0) ##### fix [#​255](https://togithub.com/simulot/immich-go/issues/255) Last percents of google puzzle solving are very slow when processing very large takeout archive The google puzzle solving is now much faster for large takeout archives. ##### fix [#​215](https://togithub.com/simulot/immich-go/issues/215) Use XDG_CONFIG_HOME for storing config The configuration file that contains the server and the key is now stored by default in following folder: - Linux `$HOME/.config/immich-go/immich-go.json` - Windows `%AppData%\immich-go\immich-go.json` - Apple `$HOME/Library/Application Support/immich-go/immich-go.json` ##### Store the log files into sensible dir for user's system The default log file is: - Linux `$HOME/.cache/immich-go/immich-go_YYYY-MM-DD_HH-MI-SS.log` - Windows `%LocalAppData%\immich-go\immich-go_YYYY-MM-DD_HH-MI-SS.log` - Apple `$HOME/Library/Caches/immich-go/immich-go_YYYY-MM-DD_HH-MI-SS.log` ##### Feat: \[[#​249](https://togithub.com/simulot/immich-go/issues/249)] Fix Display the path of log file name The log file name is printed when the program exits. #### Changelog - [`87471f6`](https://togithub.com/simulot/immich-go/commit/87471f6) Edit releases.md - [`d4269ee`](https://togithub.com/simulot/immich-go/commit/d4269ee) Fix Display the path of log file name ([#​251](https://togithub.com/simulot/immich-go/issues/251)) - [`49bf673`](https://togithub.com/simulot/immich-go/commit/49bf673) Fix Display the path of log file name ([#​256](https://togithub.com/simulot/immich-go/issues/256)) - [`73fde8d`](https://togithub.com/simulot/immich-go/commit/73fde8d) Fixes [#​255](https://togithub.com/simulot/immich-go/issues/255) - [`9b70932`](https://togithub.com/simulot/immich-go/commit/9b70932) Merge branch 'main' into simulot/issue215 - [`734bbeb`](https://togithub.com/simulot/immich-go/commit/734bbeb) Merge pull request [#​257](https://togithub.com/simulot/immich-go/issues/257) from simulot:simulot/issue255 - [`8805ca7`](https://togithub.com/simulot/immich-go/commit/8805ca7) Merge pull request [#​258](https://togithub.com/simulot/immich-go/issues/258) from simulot:simulot/issue215 - [`c63dc09`](https://togithub.com/simulot/immich-go/commit/c63dc09) Merge pull request [#​260](https://togithub.com/simulot/immich-go/issues/260) from simulot:simulot/issue259 - [`e4d1643`](https://togithub.com/simulot/immich-go/commit/e4d1643) Set a timeout for all http queries ([#​253](https://togithub.com/simulot/immich-go/issues/253)) - [`f968bd6`](https://togithub.com/simulot/immich-go/commit/f968bd6) The log doesn't show discared files Fixes [#​259](https://togithub.com/simulot/immich-go/issues/259) - [`37f4b83`](https://togithub.com/simulot/immich-go/commit/37f4b83) Use XDG_CONFIG_HOME for storing config ([#​248](https://togithub.com/simulot/immich-go/issues/248)) - [`755e1b7`](https://togithub.com/simulot/immich-go/commit/755e1b7) Use XDG_CONFIG_HOME for storing config Fixes [#​215](https://togithub.com/simulot/immich-go/issues/215) </details> <details> <summary>snyk/cli (snyk/cli)</summary> ### [`v1.1291.1`](https://togithub.com/snyk/cli/releases/tag/v1.1291.1) [Compare Source](https://togithub.com/snyk/cli/compare/v1.1291.0...v1.1291.1) The Snyk CLI is being deployed to different deployment channels, users can select the stability level according to their needs. For details please see [this documentation](https://docs.snyk.io/snyk-cli/releases-and-channels-for-the-snyk-cli) ##### Bug Fixes - **dependencies:** Upgrade go-getter to v1.7.4 to fix vulnerabilities ([#​5252](https://togithub.com/snyk/snyk/issues/5252)) </details> <details> <summary>twpayne/chezmoi (twpayne/chezmoi)</summary> ### [`v2.48.2`](https://togithub.com/twpayne/chezmoi/compare/v2.48.1...v2.48.2) [Compare Source](https://togithub.com/twpayne/chezmoi/compare/v2.48.1...v2.48.2) </details> <details> <summary>weaveworks/eksctl (weaveworks/eksctl)</summary> ### [`v0.180.0`](https://togithub.com/eksctl-io/eksctl/releases/tag/v0.180.0): eksctl 0.180.0 [Compare Source](https://togithub.com/weaveworks/eksctl/compare/0.179.0-rc.0...0.180.0) ### Release v0.180.0 #### 🐛 Bug Fixes - Add option to create service account for pod identities which defaults to `false` ([#​7784](https://togithub.com/weaveworks/eksctl/issues/7784)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 4pm on thursday" in timezone America/Los_Angeles, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/scottames/dots). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4zNzcuOCIsInVwZGF0ZWRJblZlciI6IjM3LjM3Ny44IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiXX0=--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: scottames-github-bot[bot] <162828115+scottames-github-bot[bot]@users.noreply.github.com>
This PR gets rid of our shim and migrates all of its functionality to run directly in the engine instead.
Original motivation: as part of the general engine refactoring/cleanup I've been working through I realized we would be able to simplify a lot if we served nested clients (e.g. module functions and nested execs) directly from our executor. Among other things, it would allow us to be more "stateless" and not need to pre-register nested clients (which is a confusing and bug prone set of code paths).
But at that point it felt like the right time to just bite the bullet and rm the shim entirely, which is possible now thanks to our new worker/executor and has a ton of other side benefits, e.g.
Status: I've migrated everything now, just some cleanup and testing left:
ONLCR
(afaik anyways) and those work interactively. cc @vitopython -m http.server
service we use in tests no longer responds toSIGTERM
(though it does respond toSIGINT
). This explains why and how it's related to the fact that we now allow user processes to be init.systemd
, etc.)I will of course also cleanup the commit history here with more thorough explanations of the changes.