Prevent and recover from wedged Wine-hosted processes

Prompted by a real incident: an MSBuild node-reuse worker (its own
/nodeReuse:true default) survived a build getting interrupted, came
back deadlocked, and got reused by the next `msbuild` invocation -
which then failed with a confusing, unrelated-looking
`System.TypeLoadException` on Microsoft.VisualStudio.Telemetry on
every call for hours, until the stale process was killed by hand.
That's exactly the "unrelated blocker" noted in this repo's own
earlier session notes (CLAUDE.md) while debugging a real project's
build - it wasn't a missing dependency, it was a corrupted reused
process.

Two changes:

- vintner now forces /nodeReuse:false on every msbuild invocation
  (unless the caller already passed their own /nodeReuse or /nr
  switch), so a wedged worker can never poison a later, unrelated
  build in the first place. Costs each invocation the couple-hundred-
  ms/node startup time node reuse exists to save.

- VINTNER_TIMEOUT (a duration string, e.g. "30m") bounds how long any
  single tool invocation is allowed to run, for the case something
  wedges that isn't MSBuild-specific. Every exec.Command site in
  internal/wrapper now goes through a shared newToolCommand
  constructor that, when the timeout is set, kills the *whole*
  process group (not just the immediate `wine` process - a wedged
  child surviving under it is exactly the scenario this needs to
  reach) via a context deadline, and reports a clear "timed out after
  Xm" message (exit 124, matching the timeout(1) convention) instead
  of a bare "signal: killed". Unset by default - every real build
  observed stays unbounded, matching Windows' own behavior.

Verified end-to-end, not just at the unit level: a real `sleep 30`
through the `cmd` native wrapper with VINTNER_TIMEOUT=1s was killed
within the deadline and reported the timeout clearly (exit 124); a
real `cl` invocation with the same 1s timeout finished normally
(0.26s) without being mistaken for a hang.
This commit is contained in:
Cheviiot
2026-07-25 15:58:22 +10:00
parent bdea270d71
commit f6b9a0811a
7 changed files with 313 additions and 39 deletions
+33
View File
@@ -218,6 +218,39 @@ func msbuildGlobalArgs(cfg *wineenv.Config, args []string) []string {
return out
}
var reNodeReuse = regexp.MustCompile(`(?i)^[-/](nodereuse|nr):`)
// msbuildNodeReuseArgs returns ["/nodeReuse:false"] unless args already pins
// node reuse one way or the other.
//
// MSBuild's node-reuse worker processes (its own /nodeReuse:true default)
// don't behave like a normal child process here: they're meant to outlive
// the parent msbuild.exe invocation that spawned them, waiting around under
// Wine for the *next* msbuild call to reuse them - so nothing about
// vintner's own process-lifetime handling (see signals.go) touches them,
// and there's no parent process left to notice if one wedges. If a build is
// interrupted (Ctrl-C, a killed session, a crashed Wine transport) mid-
// compile, the worker can be left holding a half-open pipe/mutex,
// permanently deadlocked rather than exited - confirmed in practice: a
// stale reused node kept throwing an unrelated-looking
// `System.TypeLoadException` on Microsoft.VisualStudio.Telemetry on every
// subsequent build, for hours, until it was killed by hand and the next
// build got a fresh node. Forcing node reuse off means every invocation
// gets a clean process, so a wedged one can never poison a later,
// unrelated build - at the cost of the couple-hundred-ms/node startup time
// node reuse exists to save. Callers who deliberately want reuse (e.g.
// running many builds back to back and are prepared to clean up wedged
// nodes themselves) can still pass their own /nodeReuse or /nr switch to
// override this.
func msbuildNodeReuseArgs(args []string) []string {
for _, a := range args {
if reNodeReuse.MatchString(a) {
return nil
}
}
return []string{"/nodeReuse:false"}
}
func msbuildPlatform(arch string) string {
switch arch {
case "x86":