Jobs, parallelism and remoting

PowerShell · Advanced · 12 min

Parallel and remote execution turn slow, one-machine-at-a-time scripts into fleet automation.

PowerShell has several ways to run work concurrently:

  • Start-Job runs a script block in a separate background process. Isolated but heavy (a new pwsh starts each time).
  • Start-ThreadJob runs it on a thread in the same process: much lighter, the usual choice. Both return job objects; collect results with Receive-Job (-Wait to block) and clean up with Remove-Job.
  • ForEach-Object -Parallel (PowerShell 7) processes pipeline items concurrently in runspaces, with -ThrottleLimit (default 5) capping concurrency.

Code in a job or a parallel block cannot see your variables. Pass values with the `$using:` scope modifier, which copies them in: $using:token. Do not modify shared collections from parallel blocks; output results and collect them instead.

Parallelism pays off for I/O-bound work like web calls, remote commands and file copies. For tiny CPU tasks, the overhead of runspaces can make it slower than a plain loop.

Remoting runs commands on other machines: Invoke-Command -ComputerName srv1, srv2 -ScriptBlock { ... } executes in parallel on all of them over WinRM (Windows), and -HostName uses SSH, which works cross-platform. Enter-PSSession gives an interactive remote prompt; New-PSSession keeps a reusable connection. Results come back as deserialized objects (properties only, no methods).

Example

$job = Start-ThreadJob { Start-Sleep -Milliseconds 300; (Get-Date).Year }
Receive-Job $job -Wait -AutoRemoveJob          # 2026
$pj = Start-Job { $PID }                       # a separate process
(Receive-Job $pj -Wait) -ne $PID               # True
Remove-Job $pj
$factor = 10
1..5 | ForEach-Object -Parallel {
  Start-Sleep -Milliseconds (Get-Random -Maximum 200)
  $_ * $using:factor                           # $using: copies a local value in
} -ThrottleLimit 3                             # 10..50, in any order
# Remoting (WinRM on Windows, or SSH everywhere):
# Invoke-Command -HostName web1, web2 -UserName ops -ScriptBlock { Get-Uptime }
# Enter-PSSession -HostName web1               # interactive remote prompt
# $s = New-PSSession -HostName web1 -UserName ops
# Invoke-Command -Session $s { $env:HOSTNAME }

Remember

  • Start-ThreadJob is light, Start-Job isolates in a process
  • ForEach-Object -Parallel with -ThrottleLimit for concurrent pipelines
  • $using:var passes local values into jobs, parallel blocks and remote sessions

Watch out for

  • Referencing $myVar inside -Parallel or a job without $using: and getting $null.

Check yourself

  1. Inside ForEach-Object -Parallel, how do you read the caller's $limit variable?
  2. Which is the lightest way to run a background task in PowerShell 7?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/foreach-object