python - Why does subprocess.Popen() with shell=True work differently on Linux vs Windows?

Question

When using subprocess.Popen(args, shell=True) to run "gcc --version" (just as an example), on Windows we get this:

>>> from subprocess import Popen
>>> Popen(['gcc', '--version'], shell=True)
gcc (GCC) 3.4.5 (mingw-vista special r3) ...

So it's nicely printing out the version as I expect. But on Linux we get this:

>>> from subprocess import Popen
>>> Popen(['gcc', '--version'], shell=True)
gcc: no input files

Because gcc hasn't received the --version option.

The docs don't specify exactly what should happen to the args under Windows, but it does say, on Unix, "If args is a sequence, the first item specifies the command string, and any additional items will be treated as additional shell arguments." IMHO the Windows way is better, because it allows you to treat Popen(arglist) calls the same as Popen(arglist, shell=True) ones.

Why the difference between Windows and Linux here?

Answer 1

Actually on Windows, it does use cmd.exe when shell=True - it prepends cmd.exe /c (it actually looks up the COMSPEC environment variable but defaults to cmd.exe if not present) to the shell arguments. (On Windows 95/98 it uses the intermediate w9xpopen program to actually launch the command).

So the strange implementation is actually the UNIX one, which does the following (where each space separates a different argument):

/bin/sh -c gcc --version

It looks like the correct implementation (at least on Linux) would be:

/bin/sh -c "gcc --version" gcc --version

Since this would set the command string from the quoted parameters, and pass the other parameters successfully.

From the sh man page section for -c:

Read commands from the command_string operand instead of from the standard input. Special parameter 0 will be set from the command_name operand and the positional parameters ($1, $2, etc.) set from the remaining argument operands.

This patch seems to fairly simply do the trick:

--- subprocess.py.orig  2009-04-19 04:43:42.000000000 +0200
+++ subprocess.py       2009-08-10 13:08:48.000000000 +0200
@@ -990,7 +990,7 @@
                 args = list(args)

             if shell:
-                args = ["/bin/sh", "-c"] + args
+                args = ["/bin/sh", "-c"] + [" ".join(args)] + args

             if executable is None:
                 executable = args[0]