Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
692 views
in Technique[技术] by (71.8m points)

python - Howto: workaround of close_fds=True and redirect stdout/stderr on windows

I had a problem: using Python 2.7, it was not possible to create a subprocess using

subprocess.Popen([.......], close_fds=True, stdout=subprocess.PIPE, ...)

On windows, because of limitations. The use of close_fds was needed in my case, as I did not wanted the subprocess to inherit from already opened files file-descriptor. This was called in a library, which meant I had no control over already opened- file descriptors (N flag).

This is a known bug, which was fixed on Python 3.4+

The question I had was: How do I use subprocess without getting

close_fds is not supported on Windows platforms if you redirect stdin/stdout/stderr

Answer below

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

This issue is fixed on Python 3.7+ by default

This is definatly a tricky hack: answer is to iterate through already opened file descriptors, just before using the subprocess module.

def _hack_windows_subprocess():
    """HACK: python 2.7 file descriptors.
    This magic hack fixes https://bugs.python.org/issue19575
    by adding HANDLE_FLAG_INHERIT to all already opened file descriptors.
    """
    # See https://github.com/secdev/scapy/issues/1136
    import stat
    from ctypes import windll, wintypes
    from msvcrt import get_osfhandle

    HANDLE_FLAG_INHERIT = 0x00000001

    for fd in range(100):
        try:
            s = os.fstat(fd)
        except:
            continue
        if stat.S_ISREG(s.st_mode):
            handle = wintypes.HANDLE(get_osfhandle(fd))
            mask   = wintypes.DWORD(HANDLE_FLAG_INHERIT)
            flags  = wintypes.DWORD(0)
            windll.kernel32.SetHandleInformation(handle, mask, flags)

Here is a sample that would crash without it:

import os, subprocess
f = open("a.txt", "w")
subprocess.Popen(["cmd"], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
f.close()
os.remove(f.name)

Traceback (most recent call last):

File "stdin", line 1, in module

WindowsError: [Error 32] Le processus ne peut pas accúder au fichier car ce fichier est utilisú par un autre processus: 'a.txt'

Now with the fix:

import os, subprocess
f = open("a.txt", "w")
_hack_windows_subprocess()
subprocess.Popen(["cmd"], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
f.close()
os.remove(f.name)

Works.

Hope i helped


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...