I am running Python 3.4.3 on Linux 3.16.0. I want to use subprocess.Popen
to run a command with a long single argument (a complex Bash invocation), roughly 200KiB.
According to getconf
and xargs
, this should be well within my limits:
$ getconf ARG_MAX
2097152
$ xargs --show-limits < /dev/null
Your environment variables take up 3364 bytes
POSIX upper limit on argument length (this system): 2091740
POSIX smallest allowable upper limit on argument length (all systems): 4096
Maximum length of command we could actually use: 2088376
Size of command buffer we are actually using: 131072
However, Python fails with quite smaller limits:
>>> subprocess.Popen('echo %s > /dev/null' % ('a' * (131072-4096)), shell=True, executable='/bin/bash')
<subprocess.Popen object at 0x7f4613b58410>
>>> subprocess.Popen('echo %s > /dev/null' % ('a' * (262144-4096)), shell=True, executable='/bin/bash')
Traceback (most recent call last):
[...]
OSError: [Errno 7] Argument list too long
Note that the Python limit is roughly the same as the "actually using" command buffer xargs
reports. This suggests that xargs
is somehow smart enough to start with a smaller limit and increase it as needed, but Python is not.
Questions:
- Why is the Python limit smaller than the OS limit of 2MiB?
- Can I increase the Python limit?
- If so, how?
See Question&Answers more detail:
os 与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…