Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
445 views
in Technique[技术] by (71.8m points)

python - Can someone explain pipe buffer deadlock?

Python documentation to Popen states:

Warning Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.

Now, I'm trying to figure out how this deadlock can occur and why.

My mental model: subproccess produces something to stdout/err, which is buffered and after buffer is filled, it's flushed to stdout/err of subproccess, which is send through pipe to parent process.

From what documentation states, pipe has it's own buffer and when it's filled or subprocess terminates, it's flushed to to the parent process.

Either way (with pipe buffer or not), I'm not entirely sure how deadlock can occur. Only thing I can think of is some kind of "global" OS pipe buffer processes will be striving for, which sounds strange. Another is that more processes will share same pipe, which should not happen on it's own.

Can someone please explain this?

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

Careful, this has a subtle mistake in it.

My mental model: subproccess produces something to stdout/err, which is buffered and after buffer is filled, it's flushed to stdout/err of subproccess, which is send through pipe to parent process.

The buffer is shared by parent and child process.

Subprocess produces something to stdout, which is the same buffer the parent process is supposed to be reading from.

When the buffer is filled, writing stops until the buffer is emptied. Flush doesn't mean anything to a pipe, since two processes share the same buffer.

Flushing to disk means that the device driver must push the bytes down to the device. Flushing a socket means to tell TCP/IP to stop waiting to accumulate a buffer and send stuff. Flushing to a console means stop waiting for a newline and push the bytes through the device driver to the device.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...