Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
753 views
in Technique[技术] by (71.8m points)

bash - Is it possible to distribute STDIN over parallel processes?

Given the following example input on STDIN:

foo
bar bar
baz
===
qux
bla
===
def
zzz yyy

Is it possible to split it on the delimiter (in this case '===') and feed it over stdin to a command running in parallel?

So the example input above would result in 3 parallel processes (for example a command called do.sh) where each instance received a part of the data on STDIN, like this:

do.sh (instance 1) receives this over STDIN:

foo
bar bar
baz

do.sh (instance 2) receives this over STDIN:

qux
bla

do.sh (instance 3) receives this over STDIN:

def
zzz yyy

I suppose something like this is possible using xargs or GNU parallel, but I do not know how.

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

GNU Parallel can do that from version 20110205.

cat | parallel --pipe --recend '===
' --rrs do_stuff

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...