Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
351 views
in Technique[技术] by (71.8m points)

python - How can I get non-blocking socket connect()'s?

I have a quite simple problem here. I need to communicate with a lot of hosts simultaneously, but I do not really need any synchronization because each request is pretty self sufficient.

Because of that, I chose to work with asynchronous sockets, rather than spamming threads. Now I do have a little problem:

The async stuff works like a charm, but when I connect to 100 hosts, and I get 100 timeouts (timeout = 10 secs) then I wait 1000 seconds, just to find out all my connections failed.

Is there any way to also get non blocking socket connects? My socket is already set to nonBlocking, but calls to connect() are still blocking.

Reducing the timeout is not an acceptable solution.

I am doing this in Python, but I guess the programming language doesnt really matter in this case.

Do I really need to use threads?

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

Use the select module. This allows you to wait for I/O completion on multiple non-blocking sockets. Here's some more information on select. From the linked-to page:

In C, coding select is fairly complex. In Python, it's a piece of cake, but it's close enough to the C version that if you understand select in Python, you'll have little trouble with it in C.

ready_to_read, ready_to_write, in_error = select.select(
                  potential_readers, 
                  potential_writers, 
                  potential_errs, 
                  timeout)

You pass select three lists: the first contains all sockets that you might want to try reading; the second all the sockets you might want to try writing to, and the last (normally left empty) those that you want to check for errors. You should note that a socket can go into more than one list. The select call is blocking, but you can give it a timeout. This is generally a sensible thing to do - give it a nice long timeout (say a minute) unless you have good reason to do otherwise.

In return, you will get three lists. They have the sockets that are actually readable, writeable and in error. Each of these lists is a subset (possibly empty) of the corresponding list you passed in. And if you put a socket in more than one input list, it will only be (at most) in one output list.

If a socket is in the output readable list, you can be as-close-to-certain-as-we-ever-get-in-this-business that a recv on that socket will return something. Same idea for the writeable list. You'll be able to send something. Maybe not all you want to, but something is better than nothing. (Actually, any reasonably healthy socket will return as writeable - it just means outbound network buffer space is available.)

If you have a "server" socket, put it in the potential_readers list. If it comes out in the readable list, your accept will (almost certainly) work. If you have created a new socket to connect to someone else, put it in the potential_writers list. If it shows up in the writeable list, you have a decent chance that it has connected.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...