Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
694 views
in Technique[技术] by (71.8m points)

node.js - NodeJS - What does "socket hang up" actually mean?

I'm building a web scraper with Node and Cheerio, and for a certain website I'm getting the following error (it only happens on this one website, no others that I try to scrape.

It happens at a different location every time, so sometimes it's url x that throws the error, other times url x is fine and it's a different url entirely:

    Error!: Error: socket hang up using [insert random URL, it's different every time]

Error: socket hang up
    at createHangUpError (http.js:1445:15)
    at Socket.socketOnEnd [as onend] (http.js:1541:23)
    at Socket.g (events.js:175:14)
    at Socket.EventEmitter.emit (events.js:117:20)
    at _stream_readable.js:910:16
    at process._tickCallback (node.js:415:13)

This is very tricky to debug, I don't really know where to start. To begin, what IS a socket hang up error? Is it a 404 error or similar? Or does it just mean that the server refused a connection?

I can't find an explanation of this anywhere!

EDIT: Here's a sample of code that is (sometimes) returning errors:

function scrapeNexts(url, oncomplete) {
    request(url, function(err, resp, body) {

        if (err) {
            console.log("Uh-oh, ScrapeNexts Error!: " + err + " using " + url);
            errors.nexts.push(url);
        }
        $ = cheerio.load(body);
        // do stuff with the '$' cheerio content here
    });
}

There is no direct call to close the connection, but I'm using Node Request which (as far as I can tell) uses http.get so this is not required, correct me if I'm wrong!

EDIT 2: Here's an actual, in-use bit of code that is causing errors. prodURL and other variables are mostly jquery selectors that are defined earlier. This uses the async library for Node.

function scrapeNexts(url, oncomplete) {
    request(url, function (err, resp, body) {

        if (err) {
            console.log("Uh-oh, ScrapeNexts Error!: " + err + " using " + url);
            errors.nexts.push(url);
        }
        async.series([
                function (callback) {
                    $ = cheerio.load(body);
                    callback();
                },
                function (callback) {
                    $(prodURL).each(function () {
                        var theHref = $(this).attr('href');
                        urls.push(baseURL + theHref);
                    });
                    var next = $(next_select).first().attr('href');
                    oncomplete(next);
                }
            ]);
    });
}
Question&Answers:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

There are two cases when socket hang up gets thrown:

When you are a client

When you, as a client, send a request to a remote server, and receive no timely response. Your socket is ended which throws this error. You should catch this error and decide how to handle it: whether retry the request, queue it for later, etc.

When you are a server/proxy

When you, as a server, perhaps a proxy server, receive a request from a client, then start acting upon it (or relay the request to the upstream server), and before you have prepared the response, the client decides to cancel/abort the request.

This stack trace shows what happens when a client cancels the request.

Trace: { [Error: socket hang up] code: 'ECONNRESET' }
    at ClientRequest.proxyError (your_server_code_error_handler.js:137:15)
    at ClientRequest.emit (events.js:117:20)
    at Socket.socketCloseListener (http.js:1526:9)
    at Socket.emit (events.js:95:17)
    at TCP.close (net.js:465:12)

Line http.js:1526:9points to the same socketCloseListener mentioned by @Blender, particularly:

// This socket error fired before we started to
// receive a response. The error needs to
// fire on the request.
req.emit('error', createHangUpError());

...

function createHangUpError() {
  var error = new Error('socket hang up');
  error.code = 'ECONNRESET';
  return error;
}

This is a typical case if the client is a user in the browser. The request to load some resource/page takes long, and users simply refresh the page. Such action causes the previous request to get aborted which on your server side throws this error.

Since this error is caused by the wish of a client, they don't expect to receive any error message. So, no need to consider this error as critical. Just ignore it. This is encouraged by the fact that on such error the res socket that your client listened to is, though still writable, destroyed.

console.log(res.socket.destroyed); //true

So, no point to send anything, except explicitly closing the response object:

res.end();

However, what you should do for sure if you are a proxy server which has already relayed the request to the upstream, is to abort your internal request to the upstream, indicating your lack of interest in the response, which in turn will tell the upstream server to, perhaps, stop an expensive operation.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...