javascript - Process chain of functions without UI block

Question

I need to perform several functions in my JavaScript/jQuery, but I want to avoid blocking the UI.

AJAX is not a viable solution, because of the nature of the application, those functions will easily reach the thousands. Doing this asynchroniously will kill the browser.

So, I need some way of chaining the functions the browser needs to process, and only send the next function after the first has finished.

The algorithm is something like this

For steps from 2 to 15

HTTP:GET amount of items for current step (ranges somewhere from a couple of hundred to multiple thousands)

For every item, HTTP:GET the results

As you see, I have two GET-request-"chains" I somehow need to manage... Especially the innermost loop crashes the browser near to instantly, if it's done asynchroniously - but I'd still like the user to be able to operate the page, so a pure (blocking) synchronous way will not work.

Answer 1

You can easily do this asynchronously without firing all requests at once. All you need to do is manage a queue. The following is pseudo-code for clarity. It's easily translatable to real AJAX requests:

// Basic structure of the request queue. It's a list of objects
// that defines ajax requests:
var request_queue = [{
    url : "some/path",
    callback : function_to_process_the_data
}];

// This function implements the main loop.
// It looks recursive but is not because each function
// call happens in an event handler:
function process_request_queue () {
    // If we have anything in the queue, do an ajax call.
    // Otherwise do nothing and let the loop end.
    if (request_queue.length) {
        // Get one request from the queue. We can either
        // shift or pop depending on weather you prefer
        // depth first or breadth first processing:
        var req = request_queue.pop();
        ajax(req.url,function(result){
            req.callback(result);
            // At the end of the ajax request process
            // the queue again:
            process_request_queue();
        }
    }
}

// Now get the ball rolling:
process_request_queue();

So basically we turn the ajax call itself into a pseudo loop. It's basically the classic continuation passing style of programming done recursively.

In your case, an example of a request would be:

request_queue.push({
    url : "path/to/OUTER/request",
    callback : function (result) {
        // You mentioned that the result of the OUTER request
        // should trigger another round of INNER requests.
        // To do this simply add the INNER requests to the queue:

        request_queue.push({
            url : result.inner_url,
            callback : function_to_handle_inner_request
        });
    }
});

This is quite flexible because you not only have the option of processing requests either breadth first or depth first (shift vs pop). But you can also use splice to add stuff to the middle of the queue or use unshift vs push to put requests at the head of the queue for high priority requests.

You can also increase the number of simultaneous requests by popping more than one request per loop. Just be sure to only call process_request_queue only once per loop to avoid exponential growth of simultaneous requests:

// Handling two simultaneous request channels:
function process_request_queue () {
    if (request_queue.length) {
        var req = request_queue.pop();
        ajax(req.url,function(result){
            req.callback(result);
            // Best to loop on the first request.
            // The second request below may never fire
            // if the queue runs out of requests.
            process_request_queue();
        }
    }
    if (request_queue.length) {
        var req = request_queue.pop();
        ajax(req.url,function(result){
            req.callback(result);
            // DON'T CALL process_request_queue here
            // We only want to call it once per "loop"
            // otherwise the "loop" would grow into a "tree"
        }
    }
}