node.js - How to wait for async actions inside AWS Lambda?

Question

I am trying to process uploaded file in S3. Since getObject is asyncronous main function ends before processing is done, and AWS kills lambda in 3-4 seconds.

Even worse, processing method also has async operations in it - it makes http calls.

On high level, my code looks like:

exports.handler = function(event, context) {
    // Get the object from the event and show its content type
    var bucket = event.Records[0].s3.bucket.name;
    var key = event.Records[0].s3.object.key;
    var params = {
        Bucket: bucket,
        Key: key
    };
    s3.getObject(params, function(err, data) {
        if (err) {
             ...
        } else {
            processFile(data.Body.toString(), 0);
            console.log("ok");
        }
    });
    //need to wait here till processFile is done
};

processFile = function(content, start) {
  ... build url to call
  http.get(url, function(res) {  
    console.log("Got response: " + res.statusCode + ");
    processFile(content, start + 1);
  });
}

I find out that there is async in nodejs but it is not included by amazon; Both require('async') or require('sleep) causes errors.

Lambda timeout configured to 60 seconds, but it exits in 3-4 seconds.

Answer 1

The life of a dev is constantly changing and we now have NodeJS 8 on lambda. For anyone looking at this now check out:

Lambda node 8.10 vs node 6.10 comparison: https://aws.amazon.com/blogs/compute/node-js-8-10-runtime-now-available-in-aws-lambda/

Basics of JS async: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function

Even more aws sdk examples: https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/using-promises.html

Details on wtf the .promise() method is in the first link: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Request.html#promise-property

Here is my take at a basic example (try pasting into your own lambda):

exports.handler = async (event) => {    
    function wait(){
        return new Promise((resolve, reject) => {
            setTimeout(() => resolve("hello"), 2000)
        });
    }
    
    console.log(await wait());
    console.log(await wait());
    console.log(await wait());
    console.log(await wait());
    console.log(await wait());
    console.log(await wait());
    
    return 'exiting'
};