node.js - AWS Lambda making video thumbnails

Question

I want make thumbnails from videos uploaded to S3, I know how to make it with Node.js and ffmpeg.

According to this forum post I can add libraries:

ImageMagick is the only external library that is currently provided by default, but you can include any additional dependencies in the zip file you provide when you create a Lambda function. Note that if this is a native library or executable, you will need to ensure that it runs on Amazon Linux.

But how can I put static ffmpeg binary on aws lambda?

And how can I call from Node.js this static binary (ffmpeg) with AWS Lambda?

I'm newbie with amazon AWS and Linux

Can anyone help me?

Answer 1

The process as outlined by Naveen is correct, but it glosses over a detail that can be pretty painful - including the ffmpeg binary in the zip and accessing it within your lambda function.

I just went through this, it went like this:

1. Include the ffmpeg static binary in your zipped lambda function package (I have a gulp task to copy this into the /dist every time it builds)
2. When your function is called, move the binary to a /tmp/ dir and chmod it to give yourself access (Update Feb 2017: it's reported that this is no longer necessary, re: @loretoparisi and @allen's answers).
3. update your PATH to include the ffmpeg executable (I used fluent-ffmpeg which lets you set two env vars to handle that more easily.

Let me know if more detail is necessary, I can update this answer.

The copy and chmod (step 2) is obviously not ideal.... would love to know if anyone's found a better way to handle this, or if this is typical for this architecture style.

(2nd Update, writing it before the first update b/c it's more relevant):

The copy + chmod step is no longer necessary, as @Allen pointed out – I'm executing ffmpeg in Lambda functions directly from /var/task/ with no trouble at this point. Be sure to chmod 755 whatever binaries before uploading them to Lambda (also as @Allen pointed out).

I'm no longer using fluent-ffmpeg to do the work. Rather, I'm updating the PATH to include the process.env['LAMBDA_TASK_ROOT'] and executing simple bash scripts.

At the top of your Lambda function:

process.env['PATH'] = process.env['PATH'] + "/" + process.env['LAMBDA_TASK_ROOT']

For an example that uses ffmpeg: lambda-pngs-to-mp4.

For a slew of useful lambda components: lambduh.

The below update left in for posterity, but no longer necessary:

UPDATE WITH MORE DETAIL:

I downloaded the static ffmpeg binary here. Amazon recommends booting up an EC2 and building a binary for your use on there, because that environment will be the same as the conditions Lambda runs on. Probably a good idea, but more work, and this static download worked for me.

I pulled only the ffmpeg binary into my project's to-be-archived /dist folder.

When you upload your zip to lambda, it lives at /var/task/. For whatever reason, I ran into access issues trying to use the binary at that location, and more issues trying to edit permissions on the file there. A quick work-around is to move the binary to /tmp/ and chmod permissions on it there.

In Node, you can run shell via a child_process. What I did looks like this:

require('child_process').exec(
  'cp /var/task/ffmpeg /tmp/.; chmod 755 /tmp/ffmpeg;',
  function (error, stdout, stderr) {
    if (error) {
      //handle error
    } else {
      console.log("stdout: " + stdout)
      console.log("stderr: " + stderr)
      //handle success
    }
  }
)

This much should give you an executable ffmpeg binary in your lambda function – but you still need to make sure it's on your $PATH.

I abandoned fluent-ffmpeg and using node to launch ffmpeg commands in favor of just launching a bash script out of node, so for me, I had to add /tmp/ to my path at the top of the lambda function:

process.env.PATH = process.env.PATH + ':/tmp/'

If you use fluent-ffmpeg, you can set the path to ffmpeg via:

process.env['FFMPEG_PATH'] = '/tmp/ffmpeg';

Somewhat related/shameless self-plug: I'm working on a set of modules to make building Lambda functions out of composable modules easier under the name Lambduh. Might save some time getting these things together. A quick example: handling this scenario with lambduh-execute would be as simple as:

promises.push(execute({
  shell: "cp /var/task/ffmpeg /tmp/.; chmod 755 /tmp/ffmpeg",
})

Where promises is an array of promises to be run.