java - How do I get the raw Android camera buffer in C using JNI?

Question

I've been searching Google and StackOverflow exhaustively and cannot find this. Maybe I'm missing something obvious. Thanks!

(This is because the Java implementation of the preview callback [even with buffer] is too inefficient.)

Answer 1

I made a little investigation on topic. This presentation (from p.277, Chinese) helped a lot.

Camera preview call chain

As others mentioned, you can get a buffer using a Camera.setPreviewCallback method.
Here's how it happens there (a verbose version):

1. User calls Camera.startPreview() which is a native function.
2. android_hardware_Camera_startPreview calls startPreview method of C++ Camera class.
3. Camera calls a startPreview method of ICamera interface
4. ICamera makes an IPC call to remote client.
5. It calls a setCameraMode method of CameraService class.
6. CameraService sets a window to display a preview and calls a startPreview method of CameraHardwareInterface class.
7. The latter tries to call a start_preview method on particular camera_device_t device.
   I didn't looked up further but it should perform a call to the driver.
8. When image arrives, dataCallback of CameraService is invoked.
9. It passes data to handlePreviewData method of client.
10. Client either copies the buffer or sends it directly to the ICameraClient.
11. ICameraClient sends it over IPC to the Camera.
12. Camera calls a registered listener and passes buffer to JNI.
13. It invokes a callback in Java class. If user provided a buffer with Camera.addCallbackBuffer then it copies to the buffer first.
14. Finally Java class Camera handles the message and invokes a onPreviewFrame method of Camera.PreviewCallback.

As you can see 2 IPC calls were invoked and buffer was copied at least twice on steps 10, 11. First instance of raw buffer which is returned by camera_device_t is hosted in another process and you cannot access it due to security checks in CameraService.

Preview surface

However, when you set a preview surface using either Camera.setPreviewTexture or Camera.setPreviewDisplay it is be passed directly to the camera device and refreshed in realtime without participation of all the chain above. As it's documentation says:

Handle onto a raw buffer that is being managed by the screen compositor.

Java class Surface has a method to retrieve it's contents:

public static native Bitmap screenshot(int width, int height, int minLayer, int maxLayer);

But this API is hidden. See i.e. this question for a way to use it.