android - MediaPipe how to use the BoxTracking aar?

Question

Being new to MediaPipe, I am not familiar with concept of graph, node, subgraph etc.

After building an aar file of BoxTracking, unable to run it within a Android Studio gradle based project due to some unknown input and output parameters required by model

On comparing HandTracking graph and BoxTracking graph using the visualizer tool and with a working project of HandTracking with aar file added as lib, I added new required input streams and side packets as seen in the graph.

Results are always some errors, mainly due to something wrong in inputs or BoxTracking being a subgraph which is used directly. How to know which input is required and data type of input to run this?

2021-02-05 21:15:23.477 22514-22564/com.example.mediapipemultihandstrackingapp E/FrameProcessor: Mediapipe error: 
com.google.mediapipe.framework.MediaPipeException: internal: Graph has errors: 
Calculator::Open() for node "objectdetectionsubgraphgpu__TfLiteInferenceCalculator" failed: ; could not read asset: ssdlite_object_detection.tfliteer_util.cc:158) 
    at com.google.mediapipe.framework.Graph.nativeMovePacketToInputStream(Native Method)
    at com.google.mediapipe.framework.Graph.addConsumablePacketToInputStream(Graph.java:360)
    at com.google.mediapipe.components.FrameProcessor.onNewFrame(FrameProcessor.java:442)
    at com.google.mediapipe.components.ExternalTextureConverter$RenderThread.renderNext(ExternalTextureConverter.java:364)
    at com.google.mediapipe.components.ExternalTextureConverter$RenderThread.lambda$onFrameAvailable$0$ExternalTextureConverter$RenderThread(ExternalTextureConverter.java:309)
    at com.google.mediapipe.components.-$$Lambda$ExternalTextureConverter$RenderThread$Y1vV_XyLsWZ0ebOvq-iwjQ0H3Sw.run(Unknown Source:4)
    at android.os.Handler.handleCallback(Handler.java:883)
    at android.os.Handler.dispatchMessage(Handler.java:100)
    at android.os.Looper.loop(Looper.java:237)
    at com.google.mediapipe.glutil.GlThread.run(GlThread.java:141)

question from:https://stackoverflow.com/questions/66066560/mediapipe-how-to-use-the-boxtracking-aar

Answer 1

Datatypes required as input and output was not included in the default build, the build configurations has to be modified to include box_tracker.proto and its dependencies.

https://github.com/google/mediapipe/issues/1624