Cannot convert Tensorflow .pb frozen graph to tensorflow lite due to strange 'utf-8' codec error on Colab

Question

I have am ONNX model that I converted to tensorflow, that conversion went ahead without any problems, but now I want to convert this .pb file to tf lite using the following code

import tensorflow as tf

TF_PATH = "/content/tf_model/saved_model.pb" # where the forzen graph is stored
TFLITE_PATH = "./model.tflite"

# make a converter object from the saved tensorflow file
converter = tf.compat.v1.lite.TFLiteConverter.from_frozen_graph(TF_PATH,  # TensorFlow freezegraph .pb model file
                                                      input_arrays=['input_ids'], # name of input arrays as defined in torch.onnx.export function before.
                                                      output_arrays=['logits'], # name of output arrays defined in torch.onnx.export function before.
                                                      )


converter.experimental_new_converter = True

converter.target_spec.supported_ops = [tf.compat.v1.lite.OpsSet.TFLITE_BUILTINS,
                                       tf.compat.v1.lite.OpsSet.SELECT_TF_OPS]

tf_lite_model = converter.convert()
# Save the model.
with open(TFLITE_PATH, 'wb') as f:
    f.write(tf_lite_model)

But when I run this cell on Colab I get the error:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa3 in position 3: invalid start byte

And directs towards the line: converter = tf.compat.v1.lite.TFLiteConverter.from_frozen_graph().

I cant seem to figure out what is causing this..

Answer 1

Looks like your frozen_graph is not frozen graph but saved_model format. If I guess right all you need is to change conversion method: you are looking for convert from SavedModel

Assuming you are using TF2 and it will be:

import tensorflow as tf

TF_PATH = "/content/tf_model/" # where the saved_model is stored - but folder name
TFLITE_PATH = "./model.tflite"

# make a converter object from the saved tensorflow file
converter = tf.lite.TFLiteConverter.from_saved_model(TF_PATH)

converter.experimental_new_converter = True

converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS,
                                       tf.lite.OpsSet.SELECT_TF_OPS]

tf_lite_model = converter.convert()
# Save the model.
open(TFLITE_PATH, "wb").write(tf_lite_model)