swift - Espresso exception: "Invalid argument":general shape kernel while loading mlmodel

Question

I converted my mlmodel from tf.keras. The goal is to recognize handwritten text from the image When I run it using this code:

func performCoreMLImageRecognition(_ image: UIImage) {
     
        let model = try! HTRModel()
        // process input image
        let scale = image.scaledImage(200)
        let sized = scale?.resize(size: CGSize(width: 200, height: 50))
        let gray = sized?.rgb2GrayScale()
        
        guard let pixelBuffer = sized?.pixelBufferGray(width: 200, height: 50) else { fatalError("Cannot convert image to pixelBufferGray")}
        
        UIImageWriteToSavedPhotosAlbum(gray! ,
                                        self,
                                        #selector(self.didFinishSavingImage(_:didFinishSavingWithError:contextInfo:)),
                                        nil)
        
        let mlArray = try! MLMultiArray(shape: [1, 1], dataType: MLMultiArrayDataType.float32)
        let htrinput = HTRInput(image: pixelBuffer, label: mlArray)
        if let prediction = try? model.prediction(input: htrinput) {
            print(prediction)
        }
    }

I get the following error:

[espresso] [Espresso::handle_ex_plan] exception=Espresso exception: "Invalid argument": generic_reshape_kernel: Invalid bottom shape (64 12 1 1 1) for reshape to (768 50 -1 1 1) status=-6
2021-01-21 20:23:50.712585+0900 Guided Camera[7575:1794819] [coreml] Error computing NN outputs -6
2021-01-21 20:23:50.712611+0900 Guided Camera[7575:1794819] 
[coreml] Failure in -executePlan:error:.

Here is the model configuration The model ran perfectly fine. Where am I going wrong in this. I am not well versed with swift and need help. What does this error mean and How do I resolve this error?

question from:https://stackoverflow.com/questions/65840137/espresso-exception-invalid-argumentgeneral-shape-kernel-while-loading-mlmode

Answer 1

Sometimes during the conversion from Keras (or whatever) to Core ML, the converter doesn't understand how to handle certain operations, which results in a model that doesn't work.

In your case, there is a layer that outputs a tensor with shape (64, 12, 1, 1, 1) while there is a reshape layer that expects something that can be reshaped to (768, 50, -1, 1, 1).

You'll need to find out which layer does this reshape and then examine the Core ML model why it gets an input tensor that is not the correct size. Just because it works OK in Keras does not mean the conversion to Core ML was flawless.

You can examine the Core ML model with Netron, an open source model viewer.

(Note that 64x12 = 768, so the issue appears to be with the 50 in that tensor.)