python - OpenCV aruco detection fails on estimatePoseBoard()

Question

I'm trying to detect a 3x3 Aruco board.

The code runs until the params on line 33 are met, then line 35 throws the same error regardless of how many markers are in frame (i.e. between 1 and all 9)

The output is

width: 3
height: 3
marker_size: 0.05
marker_separation: 0.05
cameraMat: D:ArucoCameraCalibL515_camMat_0.37.dat
distCoeff: D:ArucoCameraCalibL515_distCoeffs_0.37.dat
output:
  width: 700
  height: 700
  margin: 10

[597.43121351   0.         317.54758687   0.         597.50401806
 264.76581981   0.           0.           1.        ]
[ 0.07009082  0.15721941  0.00925155 -0.00357965 -0.9931981 ]
None None
Traceback (most recent call last):
  File "d:ArucoCameraPosepose.py", line 35, in mainFunc
    success, rvec, tvec = cv.aruco.estimatePoseBoard(corners, ids, board, cameraMatrix, distCoeffs, rvec, tvec)
cv2.error: OpenCV(4.5.1) C:UsersappveyorAppDataLocalTemp1pip-req-build-i1s8y2i1opencvmodulescoresrcconvert_c.cpp:112: error: (-215:Assertion failed) src.size == dst.size && src.channels() == dst.channels() in function 'cvConvertScale'

The whole script is

import cv2 as cv
from omegaconf import DictConfig, OmegaConf
import numpy as np
import hydra

vid = cv.VideoCapture(1) 

@hydra.main(config_name='config')
def mainFunc(cfg: DictConfig) -> None:
    print(OmegaConf.to_yaml(cfg))

    # camera parameters are read from somewhere
    cameraMatrix = np.fromfile("D:ArucoCameraCalibL515_camMat_0.37.dat")
    print(cameraMatrix)
    distCoeffs = np.fromfile("D:ArucoCameraCalibL515_distCoeffs_0.37.dat")
    print(distCoeffs)

    # create the board
    markerDict = cv.aruco.getPredefinedDictionary(cv.aruco.DICT_4X4_50)
    board = cv.aruco.GridBoard_create(cfg.width,cfg.height, cfg.marker_size, cfg.marker_separation, markerDict)

    rvec = None
    tvec = None
    while(True):
        # get frame from camera
        ret, inputImg = vid.read() 
        gray = cv.cvtColor(inputImg, cv.COLOR_BGR2GRAY)

        # detect markers
        corners, ids, reject = cv.aruco.detectMarkers(gray, markerDict, cameraMatrix=cameraMatrix, distCoeff=distCoeffs)

        # if at least one marker detected
        if (len(corners) > 0) and (len(ids) > 0):
            print(rvec, tvec)
            success, rvec, tvec = cv.aruco.estimatePoseBoard(corners, ids, board, cameraMatrix, distCoeffs, rvec, tvec)

            if success > 0:
                gray = cv.aruco.drawAxis(gray, cameraMatrix, distCoeffs, rvec, tvec, 0.05)
                
        cv.imshow('frame', gray)
        #print(gray.shape)
        cv.waitKey(1)

if __name__ == "__main__":
    mainFunc()

I've also tried scripts written by others (Link) that claim to do the same thing, they fail in the same way on the same function call

Answer 1

Solved, if anyone else runs into this issue:

np.fromfile("D:ArucoCameraCalibL515_camMat_0.37.dat")

has the shape [9], not [3,3] as the matrix was before saving, so the camera matrix wasn't in the correct shape when passed into the function

Curiously cv.aruco.detectMarkers() does not throw any errors and returns normally when passed the incorrectly shaped matrix

Fixed by swapping to using np.save and np.load in both the camera calibration and this file, then redoing the calibration so the correct format was saved