CNN segmentation using Caffe, OpenCV and C#

Question

I have a project in C# where I need to use image segmentation. I learned about DNN and OpenCV mostly from pyimagesearch.com (Thanks Adrian! Great site!) However, I do not understand how should I extract individual segments from the result. Here is the part of the code in python:

blob = cv2.dnn.blobFromImage(frame, 1 / 255.0, (1024, 512), 0, swapRB=True, crop=False)
net.setInput(blob)
start = time.time()
output = net.forward()
end = time.time()
# infer the total number of classes along with the spatial
# dimensions of the mask image via the shape of the output array
(numClasses, height, width) = output.shape[1:4]
# our output class ID map will be num_classes x height x width in
# size, so we take the argmax to find the class label with the
# largest probability for each and every (x, y)-coordinate in the
# image
classMap = np.argmax(output[0], axis=0)
# given the class ID map, we can map each of the class IDs to its
# corresponding color
mask = COLORS[classMap]
# resize the mask such that its dimensions match the original size
# of the input frame
mask = cv2.resize(mask, (frame.shape[1], frame.shape[0]), interpolation=cv2.INTER_NEAREST)
# perform a weighted combination of the input frame with the mask
# to form an output visualization
output = ((0.3 * frame) + (0.7 * mask)).astype("uint8")

I'm able to create the net, load the .caffemodel, create blob and run "forward()" But I do not know how to write an equivalent of

(numClasses, height, width) = output.shape[1:4]
classMap = np.argmax(output[0], axis=0)
mask = COLORS[classMap]

in C# (I'm using EMGU wrapper for OpenCV)

And here is my code:

Emgu.CV.Dnn.Net segNet = Emgu.CV.Dnn.DnnInvoke.ReadNetFromCaffe("segmentation_deploy.prototxt", "fcn8s-heavy-pascal.caffemodel");
Mat rawImage = Emgu.CV.CvInvoke.Imread(_hideImagePath, ImreadModes.AnyColor);
double scale = 1/255;
Mat blob = Emgu.CV.Dnn.DnnInvoke.BlobFromImage(rawImage, scale, rawImage.Size, new MCvScalar(104.00698793, 116.66876762, 122.67891434), true, false);
segNet.SetInput(blob);
Mat results = segNet.Forward();
Mat tmpMat = results.Reshape(0, results.SizeOfDimension[2]);
ImageViewer vTL1 = new ImageViewer(tmpMat, "tmpMat");
vTL1.Show();

It shows only wide black rectangle. Thanks in advance for any help.

question from:https://stackoverflow.com/questions/65875027/cnn-segmentation-using-caffe-opencv-and-c-sharp

Answer 1

Waitting for answers