Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
755 views
in Technique[技术] by (71.8m points)

c# - Basler USB Camera byte buffer to image conversion

I have Basler acA3800 USB camera.

            IGrabResult grabResult = camera.StreamGrabber.RetrieveResult(5000, TimeoutHandling.ThrowException);
            // Image grabbed successfully?
            if (grabResult.GrabSucceeded)
            {
                byte[] buffer = grabResult.PixelData as byte[];
                ImageWindow.DisplayImage(0, grabResult);

                pictureBox1.Image = ImageFromRawBgraArray(buffer,3840,2748,PixelFormat.Format8bppIndexed);
                MessageBox.Show(grabResult.PixelTypeValue.ToString());
            }

This code section shows image self window.

I have pixel data of captured image. Original image is good but when I convert it to Image, it corrupt. And here is my conversion function.

    public Image ImageFromRawBgraArray(byte[] arr, int width, int height, PixelFormat pixelFormat)
    {
        var output = new Bitmap(width, height, pixelFormat);
        var rect = new Rectangle(0, 0, width, height);
        var bmpData = output.LockBits(rect, ImageLockMode.ReadWrite, output.PixelFormat);

        // Row-by-row copy
        var arrRowLength = width * Image.GetPixelFormatSize(output.PixelFormat) / 8;
        var ptr = bmpData.Scan0;
        for (var i = 0; i < height; i++)
        {
            Marshal.Copy(arr, i * arrRowLength, ptr, arrRowLength);
            ptr += bmpData.Stride;
        }

        output.UnlockBits(bmpData);
        return output;
    }

Original image

Image from byte buffer

I think it is about pixel type. I have selected pixel format of PixelFormat.Format8bppIndexed. The others is not working.

                MessageBox.Show(grabResult.PixelTypeValue.ToString());

This messagebox gives me the pixeltype. and it says "BayerBG8". What does it mean? What should I do to get clear image?

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

The odd colours you are getting are because Format8bppIndexed is paletted, and you never edit the palette, meaning it retains the default generated Windows palette. But in your case, this palette is irrelevant, because the image is not an 8-bit indexed format; it needs to be processed to convert it to RGB.

A quick google for BayerBG8 got me this page. The Bayer section there shows it's a rather peculiar transformation to use specifically patterned indices on the image as R, G and B.

Bayer pattern

Wikipedia has a whole article on how this stuff is generally processed, but this YouTube video shows the basics:

Bayer to RGB Demosaicing

Note that this is a sliding window; for the first pixel, the colours are

R G
G B

but for the second pixel, they'll be

G R
B G

and for one row down, the first one will use

G B
R G

You'll end up with an image that is one pixel less wide and high than the given dimensions, since the last pixel on each row and all pixels on the last row won't have the neighbouring data needed to get their full pixel data. There are apparently more advanced algorithms to get around that, but for this method I'll just go over the basic sliding window method.

public static Byte[] BayerToRgb(Byte[] arr, ref Int32 width, ref Int32 height, ref Int32 stride, Boolean greenFirst, Boolean blueRowFirst)
{
    Int32 actualWidth = width - 1;
    Int32 actualHeight = height - 1;
    Int32 actualStride = actualWidth*3;
    Byte[] result = new Byte[actualStride*actualHeight];
    for (Int32 y = 0; y < actualHeight; y++)
    {
        Int32 curPtr = y*stride;
        Int32 resPtr = y*actualStride;
        Boolean blueRow = y % 2 == (blueRowFirst ? 0 : 1);
        for (Int32 x = 0; x < actualWidth; x++)
        {
            // Get correct colour components from sliding window
            Boolean isGreen = (x + y) % 2 == (greenFirst ? 0 : 1);
            Byte cornerCol1 = isGreen ? arr[curPtr + 1] : arr[curPtr];
            Byte cornerCol2 = isGreen ? arr[curPtr + stride] : arr[curPtr + stride + 1];
            Byte greenCol1 = isGreen ? arr[curPtr] : arr[curPtr + 1];
            Byte greenCol2 = isGreen ? arr[curPtr + stride + 1] : arr[curPtr + stride];
            Byte blueCol = blueRow ? cornerCol1 : cornerCol2;
            Byte redCol = blueRow ? cornerCol2 : cornerCol1;
            // 24bpp RGB is saved as [B, G, R].
            // Blue
            result[resPtr + 0] = blueCol;
            // Green
            result[resPtr + 1] = (Byte) ((greenCol1 + greenCol2)/2);
            // Red
            result[resPtr + 2] = redCol;
            curPtr++;
            resPtr+=3;
        }
    }
    height = actualHeight;
    width = actualWidth;
    stride = actualStride;
    return result;
}

The parameters greenFirst and blueRowFirst indicate whether green is the first encountered pixel on the image, and whether the blue pixels are on the first or second row. For your "BG" format, both of these should be false.

From the result of this, with the adjusted width, height and stride, you can convert that to a new image using the method you already used, but with Format24bppRgb as pixel format.

Personally I use a somewhat more advanced method that takes the input stride into account and can handle indexed content. If you're interested, that method can be found here.

Note that the demosaicing method above is very basic, and will show many of the expected artifacts. There are more advanced methods out there to analyse the data and get more accurate results based on how the image was taken, but it'll probably cost you quite some research to figure all that out and implement it yourself.

Here's a little test I did, starting from a Bayer-filtered image I found online (first image) which I converted to an 8-bit array (shown here as grayscale; second image). As you can see, my own demosaicing (third image) is far less accurate than the corrected version they got out of it (fourth image), and, notably, is one pixel smaller and thus shows a white border.

Bayer demosaicing test

(Note that, unlike the examples above, this image starts with a green pixel, meaning the parameters to decode it had to be adjusted)


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...