Quantcast

OpenCv Mat to GstBuffer -> appsrc, possible buffer underflow?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

OpenCv Mat to GstBuffer -> appsrc, possible buffer underflow?

borisRobson
This post was updated on .
Hi all

I have a project which aims to de-warp a fisheye camera, using two gstreamer pipelines with opencv processing in between:
|   input-pipe                              |   OpenCV processing           |  output-pipe
|  rtsprc -> decodebin -> appsink  |   gst-buf -> cvMat -> gstbuf |  appsrc -> videoparse -> videoconvert ->   xvimagesink

I can save the image from the cvMat and the result is as I expect, however the image shown in xvimagesink is grayscale and constantly shifting upwards and to the left. (Best I could do on description there)




The code where I suspect the problem is:
(conv is the Mat shown in the attached image)
//get outputize & allocate mem block
        gint outSize = conv.cols * conv.rows * inBpp;
        GstMemory *outMem = gst_allocator_alloc(NULL, outSize, NULL);

        //assign frame data to mem
        GstMapInfo outMap;
        if(!gst_memory_map(outMem, &outMap, GST_MAP_WRITE))
        {
                g_printerr("could not map outMem\n");
                return;
        }

        memcpy(outMap.data, conv.data, outMap.size);

        gst_memory_unmap(outMem, &outMap);

        GstBuffer *outbuf;
        outbuf = gst_buffer_new();

        gst_buffer_insert_memory(outbuf, -1,outMem);

        GstVideoMeta *outmeta = new GstVideoMeta();
        outmeta = gst_buffer_add_video_meta(outbuf,GST_VIDEO_FRAME_FLAG_NONE, GST_VIDEO_FORMAT_RGB, conv.cols, conv.rows );

        //set timestamp values
        GST_BUFFER_DURATION(outbuf) = (GstClockTime)40000000;
        GST_BUFFER_DTS(outbuf) = GST_CLOCK_TIME_NONE;
        GST_BUFFER_PTS(outbuf) = timestamp;
        timestamp += GST_BUFFER_DURATION(outbuf);

        GST_BUFFER_FLAG_SET(outbuf, GST_BUFFER_FLAG_LIVE);

        //ref buffer to give copy to appsrc
        gst_buffer_ref(outbuf);

        GstFlowReturn ret;
        ret = gst_app_src_push_buffer((GstAppSrc*)asrc, outbuf);
        if(ret != GST_FLOW_OK)
        {
                g_printerr("could not push buffer\n");
                g_printerr("ret enum: %i\n", ret);
        }

        //dec. ref count so that we can edit data on next run
        gst_buffer_unref(outbuf);


I have tried adding extra conversion stages in my out-pipe but to no effect.
Has anyone had experience with this before or is able to spot some rookie error I'm making?
Thanks
Brandon


                       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OpenCv Mat to GstBuffer -> appsrc, possible buffer underflow?

Mailing List SVR
Hi,

you can use the gstreamer dewarp plugin available in git master

https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/opencv/gstdewarp.cpp

please report here if you have any issues with your fisheye image,

Nicola


Il 03/02/2017 14:06, borisRobson ha scritto:

> Hi all
>
> I have a project which aims to de-warp a fisheye camera, using two gstreamer
> pipelines with opencv processing in between:
> |   input-pipe                              |   OpenCV processing
> |  output-pipe
> |  rtsprc -> decodebin -> appsink  |   gst-buf -> cvMat -> gstbuf |  appsrc
> -> videoparse -> videoconvert ->
> xvimagesink
>
> I can save the image from the cvMat and the result is as I expect, however
> the image shown in xvimagesink is grayscale and constantly shifting upwards
> and to the left. (Best I could do on description there)
>
> <http://gstreamer-devel.966125.n4.nabble.com/file/n4681701/xvimagesink.png>
> <http://gstreamer-devel.966125.n4.nabble.com/file/n4681701/dewarped.png>
>
> The code where I suspect the problem is:
> (conv is the Mat shown in the attached image)
> //get outputize & allocate mem block
> gint outSize = conv.cols * conv.rows * inBpp;
> GstMemory *outMem = gst_allocator_alloc(NULL, outSize, NULL);
>
> //assign frame data to mem
> GstMapInfo outMap;
> if(!gst_memory_map(outMem, &outMap, GST_MAP_WRITE))
> {
> g_printerr("could not map outMem\n");
> return;
> }
>
> memcpy(outMap.data, conv.data, outMap.size);
>
> gst_memory_unmap(outMem, &outMap);
>
> GstBuffer *outbuf;
> outbuf = gst_buffer_new();
>
> gst_buffer_insert_memory(outbuf, -1,outMem);
>
> GstVideoMeta *outmeta = new GstVideoMeta();
> outmeta = gst_buffer_add_video_meta(outbuf,GST_VIDEO_FRAME_FLAG_NONE,
> GST_VIDEO_FORMAT_RGB, conv.cols, conv.rows );
>
> //set timestamp values
> GST_BUFFER_DURATION(outbuf) = (GstClockTime)40000000;
> GST_BUFFER_DTS(outbuf) = GST_CLOCK_TIME_NONE;
> GST_BUFFER_PTS(outbuf) = timestamp;
> timestamp += GST_BUFFER_DURATION(outbuf);
>
> GST_BUFFER_FLAG_SET(outbuf, GST_BUFFER_FLAG_LIVE);
>
> //ref buffer to give copy to appsrc
> gst_buffer_ref(outbuf);
>
> GstFlowReturn ret;
> ret = gst_app_src_push_buffer((GstAppSrc*)asrc, outbuf);
> if(ret != GST_FLOW_OK)
> {
> g_printerr("could not push buffer\n");
> g_printerr("ret enum: %i\n", ret);
> }
>
> //dec. ref count so that we can edit data on next run
> gst_buffer_unref(outbuf);
>
>
> I have tried adding extra conversion stages in my out-pipe but to no effect.
> Has anyone had experience with this before or is able to spot some rookie
> error I'm making?
> Thanks
> Brandon
>
>
>                          
>
>
>
>
> --
> View this message in context: http://gstreamer-devel.966125.n4.nabble.com/OpenCv-Mat-to-GstBuffer-appsrc-possible-buffer-underflow-tp4681701.html
> Sent from the GStreamer-devel mailing list archive at Nabble.com.
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OpenCv Mat to GstBuffer -> appsrc, possible buffer underflow?

borisRobson
Hi Nicola,

I'm currently limited to v1.2.4 unfortunately. I came across that plugin yesterday and thought I'd hit the jackpot until I tried to compile it.

Thanks for the quick reply though
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OpenCv Mat to GstBuffer -> appsrc, possible buffer underflow?

filnet
In reply to this post by borisRobson
Hi,

OpenCV will often convert images to grey scale before applying some algorithm on it.
The shifting is probablu due to a stride issue.


Did you look at the various opencv plugins in gst-plugins-bad ?
It has all the code samples you need to convert cvMat to GstBuffer.

I am working on a OpenCV based camera calibration/undistort plugin for gstreamer.

While working on that plugin, I also struggled with gstreamer vs opencv image formats.



Le Vendredi 3 février 2017 14h09, borisRobson <[hidden email]> a écrit :


Hi all

I have a project which aims to de-warp a fisheye camera, using two gstreamer
pipelines with opencv processing in between:
|  input-pipe                              |  OpenCV processing         
|  output-pipe
|  rtsprc -> decodebin -> appsink  |  gst-buf -> cvMat -> gstbuf |  appsrc
-> videoparse -> videoconvert ->                                                                                               
xvimagesink

I can save the image from the cvMat and the result is as I expect, however
the image shown in xvimagesink is grayscale and constantly shifting upwards
and to the left. (Best I could do on description there)

<http://gstreamer-devel.966125.n4.nabble.com/file/n4681701/xvimagesink.png>
<http://gstreamer-devel.966125.n4.nabble.com/file/n4681701/dewarped.png>

The code where I suspect the problem is:
(conv is the Mat shown in the attached image)
//get outputize & allocate mem block
    gint outSize = conv.cols * conv.rows * inBpp;
    GstMemory *outMem = gst_allocator_alloc(NULL, outSize, NULL);   

    //assign frame data to mem
    GstMapInfo outMap;
    if(!gst_memory_map(outMem, &outMap, GST_MAP_WRITE))
    {
        g_printerr("could not map outMem\n");
        return;
    }

    memcpy(outMap.data, conv.data, outMap.size);

    gst_memory_unmap(outMem, &outMap);

    GstBuffer *outbuf;
    outbuf = gst_buffer_new();

    gst_buffer_insert_memory(outbuf, -1,outMem);

    GstVideoMeta *outmeta = new GstVideoMeta();
    outmeta = gst_buffer_add_video_meta(outbuf,GST_VIDEO_FRAME_FLAG_NONE,
GST_VIDEO_FORMAT_RGB, conv.cols, conv.rows );

    //set timestamp values
    GST_BUFFER_DURATION(outbuf) = (GstClockTime)40000000;
    GST_BUFFER_DTS(outbuf) = GST_CLOCK_TIME_NONE;
    GST_BUFFER_PTS(outbuf) = timestamp;
    timestamp += GST_BUFFER_DURATION(outbuf);

    GST_BUFFER_FLAG_SET(outbuf, GST_BUFFER_FLAG_LIVE);

    //ref buffer to give copy to appsrc
    gst_buffer_ref(outbuf);

    GstFlowReturn ret;
    ret = gst_app_src_push_buffer((GstAppSrc*)asrc, outbuf);
    if(ret != GST_FLOW_OK)
    {
        g_printerr("could not push buffer\n");
        g_printerr("ret enum: %i\n", ret);
    }

    //dec. ref count so that we can edit data on next run
    gst_buffer_unref(outbuf);


I have tried adding extra conversion stages in my out-pipe but to no effect.
Has anyone had experience with this before or is able to spot some rookie
error I'm making?
Thanks
Brandon


                       




--
View this message in context: http://gstreamer-devel.966125.n4.nabble.com/OpenCv-Mat-to-GstBuffer-appsrc-possible-buffer-underflow-tp4681701.html
Sent from the GStreamer-devel mailing list archive at Nabble.com.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel



_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OpenCv Mat to GstBuffer -> appsrc, possible buffer underflow?

borisRobson
Just had a look into stride (I'm new to this level of detail of videoframes) and managed to fix my app with one line of code!

Thanks
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OpenCv Mat to GstBuffer -> appsrc, possible buffer underflow?

Mikl

Hello,


Can you give more information about your solution?


What line?

Where?


For future readers. :)


Mikl


From: gstreamer-devel <[hidden email]> on behalf of borisRobson <[hidden email]>
Sent: Monday, February 6, 2017 9:36:30 AM
To: [hidden email]
Subject: Re: OpenCv Mat to GstBuffer -> appsrc, possible buffer underflow?
 
Just had a look into stride (I'm new to this level of detail of videoframes)
and managed to fix my app with one line of code!

Thanks



--
View this message in context: http://gstreamer-devel.966125.n4.nabble.com/OpenCv-Mat-to-GstBuffer-appsrc-possible-buffer-underflow-tp4681701p4681740.html
Sent from the GStreamer-devel mailing list archive at Nabble.com.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OpenCv Mat to GstBuffer -> appsrc, possible buffer underflow?

borisRobson
I can't stand when people don't post their solutions, didn't even realise I forgot to!

I create two arrays initialised to 0 :
        offset = new gsize[4] {0};
        stride = new gint[4] {0};

Then later when creating the buffer:
        GstVideoMeta *outmeta = new GstVideoMeta();

        ostrm->stride[0] = conv.cols * inBpp;
        outmeta = gst_buffer_add_video_meta_full(outbuf,GST_VIDEO_FRAME_FLAG_NONE, GST_VIDEO_FORMAT_RGB, conv.cols, conv.rows, 1, ostrm->offset, ostrm->stride );

Set the stride value to image width * bytes per plane.

More info on stride:
https://msdn.microsoft.com/en-us/library/windows/desktop/aa473780(v=vs.85).aspx

Hope this helps someone eventually
Loading...