Overlay without major performance hit?

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Overlay without major performance hit?

Michael Tyson
Hello,

I’m trying to use cairooverlay to add some content to live video frames displayed on a Raspberry Pi.

I’ve not been able to get video frames to appear onscreen while drawing from udpsrc, so I did some testing using get-inspect and a file-based test video:

gst-launch-1.0 filesrc location=big_buck_bunny_720p_H264_AAC_25fps_3400K.MP4 ! qtdemux ! h264parse ! omxh264dec ! autovideoconvert ! cairooverlay ! autovideoconvert ! eglglessink

I discovered that including cairooverlay in the video pipeline introduces a massive performance hit at 720p: The video playback goes from the video’s native framerate to about 1 or 2 frames per second. I guess cairooverlay doesn’t work with the zero-copy stuff happening between omxh264dec and eglglessink.

Curious about using appsrc and videomixer instead, I tried another pipeline, using videotestsrc (in place of appsrc) and a videomixer:

gst-launch-1.0 filesrc location=big_buck_bunny_720p_H264_AAC_25fps_3400K.MP4 ! qtdemux ! h264parse ! omxh264dec ! autovideoconvert ! videomixer name=mix !  eglglessink sync=false videotestsrc ! mix.

The same thing happened: Frame rate dropped below usable values.

I’m wondering: Is there any way, using the Gstreamer framework, that one can achieve acceptable frame rates while applying an overlay?

Cheers,
Michael

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Overlay without major performance hit?

adrien_sch
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: Overlay without major performance hit?

Michael Tyson
Ah, I was wondering about using OpenGL. I want to render just a few pieces of text, plus a line drawing, so OpenGL rendering would suffice.

I was looking through the eglglessink source to see if there was a convenient way to access the egl context. Would this be the path forward, or is there another element type that one would use? glfilterapp, perhaps? Would that retain the performance of the omxh264dec/eglglessink combo? Or would it be better (or safer, perhaps, making less assumptions) to gain access to eglglessink’s egl context?

Cheers!
Michael


On 15 Mar 2014, at 8:34 pm, Adrien Schwartzentruber <[hidden email]> wrote:

One of the limitation with the cairooverlay element is that it require RGB format, so it will always downgrade your pipeline speed (if convert is needed, that is usually the case). This fact, is due to the under cairo image format limitation.

I think that the solution depends of what king of content you want to add to the stream. How complex ? Static ? 

I never tried, but I think that you can compose images with opengl element. If I was you, I will try to look in this direction.  


On Sat, Mar 15, 2014 at 8:12 AM, Michael Tyson <[hidden email]> wrote:
Hello,

I’m trying to use cairooverlay to add some content to live video frames displayed on a Raspberry Pi.

I’ve not been able to get video frames to appear onscreen while drawing from udpsrc, so I did some testing using get-inspect and a file-based test video:

gst-launch-1.0 filesrc location=big_buck_bunny_720p_H264_AAC_25fps_3400K.MP4 ! qtdemux ! h264parse ! omxh264dec ! autovideoconvert ! cairooverlay ! autovideoconvert ! eglglessink

I discovered that including cairooverlay in the video pipeline introduces a massive performance hit at 720p: The video playback goes from the video’s native framerate to about 1 or 2 frames per second. I guess cairooverlay doesn’t work with the zero-copy stuff happening between omxh264dec and eglglessink.

Curious about using appsrc and videomixer instead, I tried another pipeline, using videotestsrc (in place of appsrc) and a videomixer:

gst-launch-1.0 filesrc location=big_buck_bunny_720p_H264_AAC_25fps_3400K.MP4 ! qtdemux ! h264parse ! omxh264dec ! autovideoconvert ! videomixer name=mix !  eglglessink sync=false videotestsrc ! mix.

The same thing happened: Frame rate dropped below usable values.

I’m wondering: Is there any way, using the Gstreamer framework, that one can achieve acceptable frame rates while applying an overlay?

Cheers,
Michael

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Overlay without major performance hit?

Mailing List SVR
In reply to this post by Michael Tyson
Hi,

I suggest to use an hw accelerated overlay, take a look here:

https://github.com/popcornmix/omxplayer/blob/master/SubtitleRenderer.cpp#L391

eglglessink use 0 as layer value, you have to use a value > 0 to display
the overlay on the top of the one created by eglglessink,

I'm not aware of any gstreamer element that do this, you have to create
the overlay manually or create a new gstreamer element for that

Nicola


Il 15/03/2014 08:12, Michael Tyson ha scritto:

> Hello,
>
> I’m trying to use cairooverlay to add some content to live video frames displayed on a Raspberry Pi.
>
> I’ve not been able to get video frames to appear onscreen while drawing from udpsrc, so I did some testing using get-inspect and a file-based test video:
>
> gst-launch-1.0 filesrc location=big_buck_bunny_720p_H264_AAC_25fps_3400K.MP4 ! qtdemux ! h264parse ! omxh264dec ! autovideoconvert ! cairooverlay ! autovideoconvert ! eglglessink
>
> I discovered that including cairooverlay in the video pipeline introduces a massive performance hit at 720p: The video playback goes from the video’s native framerate to about 1 or 2 frames per second. I guess cairooverlay doesn’t work with the zero-copy stuff happening between omxh264dec and eglglessink.
>
> Curious about using appsrc and videomixer instead, I tried another pipeline, using videotestsrc (in place of appsrc) and a videomixer:
>
> gst-launch-1.0 filesrc location=big_buck_bunny_720p_H264_AAC_25fps_3400K.MP4 ! qtdemux ! h264parse ! omxh264dec ! autovideoconvert ! videomixer name=mix !  eglglessink sync=false videotestsrc ! mix.
>
> The same thing happened: Frame rate dropped below usable values.
>
> I’m wondering: Is there any way, using the Gstreamer framework, that one can achieve acceptable frame rates while applying an overlay?
>
> Cheers,
> Michael
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Overlay without major performance hit?

Michael Tyson
Oh, that’s perfect! Thank you, Nicola - just what I need.



On 15 Mar 2014, at 10:38 pm, Mailing List SVR <[hidden email]> wrote:

> Hi,
>
> I suggest to use an hw accelerated overlay, take a look here:
>
> https://github.com/popcornmix/omxplayer/blob/master/SubtitleRenderer.cpp#L391
>
> eglglessink use 0 as layer value, you have to use a value > 0 to display the overlay on the top of the one created by eglglessink,
>
> I'm not aware of any gstreamer element that do this, you have to create the overlay manually or create a new gstreamer element for that
>
> Nicola
>
>
> Il 15/03/2014 08:12, Michael Tyson ha scritto:
>> Hello,
>>
>> I’m trying to use cairooverlay to add some content to live video frames displayed on a Raspberry Pi.
>>
>> I’ve not been able to get video frames to appear onscreen while drawing from udpsrc, so I did some testing using get-inspect and a file-based test video:
>>
>> gst-launch-1.0 filesrc location=big_buck_bunny_720p_H264_AAC_25fps_3400K.MP4 ! qtdemux ! h264parse ! omxh264dec ! autovideoconvert ! cairooverlay ! autovideoconvert ! eglglessink
>>
>> I discovered that including cairooverlay in the video pipeline introduces a massive performance hit at 720p: The video playback goes from the video’s native framerate to about 1 or 2 frames per second. I guess cairooverlay doesn’t work with the zero-copy stuff happening between omxh264dec and eglglessink.
>>
>> Curious about using appsrc and videomixer instead, I tried another pipeline, using videotestsrc (in place of appsrc) and a videomixer:
>>
>> gst-launch-1.0 filesrc location=big_buck_bunny_720p_H264_AAC_25fps_3400K.MP4 ! qtdemux ! h264parse ! omxh264dec ! autovideoconvert ! videomixer name=mix !  eglglessink sync=false videotestsrc ! mix.
>>
>> The same thing happened: Frame rate dropped below usable values.
>>
>> I’m wondering: Is there any way, using the Gstreamer framework, that one can achieve acceptable frame rates while applying an overlay?
>>
>> Cheers,
>> Michael
>>
>> _______________________________________________
>> gstreamer-devel mailing list
>> [hidden email]
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Overlay without major performance hit?

Nicolas Dufresne-3
In reply to this post by Michael Tyson
Le samedi 15 mars 2014 à 18:12 +1100, Michael Tyson a écrit :
> Hello,
>
> I’m trying to use cairooverlay to add some content to live video frames displayed on a Raspberry Pi.

You could give a try to the integration of the resize component right
after the decoder. I think it's not yet ready for prime time, and ti
goes exactly into those lines, which is to convert to RGB and scale
before putting into cairo context (in Webkit).

cheers,
Nicolas

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (205 bytes) Download Attachment