Combining 2 Video Streams Side by Side, Getting Choppy Results

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Combining 2 Video Streams Side by Side, Getting Choppy Results

Jim Ruxton

I am trying to combine 2 video streams. One from my laptop camera and one from an external webcam. I'm using the following pipeline and getting very choppy results. Neither my CPU or GPU seem to be working hard. Any ideas how to make this less choppy?

./gst-launch-1.0 v4l2src device=/dev/video4 ! videoscale ! videoconvert ! video/x-raw,format=YUY2, framerate=30/1, width=640, height=480! alpha alpha=1 ! videobox border-alpha=0 left=-640 ! 
videomixer name=Mix ! videoconvert ! autovideosink v4l2src device=/dev/video2 ! videoscale ! videoconvert ! video/x-raw, format=YUY2, framerate=30/1, width=640, height=480! alpha alpha=1 ! videobox border-alpha=0 right=-640 ! Mix.

Thanks,

Jim



_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Combining 2 Video Streams Side by Side, Getting Choppy Results

Jim Ruxton

Just discovered that putting sync=false after autovideosink solved my problem.

Jim

On 2021-02-16 2:50 a.m., Jim Ruxton wrote:

I am trying to combine 2 video streams. One from my laptop camera and one from an external webcam. I'm using the following pipeline and getting very choppy results. Neither my CPU or GPU seem to be working hard. Any ideas how to make this less choppy?

./gst-launch-1.0 v4l2src device=/dev/video4 ! videoscale ! videoconvert ! video/x-raw,format=YUY2, framerate=30/1, width=640, height=480! alpha alpha=1 ! videobox border-alpha=0 left=-640 ! 
videomixer name=Mix ! videoconvert ! autovideosink v4l2src device=/dev/video2 ! videoscale ! videoconvert ! video/x-raw, format=YUY2, framerate=30/1, width=640, height=480! alpha alpha=1 ! videobox border-alpha=0 right=-640 ! Mix.

Thanks,

Jim



_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Combining 2 Video Streams Side by Side, Getting Choppy Results

Ian Davidson

I don't see any queue in your pipeline. A queue causes the pipeline to have a new thread - so all of your pipeline is one thread and is therefore only using one processor.  Experiment with putting a queue after each videoscale.

On 16/02/2021 09:02, Jim Ruxton wrote:

Just discovered that putting sync=false after autovideosink solved my problem.

Jim

On 2021-02-16 2:50 a.m., Jim Ruxton wrote:

I am trying to combine 2 video streams. One from my laptop camera and one from an external webcam. I'm using the following pipeline and getting very choppy results. Neither my CPU or GPU seem to be working hard. Any ideas how to make this less choppy?

./gst-launch-1.0 v4l2src device=/dev/video4 ! videoscale ! videoconvert ! video/x-raw,format=YUY2, framerate=30/1, width=640, height=480! alpha alpha=1 ! videobox border-alpha=0 left=-640 ! 
videomixer name=Mix ! videoconvert ! autovideosink v4l2src device=/dev/video2 ! videoscale ! videoconvert ! video/x-raw, format=YUY2, framerate=30/1, width=640, height=480! alpha alpha=1 ! videobox border-alpha=0 right=-640 ! Mix.

Thanks,

Jim



_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

  


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Combining 2 Video Streams Side by Side, Getting Choppy Results

Mathieu Duponchelle
That is a good point, you should also use compositor instead of videomixer,
especially for live streams. In any case, using sync=false on the sinks is not
a desirable solution, rather an admission of defeat :)

Cheers,

--
Mathieu Duponchelle · https://www.centricular.com

On 2/16/21 11:20 AM, Ian Davidson wrote:

I don't see any queue in your pipeline. A queue causes the pipeline to have a new thread - so all of your pipeline is one thread and is therefore only using one processor.  Experiment with putting a queue after each videoscale.

On 16/02/2021 09:02, Jim Ruxton wrote:

Just discovered that putting sync=false after autovideosink solved my problem.

Jim

On 2021-02-16 2:50 a.m., Jim Ruxton wrote:

I am trying to combine 2 video streams. One from my laptop camera and one from an external webcam. I'm using the following pipeline and getting very choppy results. Neither my CPU or GPU seem to be working hard. Any ideas how to make this less choppy?

./gst-launch-1.0 v4l2src device=/dev/video4 ! videoscale ! videoconvert ! video/x-raw,format=YUY2, framerate=30/1, width=640, height=480! alpha alpha=1 ! videobox border-alpha=0 left=-640 ! 
videomixer name=Mix ! videoconvert ! autovideosink v4l2src device=/dev/video2 ! videoscale ! videoconvert ! video/x-raw, format=YUY2, framerate=30/1, width=640, height=480! alpha alpha=1 ! videobox border-alpha=0 right=-640 ! Mix.

Thanks,

Jim



_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Combining 2 Video Streams Side by Side, Getting Choppy Results

Nicolas Dufresne-5
In reply to this post by Jim Ruxton
Le mardi 16 février 2021 à 02:50 -0500, Jim Ruxton a écrit :

I am trying to combine 2 video streams. One from my laptop camera and one from an external webcam. I'm using the following pipeline and getting very choppy results. Neither my CPU or GPU seem to be working hard. Any ideas how to make this less choppy?

./gst-launch-1.0 v4l2src device=/dev/video4 ! videoscale ! videoconvert ! video/x-raw,format=YUY2, framerate=30/1, width=640, height=480! alpha alpha=1 ! videobox border-alpha=0 left=-640 ! 
videomixer name=Mix ! videoconvert ! autovideosink v4l2src device=/dev/video2 ! videoscale ! videoconvert ! video/x-raw, format=YUY2, framerate=30/1, width=640, height=480! alpha alpha=1 ! videobox border-alpha=0 right=-640 ! Mix.

First recommendation, move away from videomixer and use compositor (videomixer is just a backward shm on top of compositor). As you have a live pipeline, you should release the composition pressure by configurating a latency on the compositor element.  The latency is in nano-second, one of two frames of latency should be fine in general, but the default is none, and would only worked with perfectly synched sources which has accurate latency (v4l2src does not have accurate latency, it simply claims 1 frame, alway).

Second recommendation, consider adding a queue before you display sink, this will improve the timeout logic inside the compositor, by giving it a bit more freedome (with thread seperation).

Thanks,

Jim


_______________________________________________
gstreamer-devel mailing list


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Combining 2 Video Streams Side by Side, Getting Choppy Results

Jim Ruxton
In reply to this post by Mathieu Duponchelle

Thanks so much for the tips. I first put queue in after the videoscales but it didn't seem to help. Using compositor seemed to fix it. I'm going to have to read up on compositor more to understand why that works.

Cheers,

Jim

On 2021-02-16 9:14 a.m., Mathieu Duponchelle wrote:
That is a good point, you should also use compositor instead of videomixer,
especially for live streams. In any case, using sync=false on the sinks is not
a desirable solution, rather an admission of defeat :)

Cheers,

--
Mathieu Duponchelle · https://www.centricular.com

On 2/16/21 11:20 AM, Ian Davidson wrote:

I don't see any queue in your pipeline. A queue causes the pipeline to have a new thread - so all of your pipeline is one thread and is therefore only using one processor.  Experiment with putting a queue after each videoscale.

On 16/02/2021 09:02, Jim Ruxton wrote:

Just discovered that putting sync=false after autovideosink solved my problem.

Jim

On 2021-02-16 2:50 a.m., Jim Ruxton wrote:

I am trying to combine 2 video streams. One from my laptop camera and one from an external webcam. I'm using the following pipeline and getting very choppy results. Neither my CPU or GPU seem to be working hard. Any ideas how to make this less choppy?

./gst-launch-1.0 v4l2src device=/dev/video4 ! videoscale ! videoconvert ! video/x-raw,format=YUY2, framerate=30/1, width=640, height=480! alpha alpha=1 ! videobox border-alpha=0 left=-640 ! 
videomixer name=Mix ! videoconvert ! autovideosink v4l2src device=/dev/video2 ! videoscale ! videoconvert ! video/x-raw, format=YUY2, framerate=30/1, width=640, height=480! alpha alpha=1 ! videobox border-alpha=0 right=-640 ! Mix.

Thanks,

Jim



_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Combining 2 Video Streams Side by Side, Getting Choppy Results

Nicolas Dufresne-5
Le mardi 16 février 2021 à 12:34 -0500, Jim Ruxton a écrit :
> Thanks so much for the tips. I first put queue in after the videoscales but it
> didn't seem to help. Using compositor seemed to fix it. I'm going to have to
> read up on compositor more to understand why that works.

I won't get into details, but compositor is a rewrite of videomixer to solve the
live use cases.

> Cheers,
> Jim
> On 2021-02-16 9:14 a.m., Mathieu Duponchelle wrote:
>  
> >   That is a good point, you should also use compositor instead of
> > videomixer,
> >  especially for live streams. In any case, using sync=false on the sinks is
> > not
> >  a desirable solution, rather an admission of defeat :)
> >  
> >  Cheers,
> >  
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel