adding filesrc inputs dinamycally to videomixer

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

adding filesrc inputs dinamycally to videomixer

Lautaro Woites
Hi people,

I'm trying to add inputs to videomixer from filesrc dynamically.
I have a videotestsrc as background (with is-live=TRUE).
When I want to add a new input I create a bin with
filesrc, decodebin, videorate and videoconvert.
Then I request a pad on the mixer and I link it to the bin.

It works  OK but not as I need. Every time I add a new input to the mixer the video file start from the current running time and not from the beginning as I expected.

I've tried to replace the first SEGMENT event at the decodebin src pad with a pad probe and replace it with a new event where start=0 and end=-1. But it seems to not make any difference.
Also I've tried to replace the segment with a new one where the segment.base=<running_time_from_the_pipeline>, but the pipeline freezes. The log shows that the new videomixer:sink receives the segment event and then both mixer's sinks sent a qos event.

I've tried two ways for replace the segment event, the first one was to drop the first event(returning GST_PAD_PROBE_DROP) and send a new one from the app. (Seeing the logs the videomixer receives the two segment events).
The other way was to replace de event on the GstPadProbeInfo on the probe.


I'm using gst 1.2.2 and a C application.

The example's code is here: http://pastebin.com/KGyjky1R

Every suggestion is welcomed
Thanks!

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: adding filesrc inputs dinamycally to videomixer

Sebastian Dröge-3
On Di, 2014-01-14 at 17:41 -0200, Lautaro Woites wrote:

> Hi people,
>
> I'm trying to add inputs to videomixer from filesrc dynamically.
> I have a videotestsrc as background (with is-live=TRUE).
> When I want to add a new input I create a bin with
> filesrc, decodebin, videorate and videoconvert.
> Then I request a pad on the mixer and I link it to the bin.
>
> It works  OK but not as I need. Every time I add a new input to the mixer
> the video file start from the current running time and not from the
> beginning as I expected.
>
> I've tried to replace the first SEGMENT event at the decodebin src pad with
> a pad probe and replace it with a new event where start=0 and end=-1. But
> it seems to not make any difference.
> Also I've tried to replace the segment with a new one where the
> segment.base=<running_time_from_the_pipeline>, but the pipeline freezes.
> The log shows that the new videomixer:sink receives the segment event and
> then both mixer's sinks sent a qos event.
>
> I've tried two ways for replace the segment event, the first one was to
> drop the first event(returning GST_PAD_PROBE_DROP) and send a new one from
> the app. (Seeing the logs the videomixer receives the two segment events).
> The other way was to replace de event on the GstPadProbeInfo on the probe.
Easier would probably be to adjust the pad offsets on the new pads, i.e.
using gst_pad_set_offset(). Note that this only works for positive
offsets currently (which is what you want), unless you use 1.2.3 (or
latest 1.2 branch or git master branch).

If you want to adjust the running time manually by intercepting the
segment events it should in theory work if you add the running time when
the videomixer pad was added to segment.base. Just overriding any
previous value there will break synchronisation.
If you get problems with QoS you should check what videomixer calculates
as running time for the buffers on the new pad and if they are what you
expect. If not debug if the segments received in videomixer are what you
expect and the buffer timestamps too.

Also check the code of gst_segment_to_running_time() to understand how
this translation works, or see draft-synchronization.txt in the
GStreamer design documents.

--
Sebastian Dröge, Centricular Ltd - http://www.centricular.com
Expertise, Straight from the Source

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (968 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: adding filesrc inputs dinamycally to videomixer

Lautaro Woites



2014/1/15 Sebastian Dröge <[hidden email]>
On Di, 2014-01-14 at 17:41 -0200, Lautaro Woites wrote:
> Hi people,
>
> I'm trying to add inputs to videomixer from filesrc dynamically.
> I have a videotestsrc as background (with is-live=TRUE).
> When I want to add a new input I create a bin with
> filesrc, decodebin, videorate and videoconvert.
> Then I request a pad on the mixer and I link it to the bin.
>
> It works  OK but not as I need. Every time I add a new input to the mixer
> the video file start from the current running time and not from the
> beginning as I expected.
>
> I've tried to replace the first SEGMENT event at the decodebin src pad with
> a pad probe and replace it with a new event where start=0 and end=-1. But
> it seems to not make any difference.
> Also I've tried to replace the segment with a new one where the
> segment.base=<running_time_from_the_pipeline>, but the pipeline freezes.
> The log shows that the new videomixer:sink receives the segment event and
> then both mixer's sinks sent a qos event.
>
> I've tried two ways for replace the segment event, the first one was to
> drop the first event(returning GST_PAD_PROBE_DROP) and send a new one from
> the app. (Seeing the logs the videomixer receives the two segment events).
> The other way was to replace de event on the GstPadProbeInfo on the probe.

Easier would probably be to adjust the pad offsets on the new pads, i.e.
using gst_pad_set_offset(). Note that this only works for positive
offsets currently (which is what you want), unless you use 1.2.3 (or
latest 1.2 branch or git master branch).


Thank you very very match Sebastian!
I  set the offset on the pad and the result was that the video played for 1 second and then dissapeared (because the arrival of EOS to the pad).
The log shows me that the decoder was dropping frames because QOS events.
So now I'm dropping the QOS events at decoder's src pad and finally it works!!!!

Why the videomixer is sending QOS events upstream? Viewing the logs I read that the buffers arrives to late (I compared log's message time  with mixer's mesage) but I've tried to set the pad offset to a value greater than the current_running_time and the QOS still arriving to the decoder. Also I've tried to pause the pipe before adding the new input and then play it again.

Is it a good idea to drop the QOS events on the decoder's src pad?

 
If you want to adjust the running time manually by intercepting the
segment events it should in theory work if you add the running time when
the videomixer pad was added to segment.base. Just overriding any
previous value there will break synchronisation.
 
I've tried this approach too and I have QOS on the decoder, ignoring it like on the previous approach it didn't work. The videotestsrc plays normally but, the new input plays for a second and then freezes. Then periodically updates one frame. The update period seems to be the running_time when I added the new input.
 
If you get problems with QoS you should check what videomixer calculates
as running time for the buffers on the new pad and if they are what you
expect. If not debug if the segments received in videomixer are what you
expect and the buffer timestamps too.   
Also check the code of gst_segment_to_running_time() to understand how
this translation works, or see draft-synchronization.txt in the
GStreamer design documents.

 
Thanks! I will read it.
 

Thank you again for your time!

 
--
Sebastian Dröge, Centricular Ltd - http://www.centricular.com
Expertise, Straight from the Source

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel



_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: adding filesrc inputs dinamycally to videomixer

Sebastian Dröge-3
On Do, 2014-01-16 at 18:57 -0200, Lautaro Woites wrote:

> 2014/1/15 Sebastian Dröge <[hidden email]>
>
> > On Di, 2014-01-14 at 17:41 -0200, Lautaro Woites wrote:
> > > Hi people,
> > >
> > > I'm trying to add inputs to videomixer from filesrc dynamically.
> > > I have a videotestsrc as background (with is-live=TRUE).
> > > When I want to add a new input I create a bin with
> > > filesrc, decodebin, videorate and videoconvert.
> > > Then I request a pad on the mixer and I link it to the bin.
> > >
> > > It works  OK but not as I need. Every time I add a new input to the mixer
> > > the video file start from the current running time and not from the
> > > beginning as I expected.
> > >
> > > I've tried to replace the first SEGMENT event at the decodebin src pad
> > with
> > > a pad probe and replace it with a new event where start=0 and end=-1. But
> > > it seems to not make any difference.
> > > Also I've tried to replace the segment with a new one where the
> > > segment.base=<running_time_from_the_pipeline>, but the pipeline freezes.
> > > The log shows that the new videomixer:sink receives the segment event and
> > > then both mixer's sinks sent a qos event.
> > >
> > > I've tried two ways for replace the segment event, the first one was to
> > > drop the first event(returning GST_PAD_PROBE_DROP) and send a new one
> > from
> > > the app. (Seeing the logs the videomixer receives the two segment
> > events).
> > > The other way was to replace de event on the GstPadProbeInfo on the
> > probe.
> >
> > Easier would probably be to adjust the pad offsets on the new pads, i.e.
> > using gst_pad_set_offset(). Note that this only works for positive
> > offsets currently (which is what you want), unless you use 1.2.3 (or
> > latest 1.2 branch or git master branch).
> >
> >
> Thank you very very match Sebastian!
> I  set the offset on the pad and the result was that the video played for 1
> second and then dissapeared (because the arrival of EOS to the pad).
> The log shows me that the decoder was dropping frames because QOS events.
> So now I'm dropping the QOS events at decoder's src pad and finally it
> works!!!!
>
> Why the videomixer is sending QOS events upstream? Viewing the logs I read
> that the buffers arrives to late (I compared log's message time  with
> mixer's mesage) but I've tried to set the pad offset to a value greater
> than the current_running_time and the QOS still arriving to the decoder.
> Also I've tried to pause the pipe before adding the new input and then play
> it again.
>
> Is it a good idea to drop the QOS events on the decoder's src pad?
No :)

Can you file a bug at http://bugzilla.gnome.org with a test case? I
assume this happens because of the QoS translation that happens in
videomixer. It might not handle the offsets properly yet.

--
Sebastian Dröge, Centricular Ltd - http://www.centricular.com
Expertise, Straight from the Source

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (968 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: adding filesrc inputs dinamycally to videomixer

Lautaro Woites



2014/1/19 Sebastian Dröge <[hidden email]>
On Do, 2014-01-16 at 18:57 -0200, Lautaro Woites wrote:
> 2014/1/15 Sebastian Dröge <[hidden email]>
>
> > On Di, 2014-01-14 at 17:41 -0200, Lautaro Woites wrote:
> > > Hi people,
> > >
> > > I'm trying to add inputs to videomixer from filesrc dynamically.
> > > I have a videotestsrc as background (with is-live=TRUE).
> > > When I want to add a new input I create a bin with
> > > filesrc, decodebin, videorate and videoconvert.
> > > Then I request a pad on the mixer and I link it to the bin.
> > >
> > > It works  OK but not as I need. Every time I add a new input to the mixer
> > > the video file start from the current running time and not from the
> > > beginning as I expected.
> > >
> > > I've tried to replace the first SEGMENT event at the decodebin src pad
> > with
> > > a pad probe and replace it with a new event where start=0 and end=-1. But
> > > it seems to not make any difference.
> > > Also I've tried to replace the segment with a new one where the
> > > segment.base=<running_time_from_the_pipeline>, but the pipeline freezes.
> > > The log shows that the new videomixer:sink receives the segment event and
> > > then both mixer's sinks sent a qos event.
> > >
> > > I've tried two ways for replace the segment event, the first one was to
> > > drop the first event(returning GST_PAD_PROBE_DROP) and send a new one
> > from
> > > the app. (Seeing the logs the videomixer receives the two segment
> > events).
> > > The other way was to replace de event on the GstPadProbeInfo on the
> > probe.
> >
> > Easier would probably be to adjust the pad offsets on the new pads, i.e.
> > using gst_pad_set_offset(). Note that this only works for positive
> > offsets currently (which is what you want), unless you use 1.2.3 (or
> > latest 1.2 branch or git master branch).
> >
> >
> Thank you very very match Sebastian!
> I  set the offset on the pad and the result was that the video played for 1
> second and then dissapeared (because the arrival of EOS to the pad).
> The log shows me that the decoder was dropping frames because QOS events.
> So now I'm dropping the QOS events at decoder's src pad and finally it
> works!!!!
>
> Why the videomixer is sending QOS events upstream? Viewing the logs I read
> that the buffers arrives to late (I compared log's message time  with
> mixer's mesage) but I've tried to set the pad offset to a value greater
> than the current_running_time and the QOS still arriving to the decoder.
> Also I've tried to pause the pipe before adding the new input and then play
> it again.
>
> Is it a good idea to drop the QOS events on the decoder's src pad?

No :)

Can you file a bug at http://bugzilla.gnome.org with a test case? I
assume this happens because of the QoS translation that happens in
videomixer. It might not handle the offsets properly yet.


I reduced the example and I seen that if I use fakesink as the pipeline's sink the QOS events are not sent.
Also if I use some combinations of    mixer ! someencoder ! somemuxer! filesink as the pipeline's sink the QOS events are not sent.
So I think that the videomixer is not the problem. I think the xvimagesink is sending the QOS events and
the mixer is only forwarding them.

Do you want me to upload the example anyway?

One more question about this, why the xvimagesink is sending QOS events? The CPU is idle and I put queues before every mixer's
sink pad and after the mixer's src pad.

P.D: sorry if I made you think that there was a bug :(

Thanks a lot for your answers.

 
--
Sebastian Dröge, Centricular Ltd - http://www.centricular.com
Expertise, Straight from the Source

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel



_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: adding filesrc inputs dinamycally to videomixer

Sebastian Dröge-3
On Mo, 2014-01-20 at 13:21 -0200, Lautaro Woites wrote:

> > Can you file a bug at http://bugzilla.gnome.org with a test case? I
> > assume this happens because of the QoS translation that happens in
> > videomixer. It might not handle the offsets properly yet.
> >
> >
> I reduced the example and I seen that if I use fakesink as the pipeline's
> sink the QOS events are not sent.
> Also if I use some combinations of    mixer ! someencoder ! somemuxer!
> filesink as the pipeline's sink the QOS events are not sent.
> So I think that the videomixer is not the problem. I think the xvimagesink
> is sending the QOS events and
> the mixer is only forwarding them.
Yes, that's correct. videomixer is only forwarding the event but that
should actually be fine as it operates on the running time of all
streams.

> Do you want me to upload the example anyway?

Yes please, also a debug log with GST_DEBUG=6 could be helpful.

> One more question about this, why the xvimagesink is sending QOS events?
> The CPU is idle and I put queues before every mixer's
> sink pad and after the mixer's src pad.

xvimagesink is sending a QoS even for every single frame it receives.
Upstream elements can then decide based on the timestamps in the QoS
event if they should drop frames and skip forward (because they would
arrive too late in the sink anyway) or degrade quality of a filter
or ...

--
Sebastian Dröge, Centricular Ltd - http://www.centricular.com
Expertise, Straight from the Source

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (968 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: adding filesrc inputs dinamycally to videomixer

Lautaro Woites
I've already reported the bug, is on: https://bugzilla.gnome.org/show_bug.cgi?id=722697

Hope it helps.


2014/1/21 Sebastian Dröge <[hidden email]>
On Mo, 2014-01-20 at 13:21 -0200, Lautaro Woites wrote:

> > Can you file a bug at http://bugzilla.gnome.org with a test case? I
> > assume this happens because of the QoS translation that happens in
> > videomixer. It might not handle the offsets properly yet.
> >
> >
> I reduced the example and I seen that if I use fakesink as the pipeline's
> sink the QOS events are not sent.
> Also if I use some combinations of    mixer ! someencoder ! somemuxer!
> filesink as the pipeline's sink the QOS events are not sent.
> So I think that the videomixer is not the problem. I think the xvimagesink
> is sending the QOS events and
> the mixer is only forwarding them.

Yes, that's correct. videomixer is only forwarding the event but that
should actually be fine as it operates on the running time of all
streams.

> Do you want me to upload the example anyway?

Yes please, also a debug log with GST_DEBUG=6 could be helpful.

> One more question about this, why the xvimagesink is sending QOS events?
> The CPU is idle and I put queues before every mixer's
> sink pad and after the mixer's src pad.

xvimagesink is sending a QoS even for every single frame it receives.
Upstream elements can then decide based on the timestamps in the QoS
event if they should drop frames and skip forward (because they would
arrive too late in the sink anyway) or degrade quality of a filter
or ...

--
Sebastian Dröge, Centricular Ltd - http://www.centricular.com
Expertise, Straight from the Source

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel



_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel