AV sync with hardware decoder and segment events

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

AV sync with hardware decoder and segment events

Juergen Sachs

Hallo together,

We have some problems to get AV in sync with our hardware decoders with some kind MP4 files.

 

We have implemented audio- and video sinks based on a hardware decoder. This decoder(s) have a (common) clock (STC) . When I feed audio and video buffers with timestamps (PTS), the hardware will output the decoded frames according to this STC.

To build the PTS for the hardware from the buffer PTS we

-          In SEGMENT event store the segment in the element and reset the hardware STC to 0 (both for audio and video)

-          In push() convert the PTS for the decoder with

o   PTS = gst_segment_to_running_time (&thiz->seg, GST_FORMAT_TIME, GST_BUFFER_PTS (buffer));

… and send the buffer with this new PTS to hardware

 

Is this correct?

 

As written above, this works for most streams, but we found some, where AV is not in sync. (compared to the playback of this streams with VLC or Windows Mediaplayer). The special of this streams seems to be the elst-box in the MP4 file.

Mostly I get two segment events before the first video buffer, for example

time segment start=0:00:00.000000000, offset=0:00:00.000000000, stop=99:99:99.999999999, rate=1.000000, applied_rate=1.000000, flags=0x00, time=0:00:00.000000000, base=0:00:00.000000000, position 0:00:00.000000000, duration 99:99:99.999999999

time segment start=0:00:00.083333333, offset=0:00:00.000000000, stop= 0:05:00.083333333, rate=1.000000, applied_rate=1.000000, flags=0x00, time=0:00:00.000000000, base=0:00:00.000000000, position 0:00:00.083333333, duration 99:99:99.999999999

render l=        1120 t=0:00:00.083333333 flags=00004040 free=         100

In the wrong case, I get 3 events like this

time segment start=0:00:00.000000000, offset=0:00:00.000000000, stop=99:99:99.999999999, rate=1.000000, applied_rate=1.000000, flags=0x00, time=0:00:00.000000000, base=0:00:00.000000000, position 0:00:00.000000000, duration 99:99:99.999999999

time segment start=0:00:00.000000000, offset=0:00:00.000000000, stop= 0:00:00.080000000, rate=1.000000, applied_rate=1.000000, flags=0x00, time=0:00:00.000000000, base=0:00:00.000000000, position 0:00:00.000000000, duration 99:99:99.999999999

time segment start=0:00:00.080000000, offset=0:00:00.000000000, stop= 0:04:30.400000000, rate=1.000000, applied_rate=1.000000, flags=0x00, time=0:00:00.080000000, base=0:00:00.000000000, position 0:00:00.080000000, duration 99:99:99.999999999

render l=       34461 t=0:00:00.080000000 flags=00000040 free=         100

 

Thanks for any help or confirmation, that I’m not totally wrong. I’m not sure, if my STC/PTS calculation is correct or if possibly the is an issue in qtdemux.

 

Kind regards

/Juergen

 


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Loading...