There are some known issues (explained by Edward in above links) that we can not just remove audio pad and add new audio pad. We would have to create new video and audio pad, push eos on old video pad and old audio pad, remove old video pad and remove old audio pad.And then start pushing packets on the new video pad and new audio pad. Currently my gstreamer version is 1.4.5 which is based on decodebin2.
In such scenario just before track change:
- significant video of future timestamps is already buffered in the video chain (old chain)
- significant audio of future timestamps is already buffered in the audio chain (old chain)
Now when switching to push buffers on new pads, for audio case, we have capability to "rewind" the PTS as per the current presentation time. But for video case, the video packets would have timestamps which is much ahead as compared to audio. I'm not sure if hlsdemux also rewinds the video when it performs the track switch ?
I think it will create problem at synchronisation stage when video is much ahead.
How such issues are handled at hlsdemux or inside any other demux plugin
which perform decode group switch ? Do they also "rewind" video before pushing on video chain ?
Do we need to also FLUSH old video and audio chains also before start pushing buffers on new video and audio pad ?
Now we also have a situation when there is no codec change we are trying to flush audio without removing audio pad and observing synchronization issues:
Re: query about audio track switch with decodebin2 (single audio decode, not multiple decode as standard gstreamer)
Thanks but moving to 1.12 is currently not easy due to some priorities. But we can analyze provided we have confirmed status based on behaviour with decodebin2 especially switching audio track use case with hlsdemux with decodebin2..does it download all streams from manifest and expose all audio pads and rely on switching with input selector ? Possibly no. Then in that case, does hlsdemux remove current video and current audio and the create new pads, push eos on old pads and then let it drain before it connects new decodegroup with playsink. In this process if it waits for drain then there would be long track switch latency... Also the timestamp of audio of switched track would be from behind then sync issues can't be avoided for the same video..is it correct understanding and how hlsdemux handles it or if there are known such limitations with hlsdemux and decodebin2. Looking forward to key pointers and insights from experts.