Muxing videos with 'sparse' audio streams using mpetgsmux

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Muxing videos with 'sparse' audio streams using mpetgsmux

Ben Muzal

I have a somewhat unusual issue and am looking for advise on the best way to proceed. I am using gstreamer 1.14, and I am familiar with gstreamer development enough to have made my own plugin or two and I already deploy a custom package of gst-plugins-bad to handle some poorly labeled metadata streams in MPEG-TS files.

The next task I have is to take a live MPEG-TS stream, demux it, do some modifications to the metadata stream, then remux it as MPEG-TS again, and finally split it into files using splitmuxsink to get files that start on good keyframe boundaries where possible.

The issue is that a significant number of the input streams have an audio stream listed in the PMT, but don't actually have any audio or only have audio every once in a while. Basically the audio stream is sparse. This is not to say that there is silent audio, instead there are no packets at all on the PID that claims to be audio.  Also, there is not a way to tell if there is not going to be audio until I don't get it. I wish I had some sort of control over the input streams, but I do not.

Because audio streams are never marked as sparse,  GstCollectPads in mpetgsmux will wait forever for the first packet of an audio stream which may never get there. This locks up the pipeline.

I figure that I either need to modify tsdemux to automatically detect and mark the misbehaved  audio streams as sparse if they don't have packets after a while, or update GstCollectPads to have a mode that allows waiting pads to time out. There may be other options that I have not considered as well.

What is the most logical way to proceed within the gstreamer framework that is not going to cause unexpected behavior elsewhere? It seems that tsdemux being honest about if a stream is sparse or not makes initial sense, but will downstream elements (like muxers) be able to handle getting delayed codec information?

Thank you.

gstreamer-devel mailing list
[hidden email]