Hello, i would like to know how can i sync data and video with GStreamer.
I capture video from camera and for each frame i also get data (for example:
angles of camera gimbal).
I would like to send the video with the data synchronized by GStreamer.
How can i do that?
I guess most simply you can send them as separate streams indexed using the
frame number or timestamp, then sync the data when you need to.
A more complex solution would be to embed the data as metadata, either as a
GstMeta objects or codec-specific (I used H264's SEI format to save IMU data
in an H264 stream). I don't think that GstMeta objects are accessible
outside of the pipeline though (i.e. if you stream the video over UDP and
receive it somewhere else, the GstMeta will be lost).