keeping multiple audio streams synchronized when applying LADSPA filters

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

keeping multiple audio streams synchronized when applying LADSPA filters

Charlie Laub

I have written a text-based, interactive gstreamer application to stream audio across my LAN from a "music server" computer to multiple "client" computers. More info and code here:

http://audio.claub.net/GSASysCon.html

My question is about the client side of this system.

 

The client computers are located in/at/near active loudspeakers and I have been passing gstreamer’s audio output, via an alsa loopback, to another program (ecasound) that performs routing and implements LADSPA plugins. This comprises audio DSP and the crossover for the loudspeaker system.

 

When testing some recent single board computers, the OS releases came without the alsa loopback. Despite trying other ways to pass audio between gstreamer and ecasound on the client I have not been able to duplicate the low-latency performance of the loopback, which has created a roadblock for me. But just recently I learned that gstreamer is able to implement LADSPA plugins, including LADSPA plugins that are external to gstreamer (I wrote my own). This means that I could possibly implement the “DSP crossover” functionality right in gstreamer itself and I would no longer need to rely on the alsa loopback (or ecasound). To do this I need to duplicate ecasound’s routing functions, and its time alignment of the various channels obtained by splitting the input.

 

Currently in ecasound, each input channel (e.g. mono or L&R stereo) is duplicated N times (once per band of the loudspeaker crossover system), a varying number of LADSPA filters are applied to each channel. There may be other operations like merging/mxing channels, etc. At the end of the process all channels are output via a multichannel DAC. I understand that in gstreamer I can use 'deinterleave' and 'tee' to copy/split the input channels. My concern is that varying latency will result depending on how many LADSPA filters are applied to each of the N channels. Each LADSPA filter has some finite processing overhead and this overhead varies by filter type and will change depending on the computing hardware employed.

 

How do I properly interleave the channels back together after applying an arbitrary number of LADSPA filters and other operations to each channel so that the timing is correct? Should I employ a multiqueue just prior to interleave? What other approaches are needed so that the N channels remain properly synchronized?

 

If I am able to migrate from alsa_loopback+ecasound to running everything via gstreamer that will streamline and improve the application in many ways. Any advice and guidance to that end is appreciated.

 

-Charlie

 

 


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: keeping multiple audio streams synchronized when applying LADSPA filters

Nicolas Dufresne-5


Le 11 déc. 2017 6:18 PM, "Charlie Laub" <[hidden email]> a écrit :

I have written a text-based, interactive gstreamer application to stream audio across my LAN from a "music server" computer to multiple "client" computers. More info and code here:

http://audio.claub.net/GSASysCon.html

My question is about the client side of this system.

 

The client computers are located in/at/near active loudspeakers and I have been passing gstreamer’s audio output, via an alsa loopback, to another program (ecasound) that performs routing and implements LADSPA plugins. This comprises audio DSP and the crossover for the loudspeaker system.

 

When testing some recent single board computers, the OS releases came without the alsa loopback. Despite trying other ways to pass audio between gstreamer and ecasound on the client I have not been able to duplicate the low-latency performance of the loopback, which has created a roadblock for me. But just recently I learned that gstreamer is able to implement LADSPA plugins, including LADSPA plugins that are external to gstreamer (I wrote my own). This means that I could possibly implement the “DSP crossover” functionality right in gstreamer itself and I would no longer need to rely on the alsa loopback (or ecasound). To do this I need to duplicate ecasound’s routing functions, and its time alignment of the various channels obtained by splitting the input.

 

Currently in ecasound, each input channel (e.g. mono or L&R stereo) is duplicated N times (once per band of the loudspeaker crossover system), a varying number of LADSPA filters are applied to each channel. There may be other operations like merging/mxing channels, etc. At the end of the process all channels are output via a multichannel DAC. I understand that in gstreamer I can use 'deinterleave' and 'tee' to copy/split the input channels. My concern is that varying latency will result depending on how many LADSPA filters are applied to each of the N channels. Each LADSPA filter has some finite processing overhead and this overhead varies by filter type and will change depending on the computing hardware employed.

 

How do I properly interleave the channels back together after applying an arbitrary number of LADSPA filters and other operations to each channel so that the timing is correct? Should I employ a multiqueue just prior to interleave? What other approaches are needed so that the N channels remain properly synchronized?


As long as the segment and timestamp are carried (should be handled by the ladspa wrapper plugin) you can later recombine the output with audiointerleave a rewrite of interleave that do proper synchronization. Note that ladspa (same for lv2) wrappers might not cover 100% of the existing filters, I would recommend to tests your filters individually first to make sure they work. If not, ping us again ;-)

 

If I am able to migrate from alsa_loopback+ecasound to running everything via gstreamer that will streamline and improve the application in many ways. Any advice and guidance to that end is appreciated.

 

-Charlie

 

 


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel



_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

RE: keeping multiple audio streams synchronized when applying LADSPA filters

Charlie Laub

Audiointerleave is not something that I have seen before but I will give it a try. Thanks for mentioning it.

 

I will test the synchronization by generating a pure tone sine wave source, tee-ing it into two channels/streams, applying some LADSPA filters to only one of the channels, then mixing them back together with audiomixer (related to audiointerleave). Audiomixer is also supposed to preserve timing like audiointerleave so it is a good proxy. I will choose filters that undo each other (e.g. gain = +6dB follow by gain = -6dB) and I will invert the waveform . During mixing they should cancel each other if the timing is correct. This will be easy to verify by ear by sending the mixed audio to headphones.

 

From: gstreamer-devel [mailto:[hidden email]] On Behalf Of Nicolas Dufresne
Sent: Monday, December 11, 2017 8:10 PM
To: Discussion of the development of and with GStreamer <[hidden email]>
Subject: Re: keeping multiple audio streams synchronized when applying LADSPA filters

 

As long as the segment and timestamp are carried (should be handled by the ladspa wrapper plugin) you can later recombine the output with audiointerleave a rewrite of interleave that do proper synchronization. Note that ladspa (same for lv2) wrappers might not cover 100% of the existing filters, I would recommend to tests your filters individually first to make sure they work. If not, ping us again ;-)

 

 


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel