audio glitches when audio sinks are synchronized to GstSystemClock

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

audio glitches when audio sinks are synchronized to GstSystemClock

Charlie Laub

I am experiencing some behavior that seems to originate in how slaving is implemented in GstAudioBaseSink. I’d like some help in tracking this down and then issuing a bug report if that is necessary.

 

My playback pipeline uses GstSystemClock as the clock source. The reason for this clock source is so that I can use multiple alsa sinks (e.g. two or more stereo DACs) and have playback closely synchronized between them. The system clock is NTP disciplined to a local stratum 1 server. The problem I am experiencing happens no matter how many alsa sinks the pipeline is playing to (e.g. even with one sink).

 

I would like to force continual resampling to the system clock so I use the following sink properties:

provide-clock=false

slave-method=resample or skew

drift-tolerance to 1 or 10 (nanoseconds)

 

I am experiencing various audible glitches. With skew, there is a small tick or pop sound that occurs relatively frequently, every couple of seconds. When audio is muted (sending zeros) the noise is gone, so I assume this is the playback pointer being changed when sample values are non-zero. With the slave-method set to resample, the audio is problem-free for longer periods of time but then there will be several seconds of glitchy audio, which sounds like the pitch is increasing and decreasing rapidly before normal playback resumes, or it might last even longer as if the pitch is wavering all the time. With slave-method set to none, these artifacts disappear until the inevitable buffer over or underrun problems that produce a discontinuity (a dropout) in the audio.

 

I looked through the code and the audio slaving seems to be done within GstAudioBaseSink. This module (gstaudiobasesink.c) is credited to Erik Walthinsen in 2000 and to Wim Taymans in 2005. Being somewhat new to gstreamer I am not all that familiar with the code and all the clock management related stuff found in that module.

 

Can someone please help me better document this behavior and issue a bug report on it, or figure out what might be causing this behavior?


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

RE: audio glitches when audio sinks are synchronized to GstSystemClock

Charlie Laub

I’m still experiencing audible glitches when source and playback pipelines use the system clock. It’s unacceptable for an audio application.

I haven’t gotten any responses here – can someone PLEASE help me figure out a solution to this problem? Example pipelines are provided below.

 

The system consists of two computers (“server” and “client”) with audio sent via RTP/UDP over a LAN. There are two DACs on the client that I need to keep tightly synchronized, meaning 100usec or less but the audio problems happen even if there is only one DAC. I included below my previous post to this list about this problem from a couple of weeks ago.

 

# GST_ARGS on server:

#  alsasrc device='dsnoop:CARD=Loopback,DEV=1' provide-clock=false ! queue  ! audioconvert ! audio/x-raw,format=S16LE ! deinterleave name=input  \

audiointerleave name=client0_output latency=10000000 !  rtpL16pay !  udpsink host=192.168.10.101 port=1234   \

input.src_1 ! queue ! tee name=input_ch1  \

input.src_0 ! queue ! tee name=input_ch0   \

input_ch0. ! queue ! audioconvert ! 'audio/x-raw,channel-mask=(int)-3' ! client0_output.sink_0  \

input_ch1. ! queue ! audioconvert ! 'audio/x-raw,channel-mask=(int)-3' ! client0_output.sink_1  \

audiomixer name=client0_mixer2 latency=50000000 ! queue ! audioconvert ! 'audio/x-raw,channel-mask=(int)-3' ! client0_output.sink_2  \

input_ch0. ! queue ! audioconvert ! client0_mixer2.  \

input_ch1. ! queue ! audioconvert ! client0_mixer2.

 

 

# GST_ARGS on client 0:

# udpsrc port=1234 caps='application/x-rtp, media=(string)audio, clock-rate=(int)48000, encoding-name=(string)L16, channels=(int)3, payload=(int)96' ! \

rtpjitterbuffer latency=80 mode=1 do-lost=true do-retransmission=true ! rtpL16depay !  queue !  audioconvert ! audio/x-raw,format=S32LE !  deinterleave name=input  \

audiointerleave name=output1 latency = 10000000 ! audioconvert ! audio/x-raw,format=S16LE ! \

alsasink device='hw:CARD=DAC,DEV=0' provide-clock=false drift-tolerance=10 slave-method=resample buffer-time=50000  \

audiointerleave name=output0 latency = 10000000 ! audioconvert ! audio/x-raw,format=S16LE ! \

alsasink device='hw:CARD=DAC_1,DEV=0' provide-clock=false drift-tolerance=10 slave-method=resample buffer-time=50000   \

input.src_0 ! queue ! audioconvert ! ladspa-acdf-so-acdf type=22 fp=125 qp=1.0 ! audioconvert ! 'audio/x-raw,format=S32LE,channel-mask=(bitmask)0x1' ! output0.sink_0   \

input.src_1 ! queue ! audioconvert ! ladspa-acdf-so-acdf type=22 fp=125 qp=1.0 ! audioconvert ! 'audio/x-raw,format=S32LE,channel-mask=(bitmask)0x2' ! output0.sink_1   \

input.src_2 ! queue ! audioconvert ! ladspa-acdf-so-acdf type=21 fp=120 qp=0.707 ! \

ladspa-acdf-so-acdf type=21 fp=120 qp=0.707 ! ladspa-acdf-so-acdf type=28 db=-6 fp=35 qp=1.0 fz=50 qz=0.5 ! audioconvert ! audio/x-raw,format=S32LE ! tee name=subs  \

subs. ! queue ! audioconvert ! 'audio/x-raw,format=S32LE,channel-mask=(bitmask)0x1' ! output1.sink_0   \

subs. ! queue ! audioconvert ! 'audio/x-raw,format=S32LE,channel-mask=(bitmask)0x2' ! output1.sink_1

 

 

 

From: gstreamer-devel <[hidden email]> On Behalf Of Charlie Laub
Sent: Thursday, March 1, 2018 7:53 AM
To: [hidden email]
Subject: audio glitches when audio sinks are synchronized to GstSystemClock

 

I am experiencing some behavior that seems to originate in how slaving is implemented in GstAudioBaseSink. I’d like some help in tracking this down and then issuing a bug report if that is necessary.

 

My playback pipeline uses GstSystemClock as the clock source. The reason for this clock source is so that I can use multiple alsa sinks (e.g. two or more stereo DACs) and have playback closely synchronized between them. The system clock is NTP disciplined to a local stratum 1 server. The problem I am experiencing happens no matter how many alsa sinks the pipeline is playing to (e.g. even with one sink).

 

I would like to force continual resampling to the system clock so I use the following sink properties:

provide-clock=false

slave-method=resample or skew

drift-tolerance to 1 or 10 (nanoseconds)

 

I am experiencing various audible glitches. With skew, there is a small tick or pop sound that occurs relatively frequently, every couple of seconds. When audio is muted (sending zeros) the noise is gone, so I assume this is the playback pointer being changed when sample values are non-zero. With the slave-method set to resample, the audio is problem-free for longer periods of time but then there will be several seconds of glitchy audio, which sounds like the pitch is increasing and decreasing rapidly before normal playback resumes, or it might last even longer as if the pitch is wavering all the time. With slave-method set to none, these artifacts disappear until the inevitable buffer over or underrun problems that produce a discontinuity (a dropout) in the audio.

 

I looked through the code and the audio slaving seems to be done within GstAudioBaseSink. This module (gstaudiobasesink.c) is credited to Erik Walthinsen in 2000 and to Wim Taymans in 2005. Being somewhat new to gstreamer I am not all that familiar with the code and all the clock management related stuff found in that module.

 

Can someone please help me better document this behavior and issue a bug report on it, or figure out what might be causing this behavior?


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

RE: audio glitches when audio sinks are synchronized to GstSystemClock

Andrew Silby

I have a similar problem when do-retransmission is true. Try setting it to false.

 

From: gstreamer-devel [mailto:[hidden email]] On Behalf Of Charlie Laub
Sent: 12 March 2018 16:03
To: 'Discussion of the development of and with GStreamer' <[hidden email]>
Subject: RE: audio glitches when audio sinks are synchronized to GstSystemClock

 

I’m still experiencing audible glitches when source and playback pipelines use the system clock. It’s unacceptable for an audio application.

I haven’t gotten any responses here – can someone PLEASE help me figure out a solution to this problem? Example pipelines are provided below.

 

The system consists of two computers (“server” and “client”) with audio sent via RTP/UDP over a LAN. There are two DACs on the client that I need to keep tightly synchronized, meaning 100usec or less but the audio problems happen even if there is only one DAC. I included below my previous post to this list about this problem from a couple of weeks ago.

 

# GST_ARGS on server:

#  alsasrc device='dsnoop:CARD=Loopback,DEV=1' provide-clock=false ! queue  ! audioconvert ! audio/x-raw,format=S16LE ! deinterleave name=input  \

audiointerleave name=client0_output latency=10000000 !  rtpL16pay !  udpsink host=192.168.10.101 port=1234   \

input.src_1 ! queue ! tee name=input_ch1  \

input.src_0 ! queue ! tee name=input_ch0   \

input_ch0. ! queue ! audioconvert ! 'audio/x-raw,channel-mask=(int)-3' ! client0_output.sink_0  \

input_ch1. ! queue ! audioconvert ! 'audio/x-raw,channel-mask=(int)-3' ! client0_output.sink_1  \

audiomixer name=client0_mixer2 latency=50000000 ! queue ! audioconvert ! 'audio/x-raw,channel-mask=(int)-3' ! client0_output.sink_2  \

input_ch0. ! queue ! audioconvert ! client0_mixer2.  \

input_ch1. ! queue ! audioconvert ! client0_mixer2.

 

 

# GST_ARGS on client 0:

# udpsrc port=1234 caps='application/x-rtp, media=(string)audio, clock-rate=(int)48000, encoding-name=(string)L16, channels=(int)3, payload=(int)96' ! \

rtpjitterbuffer latency=80 mode=1 do-lost=true do-retransmission=true ! rtpL16depay !  queue !  audioconvert ! audio/x-raw,format=S32LE !  deinterleave name=input  \

audiointerleave name=output1 latency = 10000000 ! audioconvert ! audio/x-raw,format=S16LE ! \

alsasink device='hw:CARD=DAC,DEV=0' provide-clock=false drift-tolerance=10 slave-method=resample buffer-time=50000  \

audiointerleave name=output0 latency = 10000000 ! audioconvert ! audio/x-raw,format=S16LE ! \

alsasink device='hw:CARD=DAC_1,DEV=0' provide-clock=false drift-tolerance=10 slave-method=resample buffer-time=50000   \

input.src_0 ! queue ! audioconvert ! ladspa-acdf-so-acdf type=22 fp=125 qp=1.0 ! audioconvert ! 'audio/x-raw,format=S32LE,channel-mask=(bitmask)0x1' ! output0.sink_0   \

input.src_1 ! queue ! audioconvert ! ladspa-acdf-so-acdf type=22 fp=125 qp=1.0 ! audioconvert ! 'audio/x-raw,format=S32LE,channel-mask=(bitmask)0x2' ! output0.sink_1   \

input.src_2 ! queue ! audioconvert ! ladspa-acdf-so-acdf type=21 fp=120 qp=0.707 ! \

ladspa-acdf-so-acdf type=21 fp=120 qp=0.707 ! ladspa-acdf-so-acdf type=28 db=-6 fp=35 qp=1.0 fz=50 qz=0.5 ! audioconvert ! audio/x-raw,format=S32LE ! tee name=subs  \

subs. ! queue ! audioconvert ! 'audio/x-raw,format=S32LE,channel-mask=(bitmask)0x1' ! output1.sink_0   \

subs. ! queue ! audioconvert ! 'audio/x-raw,format=S32LE,channel-mask=(bitmask)0x2' ! output1.sink_1

 

 

 

From: gstreamer-devel <[hidden email]> On Behalf Of Charlie Laub
Sent: Thursday, March 1, 2018 7:53 AM
To: [hidden email]
Subject: audio glitches when audio sinks are synchronized to GstSystemClock

 

I am experiencing some behavior that seems to originate in how slaving is implemented in GstAudioBaseSink. I’d like some help in tracking this down and then issuing a bug report if that is necessary.

 

My playback pipeline uses GstSystemClock as the clock source. The reason for this clock source is so that I can use multiple alsa sinks (e.g. two or more stereo DACs) and have playback closely synchronized between them. The system clock is NTP disciplined to a local stratum 1 server. The problem I am experiencing happens no matter how many alsa sinks the pipeline is playing to (e.g. even with one sink).

 

I would like to force continual resampling to the system clock so I use the following sink properties:

provide-clock=false

slave-method=resample or skew

drift-tolerance to 1 or 10 (nanoseconds)

 

I am experiencing various audible glitches. With skew, there is a small tick or pop sound that occurs relatively frequently, every couple of seconds. When audio is muted (sending zeros) the noise is gone, so I assume this is the playback pointer being changed when sample values are non-zero. With the slave-method set to resample, the audio is problem-free for longer periods of time but then there will be several seconds of glitchy audio, which sounds like the pitch is increasing and decreasing rapidly before normal playback resumes, or it might last even longer as if the pitch is wavering all the time. With slave-method set to none, these artifacts disappear until the inevitable buffer over or underrun problems that produce a discontinuity (a dropout) in the audio.

 

I looked through the code and the audio slaving seems to be done within GstAudioBaseSink. This module (gstaudiobasesink.c) is credited to Erik Walthinsen in 2000 and to Wim Taymans in 2005. Being somewhat new to gstreamer I am not all that familiar with the code and all the clock management related stuff found in that module.

 

Can someone please help me better document this behavior and issue a bug report on it, or figure out what might be causing this behavior?


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

RE: audio glitches when audio sinks are synchronized to GstSystemClock

Charlie Laub
In reply to this post by Charlie Laub

I believe I have finally solved this problem.

 

PROBLEM RECAP:

Streaming over LAN to client, on client pipeline ends at an alsasink. Client clock source was forced to GstSystemClock by setting provide-clock=false on sink(s). When I also set slave-method=resample or slave-method=skew I would get audible glitches. With skew it was a tick or pop sound, and with resample occasionally the pitch would go up and down for several to several tens of seconds, like “wow and flutter” if anyone remembers what this is…

 

FIX:

I set sync=false on all alsasinks. Since then (several days of auditioning ago) the audible glitches have disappeared.

 

Not sure why exactly this is fixing the problem, but I thought I would leave a note about it here for posterity.


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: audio glitches when audio sinks are synchronized to GstSystemClock

Pavel Hofman
Hi Charlie,

Congrats on solving this issue, it is very useful. But I do not
understand the change you did. Did you originally have more alsasinks,
some with and some without the sync=false config?

Thanks,

Pavel.

Dne 26.3.2018 v 20:57 Charlie Laub napsal(a):

> I believe I have finally solved this problem.
>
> PROBLEM RECAP:
>
> Streaming over LAN to client, on client pipeline ends at an alsasink.
> Client clock source was forced to GstSystemClock by setting
> provide-clock=false on sink(s). When I also set slave-method=resample or
> slave-method=skew I would get audible glitches. With skew it was a tick
> or pop sound, and with resample occasionally the pitch would go up and
> down for several to several tens of seconds, like “wow and flutter” if
> anyone remembers what this is…
>
> FIX:
>
> I set sync=false on all alsasinks. Since then (several days of
> auditioning ago) the audible glitches have disappeared.
>
> Not sure why exactly this is fixing the problem, but I thought I would
> leave a note about it here for posterity.
>
>
>
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

RE: audio glitches when audio sinks are synchronized to GstSystemClock

Charlie Laub
Hi Pavel,

The client uses two alsasinks (two stereo DACs). The full pipeline is given earlier in this thread. There were audible glitches when both alsasinks had no sync parameter specified, so the value for sync was the default value, which is sync=true.

After I explicitly set sync=false for both alsasinks, the glitches went away. That is the only change I made to the pipeline.

-Charlie

-----Original Message-----
From: gstreamer-devel <[hidden email]> On Behalf Of Pavel Hofman
Sent: Monday, March 26, 2018 11:52 PM
To: Discussion of the development of and with GStreamer <[hidden email]>; Charlie Laub <[hidden email]>
Subject: Re: audio glitches when audio sinks are synchronized to GstSystemClock

Hi Charlie,

Congrats on solving this issue, it is very useful. But I do not understand the change you did. Did you originally have more alsasinks, some with and some without the sync=false config?

Thanks,

Pavel.

Dne 26.3.2018 v 20:57 Charlie Laub napsal(a):

> I believe I have finally solved this problem.
>
> PROBLEM RECAP:
>
> Streaming over LAN to client, on client pipeline ends at an alsasink.
> Client clock source was forced to GstSystemClock by setting
> provide-clock=false on sink(s). When I also set slave-method=resample
> or slave-method=skew I would get audible glitches. With skew it was a
> tick or pop sound, and with resample occasionally the pitch would go
> up and down for several to several tens of seconds, like “wow and
> flutter” if anyone remembers what this is…
>
> FIX:
>
> I set sync=false on all alsasinks. Since then (several days of
> auditioning ago) the audible glitches have disappeared.
>
> Not sure why exactly this is fixing the problem, but I thought I would
> leave a note about it here for posterity.
>
>
>
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel