How to reduce avdec_h264 latency

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

How to reduce avdec_h264 latency

paolo perotinus
Hello,

I can play a HTTP-mpegts-H264 stream, with 640x480@25fps and zerolatency settings, with this command, which I translated into C code:

"gst-launch-1.0 -v souphttpsrc location=http://127.0.0.1:8080/stream.ts ! tsdemux name=demuxer demuxer. ! h264parse ! avdec_h264 ! autovideosink"

Then I measured that the avdec_h264 decoder adds a latency of ~70ms, which corresponds to the interval between two GstBuffers with the same timestamp: the first one taken from the h264parse's src, and the second one taken from the avdec_h264's src. It corresponds to about two frames of the stream, so it seems to me that the avdec_h264 does some buffering.

Is there a way to reduce this latency and/or reduce/avoid buffering in the decoder?

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to reduce avdec_h264 latency

Olivier Crête-3
Hi,

The only way to do that is to create a stream with no B-frames, for example using the H. 264 baseline profile.

Olivier

On July 10, 2017 6:57:16 PM EDT, paolo perotinus <[hidden email]> wrote:

>Hello,
>I can play a HTTP-mpegts-H264 stream, with 640x480@25fps and
>zerolatency settings, with this command, which I translated into C
>code:
>"gst-launch-1.0 -v souphttpsrc location=http://127.0.0.1:8080/stream.ts
>! tsdemux name=demuxer demuxer. ! h264parse ! avdec_h264 !
>autovideosink"
>Then I measured that the avdec_h264 decoder adds a latency of ~70ms,
>which corresponds to the interval between two GstBuffers with the same
>timestamp: the first one taken from the h264parse's src, and the second
>one taken from the avdec_h264's src. It corresponds to about two frames
>of the stream, so it seems to me that the avdec_h264 does some
>buffering.
>Is there a way to reduce this latency and/or reduce/avoid buffering in
>the decoder?

--
Olivier Crête
[hidden email]
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to reduce avdec_h264 latency

paolo perotinus


Il Martedì 11 Luglio 2017 1:06, Olivier Crête <[hidden email]> ha scritto:


> Hi,

> The only way to do that is to create a stream with no B-frames, for example using the H. 264 baseline profile.

> Olivier

Hi Olivier,

The stream is already coded with no B-frames (zerolatency tune in ffmpeg) but I still have that latency in avdec_h264...
 

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

答复: How to reduce avdec_h264 latency

Lijia (George Lee, Euler)

Pls take a bit of time to delve into system –level thread scheduling in attempt to find clue to long latency in plugin implementation. As per experience excluding bitstream encoding option, in many situations we often can find negative impact brought by threading scheduling over Linux. Even though bitstream has been encoded very well for low-latency use-case such as removal of B-frame/sub-frame processing (parallel pipeline), implicit threading scheduling defect is hard to find.

 

Best Regards,

 

George Lee

 

发件人: gstreamer-devel [mailto:[hidden email]] 代表 paolo perotinus
发送时间: 2017711 17:45
收件人: Discussion of the development of and with GStreamer <[hidden email]>
主题: Re: How to reduce avdec_h264 latency

 

 

Il Martedì 11 Luglio 2017 1:06, Olivier Crête <[hidden email]> ha scritto:

 

> Hi,

> The only way to do that is to create a stream with no B-frames, for example using the H. 264 baseline profile.

> Olivier

 

Hi Olivier,

 

The stream is already coded with no B-frames (zerolatency tune in ffmpeg) but I still have that latency in avdec_h264...

 


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 答复: How to reduce avdec_h264 latency

paolo perotinus
I just solved the problem (thanks to Tim-Philipp Muller) by setting is-live=1 on the receiver. The latency was added  in gstavviddec.c (check the line with: "gst_query_parse_latency (query, &is_live, NULL, NULL);" ).
However I think that this option (is-live) can appear too generic and misleading, and should be removed/changed.
What do you think about?


Il Martedì 11 Luglio 2017 12:40, "Lijia (George Lee, Euler)" <[hidden email]> ha scritto:


Pls take a bit of time to delve into system –level thread scheduling in attempt to find clue to long latency in plugin implementation. As per experience excluding bitstream encoding option, in many situations we often can find negative impact brought by threading scheduling over Linux. Even though bitstream has been encoded very well for low-latency use-case such as removal of B-frame/sub-frame processing (parallel pipeline), implicit threading scheduling defect is hard to find.
 
Best Regards,
 
George Lee
 
发件人: gstreamer-devel [mailto:[hidden email]] 代表 paolo perotinus
发送时间: 2017711 17:45
收件人: Discussion of the development of and with GStreamer <[hidden email]>
主题: Re: How to reduce avdec_h264 latency
 
 
Il Martedì 11 Luglio 2017 1:06, Olivier Crête <[hidden email]> ha scritto:
 
> Hi,

> The only way to do that is to create a stream with no B-frames, for example using the H. 264 baseline profile.

> Olivier
 
Hi Olivier,
 
The stream is already coded with no B-frames (zerolatency tune in ffmpeg) but I still have that latency in avdec_h264...
 
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel



_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

答复: 答复: How to reduce avdec_h264 latency

Lijia (George Lee, Euler)

This option is-live instructs GST h264 decoder element to perform low latency mode for Live event, thus this option takes role of key configuration of pipeline behavior in response to real-time decoding. In my opinion it should not be removed but more elaborate description is also needed to avoid misleading programmer.

 

George

 

 

发件人: gstreamer-devel [mailto:[hidden email]] 代表 paolo perotinus
发送时间: 2017711 18:50
收件人: Discussion of the development of and with GStreamer <[hidden email]>
主题: Re: 答复: How to reduce avdec_h264 latency

 

I just solved the problem (thanks to Tim-Philipp Muller) by setting is-live=1 on the receiver. The latency was added  in gstavviddec.c (check the line with: "gst_query_parse_latency (query, &is_live, NULL, NULL);" ).

However I think that this option (is-live) can appear too generic and misleading, and should be removed/changed.

What do you think about?

 

Il Martedì 11 Luglio 2017 12:40, "Lijia (George Lee, Euler)" <[hidden email]> ha scritto:

 

Pls take a bit of time to delve into system –level thread scheduling in attempt to find clue to long latency in plugin implementation. As per experience excluding bitstream encoding option, in many situations we often can find negative impact brought by threading scheduling over Linux. Even though bitstream has been encoded very well for low-latency use-case such as removal of B-frame/sub-frame processing (parallel pipeline), implicit threading scheduling defect is hard to find.

 

Best Regards,

 

George Lee

 

发件人: gstreamer-devel [[hidden email]] 代表 paolo perotinus
发送时间: 2017711 17:45
收件人: Discussion of the development of and with GStreamer <[hidden email]>
主题: Re: How to reduce avdec_h264 latency

 

 

Il Martedì 11 Luglio 2017 1:06, Olivier Crête <[hidden email]> ha scritto:

 

> Hi,

> The only way to do that is to create a stream with no B-frames, for example using the H. 264 baseline profile.

> Olivier

 

Hi Olivier,

 

The stream is already coded with no B-frames (zerolatency tune in ffmpeg) but I still have that latency in avdec_h264...

 

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

 


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to reduce avdec_h264 latency

Vinod Kesti
In reply to this post by paolo perotinus
See if fine tuning max-threads in avdec_h264 reduces the latency.

Note: CPU need more than 1 thread to decode HD streams. Check how many threads required to decode stream
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 答复: 答复: How to reduce avdec_h264 latency

paolo perotinus
In reply to this post by Lijia (George Lee, Euler)
More precisely, in gstavviddec.c there are the following lines:

    query = gst_query_new_latency ();
    is_live = FALSE;
    /* Check if upstream is live. If it isn't we can enable frame based
     * threading, which is adding latency */
    if (gst_pad_peer_query (GST_VIDEO_DECODER_SINK_PAD (ffmpegdec), query)) {
      gst_query_parse_latency (query, &is_live, NULL, NULL);
    }
    gst_query_unref (query);

    if (is_live)
      ffmpegdec->context->thread_type = FF_THREAD_SLICE;
    else
      ffmpegdec->context->thread_type = FF_THREAD_SLICE | FF_THREAD_FRAME;

Then, latency is introduced depending on "is_live" property, checked by the query. Now, I don't understand how to set the "is_live" property in the code of my receiver. I tried with:

  GstPad* pad_decoder_sink = gst_element_get_static_pad (data.video_decoder, "sink");
  GstQuery *query;
  query = gst_query_new_latency ();
  if (gst_pad_peer_query (pad_decoder, query)) {
     gst_query_set_latency (query, TRUE,  0, 0);        
  }
gst_object_unref (pad_decoder_sink);  

The  gst_query_set_latency() function is called, but gst_query_parse_latency (query, &is_live, NULL, NULL) inside gstavviddec.c returns FALSE for "is_live".  
Why? How can I set that property?










Il Martedì 11 Luglio 2017 12:57, "Lijia (George Lee, Euler)" <[hidden email]> ha scritto:


This option is-live instructs GST h264 decoder element to perform low latency mode for Live event, thus this option takes role of key configuration of pipeline behavior in response to real-time decoding. In my opinion it should not be removed but more elaborate description is also needed to avoid misleading programmer.
 
George
 
 
发件人: gstreamer-devel [mailto:[hidden email]] 代表 paolo perotinus
发送时间: 2017711 18:50
收件人: Discussion of the development of and with GStreamer <[hidden email]>
主题: Re: 答复: How to reduce avdec_h264 latency
 
I just solved the problem (thanks to Tim-Philipp Muller) by setting is-live=1 on the receiver. The latency was added  in gstavviddec.c (check the line with: "gst_query_parse_latency (query, &is_live, NULL, NULL);" ).
However I think that this option (is-live) can appear too generic and misleading, and should be removed/changed.
What do you think about?
 
Il Martedì 11 Luglio 2017 12:40, "Lijia (George Lee, Euler)" <[hidden email]> ha scritto:
 
Pls take a bit of time to delve into system –level thread scheduling in attempt to find clue to long latency in plugin implementation. As per experience excluding bitstream encoding option, in many situations we often can find negative impact brought by threading scheduling over Linux. Even though bitstream has been encoded very well for low-latency use-case such as removal of B-frame/sub-frame processing (parallel pipeline), implicit threading scheduling defect is hard to find.
 
Best Regards,
 
George Lee
 
发件人: gstreamer-devel [[hidden email]] 代表 paolo perotinus
发送时间: 2017711 17:45
收件人: Discussion of the development of and with GStreamer <[hidden email]>
主题: Re: How to reduce avdec_h264 latency
 
 
Il Martedì 11 Luglio 2017 1:06, Olivier Crête <[hidden email]> ha scritto:
 
> Hi,

> The only way to do that is to create a stream with no B-frames, for example using the H. 264 baseline profile.

> Olivier
 
Hi Olivier,
 
The stream is already coded with no B-frames (zerolatency tune in ffmpeg) but I still have that latency in avdec_h264...
 
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
 
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel



_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 答复: 答复: How to reduce avdec_h264 latency

noscgag
In reply to this post by Lijia (George Lee, Euler)
Is the "is-live" setting only available programmatically? If it will improve latency, I'd like to use it on the decode (receiver) side, but I use gst_parse_launch() to create my pipeline.

I did a gst-inspect on all the GStreamer elements (especially avdec_h264) and as far as I could tell only the source elements support the "is-live" setting (i.e. videotestsrc, souphttpsrc, gltestsrc, fakesrc, audiotestsrc, appsrc). This was with 1.10.4.

Gary



From: "Lijia (George Lee, Euler)" <[hidden email]>
To: paolo perotinus <[hidden email]>; Discussion of the development of and with GStreamer <[hidden email]>
Sent: Tuesday, July 11, 2017 3:58 AM
Subject: 答复: 答复: How to reduce avdec_h264 latency

This option is-live instructs GST h264 decoder element to perform low latency mode for Live event, thus this option takes role of key configuration of pipeline behavior in response to real-time decoding. In my opinion it should not be removed but more elaborate description is also needed to avoid misleading programmer.
 
George
 
 
发件人: gstreamer-devel [mailto:[hidden email]] 代表 paolo perotinus
发送时间: 2017711 18:50
收件人: Discussion of the development of and with GStreamer <[hidden email]>
主题: Re: 答复: How to reduce avdec_h264 latency
 
I just solved the problem (thanks to Tim-Philipp Muller) by setting is-live=1 on the receiver. The latency was added  in gstavviddec.c (check the line with: "gst_query_parse_latency (query, &is_live, NULL, NULL);" ).
However I think that this option (is-live) can appear too generic and misleading, and should be removed/changed.
What do you think about?
 
Il Martedì 11 Luglio 2017 12:40, "Lijia (George Lee, Euler)" <[hidden email]> ha scritto:
 
Pls take a bit of time to delve into system –level thread scheduling in attempt to find clue to long latency in plugin implementation. As per experience excluding bitstream encoding option, in many situations we often can find negative impact brought by threading scheduling over Linux. Even though bitstream has been encoded very well for low-latency use-case such as removal of B-frame/sub-frame processing (parallel pipeline), implicit threading scheduling defect is hard to find.
 
Best Regards,
 
George Lee
 
发件人: gstreamer-devel [[hidden email]] 代表 paolo perotinus
发送时间: 2017711 17:45
收件人: Discussion of the development of and with GStreamer <[hidden email]>
主题: Re: How to reduce avdec_h264 latency
 
 
Il Martedì 11 Luglio 2017 1:06, Olivier Crête <[hidden email]> ha scritto:
 
> Hi,

> The only way to do that is to create a stream with no B-frames, for example using the H. 264 baseline profile.

> Olivier
 
Hi Olivier,
 
The stream is already coded with no B-frames (zerolatency tune in ffmpeg) but I still have that latency in avdec_h264...
 
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
 
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel



_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Loading...