4K playback optimization help needed

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

4K playback optimization help needed

Tamas
I would like to ask for some advice regarding how to make 4K playback
smoother. I don't think the issue is that my PC can't keep up with decoding,
because if I use /sync = false/ on the video sink, the video plays fast and
very smooth. Details follow:

1. The basic pipeline I tried:

/gst-launch-1.0.exe filesrc location=4K.mkv ! matroskademux ! h265parse !
nvh265dec ! glimagesink/

This works, but the image is a bit "choppy", as in I feel I can see as the
movie proceeds from one frame onto the next, especially when the camera is
panning. Initially I assumed that the jumpiness was because of the
discrepancy between the movie's frame rate (23.967) and that of my monitor
(60), but the file plays back perfectly smooth with a video player (Windows
10 x64, MPC-BE with LAV filters using CUVID hwaccel and EVR renderer). I
don't think that MPC-BE is using motion interpolation either.

As to why I'm using these filters: d3d11 filters (d3d11h265dec ->
glimagesink or d3d11videosink) only produce a white or gray image (perhaps
because the source is 10bit), avdec_h265 CPU processing is too slow, so all
I have left is the nvcodec filters, plus I don't know any memory:GLMemory
capable video sink other than glimagesink (glimagesinkelement didn't work,
not sure why).

2. sync=false

/gst-launch-1.0.exe filesrc location=4K.mkv ! matroskademux ! h265parse !
nvh265dec ! glimagesink sync=false/

Plays back the video at high speed. Really smooth and fast, so I suspect
it's not a performance issue I'm having. I've also tried to resizing the
video stream (using gltransformation) to 2K, after the nvh265dec, that
didn't change anything either.

Here are the things I've tried to improve the situation:

3. vsync=true

/gst-launch-1.0.exe filesrc location=4K.mkv ! matroskademux ! h265parse !
nvh265dec ! glimagesink sync=true vsync=true/

Not much changed, still small jumps from frame to frame.

4. videorate

/gst-launch-1.0.exe filesrc location=4K.mkv ! matroskademux ! h265parse !
nvh265dec ! queue ! videorate ! video/x-raw(ANY), framerate=60/1 !
glimagesink sync=true vsync=true/

Or video/x-raw(memory:GLMemory) caps. This "feels" a bit better, smoother,
but not perfect.

5. videorate + sync off

/gst-launch-1.0.exe filesrc location=4K.mkv ! matroskademux ! h265parse !
nvh265dec ! queue ! videorate ! video/x-raw(ANY), framerate=60/1 !
glimagesink sync=false vsync=true/

I've seen another thread ( Choppy prores 4K playback
<http://gstreamer-devel.966125.n4.nabble.com/Choppy-prores-4k-playback-td4691115.html>
) in which it's mentioned that perhaps the individual buffers aren't
processed fast enough at the sink - not sure if it applies to my case, but I
assumed the above pipeline would avoid the problem of buffers being dropped
because they were displayed too slowly... even if that's not correct, so far
this has produced the best result, but it's still not as smooth as playing
it back with MPC-BE.

I've been able to get so far with gst-launch, I'm thinking maybe I could get
more information about what's really going on with programming this pipeline
in C, but I'd be happy if someone wiser could give me some ideas as to which
direction I should be going from here on.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: 4K playback optimization help needed

Tamas
I forgot to mention that the "choppy" playback with gst-launch and the smooth
one in MPC-BE take the same amount of time, so it's not like gst plays it
back slower. I have suspicion, without data to back it up though, that some
frames might be dropped for some reason, that's why the progression from one
frame to the next is visible.

I would like to know how to check what's actually going wrong in my
pipeline: are buffers being dropped even though it's not a performance
bottleneck?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: 4K playback optimization help needed

Matteo Valdina
You should check the latency of your pipeline. 
It is possibile that the computed latency is on the edge of the rendering deadline. Increase the logging should show when frames are dropped and for that reason. 

You can increase the latency of the synk with latency parameter. Try to increase the latency and add a queue before the sink. 
The queue should avoid the decoder to stall waiting the rendering. 

Best
Matteo


On Sun, Jan 26, 2020, 05:10 QwjN1Y9mJvamZJ <[hidden email]> wrote:
I forgot to mention that the "choppy" playback with gst-launch and the smooth
one in MPC-BE take the same amount of time, so it's not like gst plays it
back slower. I have suspicion, without data to back it up though, that some
frames might be dropped for some reason, that's why the progression from one
frame to the next is visible.

I would like to know how to check what's actually going wrong in my
pipeline: are buffers being dropped even though it's not a performance
bottleneck?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: 4K playback optimization help needed

Michael Gruner
Also, consider adding queues to your pipe. Maybe one before the encoder. 

On Jan 26, 2020, at 11:21, Matteo Valdina <[hidden email]> wrote:


You should check the latency of your pipeline. 
It is possibile that the computed latency is on the edge of the rendering deadline. Increase the logging should show when frames are dropped and for that reason. 

You can increase the latency of the synk with latency parameter. Try to increase the latency and add a queue before the sink. 
The queue should avoid the decoder to stall waiting the rendering. 

Best
Matteo


On Sun, Jan 26, 2020, 05:10 QwjN1Y9mJvamZJ <[hidden email]> wrote:
I forgot to mention that the "choppy" playback with gst-launch and the smooth
one in MPC-BE take the same amount of time, so it's not like gst plays it
back slower. I have suspicion, without data to back it up though, that some
frames might be dropped for some reason, that's why the progression from one
frame to the next is visible.

I would like to know how to check what's actually going wrong in my
pipeline: are buffers being dropped even though it's not a performance
bottleneck?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: 4K playback optimization help needed

Nicolas Dufresne-5
In reply to this post by Tamas
Le dimanche 26 janvier 2020 à 07:07 -0600, QwjN1Y9mJvamZJ a écrit :

> I would like to ask for some advice regarding how to make 4K playback
> smoother. I don't think the issue is that my PC can't keep up with decoding,
> because if I use /sync = false/ on the video sink, the video plays fast and
> very smooth. Details follow:
>
> 1. The basic pipeline I tried:
>
> /gst-launch-1.0.exe filesrc location=4K.mkv ! matroskademux ! h265parse !
> nvh265dec ! glimagesink/
>
> This works, but the image is a bit "choppy", as in I feel I can see as the
> movie proceeds from one frame onto the next, especially when the camera is
> panning. Initially I assumed that the jumpiness was because of the
> discrepancy between the movie's frame rate (23.967) and that of my monitor
> (60), but the file plays back perfectly smooth with a video player (Windows
> 10 x64, MPC-BE with LAV filters using CUVID hwaccel and EVR renderer). I
> don't think that MPC-BE is using motion interpolation either.
>
> As to why I'm using these filters: d3d11 filters (d3d11h265dec ->
> glimagesink or d3d11videosink) only produce a white or gray image (perhaps
> because the source is 10bit), avdec_h265 CPU processing is too slow, so all
> I have left is the nvcodec filters, plus I don't know any memory:GLMemory
> capable video sink other than glimagesink (glimagesinkelement didn't work,
> not sure why).
>
> 2. sync=false
>
> /gst-launch-1.0.exe filesrc location=4K.mkv ! matroskademux ! h265parse !
> nvh265dec ! glimagesink sync=false/
>
> Plays back the video at high speed. Really smooth and fast, so I suspect
> it's not a performance issue I'm having. I've also tried to resizing the
> video stream (using gltransformation) to 2K, after the nvh265dec, that
> didn't change anything either.
>
> Here are the things I've tried to improve the situation:
>
> 3. vsync=true
>
> /gst-launch-1.0.exe filesrc location=4K.mkv ! matroskademux ! h265parse !
> nvh265dec ! glimagesink sync=true vsync=true/

As Michael suggested, you can add a queue between vh265dec and glimagesink. Note
that the default maximum size is pretty small, so try adding it this way:

  ... ! nvh265dec ! queue max-size-bytes=0 ! glimagesink

Note that there is no "vsync" property on glimagesink in mainline GStreamer.
Please review all your applied patches as they might also be the cause of all
your troubles.

Another thing you may look at is the following fix that was merge recently:

https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/merge_requests/998

It's not much, but it removes 1 frame latency on the decoder. But this is
buffering latency, so on non-live playback, I have strong doubt it will make any
difference.

>
> Not much changed, still small jumps from frame to frame.
>
> 4. videorate
>
> /gst-launch-1.0.exe filesrc location=4K.mkv ! matroskademux ! h265parse !
> nvh265dec ! queue ! videorate ! video/x-raw(ANY), framerate=60/1 !
> glimagesink sync=true vsync=true/
>
> Or video/x-raw(memory:GLMemory) caps. This "feels" a bit better, smoother,
> but not perfect.
>
> 5. videorate + sync off
>
> /gst-launch-1.0.exe filesrc location=4K.mkv ! matroskademux ! h265parse !
> nvh265dec ! queue ! videorate ! video/x-raw(ANY), framerate=60/1 !
> glimagesink sync=false vsync=true/
>
> I've seen another thread ( Choppy prores 4K playback
> <
> http://gstreamer-devel.966125.n4.nabble.com/Choppy-prores-4k-playback-td4691115.html
> >
> ) in which it's mentioned that perhaps the individual buffers aren't
> processed fast enough at the sink - not sure if it applies to my case, but I
> assumed the above pipeline would avoid the problem of buffers being dropped
> because they were displayed too slowly... even if that's not correct, so far
> this has produced the best result, but it's still not as smooth as playing
> it back with MPC-BE.
>
> I've been able to get so far with gst-launch, I'm thinking maybe I could get
> more information about what's really going on with programming this pipeline
> in C, but I'd be happy if someone wiser could give me some ideas as to which
> direction I should be going from here on.
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: 4K playback optimization help needed

Tamas
Thanks everyone for your helpful replies.

I've tried most of what you've suggested, and while in the end the situation
hasn't improved, I might have some new information. Most importantly:


1) Without "queue" elements, some frames were being dropped due to QoS,
however, with queues included, I didn't see any "dropping frame" warnings.

Pipeline A without queues: "gst-launch-1.0.exe filesrc location=4K.mkv !
matroskademux ! h265parse ! nvh265dec ! glimagesink --gst-debug 2 2>dbg.txt"

Pipeline B with queues: "gst-launch-1.0.exe filesrc location=4K.mkv !
matroskademux ! queue max-size-bytes=0 ! h265parse ! queue max-size-bytes=0
! nvh265dec ! queue max-size-bytes=0 ! videorate ! video/x-raw(ANY),
framerate=(fraction)60/1 ! glimagesink sync=false --gst-debug 2 2>dbgq.txt"

Also, separately, I created debug logs with "video*:6", "glimagesink:6",
etc. for each involved element. Separately, as to keep log files specific to
individual components, and to avoid slowing things down too much
(--gst-debug 6 is way too slow).

a) The log file created by video*:6 had some dropped frame warnings for
pipeline A, not for B:

"0:00:01.407290000 12224 00000225466CC040 WARN            videodecoder
gstvideodecoder.c:3207:gst_video_decoder_clip_and_push_buf:<nvh265dec0>
Dropping frame due to QoS. start:0:00:00.333666664
deadline:0:00:00.333666664 earliest_time:0:00:00.360051535"

b) In addition, log files for "--gst-debug 2", both A and B, contained one
type of warning many times (copied 4 here to show time progression). This is
probably not the reason for the "jumpiness", see CPU decoding in #2.

"0:00:00.890107000  5644 000002C61149C040 WARN      matroskareadcommon
matroska-read-common.c:712:gst_matroska_read_common_parse_skip:<matroskademux0:sink>
Unknown CueTrackPositions subelement 0xf0 - ignoring
0:00:00.890587000  5644 000002C61149C040 WARN      matroskareadcommon
matroska-read-common.c:712:gst_matroska_read_common_parse_skip:<matroskademux0:sink>
Unknown CueTrackPositions subelement 0xf0 - ignoring
0:00:00.891072000  5644 000002C61149C040 WARN      matroskareadcommon
matroska-read-common.c:712:gst_matroska_read_common_parse_skip:<matroskademux0:sink>
Unknown CueTrackPositions subelement 0xf0 - ignoring
0:00:00.891540000  5644 000002C61149C040 WARN      matroskareadcommon
matroska-read-common.c:712:gst_matroska_read_common_parse_skip:<matroskademux0:sink>
Unknown CueTrackPositions subelement 0xf0 - ignoring"

No other warnings were present in any of the other log files. I suspect that
perhaps frame dropping isn't the issue here. Playback is still "jumpy", more
so than I get with MPC-BE. Tbh, I can see small steps from frame to frame in
both gstreamer and MPC-BE output, however, gstreamer playback seems to show
some bigger "jumps" all over the screen a few times a second, and MPC-BE
gives a smoother feeling overall.
Is it possible that the larger "jumps" are caused by glimagesink being
"slow", or lacking vsync? I'd love to try out another image sink, but don't
know of any that could show


2) CPU decoding is smooth

a) "gst-launch-1.0.exe filesrc location=4K.mkv ! matroskademux ! h265parse !
avdec_h265 ! d3d11videosink"

On my (strong enough) work PC, this pipeline works better, smoother than the
one above. It's just as smooth as MPC-BE, even without queues. BtW,
matroskademuxer has the same warnings for the CPU pipeline as above, the
playback is smooth nonetheless.

b) Uploading the CPU-decoded video to the GPU and displaying with
glimagesink has the same problems as above:

"gst-launch-1.0.exe filesrc location=4K.mkv ! matroskademux ! h265parse !
queue max-size-bytes=0 ! avdec_h265 ! videoconvert ! queue max-size-bytes=0
! glupload ! queue max-size-bytes=0 ! glimagesink"

c) I've also tried to do the other way, as in GPU decoding first and
replacing glimagesink with another video sink, but can't find any way to use
another video sink. E.g.:

"gst-launch-1.0.exe filesrc location=4K.mkv ! matroskademux ! h265parse !
nvh265dec ! gldownload ! videoconvert ! autovideosink" only shows a green
image. "d3d11videosink" is the same (perhaps autovideosink used that? didn't
check).

All in all, I suspect the problem is not with frames being dropped, but
somehow the display of glimagesink is "jumpy". However, glimagesink isn't
limited by performance either, because with "sync=false" it plays back
frames very quickly and with no apparent image quality issues (jumps small
or large). BtW, the "max-lateness=0", "qos=false" parameters don't help
glimagesink playback at all, if they're even relevant.

Can you suggest please:
- How to continue exploring the possible issues?
- Is there any other video sink I could try instead of glimagesink to see if
that's the problem? Alternatively, instruct  me how to make
d3d11videosink/autovideosink work after gldownload.

**********************
Let me quickly also respond to a few other things you mentioned:

3) "lateness"

@Matteo wrote:
> You should check the latency of your pipeline. It is possibile that the
> computed latency is on the edge of the rendering deadline.

Based on my, admittedly not very deep, knowledge of gstreamer, latency is
relevant to live streams, whereas (according to gstreamer docs) "For
pipelines where the only elements that synchronize against the clock are the
sinks, the latency is always 0". Was perhaps "max-lateness", what you meant?
In any case, I feel your suggestion is similar to what's been discussed in
the thread I referred to in my original post, i.e. that frames might be
dropped because some processing step takes too long (in my understanding). I
was trying to get around that by setting "sync=false" and controlling the
frame rate with the "videorate" element - not sure if it has the correct
effect, but it improved the playback quality a bit. I've also tried to apply
"max-lateness=-1" to the glimagesink, with no visible effect. That might be
because  I suspect frames aren't really being dropped(?).

4) glimagesink vsync

@Nicolas wrote:
>Note that there is no "vsync" property on glimagesink in mainline
GStreamer. Please review all your applied patches as they might also be the
cause of all your troubles.

I'm using gstreamer compiled with Cerbero master (1.17) with visualstudio,
nvcodec and intelmsdk variants enabled. No special patches I know of.
Glimagesink might not have a vsync parameter, but it's not complaining about
it either when I use it. Should it? "fullscreen=true" doesn't seem to work
with it either but it's not showing any error message when it's added.

5) recent merge removing nvcodec latency

@Nicolas wrote:
> https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/merge_requests/998
> It's not much, but it removes 1 frame latency on the decoder.

I've re-compiled gstreamer with Cerbero/master today, hopefully that
included this patch. Otherwise I'm not sure how to apply it.




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: 4K playback optimization help needed

Tamas
Need to correct/clarify some of what I wrote:

I'm doing the tests on my home PC (old & weak with GTX 1060) and my work PC
(i7 8700K, 1080Ti), both Windows 10 x64, CUDA 10.2, recent NVidia drivers.
The results I reported were a bit confusing (to me definitely) because of
obtaining them on 2 different machines.

The pipeline nvh265dec -> d3d11videosink works erratically: it results in a
green screen on my work PC; yesterday I got gray or white screen at home,
but now it's OK. CPU decoding at home was slow yesterday, whereas it's quite
smooth today. No idea why? Maybe one day I hibernated the PC, shut down the
other day, or something was running in the background. I'm sorry for the
confusion.

Results summary:

Work PC (today):
1. CPU (avdec_h265 -> d3d11videosink) is smooth (= subjectively, and as
smooth as MPC-BE)
2. GPU (nvh265dec -> glimagesink) is a bit choppy, queues / videorate /
sync=false improve a bit
3. nvh265dec -> d3d11videosink gives green screen (tried w/ and w/o
gldownload)

Home PC (yesterday):
1. CPU decoding too slow
2. choppy, better with queues / videorate / sync=false
3. results in gray or white screen

Home PC (today):
1. CPU is smooth, reports dropped frames
2. same as yesterday
3. (replacing glimagesink with d3d11videosink in #2 pipeline) works,
subjectively same choppy as #2

Despite varying results, GPU decoding with nvh265dec -> glimagesink always
results in a jumpier playback. In all cases, glimagesink sync=false produced
fast, smooth video.




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: 4K playback optimization help needed

Tamas
I really shouldn't post when sleepy. The smooth playback on CPU doesn't
report dropped frames (Home PC, today), got it yesterday when it was too
slow. Sorry for the repeated corrections.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: 4K playback optimization help needed

Nicolas Dufresne-5
Le lundi 27 janvier 2020 à 07:49 -0600, QwjN1Y9mJvamZJ a écrit :
> I really shouldn't post when sleepy. The smooth playback on CPU doesn't
> report dropped frames (Home PC, today), got it yesterday when it was too
> slow. Sorry for the repeated corrections.

Next step would be to look at the latency of the decoder.

  GST_TRACERS=latency\(flags=element\) GST_DEBUG=GST_TRACER:8 ./app 2> gst.log
  gst-stats gst.log

That should give us per element latency, we need to look at what is the latency
of nvh264dec here, and compare against avdec_h264. Another possibility is that
there is a timestamp issue.

>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: 4K playback optimization help needed

Tamas
I tried to do what you told me, but there seems to be some kind of problem.

set GST_TRACERS=latency(flags=element)
set GST_DEBUG=GST_TRACER:8
set GST_DEBUG_FILE=gst.log

pipe A: filesrc loc=4K.mkv ! matroskademux ! h265parse ! avdec_h265 !
d3d11videosink 2>gsterr.log
pipe B: ... nvh265dec ! glimagesink

Both pipes stop (as in program quits) after PREROLL.

gst.log:
"
0:00:00.241287000 10708 0000016FC2F62F90 DEBUG             GST_TRACER
gsttracer.c:162:gst_tracer_register:<registry0> update existing feature
0000016FC0E7E3C0 (latency)
0:00:00.241322000 10708 0000016FC2F62F90 DEBUG             GST_TRACER
gsttracer.c:162:gst_tracer_register:<registry0> update existing feature
0000016FC0E7E300 (log)
0:00:00.241343000 10708 0000016FC2F62F90 DEBUG             GST_TRACER
gsttracer.c:162:gst_tracer_register:<registry0> update existing feature
0000016FC0E7E240 (stats)
0:00:00.241365000 10708 0000016FC2F62F90 DEBUG             GST_TRACER
gsttracer.c:162:gst_tracer_register:<registry0> update existing feature
0000016FC0E7E180 (leaks)
0:00:00.241440000 10708 0000016FC2F62F90 TRACE             GST_TRACER
gsttracerrecord.c:111:gst_tracer_record_build_format: latency.class,
src-element-id=(structure)"scope\,\ type\=\(type\)gchararray\,\
related-to\=\(GstTracerValueScope\)element\;",
src-element=(structure)"scope\,\ type\=\(type\)gchararray\,\
related-to\=\(GstTracerValueScope\)element\;", src=(structure)"scope\,\
type\=\(type\)gchararray\,\ related-to\=\(GstTracerValueScope\)pad\;",
sink-element-id=(structure)"scope\,\ type\=\(type\)gchararray\,\
related-to\=\(GstTracerValueScope\)element\;",
sink-element=(structure)"scope\,\ type\=\(type\)gchararray\,\
related-to\=\(GstTracerValueScope\)element\;", sink=(structure)"scope\,\
type\=\(type\)gchararray\,\ related-to\=\(GstTracerValueScope\)pad\;",
time=(structure)"value\,\ type\=\(type\)guint64\,\
description\=\(string\)\"time\\\ it\\\ took\\\ for\\\ the\\\ buffer\\\ to\\\
go\\\ from\\\ src\\\ to\\\ sink\\\ ns\"\,\ min\=\(guint64\)0\,\
max\=\(guint64\)18446744073709551615\;", ts=(structure)"value\,\
type\=\(type\)guint64\,\ description\=\(string\)\"ts\\\ when\\\ the\\\
latency\\\ has\\\ been\\\ logged\"\,\ min\=\(guint64\)0\,\
max\=\(guint64\)18446744073709551615\;";
0:00:00.241467000 10708 0000016FC2F62F90 DEBUG             GST_TRACER
gsttracerrecord.c:125:gst_tracer_record_build_format: new format string:
latency, src-element-id=(string)%s, src-element=(string)%s, src=(string)%s,
sink-element-id=(string)%s, sink-element=(string)%s, sink=(string)%s,
time=(guint64)%I64u, ts=(guint64)%I64u;
0:00:00.241506000 10708 0000016FC2F62F90 TRACE             GST_TRACER
gsttracerrecord.c:111:gst_tracer_record_build_format: element-latency.class,
element-id=(structure)"scope\,\ type\=\(type\)gchararray\,\
related-to\=\(GstTracerValueScope\)element\;", element=(structure)"scope\,\
type\=\(type\)gchararray\,\ related-to\=\(GstTracerValueScope\)element\;",
src=(structure)"scope\,\ type\=\(type\)gchararray\,\
related-to\=\(GstTracerValueScope\)pad\;", time=(structure)"value\,\
type\=\(type\)guint64\,\ description\=\(string\)\"time\\\ it\\\ took\\\
for\\\ the\\\ buffer\\\ to\\\ go\\\ from\\\ src\\\ to\\\ sink\\\ ns\"\,\
min\=\(guint64\)0\,\ max\=\(guint64\)18446744073709551615\;",
ts=(structure)"value\,\ type\=\(type\)guint64\,\
description\=\(string\)\"ts\\\ when\\\ the\\\ latency\\\ has\\\ been\\\
logged\"\,\ min\=\(guint64\)0\,\ max\=\(guint64\)18446744073709551615\;";
0:00:00.241529000 10708 0000016FC2F62F90 DEBUG             GST_TRACER
gsttracerrecord.c:125:gst_tracer_record_build_format: new format string:
element-latency, element-id=(string)%s, element=(string)%s, src=(string)%s,
time=(guint64)%I64u, ts=(guint64)%I64u;
0:00:00.241570000 10708 0000016FC2F62F90 TRACE             GST_TRACER
gsttracerrecord.c:111:gst_tracer_record_build_format:
element-reported-latency.class, element-id=(structure)"scope\,\
type\=\(type\)gchararray\,\ related-to\=\(GstTracerValueScope\)element\;",
element=(structure)"scope\,\ type\=\(type\)gchararray\,\
related-to\=\(GstTracerValueScope\)element\;", live=(structure)"value\,\
type\=\(type\)gboolean\,\ description\=\(string\)\"wether\\\ the\\\ it\\\
is\\\ a\\\ live\\\ stream\\\ or\\\ not\"\;", min=(structure)"value\,\
type\=\(type\)guint64\,\ description\=\(string\)\"the\\\ minimum\\\
reported\\\ latency\"\,\ min\=\(guint64\)0\,\
max\=\(guint64\)18446744073709551615\;", max=(structure)"value\,\
type\=\(type\)guint64\,\ description\=\(string\)\"the\\\ maximum\\\
reported\\\ latency\"\,\ min\=\(guint64\)0\,\
max\=\(guint64\)18446744073709551615\;", ts=(structure)"value\,\
type\=\(type\)guint64\,\ description\=\(string\)\"ts\\\ when\\\ the\\\
latency\\\ has\\\ been\\\ reported\"\,\ min\=\(guint64\)0\,\
max\=\(guint64\)18446744073709551615\;";
0:00:00.241621000 10708 0000016FC2F62F90 DEBUG             GST_TRACER
gsttracerrecord.c:125:gst_tracer_record_build_format: new format string:
element-reported-latency, element-id=(string)%s, element=(string)%s,
live=(boolean)%i, min=(guint64)%I64u, max=(guint64)%I64u, ts=(guint64)%I64u;
0:00:01.069430000 10708 0000016FC31B3040 TRACE             GST_TRACER :0::
element-latency, element-id=(string)0000016FC0F02880,
element=(string)matroskademux0, src=(string)video_0, time=(guint64)18824000,
ts=(guint64)1069414000;
0:00:01.073533000 10708 0000016FC31B3040 TRACE             GST_TRACER :0::
element-latency, element-id=(string)0000016FC2F816C0,
element=(string)h265parse0, src=(string)src, time=(guint64)4108000,
ts=(guint64)1073522000;
"
gsterr.log:
"
(gst-launch-1.0:14708): GStreamer-CRITICAL **: 10:12:03.801:
gst_object_get_name: assertion 'GST_IS_OBJECT (object)' failed

(gst-launch-1.0:14708): GStreamer-CRITICAL **: 10:12:03.801:
gst_object_get_name: assertion 'GST_IS_OBJECT (object)' failed

(gst-launch-1.0:14708): GStreamer-CRITICAL **: 10:12:03.801:
gst_object_get_name: assertion 'GST_IS_OBJECT (object)' failed

(gst-launch-1.0:14708): GStreamer-CRITICAL **: 10:12:03.801:
gst_object_get_name: assertion 'GST_IS_OBJECT (object)' failed

...
"
same message many more times (56 in total in one case)

I tried variations on how the environment variables are defined, e.g. ""
(GST_DEBUG="GST_TRACER:8" as in the tracing docs), \( as you wrote or just
(, etc.

Also tried GST_TRACERS=latency, in which case the same thing happened
without critical error messages and the debug log missed the last 2 lines.

Also tried erroneous input, e.g. GST_TRACERS=aaaaaa, in which case the
pipeline played, but there was nothing in the log.

Also tried GST_TRACERS=leaks and =stats; both played, and the latter
produced a growing log with data. Can that data be used for something useful
here?




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: 4K playback optimization help needed

Nicolas Dufresne-5
Le lundi 27 janvier 2020 à 19:31 -0600, QwjN1Y9mJvamZJ a écrit :
> (gst-launch-1.0:14708): GStreamer-CRITICAL **: 10:12:03.801:
> gst_object_get_name: assertion 'GST_IS_OBJECT (object)' failed

That's an assertion, like a crash avoidance mechanism. This would need
to be debugged. Which GStreamer version is this ?

Nicolas

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: 4K playback optimization help needed

Tamas

Gstreamer v1.17.0 compiled yesterday with cerbero/master on win 10 x64, variants visualstudio, nvdocec and intelmsdk.

I'm using HTML formatting because I added some images, hope it works well for everyone.

Although I haven't been able to get the tracers to work, instead, I logged the PTS times on several pads in the pipeline by adding probes to them (gst_pad_add_probe(pad, GST_PAD_PROBE_TYPE_BUFFER, ...)). I don't have a thorough understanding of what PTS values mean, so I'm not sure this information is absolutely relevant, but I saw a difference between using nvh265dec and avdec_h265.

Pipeline A: filesrc location=4K.mkv ! matroskademux ! h265parse ! avdec_h265 ! d3d11videosink
Pipeline B: filesrc location=4K.mkv ! matroskademux ! h265parse ! nvh265dec ! d3d11videosink
Pipeline C: same as B just glimagesink at the end
(I tried with queue or videorate elements as well and the end result was basically the same.)

Fig.1 PTS values in logged order on videosink/sink First I logged each received buffers' PTS time on the videosink's sink pad for each of the pipelines. The PTS values are plotted as received. In the avdec_h265 pipeline, PTS increase frame-to-frame with a constant dPTS (difference), whereas in the other two pipelines every 3rd buffer shares the same PTS with the one before. Pipelines B and C are essentially the same.

PTS values on most pads in nvdec pipeline The 2nd image shows plots of PTS values logged from relevant pads in pipeline B, i.e. using nvh265dec, sorted in increasing order. Sorting was necessary as the decoder and parser elements didn't receive buffers in order; I can show the data without sorting if that's more informative. In this playback, the timings were somewhat different than on figure 1. It seems that the parser (h265parse) outputs different PTS values than it receives. I'm not showing the data for pipeline A, but in that case PTS values did not change between elements - all looked like the plot for parser-sink here.

Without a deep understanding, just looking at these numbers, I suspect that playback is choppy because of the uneven PTS increments. PTS timings don't change in pipeline A, whereas when h265parse is followed by nvh265dec, modified PTS values with uneven gradation appear from the parser's sink pad onwards.


Sent from the GStreamer-devel mailing list archive at Nabble.com.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: 4K playback optimization help needed

Tamas
I was able to solve the stuttering playback issue with a dirty fix by storing
the "good" PTS value for each buffer in a reference_timestamp_meta, and
replacing the changed "uneven" PTS values with the good ones later in the
pipeline. The "good" PTS values were stored into meta at the parser's sink,
and were retrieved at the decoder's sink (could've chosen another step in
the pipeline), where the current "bad" PTS values were replaced by them.
This in essence removed the stuttering.

Please let me know if this behaviour (="uneven" increments of PTS) is by
design or a bug. If it's a bug, then is it already known or is it new, and
should I open an issue?




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: 4K playback optimization help needed

Nicolas Dufresne-5
Le mercredi 29 janvier 2020 à 21:57 -0600, QwjN1Y9mJvamZJ a écrit :

> I was able to solve the stuttering playback issue with a dirty fix by storing
> the "good" PTS value for each buffer in a reference_timestamp_meta, and
> replacing the changed "uneven" PTS values with the good ones later in the
> pipeline. The "good" PTS values were stored into meta at the parser's sink,
> and were retrieved at the decoder's sink (could've chosen another step in
> the pipeline), where the current "bad" PTS values were replaced by them.
> This in essence removed the stuttering.
>
> Please let me know if this behaviour (="uneven" increments of PTS) is by
> design or a bug. If it's a bug, then is it already known or is it new, and
> should I open an issue?

This looks like a bug in the nvcodec decoder, please open an issue.

regards,
Nicolas

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel