Quantcast

vaapi changes h264 video dimension during runtime

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

vaapi changes h264 video dimension during runtime

tobatrance
This post was updated on .
hey there,

i'm using gstreamer 1.8.3 and gstreamer-vaapi 1.8.3. i know that my problem is a bug which was fixed in vaapi 1.10.2 but as we are using ubuntu in lts and the next one is coming in april 18 and i know that there must exist a workaround for that problem i want to know how this can be realised. ok, following setup:

i set up this pipeline: filesrc ! decodebin ! appsink. as you probably know a video file has a container width and height and a stream width and height. what i want to receive is the stream width and height. when not using vaapi i always receive the stream width and height from the GstVideoInfo object associated to the GstSample from my appsink. When using vaapi this changes during runtime. first i receive the stream width and height and after some time and changes i receive the container width and height. my question is what invokes this change and how i could grab the stream width and height before this change occurs. i know this must be possible because gst-launch-1.0 show the video correct. here is the gst-launch-1.0 verbose output:

Setting pipeline to PAUSED ...
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
Pipeline is PREROLLING ...
Got context from element 'autovideosink0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)NULL;
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:src: caps = "video/quicktime\,\ variant\=\(string\)iso"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:src: caps = "NULL"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-buffers = 5
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-time = 0
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-bytes = 2097152
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0.GstMultiQueuePad:sink_0: caps = "video/x-h264\,\ stream-format\=\(string\)avc\,\ alignment\=\(string\)au\,\ level\=\(string\)4.1\,\ profile\=\(string\)main\,\ codec_data\=\(buffer\)014d4029ffe10018674d402996520140cfcf2506060640000003004000000ca101000468eb7352\,\ max-input-size\=\(int\)64315\,\ width\=\(int\)640\,\ height\=\(int\)180\,\ framerate\=\(fraction\)25/1\,\ pixel-aspect-ratio\=\(fraction\)1/1"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0.GstMultiQueuePad:sink_0: caps = "video/x-h264\,\ stream-format\=\(string\)avc\,\ alignment\=\(string\)au\,\ level\=\(string\)4.1\,\ profile\=\(string\)main\,\ codec_data\=\(buffer\)014d4029ffe10018674d402996520140cfcf2506060640000003004000000ca101000468eb7352\,\ max-input-size\=\(int\)64315\,\ width\=\(int\)640\,\ height\=\(int\)180\,\ framerate\=\(fraction\)25/1\,\ pixel-aspect-ratio\=\(fraction\)1/1"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstVaapiDecodeBin:vaapidecodebin0.GstGhostPad:sink.GstProxyPad:proxypad4: caps = "video/x-h264\,\ stream-format\=\(string\)avc\,\ alignment\=\(string\)au\,\ level\=\(string\)4.1\,\ profile\=\(string\)main\,\ codec_data\=\(buffer\)014d4029ffe10018674d402996520140cfcf2506060640000003004000000ca101000468eb7352\,\ max-input-size\=\(int\)64315\,\ width\=\(int\)640\,\ height\=\(int\)180\,\ framerate\=\(fraction\)25/1\,\ pixel-aspect-ratio\=\(fraction\)1/1"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-buffers = 5
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-time = 0
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-bytes = 2097152
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstVaapiDecodeBin:vaapidecodebin0/GstVaapiDecode:vaapidecode.GstPad:sink: caps = "video/x-h264\,\ stream-format\=\(string\)avc\,\ alignment\=\(string\)au\,\ level\=\(string\)4.1\,\ profile\=\(string\)main\,\ codec_data\=\(buffer\)014d4029ffe10018674d402996520140cfcf2506060640000003004000000ca101000468eb7352\,\ max-input-size\=\(int\)64315\,\ width\=\(int\)640\,\ height\=\(int\)180\,\ framerate\=\(fraction\)25/1\,\ pixel-aspect-ratio\=\(fraction\)1/1"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstVaapiDecodeBin:vaapidecodebin0.GstGhostPad:sink: caps = "video/x-h264\,\ stream-format\=\(string\)avc\,\ alignment\=\(string\)au\,\ level\=\(string\)4.1\,\ profile\=\(string\)main\,\ codec_data\=\(buffer\)014d4029ffe10018674d402996520140cfcf2506060640000003004000000ca101000468eb7352\,\ max-input-size\=\(int\)64315\,\ width\=\(int\)640\,\ height\=\(int\)180\,\ framerate\=\(fraction\)25/1\,\ pixel-aspect-ratio\=\(fraction\)1/1"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-buffers = 5
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-time = 0
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-bytes = 2097152
Redistribute latency...
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstVaapiDecodeBin:vaapidecodebin0/GstVaapiDecode:vaapidecode.GstPad:src: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)180\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstVaapiDecodeBin:vaapidecodebin0/GstQueue:queue.GstPad:sink: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)180\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstVaapiDecodeBin:vaapidecodebin0/GstQueue:queue.GstPad:sink: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)180\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstVaapiDecodeBin:vaapidecodebin0/GstVaapiPostproc:vaapipostproc.GstPad:src: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)180\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstVaapiDecodeBin:vaapidecodebin0.GstGhostPad:src: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)180\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0.GstGhostPad:sink.GstProxyPad:proxypad1: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)180\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstVaapiSink:autovideosink0-actual-sink-vaapi.GstPad:sink: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)180\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0.GstGhostPad:sink: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)180\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstDecodePad:src_0.GstProxyPad:proxypad3: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)180\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstVaapiDecodeBin:vaapidecodebin0.GstGhostPad:src.GstProxyPad:proxypad5: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)180\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstVaapiDecodeBin:vaapidecodebin0/GstVaapiPostproc:vaapipostproc.GstPad:sink: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)180\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
*******************************************************************************************************
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstVaapiDecodeBin:vaapidecodebin0/GstVaapiDecode:vaapidecode.GstPad:src: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)192\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstVaapiDecodeBin:vaapidecodebin0/GstQueue:queue.GstPad:sink: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)192\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstVaapiDecodeBin:vaapidecodebin0/GstQueue:queue.GstPad:src: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)192\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstVaapiDecodeBin:vaapidecodebin0/GstVaapiPostproc:vaapipostproc.GstPad:src: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)192\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstVaapiDecodeBin:vaapidecodebin0.GstGhostPad:src: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)192\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstDecodePad:src_0: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)192\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0.GstGhostPad:sink.GstProxyPad:proxypad1: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)192\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstVaapiSink:autovideosink0-actual-sink-vaapi.GstPad:sink: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)192\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0.GstGhostPad:sink: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)192\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstDecodePad:src_0.GstProxyPad:proxypad3: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)192\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstVaapiDecodeBin:vaapidecodebin0.GstGhostPad:src.GstProxyPad:proxypad5: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)192\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstVaapiDecodeBin:vaapidecodebin0/GstVaapiPostproc:vaapipostproc.GstPad:sink: caps = "video/x-raw\(memory:VASurface\)\,\ format\=\(string\)I420\,\ width\=\(int\)640\,\ height\=\(int\)192\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ chroma-site\=\(string\)jpeg\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)25/1"
Redistribute latency...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:13.319795651
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

i inserted a line of stars which shows where the change is happening. gst-launch-1.0 is able to fetch the dimensions before the change thats why it displays the video correct and not distorted like my application does.

here's the interessting debug output of my application:

0:00:04.161618183 [335m 8602[00m 0x7f93880094a0 [37mDEBUG  [00m [00m          video-info video-info.c:300:gst_video_info_from_caps:[00m parsing caps video/x-raw(memory:VASurface), format=(string)I420, width=(int)640, height=(int)180, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)bt601, framerate=(fraction)25/1
0:00:04.161634933 [335m 8602[00m 0x7f93880094a0 [37mDEBUG  [00m [00m               vaapi gstvaapidecoder_h264.c:1422:ensure_context:[00m maximum number of views changed to 1
0:00:04.161640942 [335m 8602[00m 0x7f93880094a0 [37mDEBUG  [00m [00m               vaapi gstvaapidecoder_h264.c:1427:ensure_context:[00m DPB size increased
0:00:04.161645658 [335m 8602[00m 0x7f93880094a0 [37mDEBUG  [00m [00m               vaapi gstvaapidecoder_h264.c:1438:ensure_context:[00m profile changed to 2343632
0:00:04.161650612 [335m 8602[00m 0x7f93880094a0 [37mDEBUG  [00m [00m               vaapi gstvaapidecoder_h264.c:1488:ensure_context:[00m size changed
0:00:04.161655077 [335m 8602[00m 0x7f93880094a0 [37mDEBUG  [00m [00m               vaapi gstvaapidecoder.c:836:gst_vaapi_decoder_set_picture_size:[00m picture height changed to 192
0:00:04.161662953 [335m 8602[00m 0x7f93880094a0 [37mDEBUG  [00m [00;04m             default gstvideoutils.c:161:_gst_video_codec_state_free:[00m free state 0x7f937800a150
0:00:04.161669554 [335m 8602[00m 0x7f93880094a0 [37mDEBUG  [00m [00;01;34m          GST_MEMORY gstmemory.c:87:_gst_memory_free:[00m free memory 0x7f937800b040
0:00:04.161685949 [335m 8602[00m 0x7f93880094a0 [37mDEBUG  [00m [00;01;37;41m     GST_PERFORMANCE gstcaps.c:178:_gst_caps_copy:video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)4.1, profile=(string)main, codec_data=(buffer)014d4029ffe10018674d402996520140cfcf2506060640000003004000000ca101000468eb7352, max-input-size=(int)64315, width=(int)640, height=(int)192, framerate=(fraction)25/1, pixel-aspect-ratio=(fraction)1/1[00m doing copy 0x7f9378006ca0 -> 0x7f9378006de0

what i was able to find out was that the decodebin, resp. the gstvaapidecodebin, causes the trouble here as its sink caps have height=180 while its src caps have height=192.

i guess this is somehow related to state change? any hint on how to workaround this is welcome
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: vaapi changes h264 video dimension during runtime

tobatrance
solved. for what ever reason the gst_discoverer api offers an opportunity to workaround this. one might read print_topology() and print_stream_info() in the gst-discoverer.c
Loading...