h264parse: nal.size of incoming frame is corrupted

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

h264parse: nal.size of incoming frame is corrupted

Makoto Harada
Hello experts,

I'm now investigating the problem that h264parse does not work as expected in some cases.

My pipelines are as shown below.

(OK)
gst-launch-1.0 filesrc location=test.mp4 typefind=true '!' qtdemux name=demux '!' video/x-h264 '!' h264parse '!' video/x-h264,stream-format=byte-stream '!' filesink location=test.h264
gst-launch-1.0 filesrc location=test.mp4 typefind=true '!' qtdemux name=demux '!' video/x-h264 '!' h264parse '!' video/x-h264,stream-format=byte-stream '!' mpegtsmux ! filesink location=test.ts

(NG)
gst-launch-1.0 filesrc location=test.mp4 typefind=true '!' qtdemux name=demux '!' video/x-h264 '!' h264parse '!' video/x-h264,stream-format=byte-stream '!' mpegtsmux ! appsrc

In OK case, h264parse works as expected and the test.h264/test.ts has correct data. We can decode this data as expected.
In NG case, h264parse outputs corrupted h264 data.
I see lot's of the "not processing nal size 1" message, when GST_DEBUG="h264parse:5".

I made debugging and it turned out that nalu.size is corrupted in NG case as shown below.
Other fields are ok.

(OK)
   gst_h264_parse_handle_frame_packetized:<h264parse0> processing packet buffer of size 32
   gst_h264_parse_handle_frame_packetized:<h264parse0>nal offset : 4, nal size 28           <= ok                                              
   gst_h264_parse_handle_frame_packetized:<h264parse0>AVC nal offset 32                    
   gst_h264_parse_process_nal:<h264parse0>processing nal of type 1 Slice, size 28          

(NG)
   gst_h264_parse_handle_frame_packetized:<h264parse0>processing packet buffer of size 32
   gst_h264_parse_handle_frame_packetized:<h264parse0>nal offset : 4, nal size 1               <= nal size is corrupted to 1 !!
   gst_h264_parse_handle_frame_packetized:<h264parse0>AVC nal offset 5
   gst_h264_parse_process_nal:<h264parse0>  not processing nal size 1

nalu.size is got by the first 32 bit data of buffer in incoming frame to h264pasre plugin.
I confirmed this first 32 bit is broken at gst_h264_parse_handle_frame () while data size is expected.

version of h264parse I'm using is 1.4.1.

Question:
1. Is this known issue ? I could not find out the commit in git.
2. Who writes to this nalu.size field (first 32 bit of incoming frame to h264parse ) ?
    parent class of h264parse or qtdemux ?
    I could not figure it out the source code which does this yet.

Any feedback is highly appreciated.

Kind Regards,
Makoto


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: h264parse: nal.size of incoming frame is corrupted

Makoto Harada
> (NG)
> gst-launch-1.0 filesrc location=test.mp4 typefind=true '!' qtdemux name=demux '!' video/x-h264 '!' h264parse '!' video/x-h264,stream-format=byte-stream '!' mpegtsmux ! appsrc
>
ops.
sorry appsrc should be appsink.

Makoto

On 2017/03/16 20:03, Makoto Harada wrote:

> Hello experts,
>
> I'm now investigating the problem that h264parse does not work as expected in some cases.
>
> My pipelines are as shown below.
>
> (OK)
> gst-launch-1.0 filesrc location=test.mp4 typefind=true '!' qtdemux name=demux '!' video/x-h264 '!' h264parse '!' video/x-h264,stream-format=byte-stream '!' filesink location=test.h264
> gst-launch-1.0 filesrc location=test.mp4 typefind=true '!' qtdemux name=demux '!' video/x-h264 '!' h264parse '!' video/x-h264,stream-format=byte-stream '!' mpegtsmux ! filesink location=test.ts
>
> (NG)
> gst-launch-1.0 filesrc location=test.mp4 typefind=true '!' qtdemux name=demux '!' video/x-h264 '!' h264parse '!' video/x-h264,stream-format=byte-stream '!' mpegtsmux ! appsrc
>
> In OK case, h264parse works as expected and the test.h264/test.ts has correct data. We can decode this data as expected.
> In NG case, h264parse outputs corrupted h264 data.
> I see lot's of the "not processing nal size 1" message, when GST_DEBUG="h264parse:5".
>
> I made debugging and it turned out that nalu.size is corrupted in NG case as shown below.
> Other fields are ok.
>
> (OK)
>    gst_h264_parse_handle_frame_packetized:<h264parse0> processing packet buffer of size 32
>    gst_h264_parse_handle_frame_packetized:<h264parse0>nal offset : 4, nal size 28           <= ok                                              
>    gst_h264_parse_handle_frame_packetized:<h264parse0>AVC nal offset 32                    
>    gst_h264_parse_process_nal:<h264parse0>processing nal of type 1 Slice, size 28          
>
> (NG)
>    gst_h264_parse_handle_frame_packetized:<h264parse0>processing packet buffer of size 32
>    gst_h264_parse_handle_frame_packetized:<h264parse0>nal offset : 4, nal size 1               <= nal size is corrupted to 1 !!
>    gst_h264_parse_handle_frame_packetized:<h264parse0>AVC nal offset 5
>    gst_h264_parse_process_nal:<h264parse0>  not processing nal size 1
>
> nalu.size is got by the first 32 bit data of buffer in incoming frame to h264pasre plugin.
> I confirmed this first 32 bit is broken at gst_h264_parse_handle_frame () while data size is expected.
>
> version of h264parse I'm using is 1.4.1.
>
> Question:
> 1. Is this known issue ? I could not find out the commit in git.
> 2. Who writes to this nalu.size field (first 32 bit of incoming frame to h264parse ) ?
>     parent class of h264parse or qtdemux ?
>     I could not figure it out the source code which does this yet.
>
> Any feedback is highly appreciated.
>
> Kind Regards,
> Makoto
>
>

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: h264parse: nal.size of incoming frame is corrupted

Vivia Nikolaidou
Hi,

This looks like a bug in h264parse. Please file a bug here:
https://bugzilla.gnome.org/enter_bug.cgi?product=GStreamer (should be
gst-plugins-bad)

Make sure you include the debug output of one OK and one NG case (run
with the env var GST_DEBUG=*:6 and GST_DEBUG_NO_COLOR=1)

Thank you very much!

Best regards,

Vivia

On 17 March 2017 at 08:57, Makoto Harada <[hidden email]> wrote:

>> (NG)
>> gst-launch-1.0 filesrc location=test.mp4 typefind=true '!' qtdemux name=demux '!' video/x-h264 '!' h264parse '!' video/x-h264,stream-format=byte-stream '!' mpegtsmux ! appsrc
>>
> ops.
> sorry appsrc should be appsink.
>
> Makoto
>
> On 2017/03/16 20:03, Makoto Harada wrote:
>> Hello experts,
>>
>> I'm now investigating the problem that h264parse does not work as expected in some cases.
>>
>> My pipelines are as shown below.
>>
>> (OK)
>> gst-launch-1.0 filesrc location=test.mp4 typefind=true '!' qtdemux name=demux '!' video/x-h264 '!' h264parse '!' video/x-h264,stream-format=byte-stream '!' filesink location=test.h264
>> gst-launch-1.0 filesrc location=test.mp4 typefind=true '!' qtdemux name=demux '!' video/x-h264 '!' h264parse '!' video/x-h264,stream-format=byte-stream '!' mpegtsmux ! filesink location=test.ts
>>
>> (NG)
>> gst-launch-1.0 filesrc location=test.mp4 typefind=true '!' qtdemux name=demux '!' video/x-h264 '!' h264parse '!' video/x-h264,stream-format=byte-stream '!' mpegtsmux ! appsrc
>>
>> In OK case, h264parse works as expected and the test.h264/test.ts has correct data. We can decode this data as expected.
>> In NG case, h264parse outputs corrupted h264 data.
>> I see lot's of the "not processing nal size 1" message, when GST_DEBUG="h264parse:5".
>>
>> I made debugging and it turned out that nalu.size is corrupted in NG case as shown below.
>> Other fields are ok.
>>
>> (OK)
>>    gst_h264_parse_handle_frame_packetized:<h264parse0>  processing packet buffer of size 32
>>    gst_h264_parse_handle_frame_packetized:<h264parse0> nal offset : 4, nal size 28           <= ok
>>    gst_h264_parse_handle_frame_packetized:<h264parse0> AVC nal offset 32
>>    gst_h264_parse_process_nal:<h264parse0>processing nal of type 1 Slice, size 28
>>
>> (NG)
>>    gst_h264_parse_handle_frame_packetized:<h264parse0> processing packet buffer of size 32
>>    gst_h264_parse_handle_frame_packetized:<h264parse0> nal offset : 4, nal size 1               <= nal size is corrupted to 1 !!
>>    gst_h264_parse_handle_frame_packetized:<h264parse0> AVC nal offset 5
>>    gst_h264_parse_process_nal:<h264parse0>   not processing nal size 1
>>
>> nalu.size is got by the first 32 bit data of buffer in incoming frame to h264pasre plugin.
>> I confirmed this first 32 bit is broken at gst_h264_parse_handle_frame () while data size is expected.
>>
>> version of h264parse I'm using is 1.4.1.
>>
>> Question:
>> 1. Is this known issue ? I could not find out the commit in git.
>> 2. Who writes to this nalu.size field (first 32 bit of incoming frame to h264parse ) ?
>>     parent class of h264parse or qtdemux ?
>>     I could not figure it out the source code which does this yet.
>>
>> Any feedback is highly appreciated.
>>
>> Kind Regards,
>> Makoto
>>
>>
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: h264parse: nal.size of incoming frame is corrupted

Makoto Harada
Vivia,

Thanks for your attention.

I made a deep debug on this problem, however fall into difficult-to-understand situation.

First of all, I have to make a correction for the problematic pipeline.
------------------------------------------------------------------------------------
1. Problematic pipe line is as followings.

(NG)
        gst-launch-1.0 $VERBOSE filesrc location=$INPUT typefind=true ! qtdemux name=demux ! tee name=t \
        t. ! queue max-size-buffers=200000000 max-size-bytes=0 max-size-time=0 name=queue_byte_stream ! video/x-h264 ! h264parse  !  video/x-h264,stream-format=byte-stream ! mpegtsmux ! appsrc \
        t. ! queue max-size-buffers=0 max-size-bytes=0 max-size-time=0 name=queue_avc ! video/x-h264,stream-format=avc ! custom_dec ! custom_sink

(OK)
        gst-launch-1.0 $VERBOSE filesrc location=$INPUT typefind=true ! qtdemux name=demux ! tee name=t \
        t. ! queue max-size-buffers=200000000 max-size-bytes=0 max-size-time=0 name=queue_byte_stream ! video/x-h264 ! h264parse  !  video/x-h264,stream-format=byte-stream ! mpegtsmux ! appsrc \
        t. ! queue max-size-buffers=0 max-size-bytes=0 max-size-time=0 name=queue_avc ! video/x-h264,stream-format=avc ! fakesink
     
        gst-launch-1.0 $VERBOSE filesrc location=$INPUT typefind=true ! qtdemux name=demux ! tee name=t \
        t. ! queue max-size-buffers=200000000 max-size-bytes=0 max-size-time=0 name=queue_byte_stream ! video/x-h264 ! h264parse  !  video/x-h264,stream-format=byte-stream ! mpegtsmux ! appsrc

------------------------------------------------------------------------------------

2. Following description is valid for how to corrupt the data.

> I made debugging and it turned out that nalu.size is corrupted in NG case as shown below.
> Other fields are ok.
>
> (OK)
>    gst_h264_parse_handle_frame_packetized:<h264parse0>  processing packet buffer of size 32
>    gst_h264_parse_handle_frame_packetized:<h264parse0> nal offset : 4, nal size 28           <= ok
>    gst_h264_parse_handle_frame_packetized:<h264parse0> AVC nal offset 32
>    gst_h264_parse_process_nal:<h264parse0>processing nal of type 1 Slice, size 28
>
> (NG)
>    gst_h264_parse_handle_frame_packetized:<h264parse0> processing packet buffer of size 32
>    gst_h264_parse_handle_frame_packetized:<h264parse0> nal offset : 4, nal size 1               <= nal size is corrupted to 1 !!
>    gst_h264_parse_handle_frame_packetized:<h264parse0> AVC nal offset 5
>    gst_h264_parse_process_nal:<h264parse0>   not processing nal size 1
>
> nalu.size is got by the first 32 bit data of buffer in incoming frame to h264pasre plugin.

3. After digging into deeply,  it found out that deta corruption happens at queue_byte_stream  of "queue" element.
    - input data of queue_byte_stream of "queue" element is not corrupted.
    - output data of queue_byte_stream of "queue" element is corrupted.
        - data is fine at gst_queue_array_push_tail() in gst_queue_locked_enqueue_buffer()
         - data is corrupted at gst_queue_array_pop_head () in gst_queue_locked_dequeue ()
 
Another information regarding queue of "queue"
   - buffer underrun/overflow is not happened.
   - the output data of queue_avc  of "queue" element is not corrupted.

I can not understand at all why the data of queue is corrupted in spite of that the buffer underrun/overflow is not happened.
Even if underrun/overflow happened, the whole buffer should be dropped.

Any idea how to debug this issue furthermore ?

Kind Regards,
Makoto

On 2017/03/20 23:47, Vivia Nikolaidou wrote:

> Hi,
>
> This looks like a bug in h264parse. Please file a bug here:
> https://bugzilla.gnome.org/enter_bug.cgi?product=GStreamer (should be
> gst-plugins-bad)
>
> Make sure you include the debug output of one OK and one NG case (run
> with the env var GST_DEBUG=*:6 and GST_DEBUG_NO_COLOR=1)
>
> Thank you very much!
>
> Best regards,
>
> Vivia
>
> On 17 March 2017 at 08:57, Makoto Harada <[hidden email]> wrote:
>>> (NG)
>>> gst-launch-1.0 filesrc location=test.mp4 typefind=true '!' qtdemux name=demux '!' video/x-h264 '!' h264parse '!' video/x-h264,stream-format=byte-stream '!' mpegtsmux ! appsrc
>>>
>> ops.
>> sorry appsrc should be appsink.
>>
>> Makoto
>>
>> On 2017/03/16 20:03, Makoto Harada wrote:
>>> Hello experts,
>>>
>>> I'm now investigating the problem that h264parse does not work as expected in some cases.
>>>
>>> My pipelines are as shown below.
>>>
>>> (OK)
>>> gst-launch-1.0 filesrc location=test.mp4 typefind=true '!' qtdemux name=demux '!' video/x-h264 '!' h264parse '!' video/x-h264,stream-format=byte-stream '!' filesink location=test.h264
>>> gst-launch-1.0 filesrc location=test.mp4 typefind=true '!' qtdemux name=demux '!' video/x-h264 '!' h264parse '!' video/x-h264,stream-format=byte-stream '!' mpegtsmux ! filesink location=test.ts
>>>
>>> (NG)
>>> gst-launch-1.0 filesrc location=test.mp4 typefind=true '!' qtdemux name=demux '!' video/x-h264 '!' h264parse '!' video/x-h264,stream-format=byte-stream '!' mpegtsmux ! appsrc
>>>
>>> In OK case, h264parse works as expected and the test.h264/test.ts has correct data. We can decode this data as expected.
>>> In NG case, h264parse outputs corrupted h264 data.
>>> I see lot's of the "not processing nal size 1" message, when GST_DEBUG="h264parse:5".
>>>
>>> I made debugging and it turned out that nalu.size is corrupted in NG case as shown below.
>>> Other fields are ok.
>>>
>>> (OK)
>>>    gst_h264_parse_handle_frame_packetized:<h264parse0>  processing packet buffer of size 32
>>>    gst_h264_parse_handle_frame_packetized:<h264parse0> nal offset : 4, nal size 28           <= ok
>>>    gst_h264_parse_handle_frame_packetized:<h264parse0> AVC nal offset 32
>>>    gst_h264_parse_process_nal:<h264parse0>processing nal of type 1 Slice, size 28
>>>
>>> (NG)
>>>    gst_h264_parse_handle_frame_packetized:<h264parse0> processing packet buffer of size 32
>>>    gst_h264_parse_handle_frame_packetized:<h264parse0> nal offset : 4, nal size 1               <= nal size is corrupted to 1 !!
>>>    gst_h264_parse_handle_frame_packetized:<h264parse0> AVC nal offset 5
>>>    gst_h264_parse_process_nal:<h264parse0>   not processing nal size 1
>>>
>>> nalu.size is got by the first 32 bit data of buffer in incoming frame to h264pasre plugin.
>>> I confirmed this first 32 bit is broken at gst_h264_parse_handle_frame () while data size is expected.
>>>
>>> version of h264parse I'm using is 1.4.1.
>>>
>>> Question:
>>> 1. Is this known issue ? I could not find out the commit in git.
>>> 2. Who writes to this nalu.size field (first 32 bit of incoming frame to h264parse ) ?
>>>     parent class of h264parse or qtdemux ?
>>>     I could not figure it out the source code which does this yet.
>>>
>>> Any feedback is highly appreciated.
>>>
>>> Kind Regards,
>>> Makoto
>>>
>>>
>> _______________________________________________
>> gstreamer-devel mailing list
>> [hidden email]
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Loading...