"broken/invalid nal" with h264parse after v4l2h264enc on i.MX6

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

"broken/invalid nal" with h264parse after v4l2h264enc on i.MX6

Maurer, Marie

Hello,

 

environment is GStreamer 1.14.2 on Linux with i.MX6 (most of it latest mainlined versions)

 

I see a lot messages like the following:

 

WARN   h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame:<RecorderH264Parser> broken/invalid nal Type: 1 Slice, Size: 81703 will be dropped

 

Size is changing, but always around between 50000 and 80000.

 

I think the relevant part of my pipeline is a v4l2h264enc and after some queues and tees a h264parse.

 

Is it correct that somehow the parser does not understand the data which the v4l2h264enc is generating?

Or the data which the encoder is generating gets sometimes somehow corrupted till it is received by h264parse?

 

Any idea how it can be solved?

 

Best regards,

 

Marie

 


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6

Nicolas Dufresne-5
Le mercredi 12 décembre 2018 à 17:12 +0000, Maurer, Marie a écrit :
> Hello,
>  
> environment is GStreamer 1.14.2 on Linux with i.MX6 (most of it latest mainlined versions)

Please, try again with 1.14.4, as there is some fixes in the 1.14
stable branch. And if you can, complain to Yocto people who seem not to
stay up-to-date with stable.

>  
> I see a lot messages like the following:
>  
> WARN   h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame:<RecorderH264Parser> broken/invalid nal Type: 1 Slice, Size: 81703 will be dropped
>  
> Size is changing, but always around between 50000 and 80000.
>  
> I think the relevant part of my pipeline is a v4l2h264enc and after some queues and tees a h264parse.
>  
> Is it correct that somehow the parser does not understand the data which the v4l2h264enc is generating?
> Or the data which the encoder is generating gets sometimes somehow corrupted till it is received by h264parse?

Hard to say, though it's better to update first, and see if any of the
upstream bugfixes improves the situation.

>  
> Any idea how it can be solved?

You would have to share the pipeline you are using, the settings and
all. Ideally if you can reproduce in gst-launch-1.0 form, then I could
try reproduce locally.

>  
> Best regards,
>  
> Marie
>  
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

AW: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6

Maurer, Marie
Hi Nicolas,

many thanks for your answer. I am currently building Gstreamer 1.14.4 (and some more). When it is done, I can retest.
We don't use Yocto, but I had not yet locally updated to 1.14.4.

The complete pipeline is big, but I think I can reduce it to following overview (queue, caps removed, where not explicitly needed):

appsrc -> tee T1

T1 -> v4l2h264enc -> tee T2

T2#1 -> "Streaming"

T2#2 -> ("Video Recording to a file") h264parse -> CAPS streaming-format=avc -> ...

Video Recording to a file contains the h264parse, which reports these warnings.

We attach v4l2h264enc (to T1) only when streaming and/or video recording is enabled to save resources
and detach it when both are switched off.
We attach streaming (to T2) only, when it is enabled and detach it when streaming is disabled.
We attach "video recording to a file" (also to T2) only when user wants it and detach it when user switches it off.
At T1 we have more parts connected, which consume frames.

The main problem I have (beside the reported warnings) is that sometimes, h264parse and following caps can't be linked.

!!!Speculation!!!
This could be, but I am not sure yet, that h264parse reports streaming-format=byte-stream, which does not fit to caps with streaming-format=avc.
We use h264parse to change streaming-format byte-stream (before) to avc (afterwards).
Is there a more intelligent way needed after h264parse instead of just forcing caps to solve link problem?

Another thing I am currently unsure: When we dynamically add h264parse, is there some synchronization to frame data needed, so we start exactly at the beginning of a GOP?
But we are already on compressed side, so is it relevant at all?

Many thanks!

Best regards,

Marie

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Nicolas Dufresne
Gesendet: Mittwoch, 12. Dezember 2018 18:39
An: Discussion of the development of and with GStreamer <[hidden email]>
Betreff: Re: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6

Le mercredi 12 décembre 2018 à 17:12 +0000, Maurer, Marie a écrit :
> Hello,
>  
> environment is GStreamer 1.14.2 on Linux with i.MX6 (most of it latest
> mainlined versions)

Please, try again with 1.14.4, as there is some fixes in the 1.14 stable branch. And if you can, complain to Yocto people who seem not to stay up-to-date with stable.

>  
> I see a lot messages like the following:
>  
> WARN   h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame:<RecorderH264Parser> broken/invalid nal Type: 1 Slice, Size: 81703 will be dropped
>  
> Size is changing, but always around between 50000 and 80000.
>  
> I think the relevant part of my pipeline is a v4l2h264enc and after some queues and tees a h264parse.
>  
> Is it correct that somehow the parser does not understand the data which the v4l2h264enc is generating?
> Or the data which the encoder is generating gets sometimes somehow corrupted till it is received by h264parse?

Hard to say, though it's better to update first, and see if any of the upstream bugfixes improves the situation.

>  
> Any idea how it can be solved?

You would have to share the pipeline you are using, the settings and all. Ideally if you can reproduce in gst-launch-1.0 form, then I could try reproduce locally.

>  
> Best regards,
>  
> Marie
>  
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6

Nicolas Dufresne-5
Le jeudi 13 décembre 2018 à 14:08 +0000, Maurer, Marie a écrit :

> Hi Nicolas,
>
> many thanks for your answer. I am currently building Gstreamer 1.14.4 (and some more). When it is done, I can retest.
> We don't use Yocto, but I had not yet locally updated to 1.14.4.
>
> The complete pipeline is big, but I think I can reduce it to following overview (queue, caps removed, where not explicitly needed):
>
> appsrc -> tee T1
>
> T1 -> v4l2h264enc -> tee T2
>
> T2#1 -> "Streaming"
>
> T2#2 -> ("Video Recording to a file") h264parse -> CAPS streaming-format=avc -> ...
>
> Video Recording to a file contains the h264parse, which reports these warnings.
>
> We attach v4l2h264enc (to T1) only when streaming and/or video recording is enabled to save resources
> and detach it when both are switched off.
> We attach streaming (to T2) only, when it is enabled and detach it when streaming is disabled.
> We attach "video recording to a file" (also to T2) only when user wants it and detach it when user switches it off.
> At T1 we have more parts connected, which consume frames.
>
> The main problem I have (beside the reported warnings) is that sometimes, h264parse and following caps can't be linked.
>
> !!!Speculation!!!
> This could be, but I am not sure yet, that h264parse reports streaming-format=byte-stream, which does not fit to caps with streaming-format=avc.
> We use h264parse to change streaming-format byte-stream (before) to avc (afterwards).
> Is there a more intelligent way needed after h264parse instead of just forcing caps to solve link problem?

Normally that constraint should come form the next element, like a
muxer. If you use a recorder base on ISOMP4 (qtmux), the muxer would
force AVC conversion automatically.

>
> Another thing I am currently unsure: When we dynamically add h264parse, is there some synchronization to frame data needed, so we start exactly at the beginning of a GOP?
> But we are already on compressed side, so is it relevant at all?

Normally, h264parse will drop data until it reaches a state where at
least one SPS and one PPS has been seen. Everything before that is
dropped. I would suggest to review closely you're dynamic linking path,
you have have introduce a race.

>
> Many thanks!
>
> Best regards,
>
> Marie
>
> -----Ursprüngliche Nachricht-----
> Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Nicolas Dufresne
> Gesendet: Mittwoch, 12. Dezember 2018 18:39
> An: Discussion of the development of and with GStreamer <[hidden email]>
> Betreff: Re: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6
>
> Le mercredi 12 décembre 2018 à 17:12 +0000, Maurer, Marie a écrit :
> > Hello,
> >  
> > environment is GStreamer 1.14.2 on Linux with i.MX6 (most of it latest
> > mainlined versions)
>
> Please, try again with 1.14.4, as there is some fixes in the 1.14 stable branch. And if you can, complain to Yocto people who seem not to stay up-to-date with stable.
>
> >  
> > I see a lot messages like the following:
> >  
> > WARN   h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame:<RecorderH264Parser> broken/invalid nal Type: 1 Slice, Size: 81703 will be dropped
> >  
> > Size is changing, but always around between 50000 and 80000.
> >  
> > I think the relevant part of my pipeline is a v4l2h264enc and after some queues and tees a h264parse.
> >  
> > Is it correct that somehow the parser does not understand the data which the v4l2h264enc is generating?
> > Or the data which the encoder is generating gets sometimes somehow corrupted till it is received by h264parse?
>
> Hard to say, though it's better to update first, and see if any of the upstream bugfixes improves the situation.
>
> >  
> > Any idea how it can be solved?
>
> You would have to share the pipeline you are using, the settings and all. Ideally if you can reproduce in gst-launch-1.0 form, then I could try reproduce locally.
>
> >  
> > Best regards,
> >  
> > Marie
> >  
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6

vk_gst
In reply to this post by Maurer, Marie
Is this a new element 'v4l2h264enc ' introduced in 1.14?
On iMX6, there is also a imxvpuenc_h264 present? What is the difference
between them?
Why not use the element imxvpuenc_h264 ?


Regards



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

AW: AW: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6

Maurer, Marie
In reply to this post by Nicolas Dufresne-5
Hello Nicolas,

just a short follow-up: Same problem (warnings and error with gst_element_link) with 1.14.4.

Tried these two commits:

https://github.com/GStreamer/gst-plugins-bad/commit/4f7fe897b9be77c983f935538d30a5c90cecc947

and

https://lists.freedesktop.org/archives/gstreamer-commits/2018-December/109067.html

-> no difference


25xOK (only 3 shown here), 1x Error:

            Line 639452: 0:05:08.615060370 523 0x71d9db50 DEBUG  h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
            Line 667789: 0:05:21.600504371 523 0x9e7d20 DEBUG  h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
            Line 696382: 0:05:34.680368373 523 0x71d13950 DEBUG  h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
            Line 724887: 0:05:47.681662041 523 0x6ff35980 DEBUG  h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: (NULL)

OK:

0:05:34.679142373 523 0x71d13950 DEBUG  h264parse gsth264parse.c:334:gst_h264_parse_format_from_caps:[00m parsing caps: video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)baseline, level=(string)4, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30000/1001, interlace-mode=(string)progressive, colorimetry=(string)bt709
0:05:34.679444373 523 0x71d13950 DEBUG  h264parse gsth264parse.c:2716:gst_h264_parse_set_caps:<RecorderH264Parser> have bytestream h264
0:05:34.680368373 523 0x71d13950 DEBUG  h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
0:05:34.680629040 523 0x71d13950 DEBUG  h264parse gsth264parse.c:381:gst_h264_parse_negotiate:<RecorderH264Parser> negotiating with caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
0:05:34.680830040 523 0x71d13950 DEBUG  h264parse gsth264parse.c:334:gst_h264_parse_format_from_caps: parsing caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
0:05:34.681008373 523 0x71d13950 DEBUG  h264parse gsth264parse.c:411:gst_h264_parse_negotiate:<RecorderH264Parser> selected format avc, alignment au
0:05:34.681650373 523 0x71d13950 LOG    h264parse gsth264parse.c:1102:gst_h264_parse_handle_frame:<RecorderH264Parser> parsing new frame
0:05:34.681862373 523 0x71d13950 DEBUG  h264parse gsth264parse.c:186:gst_h264_parse_reset_frame:<RecorderH264Parser> reset frame

Error:

0:05:47.679435708 523 0x6ff35980 DEBUG  h264parse gsth264parse.c:334:gst_h264_parse_format_from_caps:[00m parsing caps: video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)baseline, level=(string)4, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30000/1001, interlace-mode=(string)progressive, colorimetry=(string)bt709
0:05:47.680866041 523 0x6ff35980 DEBUG  h264parse gsth264parse.c:2716:gst_h264_parse_set_caps:<RecorderH264Parser> have bytestream h264
0:05:47.681662041 523 0x6ff35980 DEBUG  h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: (NULL)
0:05:47.682161374 523 0x6ff35980 DEBUG  h264parse gsth264parse.c:411:gst_h264_parse_negotiate:<RecorderH264Parser> selected format byte-stream, alignment au
0:05:47.682607374 523 0x6ff35980 DEBUG  h264parse gsth264parse.c:1755:gst_h264_parse_update_src_caps:<RecorderH264Parser> sps: (nil)
0:05:47.685859374 523 0x6ff35980 LOG    h264parse gsth264parse.c:1102:gst_h264_parse_handle_frame:<RecorderH264Parser> parsing new frame
0:05:47.686587041 523 0x6ff35980 DEBUG  h264parse gsth264parse.c:186:gst_h264_parse_reset_frame:<RecorderH264Parser> reset frame
0:05:47.687081708 523 0x6ff35980 DEBUG  h264parse gsth264parse.c:1117:gst_h264_parse_handle_frame:<RecorderH264Parser> last parse position 0

Best regards,

Marie

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Nicolas Dufresne
Gesendet: Donnerstag, 13. Dezember 2018 18:39
An: Discussion of the development of and with GStreamer <[hidden email]>
Betreff: Re: AW: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6

Le jeudi 13 décembre 2018 à 14:08 +0000, Maurer, Marie a écrit :

> Hi Nicolas,
>
> many thanks for your answer. I am currently building Gstreamer 1.14.4 (and some more). When it is done, I can retest.
> We don't use Yocto, but I had not yet locally updated to 1.14.4.
>
> The complete pipeline is big, but I think I can reduce it to following overview (queue, caps removed, where not explicitly needed):
>
> appsrc -> tee T1
>
> T1 -> v4l2h264enc -> tee T2
>
> T2#1 -> "Streaming"
>
> T2#2 -> ("Video Recording to a file") h264parse -> CAPS streaming-format=avc -> ...
>
> Video Recording to a file contains the h264parse, which reports these warnings.
>
> We attach v4l2h264enc (to T1) only when streaming and/or video
> recording is enabled to save resources and detach it when both are switched off.
> We attach streaming (to T2) only, when it is enabled and detach it when streaming is disabled.
> We attach "video recording to a file" (also to T2) only when user wants it and detach it when user switches it off.
> At T1 we have more parts connected, which consume frames.
>
> The main problem I have (beside the reported warnings) is that sometimes, h264parse and following caps can't be linked.
>
> !!!Speculation!!!
> This could be, but I am not sure yet, that h264parse reports streaming-format=byte-stream, which does not fit to caps with streaming-format=avc.
> We use h264parse to change streaming-format byte-stream (before) to avc (afterwards).
> Is there a more intelligent way needed after h264parse instead of just forcing caps to solve link problem?

Normally that constraint should come form the next element, like a muxer. If you use a recorder base on ISOMP4 (qtmux), the muxer would force AVC conversion automatically.

>
> Another thing I am currently unsure: When we dynamically add h264parse, is there some synchronization to frame data needed, so we start exactly at the beginning of a GOP?
> But we are already on compressed side, so is it relevant at all?

Normally, h264parse will drop data until it reaches a state where at least one SPS and one PPS has been seen. Everything before that is dropped. I would suggest to review closely you're dynamic linking path, you have have introduce a race.

>
> Many thanks!
>
> Best regards,
>
> Marie
>
> -----Ursprüngliche Nachricht-----
> Von: gstreamer-devel
> [mailto:[hidden email]] Im Auftrag von
> Nicolas Dufresne
> Gesendet: Mittwoch, 12. Dezember 2018 18:39
> An: Discussion of the development of and with GStreamer
> <[hidden email]>
> Betreff: Re: "broken/invalid nal" with h264parse after v4l2h264enc on
> i.MX6
>
> Le mercredi 12 décembre 2018 à 17:12 +0000, Maurer, Marie a écrit :
> > Hello,
> >  
> > environment is GStreamer 1.14.2 on Linux with i.MX6 (most of it
> > latest mainlined versions)
>
> Please, try again with 1.14.4, as there is some fixes in the 1.14 stable branch. And if you can, complain to Yocto people who seem not to stay up-to-date with stable.
>
> >  
> > I see a lot messages like the following:
> >  
> > WARN   h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame:<RecorderH264Parser> broken/invalid nal Type: 1 Slice, Size: 81703 will be dropped
> >  
> > Size is changing, but always around between 50000 and 80000.
> >  
> > I think the relevant part of my pipeline is a v4l2h264enc and after some queues and tees a h264parse.
> >  
> > Is it correct that somehow the parser does not understand the data which the v4l2h264enc is generating?
> > Or the data which the encoder is generating gets sometimes somehow corrupted till it is received by h264parse?
>
> Hard to say, though it's better to update first, and see if any of the upstream bugfixes improves the situation.
>
> >  
> > Any idea how it can be solved?
>
> You would have to share the pipeline you are using, the settings and all. Ideally if you can reproduce in gst-launch-1.0 form, then I could try reproduce locally.
>
> >  
> > Best regards,
> >  
> > Marie
> >  
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

AW: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6

Maurer, Marie
In reply to this post by vk_gst
Hi,

I think imxvpuenc_h264 is in NXP proprietary environment (old linux kernel?),
whereas v4l2h264enc is the name in latest mainline environment (using one of the latest linux kernels).
I am not sure, when it was exactly introduced (or renamed?) into Gstreamer.

Best regards,

Marie


-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von vk_gst
Gesendet: Freitag, 14. Dezember 2018 10:53
An: [hidden email]
Betreff: Re: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6

Is this a new element 'v4l2h264enc ' introduced in 1.14?
On iMX6, there is also a imxvpuenc_h264 present? What is the difference between them?
Why not use the element imxvpuenc_h264 ?


Regards



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6

Nicolas Dufresne-5
In reply to this post by vk_gst
Le vendredi 14 décembre 2018 à 03:53 -0600, vk_gst a écrit :
> Is this a new element 'v4l2h264enc ' introduced in 1.14?
> On iMX6, there is also a imxvpuenc_h264 present? What is the difference
> between them?
> Why not use the element imxvpuenc_h264 ?

imxvpuenc_h264 comes from gstreamer-imx repository and rely on
downstream kernel (a Linux Kernel fork by NXP). v4l2h264enc is a
decoder based on standard Linux CODEC APIs (which in turn is base on
video4linux2 interface). To use this one, you'll need a mainline
kernel. It is a generic implement and you can use it on a wide amount
of embedded SoC running mainline Linux, notably Qualcomm Snapdragon,
Exynos 4/5+, Amlogic, IMX/Coda, etc.

>
>
> Regards
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: AW: AW: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6

Nicolas Dufresne-5
In reply to this post by Maurer, Marie
Le vendredi 14 décembre 2018 à 17:18 +0000, Maurer, Marie a écrit :

> Hello Nicolas,
>
> just a short follow-up: Same problem (warnings and error with gst_element_link) with 1.14.4.
>
> Tried these two commits:
>
> https://github.com/GStreamer/gst-plugins-bad/commit/4f7fe897b9be77c983f935538d30a5c90cecc947
>
> and
>
> https://lists.freedesktop.org/archives/gstreamer-commits/2018-December/109067.html
I don't think any of this is related. Again, can you share the pipeline
you are running, I would maybe help by testing on my IMX.6 board here.

>
> -> no difference
>
>
> 25xOK (only 3 shown here), 1x Error:
>
>             Line 639452: 0:05:08.615060370 523 0x71d9db50 DEBUG  h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
>             Line 667789: 0:05:21.600504371 523 0x9e7d20 DEBUG  h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
>             Line 696382: 0:05:34.680368373 523 0x71d13950 DEBUG  h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
>             Line 724887: 0:05:47.681662041 523 0x6ff35980 DEBUG  h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: (NULL)
>
> OK:
>
> 0:05:34.679142373 523 0x71d13950 DEBUG  h264parse gsth264parse.c:334:gst_h264_parse_format_from_caps:[00m parsing caps: video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)baseline, level=(string)4, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30000/1001, interlace-mode=(string)progressive, colorimetry=(string)bt709
> 0:05:34.679444373 523 0x71d13950 DEBUG  h264parse gsth264parse.c:2716:gst_h264_parse_set_caps:<RecorderH264Parser> have bytestream h264
> 0:05:34.680368373 523 0x71d13950 DEBUG  h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
> 0:05:34.680629040 523 0x71d13950 DEBUG  h264parse gsth264parse.c:381:gst_h264_parse_negotiate:<RecorderH264Parser> negotiating with caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
> 0:05:34.680830040 523 0x71d13950 DEBUG  h264parse gsth264parse.c:334:gst_h264_parse_format_from_caps: parsing caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
> 0:05:34.681008373 523 0x71d13950 DEBUG  h264parse gsth264parse.c:411:gst_h264_parse_negotiate:<RecorderH264Parser> selected format avc, alignment au
> 0:05:34.681650373 523 0x71d13950 LOG    h264parse gsth264parse.c:1102:gst_h264_parse_handle_frame:<RecorderH264Parser> parsing new frame
> 0:05:34.681862373 523 0x71d13950 DEBUG  h264parse gsth264parse.c:186:gst_h264_parse_reset_frame:<RecorderH264Parser> reset frame
>
> Error:
>
> 0:05:47.679435708 523 0x6ff35980 DEBUG  h264parse gsth264parse.c:334:gst_h264_parse_format_from_caps:[00m parsing caps: video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)baseline, level=(string)4, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30000/1001, interlace-mode=(string)progressive, colorimetry=(string)bt709
> 0:05:47.680866041 523 0x6ff35980 DEBUG  h264parse gsth264parse.c:2716:gst_h264_parse_set_caps:<RecorderH264Parser> have bytestream h264
> 0:05:47.681662041 523 0x6ff35980 DEBUG  h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: (NULL)
> 0:05:47.682161374 523 0x6ff35980 DEBUG  h264parse gsth264parse.c:411:gst_h264_parse_negotiate:<RecorderH264Parser> selected format byte-stream, alignment au
> 0:05:47.682607374 523 0x6ff35980 DEBUG  h264parse gsth264parse.c:1755:gst_h264_parse_update_src_caps:<RecorderH264Parser> sps: (nil)
> 0:05:47.685859374 523 0x6ff35980 LOG    h264parse gsth264parse.c:1102:gst_h264_parse_handle_frame:<RecorderH264Parser> parsing new frame
> 0:05:47.686587041 523 0x6ff35980 DEBUG  h264parse gsth264parse.c:186:gst_h264_parse_reset_frame:<RecorderH264Parser> reset frame
> 0:05:47.687081708 523 0x6ff35980 DEBUG  h264parse gsth264parse.c:1117:gst_h264_parse_handle_frame:<RecorderH264Parser> last parse position 0
>
> Best regards,
>
> Marie
>
> -----Ursprüngliche Nachricht-----
> Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Nicolas Dufresne
> Gesendet: Donnerstag, 13. Dezember 2018 18:39
> An: Discussion of the development of and with GStreamer <[hidden email]>
> Betreff: Re: AW: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6
>
> Le jeudi 13 décembre 2018 à 14:08 +0000, Maurer, Marie a écrit :
> > Hi Nicolas,
> >
> > many thanks for your answer. I am currently building Gstreamer 1.14.4 (and some more). When it is done, I can retest.
> > We don't use Yocto, but I had not yet locally updated to 1.14.4.
> >
> > The complete pipeline is big, but I think I can reduce it to following overview (queue, caps removed, where not explicitly needed):
> >
> > appsrc -> tee T1
> >
> > T1 -> v4l2h264enc -> tee T2
> >
> > T2#1 -> "Streaming"
> >
> > T2#2 -> ("Video Recording to a file") h264parse -> CAPS streaming-format=avc -> ...
> >
> > Video Recording to a file contains the h264parse, which reports these warnings.
> >
> > We attach v4l2h264enc (to T1) only when streaming and/or video
> > recording is enabled to save resources and detach it when both are switched off.
> > We attach streaming (to T2) only, when it is enabled and detach it when streaming is disabled.
> > We attach "video recording to a file" (also to T2) only when user wants it and detach it when user switches it off.
> > At T1 we have more parts connected, which consume frames.
> >
> > The main problem I have (beside the reported warnings) is that sometimes, h264parse and following caps can't be linked.
> >
> > !!!Speculation!!!
> > This could be, but I am not sure yet, that h264parse reports streaming-format=byte-stream, which does not fit to caps with streaming-format=avc.
> > We use h264parse to change streaming-format byte-stream (before) to avc (afterwards).
> > Is there a more intelligent way needed after h264parse instead of just forcing caps to solve link problem?
>
> Normally that constraint should come form the next element, like a muxer. If you use a recorder base on ISOMP4 (qtmux), the muxer would force AVC conversion automatically.
>
> > Another thing I am currently unsure: When we dynamically add h264parse, is there some synchronization to frame data needed, so we start exactly at the beginning of a GOP?
> > But we are already on compressed side, so is it relevant at all?
>
> Normally, h264parse will drop data until it reaches a state where at least one SPS and one PPS has been seen. Everything before that is dropped. I would suggest to review closely you're dynamic linking path, you have have introduce a race.
>
> > Many thanks!
> >
> > Best regards,
> >
> > Marie
> >
> > -----Ursprüngliche Nachricht-----
> > Von: gstreamer-devel
> > [mailto:[hidden email]] Im Auftrag von
> > Nicolas Dufresne
> > Gesendet: Mittwoch, 12. Dezember 2018 18:39
> > An: Discussion of the development of and with GStreamer
> > <[hidden email]>
> > Betreff: Re: "broken/invalid nal" with h264parse after v4l2h264enc on
> > i.MX6
> >
> > Le mercredi 12 décembre 2018 à 17:12 +0000, Maurer, Marie a écrit :
> > > Hello,
> > >  
> > > environment is GStreamer 1.14.2 on Linux with i.MX6 (most of it
> > > latest mainlined versions)
> >
> > Please, try again with 1.14.4, as there is some fixes in the 1.14 stable branch. And if you can, complain to Yocto people who seem not to stay up-to-date with stable.
> >
> > >  
> > > I see a lot messages like the following:
> > >  
> > > WARN   h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame:<RecorderH264Parser> broken/invalid nal Type: 1 Slice, Size: 81703 will be dropped
> > >  
> > > Size is changing, but always around between 50000 and 80000.
> > >  
> > > I think the relevant part of my pipeline is a v4l2h264enc and after some queues and tees a h264parse.
> > >  
> > > Is it correct that somehow the parser does not understand the data which the v4l2h264enc is generating?
> > > Or the data which the encoder is generating gets sometimes somehow corrupted till it is received by h264parse?
> >
> > Hard to say, though it's better to update first, and see if any of the upstream bugfixes improves the situation.
> >
> > >  
> > > Any idea how it can be solved?
> >
> > You would have to share the pipeline you are using, the settings and all. Ideally if you can reproduce in gst-launch-1.0 form, then I could try reproduce locally.
> >
> > >  
> > > Best regards,
> > >  
> > > Marie
> > >  
> > > _______________________________________________
> > > gstreamer-devel mailing list
> > > [hidden email]
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> >
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

AW: AW: AW: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6

Maurer, Marie
Hello Nicolas,

as far as I know dynamic pipeline attachment and detachment is not possible
via gst-launch-1.0 or similar tools?

The pipeline is part of a bigger layer which controls gstreamer pipeline creating, modification, destruction. It depends on other parts of the software.
So at the moment I think, I am not able to share this source code.

What other steps can I do to analyze the problem?

What I don't yet understand:

When I have

caps stream-format=(string)byte-stream -> h264parse -> caps stream-format=(string)avc

I force h264parse to use a given stream-format.

When I get a negotiation failure, then h264parse seems to not fulfill what I had requested via caps?

I already put printfs into source of h264parse, but not yet able to understand its behavior.
I assume some (wrong?) input data is responsible for stream-format change, which is then not compatible
to my following caps.

Best regards,

Marie

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Nicolas Dufresne
Gesendet: Freitag, 14. Dezember 2018 19:40
An: Discussion of the development of and with GStreamer <[hidden email]>
Betreff: Re: AW: AW: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6

Le vendredi 14 décembre 2018 à 17:18 +0000, Maurer, Marie a écrit :

> Hello Nicolas,
>
> just a short follow-up: Same problem (warnings and error with gst_element_link) with 1.14.4.
>
> Tried these two commits:
>
> https://github.com/GStreamer/gst-plugins-bad/commit/4f7fe897b9be77c983
> f935538d30a5c90cecc947
>
> and
>
> https://lists.freedesktop.org/archives/gstreamer-commits/2018-December
> /109067.html

I don't think any of this is related. Again, can you share the pipeline you are running, I would maybe help by testing on my IMX.6 board here.

>
> -> no difference
>
>
> 25xOK (only 3 shown here), 1x Error:
>
>             Line 639452: 0:05:08.615060370 523 0x71d9db50 DEBUG  h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
>             Line 667789: 0:05:21.600504371 523 0x9e7d20 DEBUG  h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
>             Line 696382: 0:05:34.680368373 523 0x71d13950 DEBUG  h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
>             Line 724887: 0:05:47.681662041 523 0x6ff35980 DEBUG  
> h264parse
> gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser>
> allowed caps: (NULL)
>
> OK:
>
> 0:05:34.679142373 523 0x71d13950 DEBUG  h264parse
> gsth264parse.c:334:gst_h264_parse_format_from_caps:[00m parsing caps:
> video/x-h264, stream-format=(string)byte-stream, alignment=(string)au,
> profile=(string)baseline, level=(string)4, width=(int)1920,
> height=(int)1080, pixel-aspect-ratio=(fraction)1/1,
> framerate=(fraction)30000/1001, interlace-mode=(string)progressive,
> colorimetry=(string)bt709
> 0:05:34.679444373 523 0x71d13950 DEBUG  h264parse
> gsth264parse.c:2716:gst_h264_parse_set_caps:<RecorderH264Parser> have
> bytestream h264
> 0:05:34.680368373 523 0x71d13950 DEBUG  h264parse
> gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser>
> allowed caps: video/x-h264, parsed=(boolean)true,
> stream-format=(string)avc, alignment=(string)au
> 0:05:34.680629040 523 0x71d13950 DEBUG  h264parse
> gsth264parse.c:381:gst_h264_parse_negotiate:<RecorderH264Parser>
> negotiating with caps: video/x-h264, parsed=(boolean)true,
> stream-format=(string)avc, alignment=(string)au
> 0:05:34.680830040 523 0x71d13950 DEBUG  h264parse
> gsth264parse.c:334:gst_h264_parse_format_from_caps: parsing caps:
> video/x-h264, parsed=(boolean)true, stream-format=(string)avc,
> alignment=(string)au
> 0:05:34.681008373 523 0x71d13950 DEBUG  h264parse gsth264parse.c:411:gst_h264_parse_negotiate:<RecorderH264Parser> selected format avc, alignment au
> 0:05:34.681650373 523 0x71d13950 LOG    h264parse gsth264parse.c:1102:gst_h264_parse_handle_frame:<RecorderH264Parser> parsing new frame
> 0:05:34.681862373 523 0x71d13950 DEBUG  h264parse
> gsth264parse.c:186:gst_h264_parse_reset_frame:<RecorderH264Parser>
> reset frame
>
> Error:
>
> 0:05:47.679435708 523 0x6ff35980 DEBUG  h264parse
> gsth264parse.c:334:gst_h264_parse_format_from_caps:[00m parsing caps:
> video/x-h264, stream-format=(string)byte-stream, alignment=(string)au,
> profile=(string)baseline, level=(string)4, width=(int)1920,
> height=(int)1080, pixel-aspect-ratio=(fraction)1/1,
> framerate=(fraction)30000/1001, interlace-mode=(string)progressive,
> colorimetry=(string)bt709
> 0:05:47.680866041 523 0x6ff35980 DEBUG  h264parse
> gsth264parse.c:2716:gst_h264_parse_set_caps:<RecorderH264Parser> have
> bytestream h264
> 0:05:47.681662041 523 0x6ff35980 DEBUG  h264parse
> gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser>
> allowed caps: (NULL)
> 0:05:47.682161374 523 0x6ff35980 DEBUG  h264parse
> gsth264parse.c:411:gst_h264_parse_negotiate:<RecorderH264Parser>
> selected format byte-stream, alignment au
> 0:05:47.682607374 523 0x6ff35980 DEBUG  h264parse gsth264parse.c:1755:gst_h264_parse_update_src_caps:<RecorderH264Parser> sps: (nil)
> 0:05:47.685859374 523 0x6ff35980 LOG    h264parse gsth264parse.c:1102:gst_h264_parse_handle_frame:<RecorderH264Parser> parsing new frame
> 0:05:47.686587041 523 0x6ff35980 DEBUG  h264parse
> gsth264parse.c:186:gst_h264_parse_reset_frame:<RecorderH264Parser>
> reset frame
> 0:05:47.687081708 523 0x6ff35980 DEBUG  h264parse
> gsth264parse.c:1117:gst_h264_parse_handle_frame:<RecorderH264Parser>
> last parse position 0
>
> Best regards,
>
> Marie
>
> -----Ursprüngliche Nachricht-----
> Von: gstreamer-devel
> [mailto:[hidden email]] Im Auftrag von
> Nicolas Dufresne
> Gesendet: Donnerstag, 13. Dezember 2018 18:39
> An: Discussion of the development of and with GStreamer
> <[hidden email]>
> Betreff: Re: AW: "broken/invalid nal" with h264parse after v4l2h264enc
> on i.MX6
>
> Le jeudi 13 décembre 2018 à 14:08 +0000, Maurer, Marie a écrit :
> > Hi Nicolas,
> >
> > many thanks for your answer. I am currently building Gstreamer 1.14.4 (and some more). When it is done, I can retest.
> > We don't use Yocto, but I had not yet locally updated to 1.14.4.
> >
> > The complete pipeline is big, but I think I can reduce it to following overview (queue, caps removed, where not explicitly needed):
> >
> > appsrc -> tee T1
> >
> > T1 -> v4l2h264enc -> tee T2
> >
> > T2#1 -> "Streaming"
> >
> > T2#2 -> ("Video Recording to a file") h264parse -> CAPS streaming-format=avc -> ...
> >
> > Video Recording to a file contains the h264parse, which reports these warnings.
> >
> > We attach v4l2h264enc (to T1) only when streaming and/or video
> > recording is enabled to save resources and detach it when both are switched off.
> > We attach streaming (to T2) only, when it is enabled and detach it when streaming is disabled.
> > We attach "video recording to a file" (also to T2) only when user wants it and detach it when user switches it off.
> > At T1 we have more parts connected, which consume frames.
> >
> > The main problem I have (beside the reported warnings) is that sometimes, h264parse and following caps can't be linked.
> >
> > !!!Speculation!!!
> > This could be, but I am not sure yet, that h264parse reports streaming-format=byte-stream, which does not fit to caps with streaming-format=avc.
> > We use h264parse to change streaming-format byte-stream (before) to avc (afterwards).
> > Is there a more intelligent way needed after h264parse instead of just forcing caps to solve link problem?
>
> Normally that constraint should come form the next element, like a muxer. If you use a recorder base on ISOMP4 (qtmux), the muxer would force AVC conversion automatically.
>
> > Another thing I am currently unsure: When we dynamically add h264parse, is there some synchronization to frame data needed, so we start exactly at the beginning of a GOP?
> > But we are already on compressed side, so is it relevant at all?
>
> Normally, h264parse will drop data until it reaches a state where at least one SPS and one PPS has been seen. Everything before that is dropped. I would suggest to review closely you're dynamic linking path, you have have introduce a race.
>
> > Many thanks!
> >
> > Best regards,
> >
> > Marie
> >
> > -----Ursprüngliche Nachricht-----
> > Von: gstreamer-devel
> > [mailto:[hidden email]] Im Auftrag
> > von Nicolas Dufresne
> > Gesendet: Mittwoch, 12. Dezember 2018 18:39
> > An: Discussion of the development of and with GStreamer
> > <[hidden email]>
> > Betreff: Re: "broken/invalid nal" with h264parse after v4l2h264enc
> > on
> > i.MX6
> >
> > Le mercredi 12 décembre 2018 à 17:12 +0000, Maurer, Marie a écrit :
> > > Hello,
> > >  
> > > environment is GStreamer 1.14.2 on Linux with i.MX6 (most of it
> > > latest mainlined versions)
> >
> > Please, try again with 1.14.4, as there is some fixes in the 1.14 stable branch. And if you can, complain to Yocto people who seem not to stay up-to-date with stable.
> >
> > >  
> > > I see a lot messages like the following:
> > >  
> > > WARN   h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame:<RecorderH264Parser> broken/invalid nal Type: 1 Slice, Size: 81703 will be dropped
> > >  
> > > Size is changing, but always around between 50000 and 80000.
> > >  
> > > I think the relevant part of my pipeline is a v4l2h264enc and after some queues and tees a h264parse.
> > >  
> > > Is it correct that somehow the parser does not understand the data which the v4l2h264enc is generating?
> > > Or the data which the encoder is generating gets sometimes somehow corrupted till it is received by h264parse?
> >
> > Hard to say, though it's better to update first, and see if any of the upstream bugfixes improves the situation.
> >
> > >  
> > > Any idea how it can be solved?
> >
> > You would have to share the pipeline you are using, the settings and all. Ideally if you can reproduce in gst-launch-1.0 form, then I could try reproduce locally.
> >
> > >  
> > > Best regards,
> > >  
> > > Marie
> > >  
> > > _______________________________________________
> > > gstreamer-devel mailing list
> > > [hidden email]
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> >
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: AW: AW: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6

Nicolas Dufresne-5
Le mardi 18 décembre 2018 à 10:48 +0000, Maurer, Marie a écrit :
> Hello Nicolas,
>
> as far as I know dynamic pipeline attachment and detachment is not possible
> via gst-launch-1.0 or similar tools?
>
> The pipeline is part of a bigger layer which controls gstreamer pipeline creating, modification, destruction. It depends on other parts of the software.
> So at the moment I think, I am not able to share this source code.
>
> What other steps can I do to analyze the problem?

Well, then save the H264 data that arrived to h264parse after
dynamically linking it into a file, and share a small samples. I
believe your dynamic linking code need to be aware of h264 data and
make sure it will send a valid H264 stream. Normally h264parse just
drops until it reaches a stable state, not sure what is happening
exactly for you.

>
> What I don't yet understand:
>
> When I have
>
> caps stream-format=(string)byte-stream -> h264parse -> caps stream-format=(string)avc
>
> I force h264parse to use a given stream-format.
>
> When I get a negotiation failure, then h264parse seems to not fulfill what I had requested via caps?
>
> I already put printfs into source of h264parse, but not yet able to understand its behavior.
> I assume some (wrong?) input data is responsible for stream-format change, which is then not compatible
> to my following caps.
>
> Best regards,
>
> Marie
>
> -----Ursprüngliche Nachricht-----
> Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Nicolas Dufresne
> Gesendet: Freitag, 14. Dezember 2018 19:40
> An: Discussion of the development of and with GStreamer <[hidden email]>
> Betreff: Re: AW: AW: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6
>
> Le vendredi 14 décembre 2018 à 17:18 +0000, Maurer, Marie a écrit :
> > Hello Nicolas,
> >
> > just a short follow-up: Same problem (warnings and error with gst_element_link) with 1.14.4.
> >
> > Tried these two commits:
> >
> > https://github.com/GStreamer/gst-plugins-bad/commit/4f7fe897b9be77c983
> > f935538d30a5c90cecc947
> >
> > and
> >
> > https://lists.freedesktop.org/archives/gstreamer-commits/2018-December
> > /109067.html
>
> I don't think any of this is related. Again, can you share the pipeline you are running, I would maybe help by testing on my IMX.6 board here.
>
> > -> no difference
> >
> >
> > 25xOK (only 3 shown here), 1x Error:
> >
> >             Line 639452: 0:05:08.615060370 523 0x71d9db50 DEBUG  h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
> >             Line 667789: 0:05:21.600504371 523 0x9e7d20 DEBUG  h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
> >             Line 696382: 0:05:34.680368373 523 0x71d13950 DEBUG  h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
> >             Line 724887: 0:05:47.681662041 523 0x6ff35980 DEBUG  
> > h264parse
> > gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser>
> > allowed caps: (NULL)
> >
> > OK:
> >
> > 0:05:34.679142373 523 0x71d13950 DEBUG  h264parse
> > gsth264parse.c:334:gst_h264_parse_format_from_caps:[00m parsing caps:
> > video/x-h264, stream-format=(string)byte-stream, alignment=(string)au,
> > profile=(string)baseline, level=(string)4, width=(int)1920,
> > height=(int)1080, pixel-aspect-ratio=(fraction)1/1,
> > framerate=(fraction)30000/1001, interlace-mode=(string)progressive,
> > colorimetry=(string)bt709
> > 0:05:34.679444373 523 0x71d13950 DEBUG  h264parse
> > gsth264parse.c:2716:gst_h264_parse_set_caps:<RecorderH264Parser> have
> > bytestream h264
> > 0:05:34.680368373 523 0x71d13950 DEBUG  h264parse
> > gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser>
> > allowed caps: video/x-h264, parsed=(boolean)true,
> > stream-format=(string)avc, alignment=(string)au
> > 0:05:34.680629040 523 0x71d13950 DEBUG  h264parse
> > gsth264parse.c:381:gst_h264_parse_negotiate:<RecorderH264Parser>
> > negotiating with caps: video/x-h264, parsed=(boolean)true,
> > stream-format=(string)avc, alignment=(string)au
> > 0:05:34.680830040 523 0x71d13950 DEBUG  h264parse
> > gsth264parse.c:334:gst_h264_parse_format_from_caps: parsing caps:
> > video/x-h264, parsed=(boolean)true, stream-format=(string)avc,
> > alignment=(string)au
> > 0:05:34.681008373 523 0x71d13950 DEBUG  h264parse gsth264parse.c:411:gst_h264_parse_negotiate:<RecorderH264Parser> selected format avc, alignment au
> > 0:05:34.681650373 523 0x71d13950 LOG    h264parse gsth264parse.c:1102:gst_h264_parse_handle_frame:<RecorderH264Parser> parsing new frame
> > 0:05:34.681862373 523 0x71d13950 DEBUG  h264parse
> > gsth264parse.c:186:gst_h264_parse_reset_frame:<RecorderH264Parser>
> > reset frame
> >
> > Error:
> >
> > 0:05:47.679435708 523 0x6ff35980 DEBUG  h264parse
> > gsth264parse.c:334:gst_h264_parse_format_from_caps:[00m parsing caps:
> > video/x-h264, stream-format=(string)byte-stream, alignment=(string)au,
> > profile=(string)baseline, level=(string)4, width=(int)1920,
> > height=(int)1080, pixel-aspect-ratio=(fraction)1/1,
> > framerate=(fraction)30000/1001, interlace-mode=(string)progressive,
> > colorimetry=(string)bt709
> > 0:05:47.680866041 523 0x6ff35980 DEBUG  h264parse
> > gsth264parse.c:2716:gst_h264_parse_set_caps:<RecorderH264Parser> have
> > bytestream h264
> > 0:05:47.681662041 523 0x6ff35980 DEBUG  h264parse
> > gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser>
> > allowed caps: (NULL)
> > 0:05:47.682161374 523 0x6ff35980 DEBUG  h264parse
> > gsth264parse.c:411:gst_h264_parse_negotiate:<RecorderH264Parser>
> > selected format byte-stream, alignment au
> > 0:05:47.682607374 523 0x6ff35980 DEBUG  h264parse gsth264parse.c:1755:gst_h264_parse_update_src_caps:<RecorderH264Parser> sps: (nil)
> > 0:05:47.685859374 523 0x6ff35980 LOG    h264parse gsth264parse.c:1102:gst_h264_parse_handle_frame:<RecorderH264Parser> parsing new frame
> > 0:05:47.686587041 523 0x6ff35980 DEBUG  h264parse
> > gsth264parse.c:186:gst_h264_parse_reset_frame:<RecorderH264Parser>
> > reset frame
> > 0:05:47.687081708 523 0x6ff35980 DEBUG  h264parse
> > gsth264parse.c:1117:gst_h264_parse_handle_frame:<RecorderH264Parser>
> > last parse position 0
> >
> > Best regards,
> >
> > Marie
> >
> > -----Ursprüngliche Nachricht-----
> > Von: gstreamer-devel
> > [mailto:[hidden email]] Im Auftrag von
> > Nicolas Dufresne
> > Gesendet: Donnerstag, 13. Dezember 2018 18:39
> > An: Discussion of the development of and with GStreamer
> > <[hidden email]>
> > Betreff: Re: AW: "broken/invalid nal" with h264parse after v4l2h264enc
> > on i.MX6
> >
> > Le jeudi 13 décembre 2018 à 14:08 +0000, Maurer, Marie a écrit :
> > > Hi Nicolas,
> > >
> > > many thanks for your answer. I am currently building Gstreamer 1.14.4 (and some more). When it is done, I can retest.
> > > We don't use Yocto, but I had not yet locally updated to 1.14.4.
> > >
> > > The complete pipeline is big, but I think I can reduce it to following overview (queue, caps removed, where not explicitly needed):
> > >
> > > appsrc -> tee T1
> > >
> > > T1 -> v4l2h264enc -> tee T2
> > >
> > > T2#1 -> "Streaming"
> > >
> > > T2#2 -> ("Video Recording to a file") h264parse -> CAPS streaming-format=avc -> ...
> > >
> > > Video Recording to a file contains the h264parse, which reports these warnings.
> > >
> > > We attach v4l2h264enc (to T1) only when streaming and/or video
> > > recording is enabled to save resources and detach it when both are switched off.
> > > We attach streaming (to T2) only, when it is enabled and detach it when streaming is disabled.
> > > We attach "video recording to a file" (also to T2) only when user wants it and detach it when user switches it off.
> > > At T1 we have more parts connected, which consume frames.
> > >
> > > The main problem I have (beside the reported warnings) is that sometimes, h264parse and following caps can't be linked.
> > >
> > > !!!Speculation!!!
> > > This could be, but I am not sure yet, that h264parse reports streaming-format=byte-stream, which does not fit to caps with streaming-format=avc.
> > > We use h264parse to change streaming-format byte-stream (before) to avc (afterwards).
> > > Is there a more intelligent way needed after h264parse instead of just forcing caps to solve link problem?
> >
> > Normally that constraint should come form the next element, like a muxer. If you use a recorder base on ISOMP4 (qtmux), the muxer would force AVC conversion automatically.
> >
> > > Another thing I am currently unsure: When we dynamically add h264parse, is there some synchronization to frame data needed, so we start exactly at the beginning of a GOP?
> > > But we are already on compressed side, so is it relevant at all?
> >
> > Normally, h264parse will drop data until it reaches a state where at least one SPS and one PPS has been seen. Everything before that is dropped. I would suggest to review closely you're dynamic linking path, you have have introduce a race.
> >
> > > Many thanks!
> > >
> > > Best regards,
> > >
> > > Marie
> > >
> > > -----Ursprüngliche Nachricht-----
> > > Von: gstreamer-devel
> > > [mailto:[hidden email]] Im Auftrag
> > > von Nicolas Dufresne
> > > Gesendet: Mittwoch, 12. Dezember 2018 18:39
> > > An: Discussion of the development of and with GStreamer
> > > <[hidden email]>
> > > Betreff: Re: "broken/invalid nal" with h264parse after v4l2h264enc
> > > on
> > > i.MX6
> > >
> > > Le mercredi 12 décembre 2018 à 17:12 +0000, Maurer, Marie a écrit :
> > > > Hello,
> > > >  
> > > > environment is GStreamer 1.14.2 on Linux with i.MX6 (most of it
> > > > latest mainlined versions)
> > >
> > > Please, try again with 1.14.4, as there is some fixes in the 1.14 stable branch. And if you can, complain to Yocto people who seem not to stay up-to-date with stable.
> > >
> > > >  
> > > > I see a lot messages like the following:
> > > >  
> > > > WARN   h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame:<RecorderH264Parser> broken/invalid nal Type: 1 Slice, Size: 81703 will be dropped
> > > >  
> > > > Size is changing, but always around between 50000 and 80000.
> > > >  
> > > > I think the relevant part of my pipeline is a v4l2h264enc and after some queues and tees a h264parse.
> > > >  
> > > > Is it correct that somehow the parser does not understand the data which the v4l2h264enc is generating?
> > > > Or the data which the encoder is generating gets sometimes somehow corrupted till it is received by h264parse?
> > >
> > > Hard to say, though it's better to update first, and see if any of the upstream bugfixes improves the situation.
> > >
> > > >  
> > > > Any idea how it can be solved?
> > >
> > > You would have to share the pipeline you are using, the settings and all. Ideally if you can reproduce in gst-launch-1.0 form, then I could try reproduce locally.
> > >
> > > >  
> > > > Best regards,
> > > >  
> > > > Marie
> > > >  
> > > > _______________________________________________
> > > > gstreamer-devel mailing list
> > > > [hidden email]
> > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > >
> > > _______________________________________________
> > > gstreamer-devel mailing list
> > > [hidden email]
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > > _______________________________________________
> > > gstreamer-devel mailing list
> > > [hidden email]
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> >
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel