Video artifacts on a receiver side after re-creating of its pipeline

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Video artifacts on a receiver side after re-creating of its pipeline

Denis Shienkov
Hi all.

I use two own applications: producer & consumer.

A producer generates the video stream and sends to the UDP sink.
A consumer receives this stream and displays a video content.

When the consumer started before the producer, then I did not get any
video artifacts on an consumer.

But, when I try to do stop/start the consumer (when the producer already
running), then I see many video artifacts on the consumer.

A possible workaround it is try to restart the producer (in this case the
consumer receives good video content).

How to solve this issue?

UPD:

A producer uses the following pipeline's equivalent:

[code]
void Producer::start()
{
     ...
     ...

     m_appsrc = ::gst_element_factory_make("appsrc", nullptr);

     ::g_object_set(G_OBJECT(m_appsrc),
                    "stream-type", GST_APP_STREAM_TYPE_STREAM,
                    "is-live", true,
                    "format", GST_FORMAT_TIME,
                    nullptr);

     const auto videoconvert =
::gst_element_factory_make("videoconvert", nullptr);

     const auto videoenc = ::gst_element_factory_make("x264enc", nullptr);

     ::gst_util_set_object_arg(G_OBJECT(videoenc), "tune", "zerolatency");

     ::g_object_set(G_OBJECT(videoenc),
                    "key-int-max", 10,
                    nullptr);

     const auto parse = ::gst_element_factory_make("h264parse", nullptr);

     ::g_object_set(G_OBJECT(parse),
                    "config-interval", -1,
                    nullptr);

     const auto payloader = ::gst_element_factory_make("rtph264pay",
nullptr);

     const auto udpsink = ::gst_element_factory_make("udpsink", nullptr);

     const auto host = destination.host().toLocal8Bit();
     const auto port = destination.port();

     ::g_object_set(G_OBJECT(udpsink),
                    "host", host.constData(),
                    "port", port,
                    nullptr);

     m_pipeline = ::gst_pipeline_new("pipeline");

     ...
     ...

     if (!m_appsrc || !videoconvert || !videoenc
             || !payloader || !udpsink) {
         qCCritical(gstProducerControl) << "Unable to create one of
pipeline element";
     }

     ::gst_bin_add_many(GST_BIN(m_pipeline),
                        m_appsrc,
                        videoconvert,
                        videoenc,
                        parse,
                        payloader,
                        udpsink,
                        nullptr);

     const auto result = ::gst_element_link_many(m_appsrc,
                                                 videoconvert,
                                                 videoenc,
                                                 parse,
                                                 payloader,
                                                 udpsink,
                                                 nullptr);
     ...
     const auto status = ::gst_element_set_state(m_pipeline,
GST_STATE_PLAYING);
     ...
}

void Foo::stop()
{
     ...
     ::gst_element_set_state(m_pipeline, GST_STATE_NULL);
     ::gst_object_unref(m_pipeline);

     m_appsrc = nullptr;
     m_pipeline = nullptr;
     ...
}
[/code]

A consumer uses the following pipeline's equivalent:

[code]
void Consumer::start()
{
     ...
     const auto udpsrc = ::gst_element_factory_make("udpsrc", nullptr);

     ::g_object_set(G_OBJECT(udpsrc),
                    "address", address.constData(),
                    "port", port,
                    nullptr);

     const auto srccaps = ::gst_caps_new_simple(
                 "application/x-rtp",
                 "", G_TYPE_STRING, "",
                 nullptr);

     ::g_object_set(G_OBJECT(udpsrc),
                    "caps", srccaps,
                    nullptr);

     const auto depay = ::gst_element_factory_make("rtph264depay", nullptr);
     const auto decodebin = ::gst_element_factory_make("decodebin",
nullptr);
     const auto videoconvert =
::gst_element_factory_make("videoconvert", nullptr);

     const auto videofilter = ::gst_element_factory_make("capsfilter",
nullptr);

     const auto videofiltercaps = ::gst_caps_new_simple(
                 "video/x-raw",
                 "format", G_TYPE_STRING, "BGRA",
                 nullptr);

     ::g_object_set(G_OBJECT(videofilter),
                    "caps", videofiltercaps,
                    nullptr);

     m_videosink = m_renderer->videoSink();

     ::g_object_set(G_OBJECT(m_videosink),
                    "sync", false,
                    nullptr);

     m_pipeline = ::gst_pipeline_new("pipeline");

     ...

     ::gst_bin_add_many(GST_BIN(m_pipeline),
                        udpsrc,
                        depay,
                        decodebin,
                        videoconvert,
                        videofilter,
                        m_videosink,
                        nullptr);

     ::gst_element_link_many(udpsrc, depay, decodebin, nullptr);
     ::gst_element_link_many(videoconvert, videofilter, m_videosink,
nullptr);

     ...

     const auto status = ::gst_element_set_state(m_pipeline,
GST_STATE_PLAYING);
     ...
}

void Consumer::stop()
{
     ::gst_element_set_state(m_pipeline, GST_STATE_NULL);
     ::gst_object_unref(m_pipeline);

     m_videosink = nullptr;
     m_pipeline = nullptr;
}
[/code]

BR,
Denis
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Video artifacts on a receiver side after re-creating of its pipeline

Tim Müller
On Wed, 2018-01-10 at 16:23 +0300, Denis Shienkov wrote:

Hi,

> I use two own applications: producer & consumer.
> A producer generates the video stream and sends to the UDP sink.
> A consumer receives this stream and displays a video content.
>
> When the consumer started before the producer, then I did not get any
> video artifacts on an consumer.
>
> But, when I try to do stop/start the consumer (when the producer
> already running), then I see many video artifacts on the consumer.
>
> A possible workaround it is try to restart the producer (in this case
> the consumer receives good video content).

Do the artefacts disappear after a few seconds (when the producer
generates a new keyframe)?

If yes, then all you need to do is force the encoder on the producer
side to generate a new keyframe whenever you add a client. You can do
that with gst_video_event_new_upstream_force_key_unit() for example.

Cheers
-Tim

--
Tim Müller, Centricular Ltd - http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

AW: Video artifacts on a receiver side after re-creating of its pipeline

Thornton, Keith
In reply to this post by Denis Shienkov
Hi, the consumer needs an SPS and PPS. This could be transmitted by setting the config-interval parameter of rtph264pay.

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Denis Shienkov
Gesendet: Mittwoch, 10. Januar 2018 14:23
An: Discussion of the development of and with GStreamer <[hidden email]>
Cc: Tim Müller <[hidden email]>
Betreff: Video artifacts on a receiver side after re-creating of its pipeline

Hi all.

I use two own applications: producer & consumer.

A producer generates the video stream and sends to the UDP sink.
A consumer receives this stream and displays a video content.

When the consumer started before the producer, then I did not get any video artifacts on an consumer.

But, when I try to do stop/start the consumer (when the producer already running), then I see many video artifacts on the consumer.

A possible workaround it is try to restart the producer (in this case the consumer receives good video content).

How to solve this issue?

UPD:

A producer uses the following pipeline's equivalent:

[code]
void Producer::start()
{
     ...
     ...

     m_appsrc = ::gst_element_factory_make("appsrc", nullptr);

     ::g_object_set(G_OBJECT(m_appsrc),
                    "stream-type", GST_APP_STREAM_TYPE_STREAM,
                    "is-live", true,
                    "format", GST_FORMAT_TIME,
                    nullptr);

     const auto videoconvert =
::gst_element_factory_make("videoconvert", nullptr);

     const auto videoenc = ::gst_element_factory_make("x264enc", nullptr);

     ::gst_util_set_object_arg(G_OBJECT(videoenc), "tune", "zerolatency");

     ::g_object_set(G_OBJECT(videoenc),
                    "key-int-max", 10,
                    nullptr);

     const auto parse = ::gst_element_factory_make("h264parse", nullptr);

     ::g_object_set(G_OBJECT(parse),
                    "config-interval", -1,
                    nullptr);

     const auto payloader = ::gst_element_factory_make("rtph264pay",
nullptr);

     const auto udpsink = ::gst_element_factory_make("udpsink", nullptr);

     const auto host = destination.host().toLocal8Bit();
     const auto port = destination.port();

     ::g_object_set(G_OBJECT(udpsink),
                    "host", host.constData(),
                    "port", port,
                    nullptr);

     m_pipeline = ::gst_pipeline_new("pipeline");

     ...
     ...

     if (!m_appsrc || !videoconvert || !videoenc
             || !payloader || !udpsink) {
         qCCritical(gstProducerControl) << "Unable to create one of pipeline element";
     }

     ::gst_bin_add_many(GST_BIN(m_pipeline),
                        m_appsrc,
                        videoconvert,
                        videoenc,
                        parse,
                        payloader,
                        udpsink,
                        nullptr);

     const auto result = ::gst_element_link_many(m_appsrc,
                                                 videoconvert,
                                                 videoenc,
                                                 parse,
                                                 payloader,
                                                 udpsink,
                                                 nullptr);
     ...
     const auto status = ::gst_element_set_state(m_pipeline,
GST_STATE_PLAYING);
     ...
}

void Foo::stop()
{
     ...
     ::gst_element_set_state(m_pipeline, GST_STATE_NULL);
     ::gst_object_unref(m_pipeline);

     m_appsrc = nullptr;
     m_pipeline = nullptr;
     ...
}
[/code]

A consumer uses the following pipeline's equivalent:

[code]
void Consumer::start()
{
     ...
     const auto udpsrc = ::gst_element_factory_make("udpsrc", nullptr);

     ::g_object_set(G_OBJECT(udpsrc),
                    "address", address.constData(),
                    "port", port,
                    nullptr);

     const auto srccaps = ::gst_caps_new_simple(
                 "application/x-rtp",
                 "", G_TYPE_STRING, "",
                 nullptr);

     ::g_object_set(G_OBJECT(udpsrc),
                    "caps", srccaps,
                    nullptr);

     const auto depay = ::gst_element_factory_make("rtph264depay", nullptr);
     const auto decodebin = ::gst_element_factory_make("decodebin",
nullptr);
     const auto videoconvert =
::gst_element_factory_make("videoconvert", nullptr);

     const auto videofilter = ::gst_element_factory_make("capsfilter",
nullptr);

     const auto videofiltercaps = ::gst_caps_new_simple(
                 "video/x-raw",
                 "format", G_TYPE_STRING, "BGRA",
                 nullptr);

     ::g_object_set(G_OBJECT(videofilter),
                    "caps", videofiltercaps,
                    nullptr);

     m_videosink = m_renderer->videoSink();

     ::g_object_set(G_OBJECT(m_videosink),
                    "sync", false,
                    nullptr);

     m_pipeline = ::gst_pipeline_new("pipeline");

     ...

     ::gst_bin_add_many(GST_BIN(m_pipeline),
                        udpsrc,
                        depay,
                        decodebin,
                        videoconvert,
                        videofilter,
                        m_videosink,
                        nullptr);

     ::gst_element_link_many(udpsrc, depay, decodebin, nullptr);
     ::gst_element_link_many(videoconvert, videofilter, m_videosink, nullptr);

     ...

     const auto status = ::gst_element_set_state(m_pipeline,
GST_STATE_PLAYING);
     ...
}

void Consumer::stop()
{
     ::gst_element_set_state(m_pipeline, GST_STATE_NULL);
     ::gst_object_unref(m_pipeline);

     m_videosink = nullptr;
     m_pipeline = nullptr;
}
[/code]

BR,
Denis
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Video artifacts on a receiver side after re-creating of its pipeline

Denis Shienkov
In reply to this post by Tim Müller
Hi Tim,

 > Do the artefacts disappear after a few seconds (when the producer
generates a new keyframe)?

No, an artifacts stays forewer, until I do not restart the producer. :(


10.01.2018 16:47, Tim Müller пишет:

> On Wed, 2018-01-10 at 16:23 +0300, Denis Shienkov wrote:
>
> Hi,
>
>> I use two own applications: producer & consumer.
>> A producer generates the video stream and sends to the UDP sink.
>> A consumer receives this stream and displays a video content.
>>
>> When the consumer started before the producer, then I did not get any
>> video artifacts on an consumer.
>>
>> But, when I try to do stop/start the consumer (when the producer
>> already running), then I see many video artifacts on the consumer.
>>
>> A possible workaround it is try to restart the producer (in this case
>> the consumer receives good video content).
> Do the artefacts disappear after a few seconds (when the producer
> generates a new keyframe)?
>
> If yes, then all you need to do is force the encoder on the producer
> side to generate a new keyframe whenever you add a client. You can do
> that with gst_video_event_new_upstream_force_key_unit() for example.
>
> Cheers
> -Tim
>

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: Video artifacts on a receiver side after re-creating of its pipeline

Denis Shienkov
In reply to this post by Thornton, Keith

Hi,

> Hi, the consumer needs an SPS and PPS. This could be transmitted by setting the config-interval parameter of rtph264pay

I already tried this:

const auto payloader = ::gst_element_factory_make("rtph264pay", nullptr);

::g_object_set(G_OBJECT(payloader),

                   "config-interval", 1,
                   nullptr);

with different values, as "-1", "1" and so on, but it does not help.



10.01.2018 17:14, Thornton, Keith пишет:
Hi, the consumer needs an SPS and PPS. This could be transmitted by setting the config-interval parameter of rtph264pay.

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [[hidden email]] Im Auftrag von Denis Shienkov
Gesendet: Mittwoch, 10. Januar 2018 14:23
An: Discussion of the development of and with GStreamer [hidden email]
Cc: Tim Müller [hidden email]
Betreff: Video artifacts on a receiver side after re-creating of its pipeline

Hi all.

I use two own applications: producer & consumer.

A producer generates the video stream and sends to the UDP sink.
A consumer receives this stream and displays a video content.

When the consumer started before the producer, then I did not get any video artifacts on an consumer.

But, when I try to do stop/start the consumer (when the producer already running), then I see many video artifacts on the consumer.

A possible workaround it is try to restart the producer (in this case the consumer receives good video content).

How to solve this issue?

UPD:

A producer uses the following pipeline's equivalent:

[code]
void Producer::start()
{
     ...
     ...

     m_appsrc = ::gst_element_factory_make("appsrc", nullptr);

     ::g_object_set(G_OBJECT(m_appsrc),
                    "stream-type", GST_APP_STREAM_TYPE_STREAM,
                    "is-live", true,
                    "format", GST_FORMAT_TIME,
                    nullptr);

     const auto videoconvert =
::gst_element_factory_make("videoconvert", nullptr);

     const auto videoenc = ::gst_element_factory_make("x264enc", nullptr);

     ::gst_util_set_object_arg(G_OBJECT(videoenc), "tune", "zerolatency");

     ::g_object_set(G_OBJECT(videoenc),
                    "key-int-max", 10,
                    nullptr);

     const auto parse = ::gst_element_factory_make("h264parse", nullptr);

     ::g_object_set(G_OBJECT(parse),
                    "config-interval", -1,
                    nullptr);

     const auto payloader = ::gst_element_factory_make("rtph264pay",
nullptr);

     const auto udpsink = ::gst_element_factory_make("udpsink", nullptr);

     const auto host = destination.host().toLocal8Bit();
     const auto port = destination.port();

     ::g_object_set(G_OBJECT(udpsink),
                    "host", host.constData(),
                    "port", port,
                    nullptr);

     m_pipeline = ::gst_pipeline_new("pipeline");

     ...
     ...

     if (!m_appsrc || !videoconvert || !videoenc
             || !payloader || !udpsink) {
         qCCritical(gstProducerControl) << "Unable to create one of pipeline element";
     }

     ::gst_bin_add_many(GST_BIN(m_pipeline),
                        m_appsrc,
                        videoconvert,
                        videoenc,
                        parse,
                        payloader,
                        udpsink,
                        nullptr);

     const auto result = ::gst_element_link_many(m_appsrc,
                                                 videoconvert,
                                                 videoenc,
                                                 parse,
                                                 payloader,
                                                 udpsink,
                                                 nullptr);
     ...
     const auto status = ::gst_element_set_state(m_pipeline,
GST_STATE_PLAYING);
     ...
}

void Foo::stop()
{
     ...
     ::gst_element_set_state(m_pipeline, GST_STATE_NULL);
     ::gst_object_unref(m_pipeline);

     m_appsrc = nullptr;
     m_pipeline = nullptr;
     ...
}
[/code]

A consumer uses the following pipeline's equivalent:

[code]
void Consumer::start()
{
     ...
     const auto udpsrc = ::gst_element_factory_make("udpsrc", nullptr);

     ::g_object_set(G_OBJECT(udpsrc),
                    "address", address.constData(),
                    "port", port,
                    nullptr);

     const auto srccaps = ::gst_caps_new_simple(
                 "application/x-rtp",
                 "", G_TYPE_STRING, "",
                 nullptr);

     ::g_object_set(G_OBJECT(udpsrc),
                    "caps", srccaps,
                    nullptr);

     const auto depay = ::gst_element_factory_make("rtph264depay", nullptr);
     const auto decodebin = ::gst_element_factory_make("decodebin",
nullptr);
     const auto videoconvert =
::gst_element_factory_make("videoconvert", nullptr);

     const auto videofilter = ::gst_element_factory_make("capsfilter",
nullptr);

     const auto videofiltercaps = ::gst_caps_new_simple(
                 "video/x-raw",
                 "format", G_TYPE_STRING, "BGRA",
                 nullptr);

     ::g_object_set(G_OBJECT(videofilter),
                    "caps", videofiltercaps,
                    nullptr);

     m_videosink = m_renderer->videoSink();

     ::g_object_set(G_OBJECT(m_videosink),
                    "sync", false,
                    nullptr);

     m_pipeline = ::gst_pipeline_new("pipeline");

     ...

     ::gst_bin_add_many(GST_BIN(m_pipeline),
                        udpsrc,
                        depay,
                        decodebin,
                        videoconvert,
                        videofilter,
                        m_videosink,
                        nullptr);

     ::gst_element_link_many(udpsrc, depay, decodebin, nullptr);
     ::gst_element_link_many(videoconvert, videofilter, m_videosink, nullptr);

     ...

     const auto status = ::gst_element_set_state(m_pipeline,
GST_STATE_PLAYING);
     ...
}

void Consumer::stop()
{
     ::gst_element_set_state(m_pipeline, GST_STATE_NULL);
     ::gst_object_unref(m_pipeline);

     m_videosink = nullptr;
     m_pipeline = nullptr;
}
[/code]

BR,
Denis
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: Video artifacts on a receiver side after re-creating of its pipeline

Baby Octopus
Seemslike some SPS/PPS issue. Can you try making producer send a multicast
stream and have multiple consumers
1. Your application
2. VLC
3.gst-launch playbin uri=<>

and see if the issue exists with all these clients or consumers?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel