h.264 Video Streaming over TCP

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

h.264 Video Streaming over TCP

pfarmer
Server and Client are on the same machine
With UDP it works.

Server:
$ gst-launch-1.0 videotestsrc horizontal-speed=5 ! x264enc tune="zerolatency" threads=1 ! rtph264pay config-interval=2 ! udpsink port=8554
Client:
$ gst-launch-1.0 udpsrc port=8554 ! application/x-rtp, payload=96 ! rtpjitterbuffer ! rtph264depay ! avdec_h264 ! xvimagesink

For some reason (firewall related) I have to use TCP. While replacing the udp sink/src by tcpserversink/tcpclientsrc

Server
gst-launch-1.0 videotestsrc horizontal-speed=5 ! x264enc tune="zerolatency" threads=1 ! rtph264pay config-interval=2 ! tcpserversink port=8554
Client:
gst-launch-1.0 tcpclientsrc port=8554 host=localhost ! application/x-rtp, payload=96 !  rtph264depay ! avdec_h264 ! xvimagesink

1.) The rtpjitterbuffer does not work. The launch fails with "tcpclientsrc0: streaming task paused, reason error (-5)"
       why is it so?
2.) The stream randomly fails afer a while with "rtph264depay0: NAL unit type 26 not supported yet" Whereas the number is different in each run. Before hand the debug info "rtph264depay0: Could not decode stream." occurs frequently. So it seems  So it seems the deepayloader does not get the access delimiter correctly. May guess is that the mtu is not set correctly for the payloader at the server side. A look in wireshark gave that some packets have sometimes a size are above 11000Bytes. This should not be the case since the payloader has a default mtu of 1400Bytes.
      What can i do let it decode the stream correctly?
3.) The packets are encapsulated in RTSP
      Is this a desired behavior? I thought the gst buffers (or rtp packets) should just be put in the tcp data block.



Reply | Threaded
Open this post in threaded view
|

Re: h.264 Video Streaming over TCP

pfarmer
author="pfarmer">
3.) The packets are encapsulated in RTSP
      Is this a desired behavior? I thought the gst buffers (or rtp packets) should just be put in the tcp data block.

That is not true. Wireshark just treated them as RTSP since i used the official RTSP port.

Reply | Threaded
Open this post in threaded view
|

Re: h.264 Video Streaming over TCP

pfarmer
Does anybody has an idea on how to stream h.264 video via TCP ?
Reply | Threaded
Open this post in threaded view
|

Re: h.264 Video Streaming over TCP

Tim-Philipp Müller-2
In reply to this post by pfarmer
On Sat, 2013-02-23 at 07:30 -0800, pfarmer wrote:

> Server
> gst-launch-1.0 videotestsrc horizontal-speed=5 ! x264enc tune="zerolatency"
> threads=1 ! rtph264pay config-interval=2 ! tcpserversink port=8554
> Client:
> gst-launch-1.0 tcpclientsrc port=8554 host=localhost ! application/x-rtp,
> payload=96 !  rtph264depay ! avdec_h264 ! xvimagesink

This will not work. The problem is that a TCP connection just transmits
a stream of bytes, unlike UDP it won't maintain the packetisation of the
input data, but RTP relies on that.

There are some things you can do:

a) skip the RTP pay/depay - you can send h264 byte-stream right over the
wire and then use h264parse to packetise it on the receiver end.

b) use rtph264pay ! gdppay ! sink  and  src ! gdpdepay ! rtph264depay

(But I'm not sure how much I'd rely on the GDP elements in 1.0, I don't
think they're production-use ready, but you can test if they work for
you in your use case)

Cheers
 -Tim

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: h.264 Video Streaming over TCP

Tim-Philipp Müller-2
On Thu, 2013-02-28 at 12:24 +0000, Tim-Philipp Müller wrote:
> There are some things you can do:
>
> a) skip the RTP pay/depay - you can send h264 byte-stream right over the
> wire and then use h264parse to packetise it on the receiver end.
>
> b) use rtph264pay ! gdppay ! sink  and  src ! gdpdepay ! rtph264depay

Oh, and of course

 c) put it in a streamable container (e.g. MPEG-PS or MPEG-TS)

Cheers
 -Tim

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: h.264 Video Streaming over TCP

pfarmer
Thanks a lot for the reply!

Tim-Philipp Müller-2 wrote
 a) skip the RTP pay/depay - you can send h264 byte-stream right over the wire and then use h264parse to packetise it on the receiver end.
 b) use rtph264pay ! gdppay ! sink  and  src ! gdpdepay ! rtph264depay
 c) put it in a streamable container (e.g. MPEG-PS or MPEG-TS)
To a)
Server:
 gst-launch-1.0 videotestsrc horizontal-speed=5 ! x264enc tune="zerolatency" threads=1 ! tcpserversink port=8554
Receiver:
 gst-launch-1.0 tcpclientsrc port=8554 host=localhost ! h264parse ! avdec_h264 ! xvimagesink

The receiver does not go from the Prerolling into the Playing state. I somehow should to say to the piple to be live which does not need to preroll. How can I do this?


To b)
Server:
 gst-launch-1.0 videotestsrc horizontal-speed=5 ! x264enc tune="zerolatency" threads=1 ! rtph264pay config-interval=1 ! gdppay ! tcpserversink port=8554
Receiver:
 gst-launch-1.0 tcpclientsrc port=8554 host=localhost ! gdpdepay ! application/x-rtp, payload=96 ! rtpjitterbuffer ! rtph264depay ! avdec_h264 ! xvimagesink

This works. But well its somewhat in a roundabout way and has quiet some overhead.


To c)
Server:
 gst-launch-1.0 videotestsrc horizontal-speed=5 ! x264enc tune="zerolatency" threads=1 ! mpegtsmux ! tcpserversink port=8554
Receiver:
 gst-launch-1.0 tcpclientsrc port=8554 host=localhost  ! tsdemux ! h264parse ! avdec_h264 ! xvimagesink
 
This works fine. Is it normal that there must be no tsparse before the tsdemux?
Reply | Threaded
Open this post in threaded view
|

Re: h.264 Video Streaming over TCP

Tim-Philipp Müller-2
On Mon, 2013-03-04 at 00:00 -0800, pfarmer wrote:

> To a)
> Server:
>  gst-launch-1.0 videotestsrc horizontal-speed=5 ! x264enc tune="zerolatency"
> threads=1 ! tcpserversink port=8554
> Receiver:
>  gst-launch-1.0 tcpclientsrc port=8554 host=localhost ! h264parse !
> avdec_h264 ! xvimagesink
>
> *The receiver does not go from the Prerolling into the Playing state. I
> somehow should to say to the piple to be live which does not need to
> preroll. How can I do this?*

Pass -v to gst-launch-1.0 and check what H264 caps are negotiated at the
sender. It should be stream-format=byte-stream, not avc. If it's avc,
add an .... ! video/x-h264,stream-format=byte-stream ! ...  after the
encoder.

Cheers
 -Tim

> To b)
> Server:
>  gst-launch-1.0 videotestsrc horizontal-speed=5 ! x264enc tune="zerolatency"
> threads=1 ! rtph264pay config-interval=1 ! gdppay ! tcpserversink port=8554
> Receiver:
>  gst-launch-1.0 tcpclientsrc port=8554 host=localhost ! gdpdepay !
> application/x-rtp, payload=96 ! rtpjitterbuffer ! rtph264depay ! avdec_h264
> ! xvimagesink
>
> This works. But well its somewhat in a roundabout way and has quiet some
> overhead.
>
>
> To c)
> Server:
>  gst-launch-1.0 videotestsrc horizontal-speed=5 ! x264enc tune="zerolatency"
> threads=1 ! mpegtsmux ! tcpserversink port=8554
> Receiver:
>  gst-launch-1.0 tcpclientsrc port=8554 host=localhost  ! tsdemux ! h264parse
> ! avdec_h264 ! xvimagesink
>  
> This works fine. Is it normal that there must be no tsparse before the
> tsdemux?
>
>
>
> --
> View this message in context: http://gstreamer-devel.966125.n4.nabble.com/h-264-Video-Streaming-over-TCP-tp4658747p4658856.html
> Sent from the GStreamer-devel mailing list archive at Nabble.com.
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: h.264 Video Streaming over TCP

Chuck Crisler-2
Tim, how does the -v option work on gst-launch? I haven't seen anything that seems to relate to that in the elements I have worked with but I know that it generates output that otherwise isn't displayed.

Chuck Crisler

On Mon, Mar 4, 2013 at 5:56 AM, Tim-Philipp Müller <[hidden email]> wrote:
On Mon, 2013-03-04 at 00:00 -0800, pfarmer wrote:

> To a)
> Server:
>  gst-launch-1.0 videotestsrc horizontal-speed=5 ! x264enc tune="zerolatency"
> threads=1 ! tcpserversink port=8554
> Receiver:
>  gst-launch-1.0 tcpclientsrc port=8554 host=localhost ! h264parse !
> avdec_h264 ! xvimagesink
>
> *The receiver does not go from the Prerolling into the Playing state. I
> somehow should to say to the piple to be live which does not need to
> preroll. How can I do this?*

Pass -v to gst-launch-1.0 and check what H264 caps are negotiated at the
sender. It should be stream-format=byte-stream, not avc. If it's avc,
add an .... ! video/x-h264,stream-format=byte-stream ! ...  after the
encoder.

Cheers
 -Tim

> To b)
> Server:
>  gst-launch-1.0 videotestsrc horizontal-speed=5 ! x264enc tune="zerolatency"
> threads=1 ! rtph264pay config-interval=1 ! gdppay ! tcpserversink port=8554
> Receiver:
>  gst-launch-1.0 tcpclientsrc port=8554 host=localhost ! gdpdepay !
> application/x-rtp, payload=96 ! rtpjitterbuffer ! rtph264depay ! avdec_h264
> ! xvimagesink
>
> This works. But well its somewhat in a roundabout way and has quiet some
> overhead.
>
>
> To c)
> Server:
>  gst-launch-1.0 videotestsrc horizontal-speed=5 ! x264enc tune="zerolatency"
> threads=1 ! mpegtsmux ! tcpserversink port=8554
> Receiver:
>  gst-launch-1.0 tcpclientsrc port=8554 host=localhost  ! tsdemux ! h264parse
> ! avdec_h264 ! xvimagesink
>
> This works fine. Is it normal that there must be no tsparse before the
> tsdemux?
>
>
>
> --
> View this message in context: http://gstreamer-devel.966125.n4.nabble.com/h-264-Video-Streaming-over-TCP-tp4658747p4658856.html
> Sent from the GStreamer-devel mailing list archive at Nabble.com.
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: h.264 Video Streaming over TCP

Tim-Philipp Müller-2
On Mon, 2013-03-04 at 09:21 -0500, Chuck Crisler wrote:

> how does the -v option work on gst-launch? I haven't seen anything
> that seems to relate to that in the elements I have worked with but I
> know that it generates output that otherwise isn't displayed.

If only you had the source code to check, right? :)

It hooks up to the "deep-notify" signal on the top-level pipeline, which
will receive all GObject ::notify signals from the object hierarchy,
which includes most notable the GstPad "caps" properties.

Cheers
 -Tim

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: h.264 Video Streaming over TCP

Chuck Crisler-2
Thank you. From my perspective, GStreamer is a rather large and intimidating project. It takes some time to get familiar with all of the various pieces and understand what they are doing. Sometimes a short explanation like this helps me get started digging to really understand. Again, Thank you.

Chuck

On Mon, Mar 4, 2013 at 9:33 AM, Tim-Philipp Müller <[hidden email]> wrote:
On Mon, 2013-03-04 at 09:21 -0500, Chuck Crisler wrote:

> how does the -v option work on gst-launch? I haven't seen anything
> that seems to relate to that in the elements I have worked with but I
> know that it generates output that otherwise isn't displayed.

If only you had the source code to check, right? :)

It hooks up to the "deep-notify" signal on the top-level pipeline, which
will receive all GObject ::notify signals from the object hierarchy,
which includes most notable the GstPad "caps" properties.

Cheers
 -Tim

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: h.264 Video Streaming over TCP

pfarmer
In reply to this post by Tim-Philipp Müller-2
Tim-Philipp Müller-2 wrote
Pass -v to gst-launch-1.0 and check what H264 caps are negotiated at the
sender. It should be stream-format=byte-stream, not avc. If it's avc,
add an .... ! video/x-h264,stream-format=byte-stream ! ...  after the
encoder.

Cheers
 -Tim
That's interesting. Somehow I have the feeling I was struggling with that already one year ago :). But indeed: if I set byte-stream=true in the properties of the encoder the caps are still negotiated with avc. Actually I am not sure what does avc stand for in this case.
In the H.264 book [1 p.245] it is described the following: First there is the Raw Byte Sequence Payload (RBSP) which consist of the H264 syntax elements. This RBSP can be padded with some trailing bits and then be put into a Network Abstraction Layer Unit (NALU).  This NALU in turn can be put into a Byte stream which adds a Strt code prefix to the NALUs.
I am wondering at if the Byte-Stream of the properties of the x264enc is the Byte stream they are talking about in the book and what is the "avc stream-format" related to the book.
But with the caps filter it is working! Thank you very much!

[1] Iain E. Richardson "THE H.264 ADVANCED VIDEO COMPRESSION STANDARD" Second Edition, ISBN: 978-0-470-51692-8