Serial port interface on frame grabber

classic Classic list List threaded Threaded
34 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Serial port interface on frame grabber

Josh Doe
Camera Link is a scientific/engineering video interface that provides
for sending/receiving data over an embedded serial comms port,
typically for changing properties of the camera like exposure, white
balance, etc. I'm not sure if anyone else has interest in this, but I
was thinking of creating a GInterface perhaps named GstSerialPort for
standardizing this.

The two hardware APIs I've looked at so far provide two functions,
read and write (though one also provides a flush function). Baud, stop
bits, etc. are all defined by the Camera Link standard.

-Josh
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Serial port interface on frame grabber

vk_gst
Hi Josh,

This is quite interesting and something that I am looking forward for as
well. Did you manage to get this Interface working?

I want to try to write the videotestsrc pattern over serial port and try
reading it onto the other serial port on another PC. Since the reading and
writing to a serial port is file operation, I tried the following to send a
video test pattern from a imx6 board to a PC connected via serial terminal.

imx_board: gst-launch-1.0 -v videotestsrc pattern=18 !
video/x-raw,width=100,height=50 ! imxvpuenc_h264 bitrate=50 ! h264parse !
rtph264pay ! filesink location=/dev/ttyUSB1

PC: gst-launch-1.0 filesrc location=/ev/ttyUSB1 ! application/x-rtp !
rtph264depay ! h264parse ! avdec_h264 ! autovideosink sync=true

However, I observe the following error at the PC side:
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
ERROR: from element /GstPipeline:pipeline0/GstFileSrc:filesrc0: Internal
data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop ():
/GstPipeline:pipeline0/GstFileSrc:filesrc0:
streaming stopped, reason error (-5)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...


On the PC: cat /dev/ttyUSB1 does show some output, but only a fixed amount
and exits. Its not  like a continous output.

Do you have any suggestions on even if this is possible or not?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Serial port interface on frame grabber

Josh Doe
On Fri, Jun 22, 2018 at 7:49 AM vk_gst <[hidden email]> wrote:
> This is quite interesting and something that I am looking forward for as
> well. Did you manage to get this Interface working?
>
> I want to try to write the videotestsrc pattern over serial port and try
> reading it onto the other serial port on another PC.
> [...]
> Do you have any suggestions on even if this is possible or not?

Interesting, but that wasn't what I was trying to do. I was trying to
send serial commands to control a camera, you're talking about sending
camera images. I've never tried this, and while I think what you're
attempting is possible, I'm not sure how to do it.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Serial port interface on frame grabber

vk_gst
Hi Josh,

Further debugging into this, I realize that using RTP might cause an issue
with timing. So I avoided using the rtph264pay and rtph264depay. Here are
the new commands:

imx6: gst-launch-1.0 -v videotestsrc pattern=18 !
video/x-raw,width=100,height=50 ! imxvpuenc_h264 bitrate=5 ! h264parse !
filesink location=/dev/ttyUSB1

PC: CAPS=video/x-h264
PC:  gst-launch-1.0 -v filesrc location=/dev/ttyUSB1 ! $CAPS ! h264parse !
avdec_h264 ! autovideosink sync=true

There are no errors observed on the imx6 board.
However, I see the following errors at the PC side :
PC:# GST_DEBUG=3 gst-launch-1.0 -v filesrc location=/dev/ttyUSB1 ! $CAPS !
h264parse ! avdec_h264 ! autovideosink sync=true
 Setting pipeline to PAUSED …
0:00:00.066439392 15475 0x556d8a01d160 WARN basesrc
gstbasesrc.c:3583:gst_base_src_start_complete: pad not activated yet
 Pipeline is PREROLLING …
0:00:21.730466251 15475 0x556d8a000940 WARN capsfilter
gstcapsfilter.c:455:gst_capsfilter_prepare_buf: error: Filter caps do not
completely specify the output format
 0:00:21.730523691 15475 0x556d8a000940 WARN capsfilter
gstcapsfilter.c:455:gst_capsfilter_prepare_buf: error: Output caps are
unfixed: video/x-h264, width=(int)[ 1, 8192 ], height=(int)[ 1, 8192 ],
framerate=(fraction)[ 0/1, 2147483647/1 ]
 0:00:21.730676173 15475 0x556d8a000940 WARN basetransform
gstbasetransform.c:2159:default_generate_output: could not get buffer from
pool: error
 0:00:21.730742223 15475 0x556d8a000940 WARN basesrc
gstbasesrc.c:3055:gst_base_src_loop: error: Internal data stream error.
 0:00:21.730775478 15475 0x556d8a000940 WARN basesrc
gstbasesrc.c:3055:gst_base_src_loop: error: streaming stopped, reason error
(-5)
 ERROR: from element /GstPipeline:pipeline0/GstCapsFilter:capsfilter0:
Filter caps do not completely specify the output format
 Additional debug info:
 gstcapsfilter.c(455): gst_capsfilter_prepare_buf ():
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0:
 Output caps are unfixed: video/x-h264, width=(int)[ 1, 8192 ],
height=(int)[ 1, 8192 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
 ERROR: pipeline doesn’t want to preroll.
 Setting pipeline to NULL …
Freeing pipeline …



The error is related to caps negotiation, which I am not sure how to solve.
Do you have a hand on this?

Also I want to try and replace the videotestsrc with a camera interface, do
you think this would work?
I am not confident regarding the behaviour of device files in Linux, so I am
not sure, even if writing to a device file and reading it using GStreamer
would work. What are your views on this?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

AW: Serial port interface on frame grabber

Thornton, Keith
Hi,
from the error message it looks like your caps str. Is incomplete. Have you tried setting with, height and framerate i.e.
PC: CAPS=video/x-h264,width=100,height=50,framerate=<your-favorite-framerate>

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von vk_gst
Gesendet: Montag, 25. Juni 2018 14:47
An: [hidden email]
Betreff: Re: Serial port interface on frame grabber

Hi Josh,

Further debugging into this, I realize that using RTP might cause an issue with timing. So I avoided using the rtph264pay and rtph264depay. Here are the new commands:

imx6: gst-launch-1.0 -v videotestsrc pattern=18 !
video/x-raw,width=100,height=50 ! imxvpuenc_h264 bitrate=5 ! h264parse !
filesink location=/dev/ttyUSB1

PC: CAPS=video/x-h264
PC:  gst-launch-1.0 -v filesrc location=/dev/ttyUSB1 ! $CAPS ! h264parse !
avdec_h264 ! autovideosink sync=true

There are no errors observed on the imx6 board.
However, I see the following errors at the PC side :
PC:# GST_DEBUG=3 gst-launch-1.0 -v filesrc location=/dev/ttyUSB1 ! $CAPS !
h264parse ! avdec_h264 ! autovideosink sync=true  Setting pipeline to PAUSED …
0:00:00.066439392 15475 0x556d8a01d160 WARN basesrc
gstbasesrc.c:3583:gst_base_src_start_complete: pad not activated yet  Pipeline is PREROLLING …
0:00:21.730466251 15475 0x556d8a000940 WARN capsfilter
gstcapsfilter.c:455:gst_capsfilter_prepare_buf: error: Filter caps do not completely specify the output format
 0:00:21.730523691 15475 0x556d8a000940 WARN capsfilter
gstcapsfilter.c:455:gst_capsfilter_prepare_buf: error: Output caps are
unfixed: video/x-h264, width=(int)[ 1, 8192 ], height=(int)[ 1, 8192 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
 0:00:21.730676173 15475 0x556d8a000940 WARN basetransform
gstbasetransform.c:2159:default_generate_output: could not get buffer from
pool: error
 0:00:21.730742223 15475 0x556d8a000940 WARN basesrc
gstbasesrc.c:3055:gst_base_src_loop: error: Internal data stream error.
 0:00:21.730775478 15475 0x556d8a000940 WARN basesrc
gstbasesrc.c:3055:gst_base_src_loop: error: streaming stopped, reason error
(-5)
 ERROR: from element /GstPipeline:pipeline0/GstCapsFilter:capsfilter0:
Filter caps do not completely specify the output format  Additional debug info:
 gstcapsfilter.c(455): gst_capsfilter_prepare_buf ():
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0:
 Output caps are unfixed: video/x-h264, width=(int)[ 1, 8192 ], height=(int)[ 1, 8192 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
 ERROR: pipeline doesn’t want to preroll.
 Setting pipeline to NULL …
Freeing pipeline …



The error is related to caps negotiation, which I am not sure how to solve.
Do you have a hand on this?

Also I want to try and replace the videotestsrc with a camera interface, do you think this would work?
I am not confident regarding the behaviour of device files in Linux, so I am not sure, even if writing to a device file and reading it using GStreamer would work. What are your views on this?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: Serial port interface on frame grabber

vk_gst
Hi Josh,

Your solution solved the problem. With your suggestion, there is no error at
the PC side. However, there is no output at the PC side as well.  I had
hoped to see because of 'autovideosink' a new window pop up and display some
video. However, here the command on the PC keeps running and there is no
output.

Does this mean, the data rate achieved is not suitbale enough to stream the
video. I have tried the same command with bitrates of 1,10,50 at the imx6
side and it did not yield any output.

Do you have any more suggestions?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: Serial port interface on frame grabber

vk_gst
In reply to this post by Thornton, Keith
Correction:
incorrectly referred to Josh instead of Keith. :)



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

AW: AW: Serial port interface on frame grabber

Thornton, Keith
In reply to this post by vk_gst
Hi,
I haven't and can't try this myself as I don't have serial ports :-(
I suggest it is now time to use the various logging and debug helpers (such as adding an identity element with silent=false to your PC pipeline) To try to find out what is going on.
regards

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von vk_gst
Gesendet: Montag, 25. Juni 2018 17:53
An: [hidden email]
Betreff: Re: AW: Serial port interface on frame grabber

Hi Josh,

Your solution solved the problem. With your suggestion, there is no error at the PC side. However, there is no output at the PC side as well.  I had hoped to see because of 'autovideosink' a new window pop up and display some video. However, here the command on the PC keeps running and there is no output.

Does this mean, the data rate achieved is not suitbale enough to stream the video. I have tried the same command with bitrates of 1,10,50 at the imx6 side and it did not yield any output.

Do you have any more suggestions?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: AW: Serial port interface on frame grabber

vk_gst
Hi Keith,

I did the following on PC:
gst-launch-1.0 -v filesrc location=/dev/ttyUSB1 ! $CAPS ! h264parse !
avdec_h264 ! autovideosink silent=false

However this did not yield any results. There is no extra content displayed
on the Terminal.
The output is same as the previous:

root@linux-GP60-2PE:/home/linux# gst-launch-1.0 -v  filesrc
location=/dev/ttyUSB0 ! $CAPS  ! h264parse  ! avdec_h264 ! autovideosink
silent=false
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps =
video/x-h264, width=(int)100, height=(int)50, framerate=(fraction)10/1
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps =
video/x-h264, width=(int)100, height=(int)50, framerate=(fraction)10/1


Do you have other suggestions to enable the debugging?

Regards








--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: AW: Serial port interface on frame grabber

vk_gst
In reply to this post by Thornton, Keith
Hi,

Ignore my previous post stating the debugging messages, because I see that I
did a mistake in the pipeline on the imx6 board where I included the element
rtph264pay in the pipeline running on imx6 board. Now I have removed the
element and these are the commands on the PC and imx6:

imx6: gst-launch-1.0 -v videotestsrc pattern=18 !
video/x-raw,width=100,height=50 ! imxvpuenc_h264 bitrate=5 ! h264parse !
filesink location=/dev/ttyUSB1

PC: CAPS=video/x-h264 ,width=100,height=50, framerate=10/1
PC:  GST_DEBUG=6 GST_DEBUG_FILE= <file location > gst-launch-1.0 -v filesrc
location=/dev/ttyUSB1 ! $CAPS ! h264parse ! avdec_h264 ! autovideosink
sync=true

Hereby I attach file2.txt
<http://gstreamer-devel.966125.n4.nabble.com/file/t378365/file2.txt>   the
new log file, which corresponds to the above commands.

The debug output shows out h264 parse error and some errors in the 'libav'.
The data is interpreted as no frame by 'libav'.

I fail to understand why the encoded data is not being received in the
format that is readable by the GStreamer. For IP it works, however for
serial it does not. Does this highly depend on the hardware interface?


Regards



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

AW: AW: AW: Serial port interface on frame grabber

Thornton, Keith
In reply to this post by Thornton, Keith
Hi,
I'm not sure what the error message no codec_data and no stream format means in this context. Where I work, we exchange codec_data and stream format using an out of band channel (rtsp / rtp) so that the receiver knows what sort of data it is receiving but I don't know what a serial to serial communication woud expect. Previously I recommended inserting an identity element into the pipeline not to setting the sink to silent=false.
That would be
gst-launch-1.0 -v filesrc location=/dev/ttyUSB1 ! $CAPS ! identity silent=false ! h264parse ! avdec_h264 ! autovideosink sync=false
This won't help further in this case though.

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von vk_gst
Gesendet: Dienstag, 26. Juni 2018 13:28
An: [hidden email]
Betreff: Re: AW: AW: Serial port interface on frame grabber

Hi Keith,

Here is a little more debug info for the command:

0:00:00.115241515  8581 0x563087d8f200 INFO              GST_STATES
gstbin.c:2504:gst_bin_element_set_state:<filesrc0> current NULL pending VOID_PENDING, desired next READY
0:00:00.115251856  8581 0x563087d8f200 INFO              GST_STATES
gstelement.c:2676:gst_element_continue_state:<filesrc0> completed state change to READY
0:00:00.115261444  8581 0x563087d8f200 INFO              GST_STATES
gstelement.c:2579:_priv_gst_element_state_changed:<filesrc0> notifying about state-changed NULL to READY (VOID_PENDING pending)
0:00:00.115274913  8581 0x563087d8f200 INFO              GST_STATES
gstbin.c:2952:gst_bin_change_state_func:<pipeline0> child 'filesrc0' changed state to 2(READY) successfully
0:00:00.115288368  8581 0x563087d8f200 INFO              GST_STATES
gstelement.c:2651:gst_element_continue_state:<pipeline0> committing state from NULL to READY, pending PAUSED, next PAUSED
0:00:00.115299121  8581 0x563087d8f200 INFO              GST_STATES
gstelement.c:2579:_priv_gst_element_state_changed:<pipeline0> notifying about state-changed NULL to READY (PAUSED pending)
0:00:00.115321866  8581 0x563087d8f200 INFO              GST_STATES
gstelement.c:2658:gst_element_continue_state:<pipeline0> continue state change READY to PAUSED, final PAUSED
0:00:00.115340443  8581 0x563087d8f200 INFO              GST_STATES
gstbin.c:2504:gst_bin_element_set_state:<autovideosink0> current READY pending VOID_PENDING, desired next PAUSED
0:00:00.115356984  8581 0x563087d8f200 INFO              GST_STATES
gstbin.c:2504:gst_bin_element_set_state:<autovideosink0-actual-sink-xvimage>
current READY pending VOID_PENDING, desired next PAUSED
0:00:00.115377702  8581 0x563087d8f200 INFO              GST_STATES
gstbin.c:2958:gst_bin_change_state_func:<autovideosink0> child 'autovideosink0-actual-sink-xvimage' is changing state asynchronously to PAUSED
0:00:00.115394251  8581 0x563087d8f200 INFO              GST_STATES
gstbin.c:2958:gst_bin_change_state_func:<pipeline0> child 'autovideosink0'
is changing state asynchronously to PAUSED
0:00:00.115406879  8581 0x563087d8f200 INFO              GST_STATES
gstbin.c:2504:gst_bin_element_set_state:<avdec_h264-0> current READY pending VOID_PENDING, desired next PAUSED
0:00:00.115478955  8581 0x563087d8f200 INFO              GST_STATES
gstelement.c:2676:gst_element_continue_state:<avdec_h264-0> completed state change to PAUSED
0:00:00.115488874  8581 0x563087d8f200 INFO              GST_STATES
gstelement.c:2579:_priv_gst_element_state_changed:<avdec_h264-0> notifying about state-changed READY to PAUSED (VOID_PENDING pending)
0:00:00.115503618  8581 0x563087d8f200 INFO              GST_STATES
gstbin.c:2952:gst_bin_change_state_func:<pipeline0> child 'avdec_h264-0'
changed state to 3(PAUSED) successfully
0:00:00.115516898  8581 0x563087d8f200 INFO              GST_STATES
gstbin.c:2504:gst_bin_element_set_state:<h264parse0> current READY pending VOID_PENDING, desired next PAUSED
0:00:00.115580248  8581 0x563087d8f200 INFO                 filesrc
gstfilesrc.c:468:gst_file_src_start:<filesrc0> opening file /dev/ttyUSB0
0:00:00.119904565  8581 0x563087d8f200 WARN                 basesrc
gstbasesrc.c:3583:gst_base_src_start_complete:<filesrc0> pad not activated yet
0:00:00.120137836  8581 0x563087d8f200 INFO              GST_STATES
gstelement.c:2676:gst_element_continue_state:<h264parse0> completed state change to PAUSED
0:00:00.120157032  8581 0x563087d8f200 INFO              GST_STATES
gstelement.c:2579:_priv_gst_element_state_changed:<h264parse0> notifying about state-changed READY to PAUSED (VOID_PENDING pending)
0:00:00.120185395  8581 0x563087d8f200 INFO              GST_STATES
gstbin.c:2952:gst_bin_change_state_func:<pipeline0> child 'h264parse0'
changed state to 3(PAUSED) successfully
0:00:00.120205986  8581 0x563087d8f200 INFO              GST_STATES
gstbin.c:2504:gst_bin_element_set_state:<capsfilter0> current READY pending VOID_PENDING, desired next PAUSED
0:00:00.120226662  8581 0x563087d8f200 INFO              GST_STATES
gstelement.c:2676:gst_element_continue_state:<capsfilter0> completed state change to PAUSED
0:00:00.120237123  8581 0x563087d8f200 INFO              GST_STATES
gstelement.c:2579:_priv_gst_element_state_changed:<capsfilter0> notifying about state-changed READY to PAUSED (VOID_PENDING pending)
0:00:00.120251129  8581 0x563087d8f200 INFO              GST_STATES
gstbin.c:2952:gst_bin_change_state_func:<pipeline0> child 'capsfilter0'
changed state to 3(PAUSED) successfully
0:00:00.120269258  8581 0x563087d8f200 INFO              GST_STATES
gstbin.c:2504:gst_bin_element_set_state:<filesrc0> current READY pending VOID_PENDING, desired next PAUSED
0:00:00.120285094  8581 0x563087d8f200 INFO                 filesrc
gstfilesrc.c:468:gst_file_src_start:<filesrc0> opening file /dev/ttyUSB0
0:00:00.121527616  8581 0x563087d8f200 INFO                 basesrc
gstbasesrc.c:1358:gst_base_src_do_seek:<filesrc0> seeking: bytes segment start=0, offset=0, stop=-1, rate=1,000000, applied_rate=1,000000, flags=0x00, time=0, base=0, position 0, duration -1
0:00:00.121566441  8581 0x563087d8f200 INFO                    task
gsttask.c:457:gst_task_set_lock: setting stream lock 0x563087fc2230 on task
0x563087fc74d0
0:00:00.121574640  8581 0x563087d8f200 INFO                GST_PADS
gstpad.c:6131:gst_pad_start_task:<filesrc0:src> created task 0x563087fc74d0
0:00:00.121653818  8581 0x563087d8f200 INFO              GST_STATES
gstelement.c:2676:gst_element_continue_state:<filesrc0> completed state change to PAUSED
0:00:00.121664466  8581 0x563087d8f200 INFO              GST_STATES
gstelement.c:2579:_priv_gst_element_state_changed:<filesrc0> notifying about state-changed READY to PAUSED (VOID_PENDING pending)
0:00:00.121678017  8581 0x563087d8f200 INFO              GST_STATES
gstbin.c:2952:gst_bin_change_state_func:<pipeline0> child 'filesrc0' changed state to 3(PAUSED) successfully
0:00:00.121717805  8581 0x56308806cd40 INFO        GST_ELEMENT_PADS
gstelement.c:917:gst_element_get_static_pad: no such pad 'sink' in element "filesrc0"
0:00:02.657744831  8581 0x56308806cd40 INFO               GST_EVENT
gstevent.c:895:gst_event_new_segment: creating segment event bytes segment start=0, offset=0, stop=-1, rate=1,000000, applied_rate=1,000000, flags=0x00, time=0, base=0, position 0, duration -1
0:00:02.657817130  8581 0x56308806cd40 INFO                 basesrc
gstbasesrc.c:2945:gst_base_src_loop:<filesrc0> marking pending DISCONT
0:00:02.658027126  8581 0x56308806cd40 INFO               GST_EVENT
gstevent.c:814:gst_event_new_caps: creating caps event video/x-h264, width=(int)100, height=(int)50, framerate=(fraction)10/1
0:00:02.658116708  8581 0x56308806cd40 ERROR              h264parse
gsth264parse.c:2588:gst_h264_parse_set_caps: video/x-h264 caps without codec_data or stream-format



And the most important are these lines:

root@linux-GP60-2PE:/home/linux# GST_DEBUG=2 gst-launch-1.0 -v filesrc
location=/dev/ttyUSB0 ! $CAPS  ! h264parse  ! avdec_h264 ! autovideosink > /home/linux/file1.txt
0:00:00.065300808  8529 0x562f49a33920 WARN                 basesrc
gstbasesrc.c:3583:gst_base_src_start_complete:<filesrc0> pad not activated yet
0:00:02.470276383  8529 0x562f49a00140 ERROR              h264parse
gsth264parse.c:2588:gst_h264_parse_set_caps: video/x-h264 caps without codec_data or stream-format


It says that the caps are without the codec_data or stream-format. Does this mean that the encoded data which is written on the imx6 board to the serial port gets converted to some other format and hence at PC, the gstreamer is unable to decode it back to a video data.





--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: AW: AW: Serial port interface on frame grabber

vk_gst
Hi,

One of the person working here at the university, had built a usb-serial-usb
device that is configurable upto 3MBaud. I tried the same thing as explained
in my previous comments, and this time the window popped up displaying a
video on the PC. However, the video frames are totally messed up and is not
the same as vidoetestsrc pattern  I send. There are a lot of slices being
dropped and the errors(the file which I attached in my earlier post) are
still present.

I have no idea on how to debug this further. The serial port does seem
challenging for a video data. :(




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: AW: AW: Serial port interface on frame grabber

vk_gst
In reply to this post by Thornton, Keith
Hi Keith,

I was thinking of PPP for this use case. This will allow me to access the
serial port with an IP address and hence I could try and use RTP. However
this will be limited by the Baudrate I guess.

I have to still figure out a way to configure PPP on imx6 and I am working
on it.
 
Any idea whether PPP might be a potential solution?

Regards



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

AW: AW: AW: AW: Serial port interface on frame grabber

Thornton, Keith
No sorry, no idea

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von vk_gst
Gesendet: Mittwoch, 27. Juni 2018 09:59
An: [hidden email]
Betreff: Re: AW: AW: AW: Serial port interface on frame grabber

Hi Keith,

I was thinking of PPP for this use case. This will allow me to access the serial port with an IP address and hence I could try and use RTP. However this will be limited by the Baudrate I guess.

I have to still figure out a way to configure PPP on imx6 and I am working on it.
 
Any idea whether PPP might be a potential solution?

Regards



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Serial port interface on frame grabber

Kaj-Michael Lang
In reply to this post by vk_gst
On Tue, 2018-06-26 at 08:32 -0700, vk_gst wrote:
> video on the PC. However, the video frames are totally messed up and
> is not
> the same as vidoetestsrc pattern  I send. There are a lot of slices
> being

You might first try with saving to a local file and checking that the
low bitrate is able to produce something that works.

--
Kaj-Michael Lang <[hidden email]>

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: AW: AW: Serial port interface on frame grabber

James Cameron
In reply to this post by vk_gst
I'm maintainer of PPTP, and I know PPP very well.

Yes, it would give you an IP transport across the serial link.

There's some additional overhead; encoding, and packet headers.  Yes,
you will lose some bandwidth.

Some pppd options can be used to reduce some of the overhead.  See man
pppd.

To compensate, you should work hard on yohur design to increase the
baud rate yet ensure the electrical signal has the right edges over
the transmission distance and cabling you are planning.

So what distance are you planning?

Is it 3.3V or 5V TTL serial, or conventional +/- 12V?  These days,
3.3V TTL level is more normal.  Very limited range without signal
conditioning and line drivers; of the order 50cm in a clean
environment, and much less in a noisy environment or bad grounding.

On Wed, Jun 27, 2018 at 12:59:11AM -0700, vk_gst wrote:

> Hi Keith,
>
> I was thinking of PPP for this use case. This will allow me to access the
> serial port with an IP address and hence I could try and use RTP. However
> this will be limited by the Baudrate I guess.
>
> I have to still figure out a way to configure PPP on imx6 and I am working
> on it.
>  
> Any idea whether PPP might be a potential solution?
>
> Regards
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--
James Cameron
http://quozl.netrek.org/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: AW: AW: Serial port interface on frame grabber

vk_gst
Hi James,

Thanks for the update.

I have a 433MHz wireless Tx/Rx that connects over serial port. The idea was
to try and use these wireless devices to stream video data on the PC from
imx6. This is the device  here
<https://www.unmannedtechshop.co.uk/100mw-ardupilot-unmanned-telemetry-kit-v2-433mhz/>  
So this works on 3.3V TTL.

With your previous test of sending and receiving the 1MB file, I performed
it and found that only around 666KB were received at the receiver side. The
binary comparison of the files revealed a number of random bytes being
missed at the receiving side.

I performed the same test, with the usb-serial-serial-usb wired device (a
small approx. 50cm cable we built long back) with 115200 baud and the file
was received correctly at the receiver. The 'shasum' also matched in this
case for the files.

So, the H264 decoder at the PC side is just not receiving enough bytes to
stream a video out of it. I do not know if using PPP here would be useful. I
can try to use RTP over UDP over serial and then transmitting it wirelessly
and see if it improves anything. But I am not sure. May be you can shed some
light on this?




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: AW: AW: Serial port interface on frame grabber

James Cameron
G'day vk_gst,

So GStreamer worked fine with a serial port that did not lose data?

If so, this is no longer a GStreamer problem per se.

Yes, I'm familiar with that class of wireless serial device.  Data can
be lost, and you have proved it is lost; the most common causes are;

- path fade,

- interfering transmissions from other devices on the shared
  frequency,

- badly configured flow control; if the firmware is configured to use
  XON/XOFF signalling from the host, it may drop all bytes 0x11 or
  0x13; so check for those bytes in the output you captured,

- missing flow control; the firmware has to buffer the data until it
  can be transmitted, and if the host continues to send data when the
  buffer is full, the data is lost; so you may have to add flow
  control lines if the host can take that,

You could work on these problems to improve the transmission test
until the shasum matches.

PPP might not help; I've not used it with lossy links for a very long
time, and I imagine it would become confused.

You could also find another RF serial module which includes delivery
guarantees.  Telemetry usually gives more recent data priority over
older data, and so doesn't need to guarantee delivery in-order.

On Wed, Jun 27, 2018 at 04:18:06AM -0700, vk_gst wrote:

> Hi James,
>
> Thanks for the update.
>
> I have a 433MHz wireless Tx/Rx that connects over serial port. The idea was
> to try and use these wireless devices to stream video data on the PC from
> imx6. This is the device  here
> <https://www.unmannedtechshop.co.uk/100mw-ardupilot-unmanned-telemetry-kit-v2-433mhz/>  
> So this works on 3.3V TTL.
>
> With your previous test of sending and receiving the 1MB file, I performed
> it and found that only around 666KB were received at the receiver side. The
> binary comparison of the files revealed a number of random bytes being
> missed at the receiving side.
>
> I performed the same test, with the usb-serial-serial-usb wired device (a
> small approx. 50cm cable we built long back) with 115200 baud and the file
> was received correctly at the receiver. The 'shasum' also matched in this
> case for the files.
>
> So, the H264 decoder at the PC side is just not receiving enough bytes to
> stream a video out of it. I do not know if using PPP here would be useful. I
> can try to use RTP over UDP over serial and then transmitting it wirelessly
> and see if it improves anything. But I am not sure. May be you can shed some
> light on this?
>
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--
James Cameron
http://quozl.netrek.org/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: AW: AW: Serial port interface on frame grabber

vk_gst
Hi James,

Thanks for the heads up.

> So GStreamer worked fine with a serial port that did not lose data?
 I only tried sending the 1MB file as you mentioned in the other post using
the wireless device both connected to the same PC and then one to imx6 board
and one to PC. The losses were observed in both cases. With the
usb-serial-usb cable, the transfer of file worked fine and the sha256sum
matched.

Streaming the video from imx6 board to PC was done with the wireless device,
but the GStreamer at the PC was not able to display any video at baud
115200. With the same baud,I performed this with the usb-serial-usb wire and
there was no video displayed. Later I tried with the usb-serial-usb cable
and baud=1MBaud and there was a video popup on PC, but with the error as
mentioned in the file
<http://gstreamer-devel.966125.n4.nabble.com/file/t378365/file2.txt>.  The
frames were missed and the video was distorted, not even close to the
videotestsrc I was sending from the imx6 board.  I also tried using a
bitrate of 5Kbps and 10 Kbps and using both (wireless device  &
usb-serial-usb cable ) to send the videotestsrc and save it to a file. The
file was unreadable. The size of file recieved in both cases were 0 bytes.
Command to write to file ( gst-launch-1.0 -v filesrc location=/dev/ttyUSB0 !
$CAPS ! h264parse ! avdec_h264 ! autovideosink sync=true) .

With 5/10 Kbps bitrate of encoded stream and 1MBaud, GStreamer displays a
distorted video. But not at lesser Baudrates. With ethernet cable and UDP,
everything works fine, should be the data transfer rate and RTP packet
matching that does the trick.

 

> badly configured flow control; if the firmware is configured to use
> XON/XOFF signalling from the host, it may drop all bytes 0x11 or
> 0x13; so check for those bytes in the output you captured,

I can see the 0x11 and 0x13 bytes in the received file, so this can be ruled
out.
However, the other things, I can give it a try, but using the flow control
might increase the latency of the video streaming.

The idea was to try and use the camera on the board and see if we could
stream live video wirelessly. I am trying to build a FPV system for Drones
that is usable for long range(as compared to short range available in
market)


>-PPP might not help; I've not used it with lossy links for a very long
> time, and I imagine it would become confused

I was considering using PPP, because I was thinking that RTP might help in
reconstructing the encoded packets at the receiver side een with some
acceptable losses. As RTP wiki says, it maintains the sequence numbers of
encoded packets to the UDP packets and helps in reconstructing. Such a
feature is not available in case of UART and hence I guess, GStreamer was
unable to reconstruct the frames from the data received.


Well now, I guess I need to try and find some other solutions. Perhaps LTE
system with GStreamer video streaming. Any views ?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: AW: AW: Serial port interface on frame grabber

James Cameron
It sounds like a fascinating problem; and the market lack of solutions
demonstrates how hard it is to solve.  I think you need to dig into
the components you are using to find out how they work.

The filesrc element is intended for use with files.  While Linux
exposes serial ports in a file-like manner, they aren't files, because
they don't behave in the same way over time.

Use strace to examine how the filesrc element is reading from the
serial port; it may be making a read syscall, with blocking, for the
value of element properly blocksize; default 4096 bytes.  This would
be the most efficient for reading from a file, but not necessarily
what you want for reading from a serial port.  After each pause in
transmission (and there will be pauses because it is a firmware driven
packet radio), there will be a remainder of bytes in the buffer yet to
be passed into the pipeline.  When enough packets arrive, the read
syscall will complete.  Try a smaller blocksize.  Add a larger queue
element.  Use the filesrc element sources to find out what it does and
why.

Split the problem.  Temporarily exclude the serial link by emulating
it between two GStreamer pipelines.  Locate or write a serial port
emulator which feeds a byte at a time and pauses for a very short time
in order to achieve 115200 baud throughput.  Perhaps the Ubuntu
package trickle could be used for ideas.

Measure the timing of arrival of serial data, and reproduce that same
timing in the emulator.

--
James Cameron
http://quozl.netrek.org/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
12