Pipeline Editor as Web App / UI

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Pipeline Editor as Web App / UI

Ralf Sippl
Hi all,

GStreamer is extremely powerful and flexible, but to the average user, it's
extremely hard to understand and use. gst-inspect-1.0, gst-launch-1.0, even
those commands are impossibly hard to type :)

What it lacks is a friendly UI. Imagine a drag-and-drop pipeline editor in
the browser, with the default pipeline sending video and audio to the
browser e.g. using the newly added webrtcbin. As default source, it would
use either the camera, or videotestsrc (or was it testvideosrc? See what I
mean??)

The app could have four layers:

Layer 0: GStreamer. This could be the local GStreamer install via the OS's
package manager. Or, if that version is too old as in my case, a gst-build
'sandbox'.

Layer 1: (optional) your own set of GStreamer plugins. As an example, I have
a set of deep learning/inference plugins (face reco, obj detection, body
pose etc.).

Layer 2: HTTP API. Simply put, a gst-inspect/gst-launch wrapper written in
Python e.g. as RESTful API. Provides all data needed by Layer 3.

Layer 3: the fancy UI, HTML5 + JS (Vue.JS, React, whatever). Gives you
access to all available GStreamer elements (in a sane way), to the pipeline
as an editable graph, to each element and its properties (click on graph
node opens dialog), launch button starts the pipeline (opens video window);
get the textual notation of the pipeline (... ! ...), see the GStreamer log
in an extra window etc.

(Layer 4: package everything as a Docker container)

The only open source project I'm aware of that resembles this is
https://github.com/bbc/brave, but it works at a higher abstraction level
(which can be both good and bad). There were other attempts at implementing
editors, all of them native (e.g. based on Qt), but none of those I'm aware
of were successful.

Is there any open source project with the aim to implement a web-based
pipeline editor? Is there any interest to work on (or fund) such a project?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
R C
Reply | Threaded
Open this post in threaded view
|

Re: Pipeline Editor as Web App / UI

R C

not only the "average user" ...  I design design special purpose HPCs (HPC clusters, and

parallel file systems) for a living ..   and I can't figure gstreamer out either, while what

I am trying to do should not even be that hard. (getting gstreamer to stream an IP camera feed

to be embedded in a html/web page.)

Some sort of gui would definitely help.


As for the docker container...   yeah good luck with that, I know a bunch of people that are trying to build a robust, production level containers, so far containers don't deliver,  especially when it comes to compatibility with older software.


Ron


On 5/5/19 3:29 AM, gripsynth wrote:
Hi all, 

GStreamer is extremely powerful and flexible, but to the average user, it's
extremely hard to understand and use. gst-inspect-1.0, gst-launch-1.0, even
those commands are impossibly hard to type :) 

What it lacks is a friendly UI. Imagine a drag-and-drop pipeline editor in
the browser, with the default pipeline sending video and audio to the
browser e.g. using the newly added webrtcbin. As default source, it would
use either the camera, or videotestsrc (or was it testvideosrc? See what I
mean??) 

The app could have four layers: 

Layer 0: GStreamer. This could be the local GStreamer install via the OS's
package manager. Or, if that version is too old as in my case, a gst-build
'sandbox'. 

Layer 1: (optional) your own set of GStreamer plugins. As an example, I have
a set of deep learning/inference plugins (face reco, obj detection, body
pose etc.). 

Layer 2: HTTP API. Simply put, a gst-inspect/gst-launch wrapper written in
Python e.g. as RESTful API. Provides all data needed by Layer 3. 

Layer 3: the fancy UI, HTML5 + JS (Vue.JS, React, whatever). Gives you
access to all available GStreamer elements (in a sane way), to the pipeline
as an editable graph, to each element and its properties (click on graph
node opens dialog), launch button starts the pipeline (opens video window);
get the textual notation of the pipeline (... ! ...), see the GStreamer log
in an extra window etc. 

(Layer 4: package everything as a Docker container) 

The only open source project I'm aware of that resembles this is
https://github.com/bbc/brave, but it works at a higher abstraction level
(which can be both good and bad). There were other attempts at implementing
editors, all of them native (e.g. based on Qt), but none of those I'm aware
of were successful. 

Is there any open source project with the aim to implement a web-based
pipeline editor? Is there any interest to work on (or fund) such a project?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Pipeline Editor as Web App / UI

Ralf Sippl
"Some sort of gui" is what didn't work that well in the past. It needs to be
a GUI which is fairly easy to set up and use by new users, works on every
OS, and is flexible enough to support more complex setups. Like, run locally
or remote (or on a Raspberry Pi), headless with scripts talking to the REST
API, with the default JS UI or with an UI tailored to your needs etc.

As for the issues you mentioned, a WebRTC based solution should be ok for
you, if not directly using webrtcbin (which BTW works well for me), then via
a server like Janus. In your thread, I just posted a pipeline which streams
to janus via udpsink.

Regarding Docker, projects like https://github.com/mpromonet/webrtc-streamer
show that containers work well, if done right...



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel