Besides kmssink with the connector-id property, there are also several other approaches to specify output display for GStreamer video playback:
- Use weston
kiosk shell and gst-launch-1.0 --prog-name argument.
- Use
gtkwaylandsink plugin and then use GTK framework to assign the GTK window to different display.
To elaborate:
Approach 1: waylandsink with app-ids in Weston kiosk shell
The Weston kiosk shell always shows Wayland surfaces in fullscreen. It also allows you to specify which application goes to which display output in weston.ini file by specifying app-ids.
The gst-launch-1.0 in GStreamer 1.24 has added an argument --prog-name that allows you to specify the application id for Wayland sink plugins in the pipeline.
As a note, the development branch of IoT Yocto v25.1-dev (based on Yocto 5.0 scarthgap and GStreamer 1.22) recently has backported the 1.24 patch to meta-mediatek-bsp. You can check the recipe patch here.
Steps
-
Either apply the patch, or upgrade to GStreamer 1.24 to have --prog-name.
-
Add the following “output” section to weston.ini (/etc/xdg/weston/weston.ini):
[output]
name=DSI-1
app-ids=gst.app.dsi
[output]
name=HDMI-A-1
app-ids=gst.app.hdmi
Note that you need to match the name to the connector names reported from DRM, e.g. modeprint mediatek.
- Run Weston with kiosk shell:
systemctl stop weston
weston --shell=kiosk --backend=drm &
Note: you might want to modify your /lib/systemd/system/weston.service unit file.
- You can then launch two
gst-launch-1.0 pipelines and specify which display output to appear:
gst-launch-1.0 --prog-name gst-app-hdmi videotestsrc ! waylandsink &
gst-launch-1.0 --prog-name gst-app-dsi videotestsrc pattern=ball ! waylandsink &
In the example above, you should see an SMPTE pattern shown in the DSI panel, and a bouncing ball in the HDMI monitor.
Approach 2: Write a GTK application with gtkwaylandsink
In GStreamer 1.22 the gtkwaylandsink plugin allows you to attach a Wayland sink surface to a GTK3 widget.
With this plugin you can use GTK3 framework to assign window size and move window around different display outputs.
Note that in IoT Yocto v25.0 the gtkwaylandsink is not enabled by default. You can refer to this commit to enable it in your image.
With the plugin enabled, you can use the GStreamer and GTK3 Python binding to (somewhat) easily write a simple GUI program with a GStreamer pipeline. Some key steps:
- gtkwaylandsink is compatible to GTK3 only:
gi.require_version('Gtk', '3.0')
gi.require_version('Gst', '1.0')
from gi.repository import Gtk, Gst, Gdk, GObject
- You need to use
gtkwaylandsink in your pipeline, and then retrieve the widget property from the sink object, then attach the GTK widget to your application window:
pipeline = Gst.parse_launch("videotestsrc ! videoconvert ! gtkwaylandsink name=gst_sink")
video_widget = pipeline.get_by_name("gst_sink").get_property("widget")
- Use
Gtk.Window.fullscreen to specify a window in fullscreen mode.
A simple Python demo script can be found here, and also a copy below:
import gi
import sys
gi.require_version('Gtk', '3.0')
gi.require_version('Gst', '1.0')
from gi.repository import Gtk, Gst, Gdk, GObject
GST_LAUNCH_CMD = "videotestsrc ! videoconvert ! gtkwaylandsink name=gst_sink"
class DemoWindow(Gtk.Window):
def __init__(self):
super().__init__(title="Genio GStreamer Video Demo")
self.setup_demo_pipeline()
# Mouse events
self.connect("button-press-event", self.on_button_press)
# Touch gestures (GTK3)
self.gesture = Gtk.GestureSwipe.new(self)
self.gesture.connect('swipe', self.on_swipe)
def setup_demo_pipeline(self):
print(f"GStreamer Launch Line >>> {GST_LAUNCH_CMD}")
self.pipeline = Gst.parse_launch(GST_LAUNCH_CMD)
# Regsiter GStreamer message
bus = self.pipeline.get_bus()
bus.add_signal_watch()
bus.connect('message', self.on_gst_message)
# Watch every property of every element in the pipeline,
# such as fpsdisplaysink.
self.pipeline.add_property_deep_notify_watch(None, True)
self.gst_sink = self.pipeline.get_by_name("gst_sink")
self.video_widget = self.gst_sink.get_property("widget")
self.add(self.video_widget)
def on_gst_message(self, bus, msg):
# This function parses GStreamer messages sent from pipeline.
# See:
# https://gstreamer.freedesktop.org/documentation/gstreamer/gstmessage.html
if msg.type == Gst.MessageType.EOS:
print(f"End-Of-Stream from {msg.src.name}, Quit.")
Gtk.main_quit()
if msg.type == Gst.MessageType.ERROR:
print(msg.parse_error())
if msg.type == Gst.MessageType.INFO:
print(msg.parse_info())
if msg.type == Gst.MessageType.WARNING:
print(msg.parse_warning())
if msg.type == Gst.MessageType.PROPERTY_NOTIFY:
# Dump fpsdisplaysink message to console
n = msg.parse_property_notify()
if n.property_name == "last-message":
print(n.property_value)
def on_window_state(self, widget, event):
self.is_fullscreen = bool(event.new_window_state & Gdk.WindowState.FULLSCREEN)
def on_swipe(self, gesture, vx, vy):
if vy > 5.0:
# swipe down to window
self.unfullscreen()
elif vy < -5.0:
# swipe up to fullscreen
self.fullscreen()
def on_button_press(self, widget, event):
# left double click
if event.button == 1 and event.type == Gdk.EventType._2BUTTON_PRESS:
self.toggle_full_screen()
def do_get_preferred_height(self):
return (240, 480)
def do_get_preferred_width(self):
return (320, 640)
def toggle_full_screen(self):
if self.get_window().get_state() & Gdk.WindowState.FULLSCREEN:
self.unfullscreen()
else:
self.fullscreen()
def play(self):
self.pipeline.set_state(Gst.State.PLAYING)
def main():
print("""
This script takes a gst-launch line and embed the video pipeline
in a GTK window.
You can:
- drag the window title bar and move it to another display, or
- mouse double-click on the video region to make it fullscreen
- touch swipe up on the video region to fullscreen mode
- touch swipe down on the video region to window mode
USAGE: ./genio-gst-demo.py [GStreamer launch line without video sink]
For example:
$ ./genio-gst-demo.py videotestsrc pattern=ball
$ ./genio-gst-demo.py filesrc location=H264_1920x1080_60fps_40Mbps.mp4 ! \\
parsebin ! queue ! v4l2h264dec ! queue ! \\
v4l2convert output-io-mode=dmabuf-import ! \\
video/x-raw,format=BGRA ! queue
""")
if len(sys.argv) > 1:
gst_cmds = sys.argv[1:]
global GST_LAUNCH_CMD
GST_LAUNCH_CMD = ' '.join(gst_cmds) + ' ! fpsdisplaysink text-overlay=false video-sink="gtkwaylandsink name=gst_sink sync=false"'
Gst.init(None)
win = DemoWindow()
win.connect("destroy", Gtk.main_quit)
win.show_all()
win.play()
Gtk.main()
if __name__ == "__main__":
main()
Let me know if this works for you 