GST cookbook

From XiphWiki
Jump to navigation Jump to search

In addition to being a powerful multimedia infrastructure for applications Gstreamer is also a useful tool for general manipulations of multimedia data. By invoking gst-launch from the command-line with a custom pipeline many useful processing steps are possible.

Gstreamer also usually has good support for Xiph-related formats.

Unfortunately, it can be rather difficult to figure out an appropriate pipeline without a starting point.

Here are some useful examples:

Encode a .wav to Vorbis:

  • gst-launch filesrc location="INPUT.wav"  ! wavparse ! audioconvert ! vorbisenc ! oggmux ! filesink location="OUTPUT.ogg"

Dump a Theora video to PNGs:

  • gst-launch filesrc location="INPUT.ogv" ! oggdemux ! theoradec ! ffmpegcolorspace ! pngenc snapshot=false ! multifilesink location="OUTPUT%04d.png"

Transmux a MKV (containing vorbis and theora) to Ogg:

  • gst-launch filesrc location="INPUT.mkv" ! matroskademux name=d ! video/x-theora ! queue ! theoraparse ! oggmux name=mux ! filesink location="OUTPUT.ogv" d. ! audio/x-vorbis ! queue ! vorbisparse ! queue ! mux.

Encode a y4m to lossless Dirac in Ogg:

  • gst-launch filesrc location="INPUT.y4m" ! decodebin ! schroenc force-profile=vc2_main rate-control=lossless ! oggmux ! filesink location="OUTPUT.ogv"

Pull from a windows media stream, transcode to Ogg/Theora+Vorbis and send to a icecast server:

(may require purchasing Fluendo plugins for decoding the encumbered codecs)

  • gst-launch uridecodebin uri=mms://SOURCE.SERVER.COM/path name=d ! queue max-size-time=100000000 ! ffmpegcolorspace ! theoraenc bitrate=800 ! oggmux name=mux ! shout2send ip=YOURICECAST.SERVER.COM port=8000 password=YOURPASSWORD mount=/OUTPUTFILENAME.ogv d. ! queue max-size-time=100000000 ! audioconvert ! vorbisenc ! mux.

Capture video from a webcam, encode to an Ogg Theora file, decode and display on screen, write to a file whose name is the current date+time, and stream to an IceCast server

  • gst-launch-0.10 --eos-on-shutdown v4l2src ! 'video/x-raw-yuv, width=640, height=480' ! videorate ! 'video/x-raw-yuv, framerate=15/1' ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=150 ! oggmux ! tee name=ogged ! queue max-size-bytes=100000000 max-size-time=0 ! oggdemux ! theoradec ! xvimagesink sync=false force-aspect-ratio=true ogged. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=`date +%F_%T`.ogv ogged. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=YOURICECAST.SERVER.COM port=8000 password=YOURPASSWORD mount=/OUTPUTFILENAME.ogv streamname=YOURSTREAMNAME description=YOURDESCRIPTION genre=YOURGENRE url=YOURSTREAMURL ogged.

A v4l2 source + ALSA source -> Ogg Theora -> IceCast

  • dov4l -i [0|1] -m NTSC
  • gst-launch-0.10 --eos-on-shutdown v4l2src ! videoscale ! video/x-raw-yuv,width=320,height=240,framerate=30000/1001,interlaced=true ! queue max-size-bytes=100000000 max-size-time=0 ! gamma gamma=1.2 ! queue max-size-bytes=100000000 max-size-time=0 ! videobalance saturation=1.9 brightness=0.00 contrast=1.4 hue=0.06 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=400 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=mux alsasrc device=hw:1,0 latency-time=100 ! audioconvert ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=icecast-server.com password=hackme mount=/mountpoint.ogv
Note that the above pipeline will slowly lose audio/video synchronization due to hardware-level limitations in audio vs. video capture of media samples. FireWire and SDI-based capture does not have these limitations.

Two v4l2 sources combined side-by-side into rectangular video + ALSA -> Ogg Theora -> IceCast

  • mplayer tv:// -tv device=/dev/video0:driver=v4l2:norm=NTSC:width=320:height=240:outfmt=uyvy:input=1:buffersize=16: -ao null
  • mplayer tv:// -tv device=/dev/video1:driver=v4l2:norm=NTSC:width=320:height=240:outfmt=uyvy:input=1:buffersize=16: -ao null
  • gst-launch --eos-on-shutdown v4l2src device=/dev/video0 ! videoscale ! video/x-raw-yuv,width=320,height=240,interlaced=true ! queue max-size-bytes=100000000 max-size-time=0 ! gamma gamma=1.2 ! queue max-size-bytes=100000000 max-size-time=0 ! videobalance saturation=1.9 brightness=0.00 contrast=1.4 hue=0.06 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=15/1 ! queue max-size-bytes=100000000 max-size-time=0  ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0  ! videomixer name=mix ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=600 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=mux alsasrc device=hw:1,0 latency-time=100 ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0  ! shout2send ip=icecast-server.com password=hackme mount=/mountpoint.ogv v4l2src device=/dev/video1 ! videoscale ! video/x-raw-yuv,width=320,height=240,interlaced=true ! gamma gamma=1.2 ! queue max-size-bytes=100000000 max-size-time=0 ! videobalance saturation=1.9 brightness=0.00 contrast=1.4 hue=0.06 ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=5/1 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0  ! videobox border-alpha=0 fill=green left=-320 ! mix.

Create a video with an alpha channel from a sequence of PNG files

  • gst-launch-0.10 multifilesrc location=images%05d.png caps="image/png,framerate=1/1,pixel-aspect-ratio=1/1" num-buffers=95 ! pngdec ! videorate ! alphacolor ! "video/x-raw-yuv,format=(fourcc)AYUV" ! matroskamux ! filesink location=images_raw.mkv

Capture from a 1680x1050 GNOME Desktop, Combine with ALSA -> Ogg Theora + Vorbis -> Icecast

  • gst-launch --eos-on-shutdown ximagesrc ! capsfilter caps=video/x-raw-rgb,framerate=3/1,width=1680,height=1050  ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-rgb, width=1056, height=660 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=450 keyframe-auto=false keyframe-force=12 keyframe-freq=12 speed-level=0 drop-frames=false ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=mux alsasrc device=hw:0,0 ! audio/x-raw-init,rate=44100,channels=1 ! queue max-size-bytes=100000000 ! audioconvert ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=icecast-server.com password=hackme mount=/mountpoint.ogv
  • Icecast will need to be built from SVN (ver. 2.3.2 fails to relay data due to a bug)
  • Viewers will need better-than-average internet connectivity (around 1.5Mbits/sec)
  • The simplest way to capture audio events is to place a mic next to computer speakers


Live-stream a high-resolution Mac OSX Desktop at 1 FPS

  • Tested on Version 10.6.8, newer releases not compatible with osximagesrc

gst-launch --eos-on-shutdown osximagesrc ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! videoscale method=4-tap ! video/x-raw-yuv, width=960, height=600 ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=360 keyframe-auto=false keyframe-force=3 keyframe-freq=3 speed-level=0 drop-frames=false ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=host.com password=hackme mount=/mountpoint.ogv

If the screen resolution is set to mirrored 1024x640 (?), start the following then plug in the video display cable:

  • gst-launch --eos-on-shutdown osximagesrc ! queue max-size-time=0 max-size-bytes=100000000 ! ffmpegcolorspace ! queue max-size-time=0 max-size-bytes=100000000 ! videocrop right=256 top=32 ! queue max-size-time=0 max-size-bytes=100000000 ! videoscale ! video/x-raw-yuv, width=658, height=486 ! queue max-size-time=0 max-size-bytes=100000000 ! theoraenc bitrate=300 keyframe-auto=false keyframe-force=3 keyframe-freq=3 speed-level=0 drop-frames=false ! queue max-size-time=0 max-size-bytes=100000000 ! oggmux ! queue max-size-time=0 max-size-bytes=100000000 ! shout2send ip=host.com port=8000 password=hackme mount=/mountpoint.ogv
  • Streamed Keynote talks will stream the Presenter's View -- no switching between display mirrors
  • Full-screen Flash video doesn't stream, but doesn't halt the pipeline

Live-stream an Elphel 353L camera, combine with on-laptop audio capture, save a high-res copy to disk, and view live video on-screen

  • Tested on an 2.4GHz Core i3 running at ≥2.0GHz. Both CPU cores should be at ~50% continuously-smooth utilization after the first ~15 seconds; battery-only power or overheating may throttle the CPU thus leading to dropped samples.
  • Set the camera to 2592x1120 @ 18FPS. Adjust camera controls to ensure the camera achieves this framerate.
  • Kill/renice any resource-competitive applications (file indexing services, Firefox, Audacity, etc.)
  • qjackctl & → Start JACK in realtime priority
  • gst-launch -e rtspsrc location=rtsp://192.168.0.9:554 latency=100 ! queue max-size-bytes=100000000 max-size-time=0 ! rtpjpegdepay ! queue max-size-bytes=100000000 max-size-time=0 ! jpegdec ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=18/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=1296, height=560 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=halfres ! queue max-size-bytes=100000000 max-size-time=0 ! jpegenc ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mux jackaudiosrc connect=1 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=jackaudio ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=elphel_recording_`date +%F_%T`.mkv halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=648, height=280 ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=450 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0  ! oggmux name=livestream jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc  ! queue max-size-bytes=100000000 max-size-time=0 ! livestream. livestream. ! queue max-size-bytes=0 max-size-time=0 ! shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv halfres. ! videoscale add-borders=true ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! xvimagesink sync=false
  • Expect a ~50GB recording for 8hrs. of recording depending on image complexity (e.g., filming a night sky vs. a waterfall). The Matroska recording may be viewed directly in VLC (≥2.0 recommended) and may be transcoded to commonly-used formats via Miro Video Converter or Firefogg.
Alternatively: ffmpeg -i elphel_recording.mkv -acodec libfaac -ab 128k -pass 1 -vcodec libx264 -vpre slow -vpre main -b 8000k -threads 8 -f mp4 -y elphel_recording.mp4

Same as above, but simultaneously broadcast in H.264

If Moment, remove 'pingTimerTick' 'from moment/rtmp_connection.cpp' to avoid stream resets
Test stream availability with this tool.
Toggle 'byte-stream=[true|false]' and '[tcpclientsink|rtmpsink]' as necessary, possibly '[flvmux|mp4mux]' if needed.
'rtmpsink' isn't yet compatible with Akamai
  • Substitute 'videotestsrc is-live=true' for 'rtspsrc ... jpegdec' and 'audiotestsrc freq=432' for 'jackaudiosrc connect=1' for testing.
  • gst-launch-0.10 -e rtspsrc location=rtsp://192.168.0.9:554 latency=100 ! queue max-size-bytes=100000000 max-size-time=0 ! rtpjpegdepay ! queue max-size-bytes=100000000 max-size-time=0 ! jpegdec ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=18/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=1296, height=560 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=halfres ! queue max-size-bytes=100000000 max-size-time=0 ! jpegenc ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mkvmux jackaudiosrc connect=1 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=jackaudio ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=vorbisaudio ! queue max-size-bytes=100000000 max-size-time=0 ! mkvmux. mkvmux. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=elphel_recording_`date +%F_%T`.mkv halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=648, height=280 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=quarterres ! queue max-size-bytes=100000000 max-size-time=0 ! x264enc sliced-threads=true cabac=true intra-refresh=false quantizer=22 rc-lookahead=15 bitrate=600 tune=zerolatency byte-stream=false ! queue max-size-bytes=100000000 max-size-time=0 ! flvmux streamable=true name=flvmux jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! queue max-size-bytes=100000000 max-size-time=0 ! faac ! queue max-size-bytes=100000000 max-size-time=0 ! flvmux. flvmux. ! queue max-size-bytes=0 max-size-time=0 ! tcpclientsink host=127.0.0.1 port=6666 quarterres. ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=600 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0  ! oggmux name=livestream jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! livestream. livestream. ! queue max-size-bytes=0 max-size-time=0 ! shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! xvimagesink sync=false

Live-stream an Elphel 353L camera set at 'FullHD' JP46, combine with realtime audio, save a high-res copy to disk, and monitor live audio+video on-screen

  • Video samples
  • Setup a high-performance system: tested on Gigabyte GA-MA785GMT-UD2H + (non-overclocked) Phenom II X4 built using gNewSense. CPU utilization approaches 100%, continuously, across all four 3.2GHz cores.
  • Set the camera to 1920x1088, JPEG Image Quality @ 90%, JP46 color mode

In separate tabs:

  • jackd -R -d alsa -C -d hw:2 -r 48000 -i 1
  • gst-launch-0.10 -e rtspsrc location=rtsp://192.168.1.50:554 protocols=0x00000001 latency=100 ! rtpjpegdepay ! queue max-size-bytes=1000000000 max-size-time=0 ! videorate force-fps=18 ! queue max-size-bytes=1000000000 max-size-time=0 ! jpegdec max-errors=-1 idct-method=2 ! queue max-size-bytes=1000000000 max-size-time=0 ! jp462bayer threads=4 ! "video/x-raw-bayer, width=(int)1920, height=(int)1088, format=(string)grbg" ! queue max-size-bytes=1000000000 max-size-time=0 ! bayer2rgb2 method=0 ! queue max-size-bytes=1000000000 max-size-time=0 ! ffmpegcolorspace ! "video/x-raw-yuv, format=(fourcc)I420" ! queue max-size-bytes=1000000000 max-size-time=0 ! videobalance saturation=1.9 ! queue max-size-bytes=1000000000 max-size-time=0 ! tee name=fullHD ! queue max-size-bytes=100000000 max-size-time=0 ! jpegenc idct-method=2 ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mux jackaudiosrc connect=1 ! queue max-size-time=0 ! audio/x-raw-float,channels=1 ! queue max-size-time=0 ! tee name=jackaudio ! queue max-size-time=0 ! vorbisenc max-bitrate=80000 ! queue max-size-time=0 ! tee name=vorbisaudio ! queue max-size-time=0 ! mux. mux. ! queue max-size-bytes=1000000000 max-size-time=0 ! filesink location=elphel_recording_`date +%s`.mkv sync=false fullHD. ! queue max-size-bytes=1000000000 max-size-time=0 ! videoscale ! "video/x-raw-yuv, width=640, height=363, aspect-ratio=(fraction)1/1" ! queue max-size-bytes=1000000000 max-size-time=0 ! tee name=downsample ! queue max-size-bytes=1000000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=1000000000 max-size-time=0 ! ximagesink pixel-aspect-ratio=1/1 sync=false downsample. ! queue max-size-bytes=100000000 max-size-time=0 ! queue max-size-bytes=1000000000 max-size-time=0 ! theoraenc bitrate=550 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=livestream vorbisaudio. ! queue max-size-time=0 ! livestream. livestream. ! queue max-size-bytes=0 max-size-time=0 ! shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv jackaudio. ! queue max-size-time=0 ! audioconvert ! queue max-size-time=0 ! alsasink

Dump high-resolution image stills from Matroska containers into a folder

  • gst-launch filesrc location=recording.mkv ! queue ! matroskademux ! queue ! videorate ! image/jpeg,framerate=1/10 ! queue ! multifilesink location="image%04d.jpg" sync=false
  • Remove any colons input filenames to avoid BASH issues
  • Change the framerate to control the time interval between image stills and number of produced stills


Mix Three Video Sources Side-by-Side-by-Side

  • gst-launch --eos-on-shutdown videotestsrc pattern=6 is-live=true ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv,width=320,height=240 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=15/1 ! queue max-size-bytes=100000000 max-size-time=0 ! clockoverlay halignment=1 valignment=1 shaded-background=true font-desc="Sans Bold 80" ypad=80 ! queue max-size-bytes=100000000 max-size-time=0 ! videobox border-alpha=0 fill=green left=-320 ! queue max-size-bytes=100000000 max-size-time=0 ! videomixer name=mix ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! ximagesink sync=false videotestsrc is-live=true ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv,width=320,height=240 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=15/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videobox border-alpha=0 fill=blue left=-640 ! queue max-size-bytes=100000000 max-size-time=0 ! mix. videotestsrc pattern=11 ! "video/x-raw-yuv, width=352, height=240" ! queue max-size-bytes=100000000 max-size-time=0 ! mix.
Retrieved from "https://wiki.xiph.org/index.php?title=GST_cookbook&oldid=14076"