GST cookbook

From XiphWiki
Revision as of 11:48, 21 May 2012 by GChriss (talk | contribs) (→‎add H.264 subsection: new section)
Jump to navigation Jump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

In addition to being a powerful multimedia infrastructure for applications Gstreamer is also a useful tool for general manipulations of multimedia data. By invoking gst-launch from the command-line with a custom pipeline many useful processing steps are possible.

Gstreamer also usually has good support for Xiph-related formats.

Unfortunately, it can be rather difficult to figure out an appropriate pipeline without a starting point.

Here are some useful examples:

Encode a .wav to Vorbis:

  • gst-launch filesrc location="INPUT.wav"  ! wavparse ! audioconvert ! vorbisenc ! oggmux ! filesink location="OUTPUT.ogg"

Dump a Theora video to PNGs:

  • gst-launch filesrc location="INPUT.ogv" ! oggdemux ! theoradec ! ffmpegcolorspace ! pngenc snapshot=false ! multifilesink location="OUTPUT%04d.png"

Transmux a MKV (containing vorbis and theora) to Ogg:

  • gst-launch filesrc location="INPUT.mkv" ! matroskademux name=d ! video/x-theora ! queue ! theoraparse ! oggmux name=mux ! filesink location="OUTPUT.ogv" d. ! audio/x-vorbis ! queue ! vorbisparse ! queue ! mux.

Encode a y4m to lossless Dirac in Ogg:

  • gst-launch filesrc location="INPUT.y4m" ! decodebin ! schroenc force-profile=vc2_main rate-control=lossless ! oggmux ! filesink location="OUTPUT.ogv"

Pull from a windows media stream, transcode to Ogg/Theora+Vorbis and send to a icecast server:

(may require purchasing Fluendo plugins for decoding the encumbered codecs)

  • gst-launch uridecodebin uri=mms://SOURCE.SERVER.COM/path name=d ! queue max-size-time=100000000 ! ffmpegcolorspace ! theoraenc bitrate=800 ! oggmux name=mux ! shout2send ip=YOURICECAST.SERVER.COM port=8000 password=YOURPASSWORD mount=/OUTPUTFILENAME.ogv d. ! queue max-size-time=100000000 ! audioconvert ! vorbisenc ! mux.

Capture video from a webcam, encode to an Ogg Theora file, decode and display on screen, write to a file whose name is the current date+time, and stream to an IceCast server

  • gst-launch-0.10 --eos-on-shutdown v4l2src ! 'video/x-raw-yuv, width=640, height=480' ! videorate ! 'video/x-raw-yuv, framerate=15/1' ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=150 ! oggmux ! tee name=ogged ! queue max-size-bytes=100000000 max-size-time=0 ! oggdemux ! theoradec ! xvimagesink sync=false force-aspect-ratio=true ogged. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=`date +%F_%T`.ogv ogged. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=YOURICECAST.SERVER.COM port=8000 password=YOURPASSWORD mount=/OUTPUTFILENAME.ogv streamname=YOURSTREAMNAME description=YOURDESCRIPTION genre=YOURGENRE url=YOURSTREAMURL ogged.

A v4l2 source + ALSA source -> Ogg Theora -> IceCast

  • dov4l -i [0|1] -m NTSC
  • gst-launch-0.10 --eos-on-shutdown v4l2src ! videoscale ! video/x-raw-yuv,width=320,height=240,framerate=30000/1001,interlaced=true ! queue max-size-bytes=100000000 max-size-time=0 ! gamma gamma=1.2 ! queue max-size-bytes=100000000 max-size-time=0 ! videobalance saturation=1.9 brightness=0.00 contrast=1.4 hue=0.06 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=400 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=mux alsasrc device=hw:1,0 latency-time=100 ! audioconvert ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=icecast-server.com password=hackme mount=/mountpoint.ogv
Note that the above pipeline will slowly lose audio/video synchronization due to hardware-level limitations in audio vs. video capture of media samples. FireWire and SDI-based capture does not have these limitations.

Two v4l2 sources combined side-by-side into rectangular video + ALSA -> Ogg Theora -> IceCast

  • mplayer tv:// -tv device=/dev/video0:driver=v4l2:norm=NTSC:width=320:height=240:outfmt=uyvy:input=1:buffersize=16: -ao null
  • mplayer tv:// -tv device=/dev/video1:driver=v4l2:norm=NTSC:width=320:height=240:outfmt=uyvy:input=1:buffersize=16: -ao null
  • gst-launch --eos-on-shutdown v4l2src device=/dev/video0 ! videoscale ! video/x-raw-yuv,width=320,height=240,interlaced=true ! queue max-size-bytes=100000000 max-size-time=0 ! gamma gamma=1.2 ! queue max-size-bytes=100000000 max-size-time=0 ! videobalance saturation=1.9 brightness=0.00 contrast=1.4 hue=0.06 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=15/1 ! queue max-size-bytes=100000000 max-size-time=0  ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0  ! videomixer name=mix ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=600 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=mux alsasrc device=hw:1,0 latency-time=100 ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0  ! shout2send ip=icecast-server.com password=hackme mount=/mountpoint.ogv v4l2src device=/dev/video1 ! videoscale ! video/x-raw-yuv,width=320,height=240,interlaced=true ! gamma gamma=1.2 ! queue max-size-bytes=100000000 max-size-time=0 ! videobalance saturation=1.9 brightness=0.00 contrast=1.4 hue=0.06 ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=5/1 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0  ! videobox border-alpha=0 fill=green left=-320 ! mix.

Create a video with an alpha channel from a sequence of PNG files

  • gst-launch-0.10 multifilesrc location=images%05d.png caps="image/png,framerate=1/1,pixel-aspect-ratio=1/1" num-buffers=95 ! pngdec ! videorate ! alphacolor ! "video/x-raw-yuv,format=(fourcc)AYUV" ! matroskamux ! filesink location=images_raw.mkv

Capture from a 1680x1050 GNOME Desktop, Combine with ALSA -> Ogg Theora + Vorbis -> Icecast

  • gst-launch --eos-on-shutdown ximagesrc ! capsfilter caps=video/x-raw-rgb,framerate=3/1,width=1680,height=1050  ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-rgb, width=1056, height=660 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=450 keyframe-auto=false keyframe-force=12 keyframe-freq=12 speed-level=0 drop-frames=false ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=mux alsasrc device=hw:0,0 ! audio/x-raw-init,rate=44100,channels=1 ! queue max-size-bytes=100000000 ! audioconvert ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=icecast-server.com password=hackme mount=/mountpoint.ogv
  • Icecast will need to be built from SVN (ver. 2.3.2 fails to relay data due to a bug)
  • Viewers will need better-than-average internet connectivity (around 1.5Mbits/sec)
  • The simplest way to capture audio events is to place a mic next to computer speakers


Live-stream a high-resolution Mac OSX Desktop at 1 FPS

  • Tested on Version 10.6.8, newer releases not compatible with osximagesrc

gst-launch --eos-on-shutdown osximagesrc ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! videoscale method=4-tap ! video/x-raw-yuv, width=960, height=600 ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=360 keyframe-auto=false keyframe-force=3 keyframe-freq=3 speed-level=0 drop-frames=false ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=host.com password=hackme mount=/mountpoint.ogv

If the screen resolution is set to mirrored 1024x640 (?), start the following then plug in the video display cable:

  • gst-launch --eos-on-shutdown osximagesrc ! queue max-size-time=0 max-size-bytes=100000000 ! ffmpegcolorspace ! queue max-size-time=0 max-size-bytes=100000000 ! videocrop right=256 top=32 ! queue max-size-time=0 max-size-bytes=100000000 ! videoscale ! video/x-raw-yuv, width=658, height=486 ! queue max-size-time=0 max-size-bytes=100000000 ! theoraenc bitrate=300 keyframe-auto=false keyframe-force=3 keyframe-freq=3 speed-level=0 drop-frames=false ! queue max-size-time=0 max-size-bytes=100000000 ! oggmux ! queue max-size-time=0 max-size-bytes=100000000 ! shout2send ip=host.com port=8000 password=hackme mount=/mountpoint.ogv
  • Streamed Keynote talks will stream the Presenter's View -- no switching between display mirrors
  • Full-screen Flash video doesn't stream, but doesn't halt the pipeline

Live-stream an Elphel 353L camera, combine with on-laptop audio capture, save a high-res copy to disk, and view live video on-screen

  • Tested on an 2.4GHz Core i3 running at ≥2.0GHz. Both CPU cores should be at ~50% continuously-smooth utilization after the first ~15 seconds; battery-only power or overheating may throttle the CPU thus leading to dropped samples.
  • Set the camera to 2592x1120 @ 18FPS. Adjust camera controls to ensure the camera achieves this framerate.
  • Kill/renice any resource-competitive applications (file indexing services, Firefox, Audacity, etc.)
  • qjackctl & → Start JACK in realtime priority
  • gst-launch -e rtspsrc location=rtsp://192.168.0.9:554 latency=100 ! queue max-size-bytes=100000000 max-size-time=0 ! rtpjpegdepay ! queue max-size-bytes=100000000 max-size-time=0 ! jpegdec ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=18/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=1296, height=560 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=halfres ! queue max-size-bytes=100000000 max-size-time=0 ! jpegenc ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mux jackaudiosrc connect=1 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=jackaudio ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=elphel_recording_`date +%F_%T`.mkv halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=648, height=280 ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=450 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0  ! oggmux name=livestream jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc  ! queue max-size-bytes=100000000 max-size-time=0 ! livestream. livestream. ! queue max-size-bytes=0 max-size-time=0 ! shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv halfres. ! videoscale add-borders=true ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! xvimagesink sync=false
  • Expect a ~50GB recording for 8hrs. of recording depending on image complexity (e.g., filming a night sky vs. a waterfall). The Matroska recording may be viewed directly in VLC (≥2.0 recommended) and may be transcoded to commonly-used formats via Miro Video Converter.

Same as above, but simultaneously broadcast in H.264

If Moment, remove 'pingTimerTick' 'from moment/rtmp_connection.cpp' to avoid stream resets
Test stream availability with this tool.
Toggle 'byte-stream=[true|false]' and '[tcpclientsink|rtmpsink]' as necessary, possibly '[flvmux|mp4mux]' if needed.
'rtmpsink' isn't yet compatible with Akamai
  • Substitute 'videotestsrc is-live=true' for 'rtspsrc ... jpegdec' and 'audiotestsrc freq=432' for 'jackaudiosrc connect=1' for testing.
  • gst-launch-0.10 -e rtspsrc location=rtsp://192.168.0.9:554 latency=100 ! queue max-size-bytes=100000000 max-size-time=0 ! rtpjpegdepay ! queue max-size-bytes=100000000 max-size-time=0 ! jpegdec ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=18/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=1296, height=560 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=halfres ! queue max-size-bytes=100000000 max-size-time=0 ! jpegenc ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mkvmux jackaudiosrc connect=1 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=jackaudio ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=vorbisaudio ! queue max-size-bytes=100000000 max-size-time=0 ! mkvmux. mkvmux. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=elphel_recording_`date +%F_%T`.mkv halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=648, height=280 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=quarterres ! queue max-size-bytes=100000000 max-size-time=0 ! x264enc sliced-threads=true cabac=true intra-refresh=false quantizer=22 rc-lookahead=15 bitrate=600 tune=zerolatency byte-stream=false ! queue max-size-bytes=100000000 max-size-time=0 ! flvmux streamable=true name=flvmux jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! queue max-size-bytes=100000000 max-size-time=0 ! faac ! queue max-size-bytes=100000000 max-size-time=0 ! flvmux. flvmux. ! queue max-size-bytes=0 max-size-time=0 ! tcpclientsink host=127.0.0.1 port=6666 quarterres. ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=600 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0  ! oggmux name=livestream jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! livestream. livestream. ! queue max-size-bytes=0 max-size-time=0 ! shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! xvimagesink sync=false

Dump Stills From the Resulting High-Res Elphel Recording Into a Folder

  • gst-launch filesrc location=recording.mkv ! matroskademux ! videorate ! image/jpeg,framerate=1/10 ! multifilesink location="OUTPUT%04d.jpg"
  • Remove any colons from the input filename to avoid BASH issues
  • Change the framerate to control the time interval between image stills and number of produced stills
  • No queue elements are necessary
  • Results in a lot of disk I/O; solid state drives would be helpful here