GST cookbook: Difference between revisions

Revision as of 10:48, 21 May 2012

In addition to being a powerful multimedia infrastructure for applications Gstreamer is also a useful tool for general manipulations of multimedia data. By invoking gst-launch from the command-line with a custom pipeline many useful processing steps are possible.

Gstreamer also usually has good support for Xiph-related formats.

Unfortunately, it can be rather difficult to figure out an appropriate pipeline without a starting point.

Here are some useful examples:

Encode a .wav to Vorbis:

gst-launch filesrc location="INPUT.wav" ! wavparse ! audioconvert ! vorbisenc ! oggmux ! filesink location="OUTPUT.ogg"

Dump a Theora video to PNGs:

gst-launch filesrc location="INPUT.ogv" ! oggdemux ! theoradec ! ffmpegcolorspace ! pngenc snapshot=false ! multifilesink location="OUTPUT%04d.png"

Transmux a MKV (containing vorbis and theora) to Ogg:

gst-launch filesrc location="INPUT.mkv" ! matroskademux name=d ! video/x-theora ! queue ! theoraparse ! oggmux name=mux ! filesink location="OUTPUT.ogv" d. ! audio/x-vorbis ! queue ! vorbisparse ! queue ! mux.

Encode a y4m to lossless Dirac in Ogg:

gst-launch filesrc location="INPUT.y4m" ! decodebin ! schroenc force-profile=vc2_main rate-control=lossless ! oggmux ! filesink location="OUTPUT.ogv"

Pull from a windows media stream, transcode to Ogg/Theora+Vorbis and send to a icecast server:

(may require purchasing Fluendo plugins for decoding the encumbered codecs)

gst-launch uridecodebin uri=mms://SOURCE.SERVER.COM/path name=d ! queue max-size-time=100000000 ! ffmpegcolorspace ! theoraenc bitrate=800 ! oggmux name=mux ! shout2send ip=YOURICECAST.SERVER.COM port=8000 password=YOURPASSWORD mount=/OUTPUTFILENAME.ogv d. ! queue max-size-time=100000000 ! audioconvert ! vorbisenc ! mux.

Capture video from a webcam, encode to an Ogg Theora file, decode and display on screen, write to a file whose name is the current date+time, and stream to an IceCast server

gst-launch-0.10 --eos-on-shutdown v4l2src ! 'video/x-raw-yuv, width=640, height=480' ! videorate ! 'video/x-raw-yuv, framerate=15/1' ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=150 ! oggmux ! tee name=ogged ! queue max-size-bytes=100000000 max-size-time=0 ! oggdemux ! theoradec ! xvimagesink sync=false force-aspect-ratio=true ogged. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=`date +%F_%T`.ogv ogged. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=YOURICECAST.SERVER.COM port=8000 password=YOURPASSWORD mount=/OUTPUTFILENAME.ogv streamname=YOURSTREAMNAME description=YOURDESCRIPTION genre=YOURGENRE url=YOURSTREAMURL ogged.

A v4l2 source + ALSA source -> Ogg Theora -> IceCast

dov4l -i [0|1] -m NTSC
gst-launch-0.10 --eos-on-shutdown v4l2src ! videoscale ! video/x-raw-yuv,width=320,height=240,framerate=30000/1001,interlaced=true ! queue max-size-bytes=100000000 max-size-time=0 ! gamma gamma=1.2 ! queue max-size-bytes=100000000 max-size-time=0 ! videobalance saturation=1.9 brightness=0.00 contrast=1.4 hue=0.06 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=400 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=mux alsasrc device=hw:1,0 latency-time=100 ! audioconvert ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=icecast-server.com password=hackme mount=/mountpoint.ogv

Note that the above pipeline will slowly lose audio/video synchronization due to hardware-level limitations in audio vs. video capture of media samples. FireWire and SDI-based capture does not have these limitations.

Two v4l2 sources combined side-by-side into rectangular video + ALSA -> Ogg Theora -> IceCast

mplayer tv:// -tv device=/dev/video0:driver=v4l2:norm=NTSC:width=320:height=240:outfmt=uyvy:input=1:buffersize=16: -ao null
mplayer tv:// -tv device=/dev/video1:driver=v4l2:norm=NTSC:width=320:height=240:outfmt=uyvy:input=1:buffersize=16: -ao null
gst-launch --eos-on-shutdown v4l2src device=/dev/video0 ! videoscale ! video/x-raw-yuv,width=320,height=240,interlaced=true ! queue max-size-bytes=100000000 max-size-time=0 ! gamma gamma=1.2 ! queue max-size-bytes=100000000 max-size-time=0 ! videobalance saturation=1.9 brightness=0.00 contrast=1.4 hue=0.06 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=15/1 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! videomixer name=mix ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=600 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=mux alsasrc device=hw:1,0 latency-time=100 ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=icecast-server.com password=hackme mount=/mountpoint.ogv v4l2src device=/dev/video1 ! videoscale ! video/x-raw-yuv,width=320,height=240,interlaced=true ! gamma gamma=1.2 ! queue max-size-bytes=100000000 max-size-time=0 ! videobalance saturation=1.9 brightness=0.00 contrast=1.4 hue=0.06 ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=5/1 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! videobox border-alpha=0 fill=green left=-320 ! mix.

Create a video with an alpha channel from a sequence of PNG files

gst-launch-0.10 multifilesrc location=images%05d.png caps="image/png,framerate=1/1,pixel-aspect-ratio=1/1" num-buffers=95 ! pngdec ! videorate ! alphacolor ! "video/x-raw-yuv,format=(fourcc)AYUV" ! matroskamux ! filesink location=images_raw.mkv

Capture from a 1680x1050 GNOME Desktop, Combine with ALSA -> Ogg Theora + Vorbis -> Icecast

gst-launch --eos-on-shutdown ximagesrc ! capsfilter caps=video/x-raw-rgb,framerate=3/1,width=1680,height=1050 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-rgb, width=1056, height=660 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=450 keyframe-auto=false keyframe-force=12 keyframe-freq=12 speed-level=0 drop-frames=false ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=mux alsasrc device=hw:0,0 ! audio/x-raw-init,rate=44100,channels=1 ! queue max-size-bytes=100000000 ! audioconvert ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=icecast-server.com password=hackme mount=/mountpoint.ogv
Icecast will need to be built from SVN (ver. 2.3.2 fails to relay data due to a bug)
Viewers will need better-than-average internet connectivity (around 1.5Mbits/sec)
The simplest way to capture audio events is to place a mic next to computer speakers

Live-stream a high-resolution Mac OSX Desktop at 1 FPS

Tested on Version 10.6.8, newer releases not compatible with osximagesrc

gst-launch --eos-on-shutdown osximagesrc ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! videoscale method=4-tap ! video/x-raw-yuv, width=960, height=600 ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=360 keyframe-auto=false keyframe-force=3 keyframe-freq=3 speed-level=0 drop-frames=false ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=host.com password=hackme mount=/mountpoint.ogv

If the screen resolution is set to mirrored 1024x640 (?), start the following then plug in the video display cable:

gst-launch --eos-on-shutdown osximagesrc ! queue max-size-time=0 max-size-bytes=100000000 ! ffmpegcolorspace ! queue max-size-time=0 max-size-bytes=100000000 ! videocrop right=256 top=32 ! queue max-size-time=0 max-size-bytes=100000000 ! videoscale ! video/x-raw-yuv, width=658, height=486 ! queue max-size-time=0 max-size-bytes=100000000 ! theoraenc bitrate=300 keyframe-auto=false keyframe-force=3 keyframe-freq=3 speed-level=0 drop-frames=false ! queue max-size-time=0 max-size-bytes=100000000 ! oggmux ! queue max-size-time=0 max-size-bytes=100000000 ! shout2send ip=host.com port=8000 password=hackme mount=/mountpoint.ogv

Streamed Keynote talks will stream the Presenter's View -- no switching between display mirrors
Full-screen Flash video doesn't stream, but doesn't halt the pipeline

Live-stream an Elphel 353L camera, combine with on-laptop audio capture, save a high-res copy to disk, and view live video on-screen

Tested on an 2.4GHz Core i3 running at ≥2.0GHz. Both CPU cores should be at ~50% continuously-smooth utilization after the first ~15 seconds; battery-only power or overheating may throttle the CPU thus leading to dropped samples.
Set the camera to 2592x1120 @ 18FPS. Adjust camera controls to ensure the camera achieves this framerate.
Kill/renice any resource-competitive applications (file indexing services, Firefox, Audacity, etc.)
qjackctl & → Start JACK in realtime priority

gst-launch -e rtspsrc location=rtsp://192.168.0.9:554 latency=100 ! queue max-size-bytes=100000000 max-size-time=0 ! rtpjpegdepay ! queue max-size-bytes=100000000 max-size-time=0 ! jpegdec ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=18/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=1296, height=560 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=halfres ! queue max-size-bytes=100000000 max-size-time=0 ! jpegenc ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mux jackaudiosrc connect=1 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=jackaudio ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=elphel_recording_`date +%F_%T`.mkv halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=648, height=280 ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=450 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=livestream jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! livestream. livestream. ! queue max-size-bytes=0 max-size-time=0 ! shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv halfres. ! videoscale add-borders=true ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! xvimagesink sync=false
Expect a ~50GB recording for 8hrs. of recording depending on image complexity (e.g., filming a night sky vs. a waterfall). The Matroska recording may be viewed directly in VLC (≥2.0 recommended) and may be transcoded to commonly-used formats via Miro Video Converter.

Same as above, but simultaneously broadcast in H.264

Experimental; use latest git checkouts
Tested with crtmpserver (works) and Moment Video Server (in theory)

If Moment, remove 'pingTimerTick' 'from moment/rtmp_connection.cpp' to avoid stream resets

Test stream availability with this tool.

Toggle 'byte-stream=[true|false]' and '[tcpclientsink|rtmpsink]' as necessary, possibly '[flvmux|mp4mux]' if needed.

'rtmpsink' isn't yet compatible with Akamai

Substitute 'videotestsrc is-live=true' for 'rtspsrc ... jpegdec' and 'audiotestsrc freq=432' for 'jackaudiosrc connect=1' for testing.

gst-launch-0.10 -e rtspsrc location=rtsp://192.168.0.9:554 latency=100 ! queue max-size-bytes=100000000 max-size-time=0 ! rtpjpegdepay ! queue max-size-bytes=100000000 max-size-time=0 ! jpegdec ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=18/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=1296, height=560 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=halfres ! queue max-size-bytes=100000000 max-size-time=0 ! jpegenc ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mkvmux jackaudiosrc connect=1 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=jackaudio ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=vorbisaudio ! queue max-size-bytes=100000000 max-size-time=0 ! mkvmux. mkvmux. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=elphel_recording_`date +%F_%T`.mkv halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=648, height=280 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=quarterres ! queue max-size-bytes=100000000 max-size-time=0 ! x264enc sliced-threads=true cabac=true intra-refresh=false quantizer=22 rc-lookahead=15 bitrate=600 tune=zerolatency byte-stream=false ! queue max-size-bytes=100000000 max-size-time=0 ! flvmux streamable=true name=flvmux jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! queue max-size-bytes=100000000 max-size-time=0 ! faac ! queue max-size-bytes=100000000 max-size-time=0 ! flvmux. flvmux. ! queue max-size-bytes=0 max-size-time=0 ! tcpclientsink host=127.0.0.1 port=6666 quarterres. ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=600 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=livestream jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! livestream. livestream. ! queue max-size-bytes=0 max-size-time=0 ! shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! xvimagesink sync=false

Dump Stills From the Resulting High-Res Elphel Recording Into a Folder

gst-launch filesrc location=recording.mkv ! matroskademux ! videorate ! image/jpeg,framerate=1/10 ! multifilesink location="OUTPUT%04d.jpg"

Remove any colons from the input filename to avoid BASH issues
Change the framerate to control the time interval between image stills and number of produced stills
No queue elements are necessary
Results in a lot of disk I/O; solid state drives would be helpful here

@@ Line 57: / Line 57: @@
 *Full-screen Flash video doesn't stream, but doesn't halt the pipeline
-===Live-stream an Elphel 353L camera, combine with on-laptop audio capture, save a high-res copy to disk, and view the video on-screen===
+===Live-stream an Elphel 353L camera, combine with on-laptop audio capture, save a high-res copy to disk, and view live video on-screen===
 *Tested on an 2.4GHz Core i3 running at ≥2.0GHz.  Both CPU cores should be at ~50% continuously-smooth utilization after the first ~15 seconds; battery-only power or overheating may throttle the CPU thus leading to dropped samples.
 *Set the camera to 2592x1120 @ 18FPS.  Adjust camera controls to ensure the camera achieves this framerate.
@@ Line 65: / Line 65: @@
 *gst-launch -e <span style="color:#8B0000">rtspsrc location=rtsp://192.168.0.9:554 latency=100 ! queue max-size-bytes=100000000 max-size-time=0 ! rtpjpegdepay ! queue max-size-bytes=100000000 max-size-time=0 ! jpegdec ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=18/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=1296, height=560 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=</span>'''halfres''' ! <span style="color:#4B0082">queue max-size-bytes=100000000 max-size-time=0 ! jpegenc ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mux</span> <span style="color:#C71585">jackaudiosrc connect=1 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=</span>'''jackaudio''' ! <span style="color:#C71585">queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0</span> ! <span style="color:#4B0082">mux.</span>{{pad|1em}}'''mux.''' ! <span style="color:#FFB6C1">queue max-size-bytes=100000000 max-size-time=0 ! filesink location=elphel_recording_`date +%F_%T`.mkv</span>{{pad|1em}}'''halfres.''' ! <span style="color:#006400">queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=648, height=280 ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=450 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0  ! oggmux name=livestream</span> '''jackaudio.''' ! <span style="color:#4682B4">queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc  ! queue max-size-bytes=100000000 max-size-time=0</span> ! <span style="color:#006400">livestream.</span>{{pad|1em}}'''livestream.''' ! <span style="color:#CD5C5C;"> queue max-size-bytes=0 max-size-time=0 !  shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv</span>{{pad|1em}}'''halfres.''' ! <span style="color:#FF8C00;">videoscale add-borders=true ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! xvimagesink sync=false</span>
 *Expect a ~50GB recording for 8hrs. of recording depending on image complexity (''e.g.'', filming a night sky vs. a waterfall).  The Matroska recording may be viewed directly in VLC (≥2.0 recommended) and may be transcoded to commonly-used formats ''via'' [http://www.mirovideoconverter.com/ Miro Video Converter].
+====Same as above, but simultaneously broadcast in H.264 ====
+*Experimental; use latest git checkouts
+*Tested with [http://www.rtmpd.com/ crtmpserver] (works) and [http://momentvideo.org/ Moment Video Server] (in theory)
+:If Moment, remove 'pingTimerTick' 'from moment/rtmp_connection.cpp' to avoid stream resets
+:Test stream availability with [http://dl.dropbox.com/u/2918563/flvplayback.swf this tool.]
+:Toggle 'byte-stream=[true|false]' and '[tcpclientsink|rtmpsink]' as necessary, possibly '[flvmux|mp4mux]' if needed.
+::'rtmpsink' isn't yet compatible with Akamai
+*Substitute 'videotestsrc is-live=true' for 'rtspsrc ... jpegdec' and 'audiotestsrc freq=432' for 'jackaudiosrc connect=1' for testing.
+*gst-launch-0.10 -e rtspsrc location=rtsp://192.168.0.9:554 latency=100 ! queue max-size-bytes=100000000 max-size-time=0 ! rtpjpegdepay ! queue max-size-bytes=100000000 max-size-time=0 ! jpegdec ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=18/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=1296, height=560 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=halfres ! queue max-size-bytes=100000000 max-size-time=0 ! jpegenc ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mkvmux jackaudiosrc connect=1 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=jackaudio ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=vorbisaudio ! queue max-size-bytes=100000000 max-size-time=0 ! mkvmux. mkvmux. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=elphel_recording_`date +%F_%T`.mkv  halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=648, height=280 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=quarterres !  queue max-size-bytes=100000000 max-size-time=0 ! x264enc sliced-threads=true cabac=true intra-refresh=false quantizer=22 rc-lookahead=15 bitrate=600 tune=zerolatency byte-stream=false ! queue max-size-bytes=100000000 max-size-time=0 ! flvmux streamable=true name=flvmux jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! queue max-size-bytes=100000000 max-size-time=0 ! faac ! queue max-size-bytes=100000000 max-size-time=0 ! flvmux.  flvmux. ! queue max-size-bytes=0 max-size-time=0 ! tcpclientsink host=127.0.0.1 port=6666  quarterres. ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=600 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0  ! oggmux name=livestream jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! livestream. livestream. ! queue max-size-bytes=0 max-size-time=0 ! shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv  halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! xvimagesink sync=false
 ===Dump Stills From the Resulting High-Res Elphel Recording Into a Folder===