GST cookbook: Difference between revisions

From XiphWiki
Jump to navigation Jump to search
(→‎v4l2 -> icecast: update for what I use, include color correction)
(45 intermediate revisions by 4 users not shown)
Line 1: Line 1:
In addition to being a powerful multimedia infrastructure for applications Gstreamer is also a useful tool for general manipulations of multimedia dataBy invoking gst-launch from the command-line with a custom pipeline many useful processing steps are possible.
GStreamer is a powerful multimedia infrastructure for applications—providing a number of programming language constructs and bindings—but also provides easy-to-use rapid prototyping tools to test new multimedia pipelinesAdditionally, Gstreamer usually has good support for Xiph-related formats.


Gstreamer also usually has good support for Xiph-related formats.  
Unfortunately, it can be rather difficult to figure out an appropriate pipeline without a starting point.  This page is a collection of gst-launch pipelines that can be used on an as-needed basis.  [http://gentrans.sourceforge.net/ GEntrans] is a command-line tool helpful in visualizing queue states (i.e., static/under/overfilled), dynamically, for a given gst-launch pipeline.  Queues can have a large impact on "even" pipeline performance, for example in avoiding intermittent frame pauses as the pipeline blocks on some other operation.


Unfortunately, it can be rather difficult to figure out an appropriate pipeline without a starting point.
Texas Instruments offers a [http://processors.wiki.ti.com/index.php/Example_GStreamer_Pipelines list of example pipelines] for TI mobile devices.
 
__TOC__
 
<center>
<div style="background-color: #ffffff; border: 1px solid #000000; color: black; font-weight: bold; margin: 2em 0 1em; padding: .5em 1em; vertical-align: middle;">Note that some examples are "high-performance" pipelines which require purpose-built systems.<br>[https://gitorious.org/openvideo_reference_build 'Open Video' Reference Build Script]</div> </center>


Here are some useful examples: <!-- Don't complain about the selection of examples, I started this simply by grepping my shell history. Feel free to submit more and/or improve the existing ones --gmaxwell -->


===Encode a .wav to Vorbis:===
===Encode a .wav to Vorbis:===
Line 19: Line 23:
*gst-launch filesrc location="INPUT.y4m" ! decodebin ! schroenc force-profile=vc2_main rate-control=lossless ! oggmux ! filesink location="OUTPUT.ogv"
*gst-launch filesrc location="INPUT.y4m" ! decodebin ! schroenc force-profile=vc2_main rate-control=lossless ! oggmux ! filesink location="OUTPUT.ogv"


===Pull from a windows media stream, transcode to Ogg/Thera+Vorbis and send to a icecast server:===
===Pull from a windows media stream, transcode to Ogg/Theora+Vorbis and send to a icecast server:===
(requires purchasing fluendo plugins for decoding the encumbered codecs)
(may require purchasing Fluendo plugins for decoding the encumbered codecs)
*gst-launch uridecodebin uri=mms://SOURCE.SERVER.COM/path name=d ! queue max-size-time=100000000 ! ffmpegcolorspace ! theoraenc bitrate=800 ! oggmux name=mux ! shout2send ip=YOURICECAST.SERVER.COM port=8000 password=YOURPASSWORD mount=/OUTPUTFILENAME.ogv d. ! queue max-size-time=100000000 ! audioconvert ! vorbisenc ! mux.
*gst-launch uridecodebin uri=mms://SOURCE.SERVER.COM/path name=d ! queue max-size-time=100000000 ! ffmpegcolorspace ! theoraenc bitrate=800 ! oggmux name=mux ! shout2send ip=YOURICECAST.SERVER.COM port=8000 password=YOURPASSWORD mount=/OUTPUTFILENAME.ogv d. ! queue max-size-time=100000000 ! audioconvert ! vorbisenc ! mux.


Line 26: Line 30:
*gst-launch-0.10 --eos-on-shutdown v4l2src ! 'video/x-raw-yuv, width=640, height=480' ! videorate ! 'video/x-raw-yuv, framerate=15/1' ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=150 ! oggmux ! tee name=ogged ! queue max-size-bytes=100000000 max-size-time=0 ! oggdemux ! theoradec ! xvimagesink sync=false force-aspect-ratio=true ogged. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=`date +%F_%T`.ogv ogged. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=YOURICECAST.SERVER.COM port=8000 password=YOURPASSWORD mount=/OUTPUTFILENAME.ogv streamname=YOURSTREAMNAME description=YOURDESCRIPTION genre=YOURGENRE url=YOURSTREAMURL ogged.
*gst-launch-0.10 --eos-on-shutdown v4l2src ! 'video/x-raw-yuv, width=640, height=480' ! videorate ! 'video/x-raw-yuv, framerate=15/1' ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=150 ! oggmux ! tee name=ogged ! queue max-size-bytes=100000000 max-size-time=0 ! oggdemux ! theoradec ! xvimagesink sync=false force-aspect-ratio=true ogged. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=`date +%F_%T`.ogv ogged. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=YOURICECAST.SERVER.COM port=8000 password=YOURPASSWORD mount=/OUTPUTFILENAME.ogv streamname=YOURSTREAMNAME description=YOURDESCRIPTION genre=YOURGENRE url=YOURSTREAMURL ogged.


===Capture video from a v4l2 device, encode to an Ogg Theora file, mux with an ALSA audio source with delayed audio for a/v sync, then stream to IceCast===
===A v4l2 source + ALSA source -> Dual Ogg Theora + Ogg Vorbis Streams -> Icecast ===
*dov4l -i [0|1] -m NTSC
*dov4l -i [0|1] -m NTSC
*gst-launch-0.10 --eos-on-shutdown v4l2src ! videoscale ! video/x-raw-yuv,width=320,height=240,framerate=30000/1001,interlaced=true ! queue max-size-bytes=100000000 max-size-time=0 ! gamma gamma=1.2 ! queue max-size-bytes=100000000 max-size-time=0 ! videobalance saturation=1.9 brightness=0.00 contrast=1.4 hue=0.06 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=400 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=mux alsasrc device=hw:1,0 latency-time=100 ! audioconvert ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=icecast-server.com password=hackme mount=/mountpoint.ogv
*gst-launch-0.10 --eos-on-shutdown v4l2src device=/dev/video0 ! queue max-size-bytes=100000000 max-size-time=0 ! deinterlace mode=interlaced ! queue max-size-bytes=100000000 max-size-time=0  ! videorate ! video/x-raw-yuv,framerate=15/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv,width=360,height=240,pixel-aspect-ratio=1/1 ! queue max-size-bytes=100000000 max-size-time=0 ! gamma gamma=1.2 ! queue max-size-bytes=100000000 max-size-time=0 ! videobalance saturation=1.9 brightness=0.00 contrast=1.4 hue=0.06 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=400 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=mux alsasrc device=hw:0,0 latency-time=100 ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! queue max-size-bytes=100000000 max-size-time=0 ! audioamplify amplification=[1-6 or so] ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=vorbisaudio ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=server.com port=[80|8000] password=hackme mount=/mountpoint.ogv  vorbisaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=server.com port=[80|8000] password=hackme mount=/mountpoint.oga
 
:Note that the above pipeline will slowly lose audio/video synchronization due to hardware-level limitations in audio vs. video capture of media samples.  FireWire and SDI-based capture does not have these limitations.
:Note that the above pipeline will slowly lose audio/video synchronization due to hardware-level limitations in audio vs. video capture of media samples.  FireWire and SDI-based capture does not have these limitations.
===Two v4l2 sources combined side-by-side into rectangular video + ALSA -> Ogg Theora -> IceCast===
*mplayer tv:// -tv device=/dev/video0:driver=v4l2:norm=NTSC:width=320:height=240:outfmt=uyvy:input=1:buffersize=16: -ao null
*mplayer tv:// -tv device=/dev/video1:driver=v4l2:norm=NTSC:width=320:height=240:outfmt=uyvy:input=1:buffersize=16: -ao null
*gst-launch --eos-on-shutdown v4l2src device=/dev/video0 ! videoscale ! video/x-raw-yuv,width=320,height=240,interlaced=true ! queue max-size-bytes=100000000 max-size-time=0 ! gamma gamma=1.2 ! queue max-size-bytes=100000000 max-size-time=0 ! videobalance saturation=1.9 brightness=0.00 contrast=1.4 hue=0.06 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=15/1 ! queue max-size-bytes=100000000 max-size-time=0  ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0  ! videomixer name=mix ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=600 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=mux alsasrc device=hw:1,0 latency-time=100 ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0  ! shout2send ip=icecast-server.com password=hackme mount=/mountpoint.ogv  v4l2src device=/dev/video1 ! videoscale ! video/x-raw-yuv,width=320,height=240,interlaced=true ! gamma gamma=1.2 ! queue max-size-bytes=100000000 max-size-time=0 ! videobalance saturation=1.9 brightness=0.00 contrast=1.4 hue=0.06 !  queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=5/1 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0  ! videobox border-alpha=0 fill=green left=-320 ! mix.


===Create a video with an alpha channel from a sequence of PNG files===
===Create a video with an alpha channel from a sequence of PNG files===
*gst-launch-0.10 multifilesrc location=images%05d.png caps="image/png,framerate=1/1,pixel-aspect-ratio=1/1" num-buffers=95 ! pngdec ! videorate ! alphacolor ! "video/x-raw-yuv,format=(fourcc)AYUV" ! matroskamux ! filesink location=images_raw.mkv
*gst-launch-0.10 multifilesrc location=images%05d.png caps="image/png,framerate=1/1,pixel-aspect-ratio=1/1" num-buffers=95 ! pngdec ! videorate ! alphacolor ! "video/x-raw-yuv,format=(fourcc)AYUV" ! matroskamux ! filesink location=images_raw.mkv
===Capture from a 1680x1050 GNOME Desktop, Combine with ALSA -> Ogg Theora + Vorbis -> Icecast===
*gst-launch  --eos-on-shutdown ximagesrc ! capsfilter caps=video/x-raw-rgb,framerate=3/1,width=1680,height=1050  ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-rgb, width=1056, height=660 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=450 keyframe-auto=false keyframe-force=12 keyframe-freq=12 speed-level=0 drop-frames=false ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=mux alsasrc device=hw:0,0 ! audio/x-raw-init,rate=44100,channels=1 ! queue max-size-bytes=100000000 ! audioconvert ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=icecast-server.com  password=hackme mount=/mountpoint.ogv
*Icecast will need to be built from SVN (ver. 2.3.2 fails to relay data due to a bug)
*Viewers will need better-than-average internet connectivity (around 1.5Mbits/sec)
*The simplest way to capture audio events is to place a mic next to computer speakers
===Live-stream a high-resolution Mac OSX Desktop at 1 FPS===
*Tested on Version 10.6.8, newer releases not compatible with [https://github.com/davibe/osximagesrc osximagesrc]
gst-launch --eos-on-shutdown osximagesrc ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! videoscale method=4-tap ! video/x-raw-yuv, width=960, height=600 ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=360 keyframe-auto=false keyframe-force=3 keyframe-freq=3 speed-level=0 drop-frames=false ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=host.com password=hackme mount=/mountpoint.ogv
If the screen resolution is set to mirrored 1024x640 (?), start the following ''then'' plug in the video display cable:
*gst-launch --eos-on-shutdown osximagesrc ! queue max-size-time=0 max-size-bytes=100000000 ! ffmpegcolorspace ! queue max-size-time=0 max-size-bytes=100000000 ! videocrop right=256 top=32 ! queue max-size-time=0 max-size-bytes=100000000 ! videoscale ! video/x-raw-yuv, width=658, height=486 ! queue max-size-time=0 max-size-bytes=100000000 ! theoraenc bitrate=300 keyframe-auto=false keyframe-force=3 keyframe-freq=3 speed-level=0 drop-frames=false ! queue max-size-time=0 max-size-bytes=100000000 ! oggmux ! queue  max-size-time=0 max-size-bytes=100000000 ! shout2send ip=host.com port=8000 password=hackme mount=/mountpoint.ogv
*Streamed Keynote talks will stream the Presenter's View -- no switching between display mirrors
*Full-screen Flash video doesn't stream, but doesn't halt the pipeline
===Live-stream an Elphel 353L camera, combine with on-laptop audio capture, save a high-res copy to disk, and view live audio+video on-screen===
*Tested on an 2.4GHz Core i3 running at ≥2.0GHz.  Both CPU cores should be at ~50% continuously-smooth utilization after the first ~15 seconds; battery-only power or overheating may throttle the CPU leading to dropped frames.
*Set the camera to 2592x1120 @ 18FPS.  Adjust camera controls to ensure the camera achieves this framerate.
*Kill/renice any resource-competitive applications (file indexing services, Firefox, Audacity, etc.)
*qjackctl &  → Start JACK in realtime priority
<br />
*gst-launch-'''0.10''' -e <span style="color:#8B0000">rtspsrc location=rtsp://192.168.0.9:554 latency=100 ! queue max-size-bytes=100000000 max-size-time=0 ! rtpjpegdepay ! queue max-size-bytes=100000000 max-size-time=0 ! jpegdec ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=18/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=1296, height=560 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=</span>'''halfres''' ! <span style="color:#4B0082">queue max-size-bytes=100000000 max-size-time=0 ! jpegenc ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mux</span> <span style="color:#C71585">jackaudiosrc connect=1 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=</span>'''jackaudio''' ! <span style="color:#C71585">queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0</span> ! <span style="color:#4B0082">mux.</span>{{pad|1em}}'''mux.''' ! <span style="color:#FFB6C1">queue max-size-bytes=100000000 max-size-time=0 ! filesink location=elphel_recording_`date +%F_%T`.mkv</span>{{pad|1em}}'''halfres.''' ! <span style="color:#006400">queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=648, height=280 ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=450 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0  ! oggmux name=livestream</span> '''jackaudio.''' ! <span style="color:#4682B4">queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc  ! queue max-size-bytes=100000000 max-size-time=0</span> ! <span style="color:#006400">livestream.</span>{{pad|1em}}'''livestream.''' ! <span style="color:#CD5C5C;"> queue max-size-bytes=0 max-size-time=0 !  shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv</span>{{pad|1em}}'''halfres.''' ! <span style="color:#FF8C00;">videoscale add-borders=true ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! xvimagesink sync=false</span> jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! queue max-size-bytes=100000000 max-size-time=0 ! alsasink
<br />
*gst-launch-'''1.0''' -e  rtspsrc location=rtsp://192.168.0.9:554 latency=100 ! queue max-size-bytes=100000000 max-size-time=0 ! rtpjpegdepay ! queue max-size-bytes=100000000 max-size-time=0 ! jpegdec ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw,framerate=18/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw, width=1296, height=560 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=halfres ! queue max-size-bytes=100000000 max-size-time=0 ! jpegenc ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mux jackaudiosrc connect=1 client-name="GStreamer Input" ! queue max-size-bytes=100000000 max-size-time=0 !  tee name=jackaudio ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=vorbisaudio !  queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=elphel_recording_`date +%s`.mkv sync=false halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw,width=648, height=280 ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=500 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0  ! oggmux name=livestream vorbisaudio. !  queue max-size-bytes=100000000 max-size-time=0 ! livestream. livestream. ! queue max-size-bytes=0 max-size-time=0 ! shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv  halfres. ! videoscale add-borders=true ! queue max-size-bytes=100000000 max-size-time=0 ! videoconvert ! queue max-size-bytes=100000000 max-size-time=0 ! xvimagesink sync=false  jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! queue max-size-bytes=100000000 max-size-time=0 ! alsasink
<br />
*Expect a ~50GB recording for 8hrs. of recording depending on image complexity.  The Matroska recording may be viewed directly in VLC (≥2.0 recommended) and may be transcoded to commonly-used formats ''via'' [http://www.mirovideoconverter.com/ Miro Video Converter] or [http://firefogg.org/ Firefogg].
:Alternatively: ffmpeg -i elphel_recording.mkv -acodec libfaac -ab 128k -pass 1 -vcodec libx264 -vpre slow -vpre main -b 8000k -threads 8 -f mp4 -y elphel_recording.mp4
====Same as above, but simultaneously broadcast in H.264 ====
Tested with [http://www.rtmpd.com/ crtmpserver] (works) and [https://gitorious.org/moment-archive Moment Video Server] (in theory)
For crtmpserver:
*Remove a 'tags_written++' line from './gst/flv/gstflvmux.c' as per [https://bugzilla.gnome.org/show_bug.cgi?id=661624 Bug 661624]
*crtmpserver.lua should be customized for the streaming application (a mostly undocumented step...)
For Moment:
*Remove 'pingTimerTick' 'from moment/rtmp_connection.cpp' to avoid stream resets
For both:
*Toggle 'byte-stream=[true|false]' and '[tcpclientsink|rtmpsink]' as necessary, possibly '[flvmux|mp4mux]' if needed.
*Test stream availability with [http://dl.dropbox.com/u/2918563/flvplayback.swf this tool.]
*Substitute 'videotestsrc is-live=true' for 'rtspsrc ... jpegdec' and 'audiotestsrc freq=432' for 'jackaudiosrc connect=1' for testing.
*'rtmpsink' isn't yet(?) compatible with Akamai
*gst-launch-0.10 -e rtspsrc location=rtsp://192.168.0.9:554 latency=100 ! queue max-size-bytes=100000000 max-size-time=0 ! rtpjpegdepay ! queue max-size-bytes=100000000 max-size-time=0 ! jpegdec ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=18/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=1296, height=560 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=halfres ! queue max-size-bytes=100000000 max-size-time=0 ! jpegenc ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mkvmux jackaudiosrc connect=1 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=jackaudio ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=vorbisaudio ! queue max-size-bytes=100000000 max-size-time=0 ! mkvmux. mkvmux. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=elphel_recording_`date +%F_%T`.mkv  halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=648, height=280 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=quarterres !  queue max-size-bytes=100000000 max-size-time=0 ! x264enc sliced-threads=true cabac=true intra-refresh=false quantizer=22 rc-lookahead=15 bitrate=600 tune=zerolatency byte-stream=false ! queue max-size-bytes=100000000 max-size-time=0 ! flvmux streamable=true name=flvmux jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! queue max-size-bytes=100000000 max-size-time=0 ! faac ! queue max-size-bytes=100000000 max-size-time=0 ! flvmux.  flvmux. ! queue max-size-bytes=0 max-size-time=0 ! tcpclientsink host=127.0.0.1 port=6666  quarterres. ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=600 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0  ! oggmux name=livestream jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! livestream. livestream. ! queue max-size-bytes=0 max-size-time=0 ! shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv  halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! xvimagesink sync=false
===<div id=ElphelJP46>Live-stream an Elphel 353L camera set at 'FullHD' JP46, combine with realtime audio, save a high-res copy to disk, and monitor live audio+video on-screen</div>===
*[https://media.xiph.org/jp46_cookbook/ Video samples]
*Setup a high-performance system: tested on Gigabyte GA-MA785GMT-UD2H + (non-overclocked) Phenom II X4 [[GST_cookbook/gNewSense|built using gNewSense.]]  CPU utilization approaches 100%, continuously, across all four 3.2GHz cores.
*Set the camera to 1920x1088, JPEG Image Quality @ 90%, JP46 color mode
In separate tabs:
*jackd -R -d alsa -C -d hw:2 -r 48000 -i 1
*gst-launch-0.10 -e rtspsrc location=rtsp://192.168.1.50:554 protocols=0x00000001 latency=100 ! rtpjpegdepay ! queue max-size-bytes=1000000000 max-size-time=0 ! videorate force-fps=18 ! queue max-size-bytes=1000000000 max-size-time=0 ! jpegdec max-errors=-1 idct-method=2 ! queue max-size-bytes=1000000000 max-size-time=0 ! jp462bayer threads=4 ! "video/x-raw-bayer, width=(int)1920, height=(int)1088, format=(string)grbg" ! queue max-size-bytes=1000000000 max-size-time=0 ! bayer2rgb2 method=0 ! queue max-size-bytes=1000000000 max-size-time=0 ! ffmpegcolorspace ! "video/x-raw-yuv, format=(fourcc)I420" ! queue max-size-bytes=1000000000 max-size-time=0 ! videobalance saturation=1.9 ! queue max-size-bytes=1000000000 max-size-time=0 ! tee name=fullHD ! queue max-size-bytes=100000000 max-size-time=0 ! jpegenc idct-method=2 ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mux  jackaudiosrc connect=1 ! queue max-size-time=0 ! audio/x-raw-float,channels=1 ! queue max-size-time=0 ! tee name=jackaudio ! queue max-size-time=0 ! vorbisenc max-bitrate=80000 ! queue max-size-time=0 ! tee name=vorbisaudio ! queue max-size-time=0 ! mux.  mux. ! queue max-size-bytes=1000000000 max-size-time=0 ! filesink location=elphel_recording_`date +%s`.mkv sync=false  fullHD. ! queue max-size-bytes=1000000000 max-size-time=0 ! videoscale ! "video/x-raw-yuv, width=640, height=363, aspect-ratio=(fraction)1/1" ! queue max-size-bytes=1000000000 max-size-time=0 ! tee name=downsample ! queue max-size-bytes=1000000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=1000000000 max-size-time=0 ! ximagesink pixel-aspect-ratio=1/1 sync=false downsample. ! queue max-size-bytes=100000000 max-size-time=0 ! queue max-size-bytes=1000000000 max-size-time=0 ! theoraenc bitrate=550 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=livestream  vorbisaudio. ! queue max-size-time=0 ! livestream.  livestream. ! queue max-size-bytes=0 max-size-time=0 ! shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv  jackaudio. ! queue max-size-time=0 ! audioconvert ! queue max-size-time=0 ! alsasink
===Select an Elphel 353L camera as the video source, widescreen, in Google[+] Video Chat/Hangouts (Linux only)===
*Requires a [https://www.google.com/tools/dlpage/hangoutplugin/download.html proprietary Google plugin]
*Also requires the [https://github.com/umlaeute/v4l2loopback v4l2loopback kernel module] loaded (w/o parameters).  Note an unresolved [https://github.com/umlaeute/v4l2loopback/issues/60 v4l2loopback bug] may trigger GStreamer "Internal data flow error" pipeline failures.
*Set the camera to 2592x1120 @ 18FPS.  Adjust camera controls to ensure the camera achieves this framerate.
*gst-launch-1.0 -e rtspsrc location=rtsp://192.168.0.9:554 latency=100 ! queue max-size-bytes=100000000 max-size-time=0 ! rtpjpegdepay !  queue max-size-bytes=100000000 max-size-time=0 ! jpegdec ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! "video/x-raw,framerate=(fraction)30/1" ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! "video/x-raw,width=640,height=480" ! queue max-size-bytes=100000000 max-size-time=0 ! videoconvert ! "video/x-raw,format=(string)YUY2,pixel-aspect-ratio=(fraction)1/1" ! v4l2sink  device=/dev/video0
*Launch Google Talk/Google+ Hangout and select 'Loopback video device 0' as the video input source.  Syncroniced audio ''via'' the v4l2 API is untested -- select an ALSA audio source as appropriate.
*Re-nice 'GoogleTalkPlugin' and both 'plugin-container' processes to '-15'.  Tested on  Abrowser 26.0 + Trisquel + linux-libre-3.12 + google-talkplugin ver. 4.9.1.0-1.
===Debayer + convert to RGB a single JP46 frame using Elphel 353L===
*Use the following to extracting the highest-quality image possible (save JP4):
*Set the Elphel camera to operate in JP46 mode using full-frame imaging (''i.e.'', 2592x1936)
*"Shift+Click" on the "view image" icon in the Elphel Camera Control Interface to save an image still
*gst-launch-0.10 filesrc location="/path/to/elphelimg_nnnnnnn.jp46"  ! queue max-size-bytes=1000000000 ! jpegdec idct-method=2  ! queue max-size-bytes=1000000000 !  jp462bayer ! "video/x-raw-bayer, width=(int)2592, height=(int)1936, format=(string)grbg" ! queue max-size-bytes=1000000000 ! bayer2rgb2 method=4 ! queue max-size-bytes=1000000000 ! pngenc compression-level=9 ! queue max-size-bytes=1000000000 ! filesink location="/path/to/outputimage.png"
*Open with GIMP
===Extract a series of individual JPEGs from Matroska+M-JPEG into a folder===
*GST_DEBUG=GST_BUS:5 gst-launch-0.10  filesrc location=recording.mkv ! queue ! matroskademux ! queue ! videorate ! image/jpeg,framerate=1/10 ! queue ! multifilesink post-messages=true location="image%04d.jpg" sync=false 2>gst-bus-debug.log
*Remove any colons input filenames to avoid BASH issues
*Change the framerate to control the time interval between image stills and number of produced stills
<br />
*After the above command is run the images should be generated.  To add EXIF timestamps to each JPG:
*cat gst-bus-debug.log | grep timestamp | grep dispatch | sed -e s/^.*filename[=].string.//g  -e s/..stream.time.*//g -e s/[,].*\)/=/g -e s/000000000//g >timestamps.log
*[TODO: create a BASH script or similar for the next step]
*exiftool -DateTimeOriginal="`date -d @start-of-recording_unixtimestamp+offset-for-this-image-from-log-file --rfc-3339=seconds`" -overwrite_original *.jpg
====Run color-correction on the extracted JPEGs with G'MIC====
*for i in *jpg; do mv $i tmp.jpg ; gmic tmp.jpg --apply_curve 0,1,1,10,1,116,172,183,204,255,255 -o[1] $i,90; done
<br />
*for i in *jpg; do mv $i tmp.jpg ; gmic tmp.jpg --balance_gamma 68,68,68 -o[1] $i,90; done
*The specified reference color must be in RGB, not hex.  Use the gimp-gmic plugin to interactively find values that work well for your photoset.
===Add EXIF Metadata to generated image stills===
*This command will ''replace'' images in a directory with the same image, metadata included - be sure to have backup copies.
*"-flash#=0x0020" indicates that the camera has no flash capability.
*Generally compatible with [http://mediagoblin.org/ MediaGoblin]
*exiftool  -Author='Your name' -Copyright='copyright license + URL' -make='Elphel Inc.' -model='Elphel NC353L-369-12V'  -Xresolution=72 -Yresolution=72  -resolutionunit=inches -flash#=0x0020 -DateTimeOriginal='2012:05:04 10:00:00-5' -exposuremode='Manual' -FNumber='1.4' -UserComment='Lens or adapter used, or other notes' -GPSLatitude=42.360857 -GPSLatitudeRef=N -GPSLongitude=71.09122 -GPSLongitudeRef=W  -overwrite_original *.jpg
===Mix Three Video Sources Side-by-Side-by-Side===
*gst-launch --eos-on-shutdown videotestsrc pattern=6 is-live=true ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv,width=320,height=240 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=15/1 ! queue max-size-bytes=100000000 max-size-time=0 ! clockoverlay halignment=1 valignment=1 shaded-background=true font-desc="Sans Bold 80" ypad=80 ! queue max-size-bytes=100000000 max-size-time=0 ! videobox border-alpha=0 fill=green left=-320 ! queue max-size-bytes=100000000 max-size-time=0 ! videomixer name=mix ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! ximagesink sync=false videotestsrc is-live=true ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv,width=320,height=240 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=15/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videobox border-alpha=0 fill=blue left=-640 ! queue max-size-bytes=100000000 max-size-time=0 !  mix.  videotestsrc pattern=11 ! "video/x-raw-yuv, width=352, height=240" ! queue max-size-bytes=100000000 max-size-time=0 ! mix.

Revision as of 20:44, 27 October 2014

GStreamer is a powerful multimedia infrastructure for applications—providing a number of programming language constructs and bindings—but also provides easy-to-use rapid prototyping tools to test new multimedia pipelines. Additionally, Gstreamer usually has good support for Xiph-related formats.

Unfortunately, it can be rather difficult to figure out an appropriate pipeline without a starting point. This page is a collection of gst-launch pipelines that can be used on an as-needed basis. GEntrans is a command-line tool helpful in visualizing queue states (i.e., static/under/overfilled), dynamically, for a given gst-launch pipeline. Queues can have a large impact on "even" pipeline performance, for example in avoiding intermittent frame pauses as the pipeline blocks on some other operation.

Texas Instruments offers a list of example pipelines for TI mobile devices.

Note that some examples are "high-performance" pipelines which require purpose-built systems.
'Open Video' Reference Build Script


Encode a .wav to Vorbis:

  • gst-launch filesrc location="INPUT.wav"  ! wavparse ! audioconvert ! vorbisenc ! oggmux ! filesink location="OUTPUT.ogg"

Dump a Theora video to PNGs:

  • gst-launch filesrc location="INPUT.ogv" ! oggdemux ! theoradec ! ffmpegcolorspace ! pngenc snapshot=false ! multifilesink location="OUTPUT%04d.png"

Transmux a MKV (containing vorbis and theora) to Ogg:

  • gst-launch filesrc location="INPUT.mkv" ! matroskademux name=d ! video/x-theora ! queue ! theoraparse ! oggmux name=mux ! filesink location="OUTPUT.ogv" d. ! audio/x-vorbis ! queue ! vorbisparse ! queue ! mux.

Encode a y4m to lossless Dirac in Ogg:

  • gst-launch filesrc location="INPUT.y4m" ! decodebin ! schroenc force-profile=vc2_main rate-control=lossless ! oggmux ! filesink location="OUTPUT.ogv"

Pull from a windows media stream, transcode to Ogg/Theora+Vorbis and send to a icecast server:

(may require purchasing Fluendo plugins for decoding the encumbered codecs)

  • gst-launch uridecodebin uri=mms://SOURCE.SERVER.COM/path name=d ! queue max-size-time=100000000 ! ffmpegcolorspace ! theoraenc bitrate=800 ! oggmux name=mux ! shout2send ip=YOURICECAST.SERVER.COM port=8000 password=YOURPASSWORD mount=/OUTPUTFILENAME.ogv d. ! queue max-size-time=100000000 ! audioconvert ! vorbisenc ! mux.

Capture video from a webcam, encode to an Ogg Theora file, decode and display on screen, write to a file whose name is the current date+time, and stream to an IceCast server

  • gst-launch-0.10 --eos-on-shutdown v4l2src ! 'video/x-raw-yuv, width=640, height=480' ! videorate ! 'video/x-raw-yuv, framerate=15/1' ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=150 ! oggmux ! tee name=ogged ! queue max-size-bytes=100000000 max-size-time=0 ! oggdemux ! theoradec ! xvimagesink sync=false force-aspect-ratio=true ogged. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=`date +%F_%T`.ogv ogged. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=YOURICECAST.SERVER.COM port=8000 password=YOURPASSWORD mount=/OUTPUTFILENAME.ogv streamname=YOURSTREAMNAME description=YOURDESCRIPTION genre=YOURGENRE url=YOURSTREAMURL ogged.

A v4l2 source + ALSA source -> Dual Ogg Theora + Ogg Vorbis Streams -> Icecast

  • dov4l -i [0|1] -m NTSC
  • gst-launch-0.10 --eos-on-shutdown v4l2src device=/dev/video0 ! queue max-size-bytes=100000000 max-size-time=0 ! deinterlace mode=interlaced ! queue max-size-bytes=100000000 max-size-time=0  ! videorate ! video/x-raw-yuv,framerate=15/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv,width=360,height=240,pixel-aspect-ratio=1/1 ! queue max-size-bytes=100000000 max-size-time=0 ! gamma gamma=1.2 ! queue max-size-bytes=100000000 max-size-time=0 ! videobalance saturation=1.9 brightness=0.00 contrast=1.4 hue=0.06 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=400 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=mux alsasrc device=hw:0,0 latency-time=100 ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! queue max-size-bytes=100000000 max-size-time=0 ! audioamplify amplification=[1-6 or so] ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=vorbisaudio ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=server.com port=[80|8000] password=hackme mount=/mountpoint.ogv vorbisaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=server.com port=[80|8000] password=hackme mount=/mountpoint.oga
Note that the above pipeline will slowly lose audio/video synchronization due to hardware-level limitations in audio vs. video capture of media samples. FireWire and SDI-based capture does not have these limitations.

Two v4l2 sources combined side-by-side into rectangular video + ALSA -> Ogg Theora -> IceCast

  • mplayer tv:// -tv device=/dev/video0:driver=v4l2:norm=NTSC:width=320:height=240:outfmt=uyvy:input=1:buffersize=16: -ao null
  • mplayer tv:// -tv device=/dev/video1:driver=v4l2:norm=NTSC:width=320:height=240:outfmt=uyvy:input=1:buffersize=16: -ao null
  • gst-launch --eos-on-shutdown v4l2src device=/dev/video0 ! videoscale ! video/x-raw-yuv,width=320,height=240,interlaced=true ! queue max-size-bytes=100000000 max-size-time=0 ! gamma gamma=1.2 ! queue max-size-bytes=100000000 max-size-time=0 ! videobalance saturation=1.9 brightness=0.00 contrast=1.4 hue=0.06 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=15/1 ! queue max-size-bytes=100000000 max-size-time=0  ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0  ! videomixer name=mix ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=600 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=mux alsasrc device=hw:1,0 latency-time=100 ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0  ! shout2send ip=icecast-server.com password=hackme mount=/mountpoint.ogv v4l2src device=/dev/video1 ! videoscale ! video/x-raw-yuv,width=320,height=240,interlaced=true ! gamma gamma=1.2 ! queue max-size-bytes=100000000 max-size-time=0 ! videobalance saturation=1.9 brightness=0.00 contrast=1.4 hue=0.06 ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=5/1 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0  ! videobox border-alpha=0 fill=green left=-320 ! mix.

Create a video with an alpha channel from a sequence of PNG files

  • gst-launch-0.10 multifilesrc location=images%05d.png caps="image/png,framerate=1/1,pixel-aspect-ratio=1/1" num-buffers=95 ! pngdec ! videorate ! alphacolor ! "video/x-raw-yuv,format=(fourcc)AYUV" ! matroskamux ! filesink location=images_raw.mkv

Capture from a 1680x1050 GNOME Desktop, Combine with ALSA -> Ogg Theora + Vorbis -> Icecast

  • gst-launch --eos-on-shutdown ximagesrc ! capsfilter caps=video/x-raw-rgb,framerate=3/1,width=1680,height=1050  ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-rgb, width=1056, height=660 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=450 keyframe-auto=false keyframe-force=12 keyframe-freq=12 speed-level=0 drop-frames=false ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=mux alsasrc device=hw:0,0 ! audio/x-raw-init,rate=44100,channels=1 ! queue max-size-bytes=100000000 ! audioconvert ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=icecast-server.com password=hackme mount=/mountpoint.ogv
  • Icecast will need to be built from SVN (ver. 2.3.2 fails to relay data due to a bug)
  • Viewers will need better-than-average internet connectivity (around 1.5Mbits/sec)
  • The simplest way to capture audio events is to place a mic next to computer speakers


Live-stream a high-resolution Mac OSX Desktop at 1 FPS

  • Tested on Version 10.6.8, newer releases not compatible with osximagesrc

gst-launch --eos-on-shutdown osximagesrc ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! videoscale method=4-tap ! video/x-raw-yuv, width=960, height=600 ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=360 keyframe-auto=false keyframe-force=3 keyframe-freq=3 speed-level=0 drop-frames=false ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=host.com password=hackme mount=/mountpoint.ogv

If the screen resolution is set to mirrored 1024x640 (?), start the following then plug in the video display cable:

  • gst-launch --eos-on-shutdown osximagesrc ! queue max-size-time=0 max-size-bytes=100000000 ! ffmpegcolorspace ! queue max-size-time=0 max-size-bytes=100000000 ! videocrop right=256 top=32 ! queue max-size-time=0 max-size-bytes=100000000 ! videoscale ! video/x-raw-yuv, width=658, height=486 ! queue max-size-time=0 max-size-bytes=100000000 ! theoraenc bitrate=300 keyframe-auto=false keyframe-force=3 keyframe-freq=3 speed-level=0 drop-frames=false ! queue max-size-time=0 max-size-bytes=100000000 ! oggmux ! queue max-size-time=0 max-size-bytes=100000000 ! shout2send ip=host.com port=8000 password=hackme mount=/mountpoint.ogv
  • Streamed Keynote talks will stream the Presenter's View -- no switching between display mirrors
  • Full-screen Flash video doesn't stream, but doesn't halt the pipeline

Live-stream an Elphel 353L camera, combine with on-laptop audio capture, save a high-res copy to disk, and view live audio+video on-screen

  • Tested on an 2.4GHz Core i3 running at ≥2.0GHz. Both CPU cores should be at ~50% continuously-smooth utilization after the first ~15 seconds; battery-only power or overheating may throttle the CPU leading to dropped frames.
  • Set the camera to 2592x1120 @ 18FPS. Adjust camera controls to ensure the camera achieves this framerate.
  • Kill/renice any resource-competitive applications (file indexing services, Firefox, Audacity, etc.)
  • qjackctl & → Start JACK in realtime priority


  • gst-launch-0.10 -e rtspsrc location=rtsp://192.168.0.9:554 latency=100 ! queue max-size-bytes=100000000 max-size-time=0 ! rtpjpegdepay ! queue max-size-bytes=100000000 max-size-time=0 ! jpegdec ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=18/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=1296, height=560 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=halfres ! queue max-size-bytes=100000000 max-size-time=0 ! jpegenc ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mux jackaudiosrc connect=1 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=jackaudio ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=elphel_recording_`date +%F_%T`.mkv halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=648, height=280 ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=450 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0  ! oggmux name=livestream jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc  ! queue max-size-bytes=100000000 max-size-time=0 ! livestream. livestream. ! queue max-size-bytes=0 max-size-time=0 ! shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv halfres. ! videoscale add-borders=true ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! xvimagesink sync=false jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! queue max-size-bytes=100000000 max-size-time=0 ! alsasink


  • gst-launch-1.0 -e rtspsrc location=rtsp://192.168.0.9:554 latency=100 ! queue max-size-bytes=100000000 max-size-time=0 ! rtpjpegdepay ! queue max-size-bytes=100000000 max-size-time=0 ! jpegdec ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw,framerate=18/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw, width=1296, height=560 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=halfres ! queue max-size-bytes=100000000 max-size-time=0 ! jpegenc ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mux jackaudiosrc connect=1 client-name="GStreamer Input" ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=jackaudio ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=vorbisaudio ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=elphel_recording_`date +%s`.mkv sync=false halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw,width=648, height=280 ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=500 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0  ! oggmux name=livestream vorbisaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! livestream. livestream. ! queue max-size-bytes=0 max-size-time=0 ! shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv halfres. ! videoscale add-borders=true ! queue max-size-bytes=100000000 max-size-time=0 ! videoconvert ! queue max-size-bytes=100000000 max-size-time=0 ! xvimagesink sync=false jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! queue max-size-bytes=100000000 max-size-time=0 ! alsasink


  • Expect a ~50GB recording for 8hrs. of recording depending on image complexity. The Matroska recording may be viewed directly in VLC (≥2.0 recommended) and may be transcoded to commonly-used formats via Miro Video Converter or Firefogg.
Alternatively: ffmpeg -i elphel_recording.mkv -acodec libfaac -ab 128k -pass 1 -vcodec libx264 -vpre slow -vpre main -b 8000k -threads 8 -f mp4 -y elphel_recording.mp4

Same as above, but simultaneously broadcast in H.264

Tested with crtmpserver (works) and Moment Video Server (in theory)

For crtmpserver:

  • Remove a 'tags_written++' line from './gst/flv/gstflvmux.c' as per Bug 661624
  • crtmpserver.lua should be customized for the streaming application (a mostly undocumented step...)

For Moment:

  • Remove 'pingTimerTick' 'from moment/rtmp_connection.cpp' to avoid stream resets

For both:

  • Toggle 'byte-stream=[true|false]' and '[tcpclientsink|rtmpsink]' as necessary, possibly '[flvmux|mp4mux]' if needed.
  • Test stream availability with this tool.
  • Substitute 'videotestsrc is-live=true' for 'rtspsrc ... jpegdec' and 'audiotestsrc freq=432' for 'jackaudiosrc connect=1' for testing.
  • 'rtmpsink' isn't yet(?) compatible with Akamai
  • gst-launch-0.10 -e rtspsrc location=rtsp://192.168.0.9:554 latency=100 ! queue max-size-bytes=100000000 max-size-time=0 ! rtpjpegdepay ! queue max-size-bytes=100000000 max-size-time=0 ! jpegdec ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=18/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=1296, height=560 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=halfres ! queue max-size-bytes=100000000 max-size-time=0 ! jpegenc ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mkvmux jackaudiosrc connect=1 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=jackaudio ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=vorbisaudio ! queue max-size-bytes=100000000 max-size-time=0 ! mkvmux. mkvmux. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=elphel_recording_`date +%F_%T`.mkv halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=648, height=280 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=quarterres ! queue max-size-bytes=100000000 max-size-time=0 ! x264enc sliced-threads=true cabac=true intra-refresh=false quantizer=22 rc-lookahead=15 bitrate=600 tune=zerolatency byte-stream=false ! queue max-size-bytes=100000000 max-size-time=0 ! flvmux streamable=true name=flvmux jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! queue max-size-bytes=100000000 max-size-time=0 ! faac ! queue max-size-bytes=100000000 max-size-time=0 ! flvmux. flvmux. ! queue max-size-bytes=0 max-size-time=0 ! tcpclientsink host=127.0.0.1 port=6666 quarterres. ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=600 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0  ! oggmux name=livestream jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! livestream. livestream. ! queue max-size-bytes=0 max-size-time=0 ! shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! xvimagesink sync=false

Live-stream an Elphel 353L camera set at 'FullHD' JP46, combine with realtime audio, save a high-res copy to disk, and monitor live audio+video on-screen

  • Video samples
  • Setup a high-performance system: tested on Gigabyte GA-MA785GMT-UD2H + (non-overclocked) Phenom II X4 built using gNewSense. CPU utilization approaches 100%, continuously, across all four 3.2GHz cores.
  • Set the camera to 1920x1088, JPEG Image Quality @ 90%, JP46 color mode

In separate tabs:

  • jackd -R -d alsa -C -d hw:2 -r 48000 -i 1
  • gst-launch-0.10 -e rtspsrc location=rtsp://192.168.1.50:554 protocols=0x00000001 latency=100 ! rtpjpegdepay ! queue max-size-bytes=1000000000 max-size-time=0 ! videorate force-fps=18 ! queue max-size-bytes=1000000000 max-size-time=0 ! jpegdec max-errors=-1 idct-method=2 ! queue max-size-bytes=1000000000 max-size-time=0 ! jp462bayer threads=4 ! "video/x-raw-bayer, width=(int)1920, height=(int)1088, format=(string)grbg" ! queue max-size-bytes=1000000000 max-size-time=0 ! bayer2rgb2 method=0 ! queue max-size-bytes=1000000000 max-size-time=0 ! ffmpegcolorspace ! "video/x-raw-yuv, format=(fourcc)I420" ! queue max-size-bytes=1000000000 max-size-time=0 ! videobalance saturation=1.9 ! queue max-size-bytes=1000000000 max-size-time=0 ! tee name=fullHD ! queue max-size-bytes=100000000 max-size-time=0 ! jpegenc idct-method=2 ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mux jackaudiosrc connect=1 ! queue max-size-time=0 ! audio/x-raw-float,channels=1 ! queue max-size-time=0 ! tee name=jackaudio ! queue max-size-time=0 ! vorbisenc max-bitrate=80000 ! queue max-size-time=0 ! tee name=vorbisaudio ! queue max-size-time=0 ! mux. mux. ! queue max-size-bytes=1000000000 max-size-time=0 ! filesink location=elphel_recording_`date +%s`.mkv sync=false fullHD. ! queue max-size-bytes=1000000000 max-size-time=0 ! videoscale ! "video/x-raw-yuv, width=640, height=363, aspect-ratio=(fraction)1/1" ! queue max-size-bytes=1000000000 max-size-time=0 ! tee name=downsample ! queue max-size-bytes=1000000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=1000000000 max-size-time=0 ! ximagesink pixel-aspect-ratio=1/1 sync=false downsample. ! queue max-size-bytes=100000000 max-size-time=0 ! queue max-size-bytes=1000000000 max-size-time=0 ! theoraenc bitrate=550 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=livestream vorbisaudio. ! queue max-size-time=0 ! livestream. livestream. ! queue max-size-bytes=0 max-size-time=0 ! shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv jackaudio. ! queue max-size-time=0 ! audioconvert ! queue max-size-time=0 ! alsasink

Select an Elphel 353L camera as the video source, widescreen, in Google[+] Video Chat/Hangouts (Linux only)

  • Requires a proprietary Google plugin
  • Also requires the v4l2loopback kernel module loaded (w/o parameters). Note an unresolved v4l2loopback bug may trigger GStreamer "Internal data flow error" pipeline failures.
  • Set the camera to 2592x1120 @ 18FPS. Adjust camera controls to ensure the camera achieves this framerate.
  • gst-launch-1.0 -e rtspsrc location=rtsp://192.168.0.9:554 latency=100 ! queue max-size-bytes=100000000 max-size-time=0 ! rtpjpegdepay ! queue max-size-bytes=100000000 max-size-time=0 ! jpegdec ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! "video/x-raw,framerate=(fraction)30/1" ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! "video/x-raw,width=640,height=480" ! queue max-size-bytes=100000000 max-size-time=0 ! videoconvert ! "video/x-raw,format=(string)YUY2,pixel-aspect-ratio=(fraction)1/1" ! v4l2sink device=/dev/video0
  • Launch Google Talk/Google+ Hangout and select 'Loopback video device 0' as the video input source. Syncroniced audio via the v4l2 API is untested -- select an ALSA audio source as appropriate.
  • Re-nice 'GoogleTalkPlugin' and both 'plugin-container' processes to '-15'. Tested on Abrowser 26.0 + Trisquel + linux-libre-3.12 + google-talkplugin ver. 4.9.1.0-1.

Debayer + convert to RGB a single JP46 frame using Elphel 353L

  • Use the following to extracting the highest-quality image possible (save JP4):
  • Set the Elphel camera to operate in JP46 mode using full-frame imaging (i.e., 2592x1936)
  • "Shift+Click" on the "view image" icon in the Elphel Camera Control Interface to save an image still
  • gst-launch-0.10 filesrc location="/path/to/elphelimg_nnnnnnn.jp46"  ! queue max-size-bytes=1000000000 ! jpegdec idct-method=2  ! queue max-size-bytes=1000000000 ! jp462bayer ! "video/x-raw-bayer, width=(int)2592, height=(int)1936, format=(string)grbg" ! queue max-size-bytes=1000000000 ! bayer2rgb2 method=4 ! queue max-size-bytes=1000000000 ! pngenc compression-level=9 ! queue max-size-bytes=1000000000 ! filesink location="/path/to/outputimage.png"
  • Open with GIMP

Extract a series of individual JPEGs from Matroska+M-JPEG into a folder

  • GST_DEBUG=GST_BUS:5 gst-launch-0.10 filesrc location=recording.mkv ! queue ! matroskademux ! queue ! videorate ! image/jpeg,framerate=1/10 ! queue ! multifilesink post-messages=true location="image%04d.jpg" sync=false 2>gst-bus-debug.log
  • Remove any colons input filenames to avoid BASH issues
  • Change the framerate to control the time interval between image stills and number of produced stills


  • After the above command is run the images should be generated. To add EXIF timestamps to each JPG:
  • cat gst-bus-debug.log | grep timestamp | grep dispatch | sed -e s/^.*filename[=].string.//g -e s/..stream.time.*//g -e s/[,].*\)/=/g -e s/000000000//g >timestamps.log
  • [TODO: create a BASH script or similar for the next step]
  • exiftool -DateTimeOriginal="`date -d @start-of-recording_unixtimestamp+offset-for-this-image-from-log-file --rfc-3339=seconds`" -overwrite_original *.jpg

Run color-correction on the extracted JPEGs with G'MIC

  • for i in *jpg; do mv $i tmp.jpg ; gmic tmp.jpg --apply_curve 0,1,1,10,1,116,172,183,204,255,255 -o[1] $i,90; done


  • for i in *jpg; do mv $i tmp.jpg ; gmic tmp.jpg --balance_gamma 68,68,68 -o[1] $i,90; done
  • The specified reference color must be in RGB, not hex. Use the gimp-gmic plugin to interactively find values that work well for your photoset.

Add EXIF Metadata to generated image stills

  • This command will replace images in a directory with the same image, metadata included - be sure to have backup copies.
  • "-flash#=0x0020" indicates that the camera has no flash capability.
  • Generally compatible with MediaGoblin
  • exiftool -Author='Your name' -Copyright='copyright license + URL' -make='Elphel Inc.' -model='Elphel NC353L-369-12V' -Xresolution=72 -Yresolution=72 -resolutionunit=inches -flash#=0x0020 -DateTimeOriginal='2012:05:04 10:00:00-5' -exposuremode='Manual' -FNumber='1.4' -UserComment='Lens or adapter used, or other notes' -GPSLatitude=42.360857 -GPSLatitudeRef=N -GPSLongitude=71.09122 -GPSLongitudeRef=W -overwrite_original *.jpg

Mix Three Video Sources Side-by-Side-by-Side

  • gst-launch --eos-on-shutdown videotestsrc pattern=6 is-live=true ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv,width=320,height=240 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=15/1 ! queue max-size-bytes=100000000 max-size-time=0 ! clockoverlay halignment=1 valignment=1 shaded-background=true font-desc="Sans Bold 80" ypad=80 ! queue max-size-bytes=100000000 max-size-time=0 ! videobox border-alpha=0 fill=green left=-320 ! queue max-size-bytes=100000000 max-size-time=0 ! videomixer name=mix ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! ximagesink sync=false videotestsrc is-live=true ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv,width=320,height=240 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=15/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videobox border-alpha=0 fill=blue left=-640 ! queue max-size-bytes=100000000 max-size-time=0 ! mix. videotestsrc pattern=11 ! "video/x-raw-yuv, width=352, height=240" ! queue max-size-bytes=100000000 max-size-time=0 ! mix.