GST cookbook

GStreamer is a powerful multimedia infrastructure for applications—providing a number of programming language constructs and bindings—but also provides easy-to-use rapid prototyping tools to test new multimedia pipelines. Additionally, Gstreamer usually has good support for Xiph-related formats.

Unfortunately, it can be rather difficult to figure out an appropriate pipeline without a starting point. This page is a collection of gst-launch pipelines that can be used on an as-needed basis. GEntrans is a command-line tool helpful in visualizing queue states (i.e., static/under/overfilled), dynamically, for a given gst-launch pipeline. Queues can have a large impact on "even" pipeline performance, for example in avoiding intermittent frame pauses as the pipeline blocks on some other operation.

Texas Instruments offers a list of example pipelines for TI mobile devices.

Hold down the 'control' key in combination with BASH cursor-left (left arrow) or cursor-right (right arrow) to quickly edit the commands listed here. Note that GStreamer 0.10 (gst-launch-0.10) has been deprecated for several years and that use of the newer 1.x series (gst-launch-1.0) is strongly recommended.

Note that some examples are "high-performance" pipelines which require purpose-built systems.
'Open Video' Reference Build Script

Encode a .wav to Vorbis:

gst-launch-0.10 filesrc location="INPUT.wav" ! wavparse ! audioconvert ! vorbisenc ! oggmux ! filesink location="OUTPUT.ogg"

Dump a Theora video to PNGs:

gst-launch-0.10 filesrc location="INPUT.ogv" ! oggdemux ! theoradec ! ffmpegcolorspace ! pngenc snapshot=false ! multifilesink location="OUTPUT%04d.png"

Generate Ogg Vorbis audio from MP4 video (+progress monitoring)

gst-launch-1.0 filesrc location=original.mp4 ! queue max-size-time=0 max-size-bytes=100000000 ! qtdemux name=demux ! progressreport update-freq=5 demux.audio_0 ! queue ! decodebin ! queue ! audioconvert ! queue ! vorbisenc ! queue ! oggmux ! queue ! filesink location=vorbis-audio.oga

Transmux a MKV (containing vorbis and theora) to Ogg:

gst-launch-0.10 filesrc location="INPUT.mkv" ! matroskademux name=d ! video/x-theora ! queue ! theoraparse ! oggmux name=mux ! filesink location="OUTPUT.ogv" d. ! audio/x-vorbis ! queue ! vorbisparse ! queue ! mux.

Encode a y4m to lossless Dirac in Ogg:

gst-launch-0.10 filesrc location="INPUT.y4m" ! decodebin ! schroenc force-profile=vc2_main rate-control=lossless ! oggmux ! filesink location="OUTPUT.ogv"

Pull from a windows media stream, transcode to Ogg/Theora+Vorbis and send to an Icecast server:

(may require purchasing Fluendo plugins for decoding the encumbered codecs)

gst-launch-0.10 uridecodebin uri=mms://SOURCE.SERVER.COM/path name=d ! queue max-size-time=100000000 ! ffmpegcolorspace ! theoraenc bitrate=800 ! oggmux name=mux ! shout2send ip=YOURICECAST.SERVER.COM port=8000 password=YOURPASSWORD mount=/OUTPUTFILENAME.ogv d. ! queue max-size-time=100000000 ! audioconvert ! vorbisenc ! mux.

Capture video from a webcam, encode to an Ogg Theora file, decode and display on screen, write to a file whose name is the current date+time, and stream to an IceCast server

gst-launch-0.10 --eos-on-shutdown v4l2src ! 'video/x-raw-yuv, width=640, height=480' ! videorate ! 'video/x-raw-yuv, framerate=15/1' ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=150 ! oggmux ! tee name=ogged ! queue max-size-bytes=100000000 max-size-time=0 ! oggdemux ! theoradec ! xvimagesink sync=false force-aspect-ratio=true ogged. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=`date +%F_%T`.ogv ogged. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=YOURICECAST.SERVER.COM port=8000 password=YOURPASSWORD mount=/OUTPUTFILENAME.ogv streamname=YOURSTREAMNAME description=YOURDESCRIPTION genre=YOURGENRE url=YOURSTREAMURL ogged.

A v4l2 source + ALSA source -> Dual Ogg Theora + Ogg Vorbis Streams -> Icecast

dov4l -i [0|1] -m NTSC
gst-launch-0.10 --eos-on-shutdown v4l2src device=/dev/video0 ! queue max-size-bytes=100000000 max-size-time=0 ! deinterlace mode=interlaced ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=15/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv,width=360,height=240,pixel-aspect-ratio=1/1 ! queue max-size-bytes=100000000 max-size-time=0 ! gamma gamma=1.2 ! queue max-size-bytes=100000000 max-size-time=0 ! videobalance saturation=1.9 brightness=0.00 contrast=1.4 hue=0.06 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=400 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=mux alsasrc device=hw:0,0 latency-time=100 ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! queue max-size-bytes=100000000 max-size-time=0 ! audioamplify amplification=[1-6 or so] ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=vorbisaudio ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=server.com port=[80|8000] password=hackme mount=/mountpoint.ogv vorbisaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=server.com port=[80|8000] password=hackme mount=/mountpoint.oga

Note that the above pipeline will slowly lose audio/video synchronization due to hardware-level limitations in audio vs. video capture of media samples. FireWire and SDI-based capture does not have these limitations.

Two v4l2 sources combined side-by-side into rectangular video + ALSA -> Ogg Theora -> Icecast

mplayer tv:// -tv device=/dev/video0:driver=v4l2:norm=NTSC:width=320:height=240:outfmt=uyvy:input=1:buffersize=16: -ao null
mplayer tv:// -tv device=/dev/video1:driver=v4l2:norm=NTSC:width=320:height=240:outfmt=uyvy:input=1:buffersize=16: -ao null
gst-launch-0.10 --eos-on-shutdown v4l2src device=/dev/video0 ! videoscale ! video/x-raw-yuv,width=320,height=240,interlaced=true ! queue max-size-bytes=100000000 max-size-time=0 ! gamma gamma=1.2 ! queue max-size-bytes=100000000 max-size-time=0 ! videobalance saturation=1.9 brightness=0.00 contrast=1.4 hue=0.06 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=15/1 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! videomixer name=mix ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=600 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=mux alsasrc device=hw:1,0 latency-time=100 ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=icecast-server.com password=hackme mount=/mountpoint.ogv v4l2src device=/dev/video1 ! videoscale ! video/x-raw-yuv,width=320,height=240,interlaced=true ! gamma gamma=1.2 ! queue max-size-bytes=100000000 max-size-time=0 ! videobalance saturation=1.9 brightness=0.00 contrast=1.4 hue=0.06 ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=5/1 ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! videobox border-alpha=0 fill=green left=-320 ! mix.

Create a video with an alpha channel from a sequence of PNG files

gst-launch-0.10 multifilesrc location=images%05d.png caps="image/png,framerate=1/1,pixel-aspect-ratio=1/1" num-buffers=95 ! pngdec ! videorate ! alphacolor ! "video/x-raw-yuv,format=(fourcc)AYUV" ! matroskamux ! filesink location=images_raw.mkv

Capture from a 1280x800 (1680x1050) GNOME Desktop -> Ogg Theora -> Icecast

gst-launch-1.0 --eos-on-shutdown ximagesrc ! capsfilter caps=video/x-raw,framerate=4/1,width=1280,height=800 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw, width=1056, height=660 ! queue max-size-bytes=100000000 max-size-time=0 ! videoconvert ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=400 keyframe-auto=false keyframe-force=12 keyframe-freq=12 speed-level=1 drop-frames=false ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv

Capture from a 1680x1050 GNOME Desktop, Combine with Desktop Audio (Pulse) -> Ogg Theora + Vorbis -> Icecast

gst-launch-1.0 --eos-on-shutdown ximagesrc use-damage=false ! capsfilter caps=video/x-raw,framerate=4/1,width=1680,height=1050 ! queue max-size-bytes=100000000 max-size-time=0 ! videoconvert ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=350 keyframe-auto=false keyframe-force=12 keyframe-freq=12 speed-level=1 drop-frames=false ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=mux pulsesrc device=alsa_output.pci-0000_00_1b.0.analog-stereo.monitor ! queue max-size-bytes=100000000 ! audioconvert ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv

The pulseaudio/pulsesrc device may need to be changed depending on the machine: run 'pactl list short sources | cut -f2' to see options
Viewers will need better-than-average internet connectivity (around 1.2Mbits/sec)

Live-stream a high-resolution Mac OSX Desktop at 1 FPS

Tested on Version 10.6.8, newer releases not compatible with osximagesrc

gst-launch-0.10 --eos-on-shutdown osximagesrc ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! videoscale method=4-tap ! video/x-raw-yuv, width=960, height=600 ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=360 keyframe-auto=false keyframe-force=3 keyframe-freq=3 speed-level=0 drop-frames=false ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=host.com password=hackme mount=/mountpoint.ogv

If the screen resolution is set to mirrored 1024x640 (?), start the following then plug in the video display cable:

gst-launch-0.10 --eos-on-shutdown osximagesrc ! queue max-size-time=0 max-size-bytes=100000000 ! ffmpegcolorspace ! queue max-size-time=0 max-size-bytes=100000000 ! videocrop right=256 top=32 ! queue max-size-time=0 max-size-bytes=100000000 ! videoscale ! video/x-raw-yuv, width=658, height=486 ! queue max-size-time=0 max-size-bytes=100000000 ! theoraenc bitrate=300 keyframe-auto=false keyframe-force=3 keyframe-freq=3 speed-level=0 drop-frames=false ! queue max-size-time=0 max-size-bytes=100000000 ! oggmux ! queue max-size-time=0 max-size-bytes=100000000 ! shout2send ip=host.com port=8000 password=hackme mount=/mountpoint.ogv

Streamed Keynote talks will stream the Presenter's View -- no switching between display mirrors
Full-screen Flash video doesn't stream, but doesn't halt the pipeline

Live-stream an Elphel 353L camera, combine with on-laptop audio capture, save a high-res copy to disk, and view live audio+video on-screen

Tested on an 2.4GHz Core i3 running at ≥2.0GHz. Both CPU cores should be at ~50% continuously-smooth utilization after the first ~15 seconds; battery-only power or overheating may throttle the CPU leading to dropped frames.
Set the camera to 2592x1120 @ 18FPS. Adjust camera controls to ensure the camera achieves this framerate.
Kill/renice any resource-competitive applications (file indexing services, Firefox, Audacity, etc.)
qjackctl & → Start JACK in realtime priority

gst-launch-1.0 -e rtspsrc location=rtsp://192.168.0.9:554 latency=100 ! queue max-size-bytes=100000000 max-size-time=0 ! rtpjpegdepay ! queue max-size-bytes=100000000 max-size-time=0 ! jpegdec max-errors=-1 ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw,framerate=18/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw, width=1296, height=560 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=halfres ! queue max-size-bytes=100000000 max-size-time=0 ! jpegenc ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mux jackaudiosrc connect=1 client-name="GStreamer Input" ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=jackaudio ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=vorbisaudio ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=elphel_recording_`date +%s`.mkv sync=false halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw,width=648, height=280 ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=500 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=livestream vorbisaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! livestream. livestream. ! queue max-size-bytes=0 max-size-time=0 ! shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv halfres. ! videoscale add-borders=true ! queue max-size-bytes=100000000 max-size-time=0 ! videoconvert ! queue max-size-bytes=100000000 max-size-time=0 ! xvimagesink sync=false jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! queue max-size-bytes=100000000 max-size-time=0 ! alsasink

Expect a ~50GB recording for 8hrs. of recording depending on image complexity. The Matroska recording may be viewed directly in VLC (≥2.0 recommended) and may be transcoded to commonly-used formats via Miro Video Converter or Firefogg.

Alternatively: ffmpeg -i elphel_recording.mkv -acodec libfaac -ab 128k -pass 1 -vcodec libx264 -vpre slow -vpre main -b 8000k -threads 8 -f mp4 -y elphel_recording.mp4

Same as above, but simultaneously broadcast in H.264

Tested with crtmpserver (works) and Moment Video Server (in theory)

For crtmpserver:

Remove a 'tags_written++' line from './gst/flv/gstflvmux.c' as per Bug 661624
crtmpserver.lua should be customized for the streaming application (a mostly undocumented step...)

For Moment:

Remove 'pingTimerTick' 'from moment/rtmp_connection.cpp' to avoid stream resets

For both:

Toggle 'byte-stream=[true|false]' and '[tcpclientsink|rtmpsink]' as necessary, possibly '[flvmux|mp4mux]' if needed.
Test stream availability with this tool.
Substitute 'videotestsrc is-live=true' for 'rtspsrc ... jpegdec' and 'audiotestsrc freq=432' for 'jackaudiosrc connect=1' for testing.
'rtmpsink' isn't yet(?) compatible with Akamai

gst-launch-0.10 -e rtspsrc location=rtsp://192.168.0.9:554 latency=100 ! queue max-size-bytes=100000000 max-size-time=0 ! rtpjpegdepay ! queue max-size-bytes=100000000 max-size-time=0 ! jpegdec ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw-yuv,framerate=18/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=1296, height=560 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=halfres ! queue max-size-bytes=100000000 max-size-time=0 ! jpegenc ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mkvmux jackaudiosrc connect=1 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=jackaudio ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=vorbisaudio ! queue max-size-bytes=100000000 max-size-time=0 ! mkvmux. mkvmux. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=elphel_recording_`date +%F_%T`.mkv halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw-yuv, width=648, height=280 ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=quarterres ! queue max-size-bytes=100000000 max-size-time=0 ! x264enc sliced-threads=true cabac=true intra-refresh=false quantizer=22 rc-lookahead=15 bitrate=600 tune=zerolatency byte-stream=false ! queue max-size-bytes=100000000 max-size-time=0 ! flvmux streamable=true name=flvmux jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! queue max-size-bytes=100000000 max-size-time=0 ! faac ! queue max-size-bytes=100000000 max-size-time=0 ! flvmux. flvmux. ! queue max-size-bytes=0 max-size-time=0 ! tcpclientsink host=127.0.0.1 port=6666 quarterres. ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=600 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=livestream jackaudio. ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! livestream. livestream. ! queue max-size-bytes=0 max-size-time=0 ! shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv halfres. ! queue max-size-bytes=100000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=100000000 max-size-time=0 ! xvimagesink sync=false

Live-stream an Elphel 353L camera set at 'FullHD' JP46, combine with realtime audio, save a high-res copy to disk, and monitor live audio+video on-screen

Video samples
Setup a high-performance system: tested on Gigabyte GA-MA785GMT-UD2H + (non-overclocked) Phenom II X4 built using gNewSense. CPU utilization approaches 100%, continuously, across all four 3.2GHz cores.
Set the camera to 1920x1088, JPEG Image Quality @ 90%, JP46 color mode

In separate tabs:

jackd -R -d alsa -C -d hw:2 -r 48000 -i 1
gst-launch-0.10 -e rtspsrc location=rtsp://192.168.1.50:554 protocols=0x00000001 latency=100 ! rtpjpegdepay ! queue max-size-bytes=1000000000 max-size-time=0 ! videorate force-fps=18 ! queue max-size-bytes=1000000000 max-size-time=0 ! jpegdec max-errors=-1 idct-method=2 ! queue max-size-bytes=1000000000 max-size-time=0 ! jp462bayer threads=4 ! "video/x-raw-bayer, width=(int)1920, height=(int)1088, format=(string)grbg" ! queue max-size-bytes=1000000000 max-size-time=0 ! bayer2rgb2 method=0 ! queue max-size-bytes=1000000000 max-size-time=0 ! ffmpegcolorspace ! "video/x-raw-yuv, format=(fourcc)I420" ! queue max-size-bytes=1000000000 max-size-time=0 ! videobalance saturation=1.9 ! queue max-size-bytes=1000000000 max-size-time=0 ! tee name=fullHD ! queue max-size-bytes=100000000 max-size-time=0 ! jpegenc idct-method=2 ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mux jackaudiosrc connect=1 ! queue max-size-time=0 ! audio/x-raw-float,channels=1 ! queue max-size-time=0 ! tee name=jackaudio ! queue max-size-time=0 ! vorbisenc max-bitrate=80000 ! queue max-size-time=0 ! tee name=vorbisaudio ! queue max-size-time=0 ! mux. mux. ! queue max-size-bytes=1000000000 max-size-time=0 ! filesink location=elphel_recording_`date +%s`.mkv sync=false fullHD. ! queue max-size-bytes=1000000000 max-size-time=0 ! videoscale ! "video/x-raw-yuv, width=640, height=363, aspect-ratio=(fraction)1/1" ! queue max-size-bytes=1000000000 max-size-time=0 ! tee name=downsample ! queue max-size-bytes=1000000000 max-size-time=0 ! ffmpegcolorspace ! queue max-size-bytes=1000000000 max-size-time=0 ! ximagesink pixel-aspect-ratio=1/1 sync=false downsample. ! queue max-size-bytes=100000000 max-size-time=0 ! queue max-size-bytes=1000000000 max-size-time=0 ! theoraenc bitrate=550 speed-level=1 ! queue max-size-bytes=100000000 max-size-time=0 ! oggmux name=livestream vorbisaudio. ! queue max-size-time=0 ! livestream. livestream. ! queue max-size-bytes=0 max-size-time=0 ! shout2send ip=host.com port=[80|8000] password=hackme mount=/mountpoint.ogv jackaudio. ! queue max-size-time=0 ! audioconvert ! queue max-size-time=0 ! alsasink

Select an Elphel 353L camera as the video source, widescreen, as a video4linux loopback device / webcam (Linux-only)

Requires the v4l2loopback kernel module loaded (w/o parameters). Note an unresolved v4l2loopback bug will generate GStreamer "Internal data flow error" pipeline failures; the finial 'tee' element is a workaround for this issue.
Set the camera to 2592x1120 @ 18FPS. Adjust camera controls to ensure the camera achieves this framerate.
gst-launch-1.0 -e rtspsrc location=rtsp://192.168.0.9:554 latency=100 ! queue max-size-bytes=100000000 max-size-time=0 ! rtpjpegdepay ! queue max-size-bytes=100000000 max-size-time=0 ! jpegdec ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! "video/x-raw,framerate=(fraction)30/1" ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! "video/x-raw,width=640,height=480" ! queue max-size-bytes=100000000 max-size-time=0 ! videoconvert ! "video/x-raw,format=(string)YUY2,pixel-aspect-ratio=(fraction)1/1" ! tee ! queue max-size-bytes=100000000 max-size-time=0 ! v4l2sink device=/dev/video0
Launch pipeline above then run video-conferencing software (e.g., Firefox Hello, Skype, Google[+] Hangouts)
Synchronized audio via the v4l2 API is untested -- select an ALSA audio source as appropriate.
It may be helpful to re-nice video processes to '-15'.

Debayer + convert to RGB a single JP46 frame using Elphel 353L

Use the following to extracting the highest-quality image possible (save JP4):
Set the Elphel camera to operate in JP46 mode using full-frame imaging (i.e., 2592x1936)
"Shift+Click" on the "view image" icon in the Elphel Camera Control Interface to save an image still
gst-launch-0.10 filesrc location="/path/to/elphelimg_nnnnnnn.jp46" ! queue max-size-bytes=1000000000 ! jpegdec idct-method=2 ! queue max-size-bytes=1000000000 ! jp462bayer ! "video/x-raw-bayer, width=(int)2592, height=(int)1936, format=(string)grbg" ! queue max-size-bytes=1000000000 ! bayer2rgb2 method=4 ! queue max-size-bytes=1000000000 ! pngenc compression-level=9 ! queue max-size-bytes=1000000000 ! filesink location="/path/to/outputimage.png"
Open with GIMP

Extract a series of individual JPEGs from Matroska+M-JPEG video into a folder and monitor progress in "fast forward" mode

GST_DEBUG=GST_BUS:5 gst-launch-1.0 filesrc location=recording.mkv ! queue ! matroskademux ! queue ! videorate ! image/jpeg,framerate=1/5 ! queue ! tee name=split ! jpegdec ! queue ! videoconvert ! queue ! xvimagesink sync=false split. ! multifilesink post-messages=true location="image_%04d.jpg" sync=false &>gst-bus-debug.log
Remove any colons in input filenames to avoid BASH interpretation issues
Change the framerate to control the time interval between image stills and number of produced stills. "1/5" translates to "one image per five seconds of video."

To add EXIF timestamps to individual JPGs:
cat gst-bus-debug.log | grep timestamp | grep dispatch | sed -e s/^.*filename[=].string.//g -e s/..stream.time.*//g -e s/[,].*\)/=/g -e s/000000000//g >timestamps.log
[TODO: create a BASH script or similar for the next step]
exiftool -DateTimeOriginal="`date -d @start-of-recording_unixtimestamp+offset-for-this-image-from-log-file --rfc-3339=seconds`" -overwrite_original *.jpg

Extract video clips using specified timecodes

mkvmerge --split parts:hh:mm:ss-hh:mm:ss -o clip.mkv recording.mkv
Requires recent builds of MKVToolNix
With short-duration time codes, extract all frames from the generated clip using the above GST pipeline for "frame-by-frame" still selection
Also generally helpful to split long recordings into manageable sections for further editing

Run color-correction on the extracted JPEGs with G'MIC

for i in *jpg; do mv $i tmp.jpg ; gmic tmp.jpg --apply_curve 0,1,1,10,1,116,172,183,204,255,255 -o[1] $i,90; done

for i in *jpg; do mv $i tmp.jpg ; gmic tmp.jpg --balance_gamma 68,68,68 -o[1] $i,90; done
The specified reference color must be in RGB, not hexadecimal. Use the gimp-gmic plugin to interactively find values that work well for your particular photoset.

Add EXIF Metadata to generated image stills

This command will replace images in a directory with the same image, metadata included - be sure to have backup copies.
"-flash#=0x0020" indicates that the camera has no flash capability.
Generally compatible with MediaGoblin
exiftool -Author='Your name' -Copyright='copyright license + URL' -make='Elphel Inc.' -model='Elphel NC353L-12V' -Xresolution=72 -Yresolution=72 -resolutionunit=inches -flash#=0x0020 -DateTimeOriginal='2012:05:04 10:00:00-5' -exposuremode='Manual' -FNumber='1.4' -UserComment='Lens or adapter used, or other notes' -GPSLatitude=42.36160 -GPSLatitudeRef=N -GPSLongitude=71.09064 -GPSLongitudeRef=W -overwrite_original *.jpg

Dump all JPEGs from Matroska+M-JPEG video, run various G'MIC filters (e.g., de-blur), then recreate video

gst-launch-1.0 filesrc location=elphel_recording_.mkv ! queue ! matroskademux ! queue ! multifilesink location="image_%04d.jpg"

Run a mild de-blur filter (slow processing) on all frames:

for i in *jpg; do mv $i tmp.jpg ; gmic tmp.jpg +fx_deblur 1.36,6,17.5,0.3,1,0,0,24,0 -o[1] $i; done

Re-create the video using the image-corrected JPEGs, coping the original audio track:

gst-launch-1.0 matroskamux name=mux ! queue ! filesink location=elphel_recording_corrected.mkv multifilesrc location="elphel_recording_%04d.jpg" caps="image/jpeg,framerate=18/1,pixel-aspect-ratio=1/1" ! queue ! jpegdec ! queue ! jpegenc ! mux. filesrc location=moon.mkv ! queue ! matroskademux ! queue ! vorbisdec ! queue ! vorbisenc ! queue ! mux.

TODO: Figure out how to avoid decoding/re-encoding audio and video -- should not be a necessary step

HDMI2USB (timvideos.us) capture with ALSA audio input

The board produces 15FPS at JPEG quality 85, resulting in a large recording file size.
a/v sync is not yet addressed
gst-launch-1.0 -e v4l2src device=/dev/video1 ! image/jpeg,width=1280,height=720 ! queue max-size-bytes=100000000 max-size-time=0 ! matroskamux name=mux alsasrc device=hw:CARD=CODEC latency-time=100 ! queue max-size-bytes=100000000 max-size-time=0 ! audioconvert ! queue max-size-bytes=100000000 max-size-time=0 ! vorbisenc ! queue max-size-bytes=100000000 max-size-time=0 ! mux. mux. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=test4.mkv

Mix Three Video Sources Side-by-Side-by-Side

May be sensitive to internal element-construction race conditions using gst-launch -- this is a proof-of-concept template for programming applications.
gst-launch-1.0 --eos-on-shutdown videotestsrc pattern=6 is-live=true ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw,framerate=15/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw,width=320,height=240 ! queue max-size-bytes=100000000 max-size-time=0 ! clockoverlay halignment=1 valignment=1 shaded-background=true font-desc="Sans Bold 80" ypad=80 ! queue max-size-bytes=100000000 max-size-time=0 ! videobox border-alpha=0 fill=green left=-320 ! queue max-size-bytes=100000000 max-size-time=0 ! videomixer name=mix ! queue max-size-bytes=100000000 max-size-time=0 ! tee name=trivideo ! queue max-size-bytes=100000000 max-size-time=0 ! ximagesink sync=false videotestsrc pattern=22 ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw,framerate=15/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw,width=320,height=240 ! queue max-size-bytes=100000000 max-size-time=0 ! mix. videotestsrc is-live=true ! queue max-size-bytes=100000000 max-size-time=0 ! videorate ! video/x-raw,framerate=15/1 ! queue max-size-bytes=100000000 max-size-time=0 ! videoscale ! video/x-raw,width=320,height=240 ! queue max-size-bytes=100000000 max-size-time=0 ! videobox border-alpha=0 fill=blue left=-640 ! queue max-size-bytes=100000000 max-size-time=0 ! mix.