ogv.js

At least libogg, libvorbis, libopus, and libtheora can be built to JavaScript using emscripten pretty straightforwardly.

ogv.js is a wrapper around these with basic JavaScript interfaces and video/audio playback wrappers semi-compatible with the native HTML5 <video>/<audio> elements.

Currently (August 2015) in use for audio/video playback in Safari/IE/Edge browsers on Wikipedia and Wikimedia Commons.

Generic decoding, or even encoding, should be possible in the future but needs better APIs to be sane.

Building

See the readme.md for details of building the full ogv.js package. The standard autotools build systems are used with the emscripten compiler front-end.

You can also download a pre-built release of ogv.js.

Demo

Playback demo at: https://brionv.com/misc/ogv.js/demo/

Integration into MediaWiki demo at: https://ogvjs-testing.wmflabs.org/ (defaults to native codecs when available, still a work in progress)

JavaScript video works in all current browsers, audio works if Web Audio API is there (latest Firefox, Safari, Chrome), or with a Flash audio shim for IE.

Notes

• libtheora
  • Must pass --disable-asm for emscripten, as it misdetects the host/target?
  • needs a slight fix to a function signature to quiet an error from emscripten compiler
• libvorbis
  • experiments with using tremor instead of libvorbis showed it produces significantly smaller JS code. Performance, however, seems to be better with libvorbis, so the switch to tremor was not pursued.
• libopus
  • it works!
• Internet Explorer
  • JS player works on IE 10 and 11 only, but requires a Flash shim for audio
• Edge
  • JS player works using native Web Audio, no Flash required. Even works on Windows 10 Mobile!
• Safari
  • JS player works on OS X 10.8/Safari 6.1, OS X 10.9/Safari 7, and iOS 8
  • Safari JIT crashes on iOS 7
  • Audio must be triggered from a UI event to work on iOS (no auto play)

Performance

Performance seems adequate on recent-ish desktops/laptops in the latest browsers, and on 64-bit iOS devices, but is woefully poor for medium-sized videos on 32-bit mobile phones and tablets.

Internet Explorer

• Older versions than IE 10 not supported
• Flash audio shim required due to lack of Web Audio
• JavaScript performance is not as good as other browsers

Edge/Windows 10

• Works natively
• Faster than IE 11 but not as fast as others
  • Experimental asm.js optimizations bump performance back in line (set in about:flags)

Safari/Mac OS X

• JS has JIT bugs on 6.0, too slow on 5 and older
• Works well on 6.1/7/8/9

Safari/iOS

• 64-bit iOS 7 devices hit decode speed targets at 360p/30fps video
  • tested iPhone 5s, iPad Air
• 32-bit iOS 7 devices play audio, or video up to 160p/15fps acceptably
  • tested iPod Touch 5th-gen, iPhone 4s, iPad 3
• iOS 6 seems to support the right APIs but doesn't play correctly, and the JIT is much too slow.
• non-Safari iOS browsers always disable the JIT and are much too slow.

Other modern browsers

• Other modern browsers don't actually need this for playback as they support Ogg Theora/Vorbis or WebM natively. Still of interest for experimental transcoding or encoding scenarios though.
• Of modern browsers, Firefox performs JS best (asm.js optimizations).
• Chrome and Opera (the new Chromium-based one) perform pretty well too.

HTML5 integration

Current playback demo uses a custom wrapper 'ogvjs' element which is extended to support a subset of the HTMLMediaElement API. With some additional selection/loader code this can be used to dynamically replace a <video> or <audio> tag.

WebRTC and MediaStreams also do not appear to allow easily plugging in custom JS codecs, but audio and video frames could be sent over WebRTC data channels as ArrayBuffers.

Live encoding from getUserMedia may be possible but I suspect frame rate will be awful as there's no way to grab the bits directly... and of course encoding is likely to be slow.

Transcoding from a native HTML5 video element may be similarly complex.

Related projects

• iOS codecs