ogv.js

At least libogg, libvorbis, and libtheora can be built to JavaScript using emscripten, and to Flash with Crossbridge, with only slight modifications.

ogv.js is a wrapper around these with basic JavaScript/ActionScript3 interfaces and video/audio playback wrappers semi-compatible with the native HTML5 <video>/<audio> elements.

Currently the internal interfaces are very much in development. I (User:Brion) hope to get these in use for playback of audio and video content on Wikipedia and Wikimedia Commons in browsers that don't support Ogg or WebM natively, as our community has rejected the idea of making and serving out MP4 transcodes for those browsers.

Generic decoding, or even encoding, should be possible in the future but needs better APIs to be sane.

Building

The build scripts make slight changes to configure scripts which may or may not be necessary, need to dive in in more detail.

There's also a slight fix needed in libtheora: https://github.com/brion/theora/commit/06a0e4acf9c35f4bd31b8788a8a573cb89262333 which is probably safe to upstream (check this!)

Demo

Playback demo at: https://brionv.com/misc/ogv.js/demo/

JavaScript video works in all current browsers, audio works if Web Audio API is there (latest Firefox, Safari, Chrome), or with a Flash audio shim for IE.

Flash video/audio playback works in all current browsers and IE 9; IE 6/7/8 support is planned.

Notes

• libtheora
  • Must pass --disable-asm for emscripten, as it misdetects the host/target?
  • needs a slight fix to a function signature to quiet an error from emscripten compiler
• libvorbis
  • experiments with using tremor instead of libvorbis showed it produces significantly smaller JS code. Performance, however, seems to be better with libvorbis, so the switch to tremor was not pursued.
• Internet Explorer
  • JS player works on IE 10 and 11 only, but requires a Flash shim for audio
  • Flash player with JS wrapper works in IE 9/10/11; working on 6/7/8 support
• Safari
  • JS player works on OS X 10.8/Safari 6.1, OS X 10.9/Safari 7, and iOS 7
  • Safari JIT crash on iOS 7 64-bit seems to have been resolved by moving to the new emscripten LLVM backend
  • Audio must be triggered from a UI event to work on iOS (no autoplay)

Performance

Performance seems adequate on recent-ish desktops/laptops in the latest browsers, but is woefully poor for medium-sized videos on most mobile phones and tablets.

Internet Explorer

• JavaScript supported only on IE 10/11; needs Flash for audio and for older browsers
• JavaScript performance is not as good as other browsers, and there's a lot of GC churn visible in dev tools UI responsiveness profiling
• Flash player performance is similar to IE 11's JS on MacBook Pro, but has fewer slowdowns
• Flash player performs about the same as IE 11 JS on a Surface RT (Nvidia Tegra 3 CPU)
• Flash player performs significantly better than IE 11 JavaScript on an old Atom-based tablet where JS can't quite handle 360p/30

Safari/Mac OS X

• JS and Flash performance about the same on Safari 6.1/7; JS won't work or too slow on 6 and older
• Flash should work on old Safari (not yet tested)

Safari/iOS

• JavaScript only, no Flash
• 64-bit iOS 7 devices hit decode speed targets at 360p/30fps video
  • tested iPhone 5s
• 32-bit iOS 7 devices play audio, or video up to 160p/15fps acceptably
  • tested iPod Touch 5th-gen and iPad 3
• iOS 6 seems to support the right APIs but doesn't play correctly, and the JIT is much too slow.
• non-Safari iOS browsers always disable the JIT and are much too slow.

Other modern browsers

• Other modern browsers don't actually need this for playback as they support Ogg Theora/Vorbis or WebM natively. Still of interest for experimental transcoding or encoding scenarios though.
• Unsurprisingly, Flash version performs about the same in any browser on the same hardware.
• Of modern browsers, Firefox performs JS best (asm.js optimizations). Beware that current release Firefox 27 only uses the asm.js optimizations on the first run; this is fixed in Firefox 30 nightlies.
• Chrome and Opera (the new Chromium-based one) perform pretty well too, maybe slightly better than Safari.

HTML5 integration

Current playback demo uses a custom wrapper 'ogvjs' or 'ogvswf' element which is extended to support a subset of the HTMLMediaElement API. With some additional selection/loader code this could be used to dynamically replace a <video> or <audio> tag.

WebRTC and MediaStreams also do not appear to allow easily plugging in custom JS codecs, but audio and video frames could be sent over WebRTC data channels as ArrayBuffers.

Live encoding from getUserMedia may be possible but I suspect frame rate will be awful as there's no way to grab the bits directly... and of course encoding is likely to be slow.

Transcoding from a native HTML5 video element may be similarly complex.

Related projects

• iOS codecs