(fix caps in title :D)
|Line 79:||Line 79:|
* [[iOS]] codecs
* [[iOS]] codecs
Revision as of 21:49, 18 March 2014
Currently the internal interfaces are very much in development. I (User:Brion) hope to get these in use for playback of audio and video content on Wikipedia and Wikimedia Commons in browsers that don't support Ogg or WebM natively, as our community has rejected the idea of making and serving out MP4 transcodes for those browsers.
Generic decoding, or even encoding, should be possible in the future but needs better APIs to be sane.
The build scripts make slight changes to configure scripts which may or may not be necessary, need to dive in in more detail.
There's also a slight fix needed in libtheora: https://github.com/brion/theora/commit/06a0e4acf9c35f4bd31b8788a8a573cb89262333 which is probably safe to upstream (check this!)
Playback demo at: https://brionv.com/misc/ogv.js/demo/
Flash video/audio playback works in all current browsers and IE 9; IE 6/7/8 support is planned.
- Must pass --disable-asm for emscripten, as it misdetects the host/target?
- needs a slight fix to a function signature to quiet an error from emscripten compiler
- experiments with using tremor instead of libvorbis showed it produces significantly smaller JS code. Performance, however, seems to be better with libvorbis, so the switch to tremor was not pursued.
- Internet Explorer
- JS player works on IE 10 and 11 only, but requires a Flash shim for audio
- Flash player with JS wrapper works in IE 9/10/11; working on 6/7/8 support
- JS player works on OS X 10.8/Safari 6.1, OS X 10.9/Safari 7, and iOS 7
- Safari JIT crash on iOS 7 64-bit seems to have been resolved by moving to the new emscripten LLVM backend
- Audio must be triggered from a UI event to work on iOS (no autoplay)
Performance seems adequate on recent-ish desktops/laptops in the latest browsers, but is woefully poor for medium-sized videos on most mobile phones and tablets.
- Flash player performance is similar to IE 11's JS on MacBook Pro, but has fewer slowdowns
- Flash player performs about the same as IE 11 JS on a Surface RT (Nvidia Tegra 3 CPU)
Safari/Mac OS X
- JS and Flash performance about the same on Safari 6.1/7; JS won't work or too slow on 6 and older
- Flash should work on old Safari (not yet tested)
- 64-bit iOS 7 devices hit decode speed targets at 360p/30fps video
- tested iPhone 5s
- 32-bit iOS 7 devices play audio, or video up to 160p/15fps acceptably
- tested iPod Touch 5th-gen and iPad 3
- iOS 6 seems to support the right APIs but doesn't play correctly, and the JIT is much too slow.
- non-Safari iOS browsers always disable the JIT and are much too slow.
Other modern browsers
- Other modern browsers don't actually need this for playback as they support Ogg Theora/Vorbis or WebM natively. Still of interest for experimental transcoding or encoding scenarios though.
- Unsurprisingly, Flash version performs about the same in any browser on the same hardware.
- Of modern browsers, Firefox performs JS best (asm.js optimizations). Beware that current release Firefox 27 only uses the asm.js optimizations on the first run; this is fixed in Firefox 30 nightlies.
- Chrome and Opera (the new Chromium-based one) perform pretty well too, maybe slightly better than Safari.
Current playback demo uses a custom wrapper 'ogvjs' or 'ogvswf' element which is extended to support a subset of the HTMLMediaElement API. With some additional selection/loader code this could be used to dynamically replace a <video> or <audio> tag.
WebRTC and MediaStreams also do not appear to allow easily plugging in custom JS codecs, but audio and video frames could be sent over WebRTC data channels as ArrayBuffers.
Live encoding from getUserMedia may be possible but I suspect frame rate will be awful as there's no way to grab the bits directly... and of course encoding is likely to be slow.
Transcoding from a native HTML5 video element may be similarly complex.
- iOS codecs