XiphWiki - User contributions [en]

Vorbis

2013-11-25T16:41:26Z

Basilgohar: Removed another dead link (maresweb.de is for sale)

'''Vorbis''' is a patent-clear, fully open, general purpose audio encoding format standard that rivals or even surpasses the 'upcoming' generation of proprietary codecs ([[Wikipedia:Advanced Audio Coding|AAC]] and [[Wikipedia:TwinVQ|TwinVQ]], also known as VQF). There is no raw Vorbis stream defined, instead the Vorbis codec is typically used in the [[Ogg]] container format for audio files. Because for a long time the Ogg container was quasi exclusive for Vorbis people often refer to it as 'Ogg Vorbis'. Later the FLAC audio codec as well as the video codecs Theora and Dirac began to be used inside Ogg too. In 2010 the [[WebM]] format was defined using the Vorbis codec inside the WebM container.

libvorbis, a BSD-licensed source implementation of Vorbis as a library is available; See the [http://xiph.org/vorbis/ Ogg Vorbis page] for documentation, downloads and distribution terms.

Many hard- and software players support Ogg Vorbis; see [http://www.vorbis.com/ vorbis.com] or the links below for a list of all the players we know about.

== More information ==

* [[Vorbis Hardware]]: List of hardware-players supporting Ogg Vorbis
* [[Vorbis Software Players]]: List of media players that can play Ogg Vorbis
* [[Vorbis Software Encoders]]: List of libvorbis frontends
* [[Vorbis Decoders]]: List of decoders (e.g. Xiph, Tremor, JOrbis, etc)
* [[Vorbis Encoders]]: List of encoders (e.g. Xiph, aoTuV, GT, vorbis-java)
* [[Vorbis-tools]]: Reference tools maintained by Xiph.org
* [[Games that use Vorbis]]: List of games using Ogg Vorbis
* [[VorbisStreams]]: Stations streaming with the [[Vorbis]] codec
* [[VorbisCasts]]: Audiocasts publishing Ogg [[Vorbis]] feeds

== External links ==

* [http://www.vorbis.com/ Vorbis.com]
* [[Wikipedia: Vorbis]]
* [http://www.hydrogenaudio.org/forums/index.php?showtopic=35438 80kbps personal listening test]
* [http://www.hydrogenaudio.org/forums/index.php?showtopic=36465 180kbps personal listening test with classical music]

[[Category:Vorbis]]

Vorbis

2013-11-25T16:40:11Z

Basilgohar: Removed a dead link (rjamorin's 128kbps test, domain name is for sale)

Main Page

2013-11-25T16:38:06Z

Basilgohar: Added link to the OpusFAQ (no link to Opus before this!)

In an effort to bring open-source ideals to the world of multimedia the [[Xiph.Org Foundation]] develops a multitude of amazing products. This wiki describes our free and open protocols and software.

----

= Demonstrations of Xiph technologies =

Want to hear or see Xiph in action? These projects are using our codecs, formats, or libraries.

* [[VorbisStreams|Vorbis Streams]]: Stations streaming with the [[Vorbis]] codec
* [[Games that use Vorbis]]: Games using the Vorbis codec for music or sound effects
* [[VorbisHardware|Vorbis Hardware]]: Hardware players using the Vorbis codec
* [[VorbisSoftwarePlayers|Vorbis Software Players]]: list of media players with out-of-box support for Vorbis
* [[TheoraHardware|Theora Hardware]]: Hardware using the Theora video codec
* [[TheoraSoftwarePlayers|Theora Software Players]]: list of media players with Theora support
* [[List of Theora videos]]: Sources for video encoded with [[Theora]]

= Projects/Formats =

== Container Formats ==

* [[Ogg]]: Media container. This is our native format and the recommended container for Xiph codecs.
** [[Ogg Skeleton]]: Skeleton information on all logical content bitstreams in Ogg.
** [[MIMETypesCodecs|Specification of MIME types and respective codecs parameter]]
* [[SpeexRTP]]: RTP payload format for voice
* [[VorbisRTP]]: RTP payload format for general audio
* [[TheoraRTP]]: RTP payload format for video
* [[XSPF]]: XML Sharable Playlist Format

== Codecs ==

* '''Compressed Audio/Video Codecs:'''
** [[Vorbis]]: Audio codec with a [[Tremor|fixed point decoder]]
** [[Theora]]: Video codec
** [[FLAC]]: Free Lossless Audio Codec
** [[Speex]]: Speech codec
** [[OpusFAQ|Opus]]: Low-latency general-purpose audio codec
* '''Uncompressed Audio/Video Codecs:'''
** [[OggPCM]]: Audio codec
* '''Timed Text/Metadata Codecs:'''
** [[CMML]]: Continuous Media Markup Language, used for [http://www.annodex.net/ Annodex] and subtitles (xine, vlc, gstreamer, and DirectShow support)
**[[OggKate|Kate]]: new format for lyrics and subtitles

== Software ==

* '''Software for distributing media'''
** [[Icecast]]: Streaming server
** [[Ices]]: Source client for Icecast servers

* '''Libraries'''
** [[OggPlay]]: library for synchronised Xiph media playback
**[[XiphQT]]: Quicktime component to play the main Xiph formats
** [[VorbisCommentEdit]]: Macintosh Framework making it easy to incorporate the editing of [[VorbisComment|Vorbis Comments]]

* '''Other software'''
** [[OggComponent/VorbisComponent]]: Wrappers to integrate Vorbis into Mac OS X (does not yet support encoding)
** [http://xiph.org/paranoia/ cdparanoia]: CDDA extractor/ripper

== Community ==

*[[How to help]]
*[[Spread Open Media]]: project to promote Xiph formats.
**[[MailOgging]]: provides templates for anyone willing to contact a company requesting them to add support for Xiph formats.
*[[People]]: Who's who in Xiph.

== Work in Progress ==
* [[Work In Progress]]: codecs and software still in the research and development stages.
* [[Todo]]: To-do list for various Xiph projects.

= Project management =

* [[AdminProcesses]]: who's in charge of what project
* [[MonthlyMeeting]]: page with information on Xiph's MonthlyMeeting
* [[MailingLists]]: list of Xiph's mailing lists
* [[Bounties]]: list of bounties that you can take to improve Xiph's projects

= Resources for Video and Audio programmers =

* [[Ambisonics]]: page with technical information on Ambisonics
* [[Resources and papers on Audio, Music and Speech|Courses and papers on Audio, Music and Speech]]: page with links to MIT and other universities' content
* [[Oggless]]: for ideas on how to use the different Xiph codecs outside Ogg

= Wiki internal =

* [[Translations]]: What about some translation work
* [[Sandbox]]: Testbed for testing editing skills
* [[XiphWiki:Copyrights]]: License used for all content on the XiphWiki

SpeexRTP

2012-12-05T19:04:33Z

Basilgohar: Reverted edits by Svererenov (talk) to last revision by Saoshyant

=== Speex over RTP ===

While [[Ogg]] is the native file format for speex data, an RTP encapsulation is a natural fit with the codec's low latency design.

Fortunately, unlike [[Vorbis]] and [[Theora]], [[Speex]] has no required setup headers, so the application is easy. A [http://speex.org/drafts/draft-herlein-speex-rtp-profile-02.txt draft exists]. It just needs to be reviewed and then taken through the formal ratification process with the IETF.

[[Category:Speex]]

Icecast Server

2011-11-21T22:33:54Z

Basilgohar: added OggOpus streaming

'''Icecast''' is an open source multi-platform streaming server. It supports [[Ogg]] [[Vorbis]], Ogg [[Theora]], and [[MP3]].

== External links ==

* [http://www.icecast.org/ Icecast homepage]
* [http://dir.xiph.org/index.php Stream directory]
* [http://www.nabble.com/Icecast-f2880.html Icecast archive / forum] - an Icecast mailing list archive that combines both user and dev lists. It is hosted by [http://www.nabble.com/ Nabble]. You can search or browse Icecast discussions here.

== Development ==

*trunk http://svn.xiph.org/icecast/trunk/icecast
*kh-branch http://svn.xiph.org/icecast/branches/kh/icecast
**diff to trunk
***fast pre-buffering aka burst-on-connect. State a burst size in bytes to indicate how much should be sent at listener connect.
***mp3 accepts artist and title separately on the url.
***program invocation at stream start and end, per mount based.
***on-demand relays, activated on first listener, disconnected when listenersfalls to 0. Available for master relays as well.
***multiple Ogg codec streaming. Current codecs handled are Theora, Vorbis, Speex, Writ.
***Clients are started at theora key frame if theora is being streamed.
***Added URL and command based listener authentication
***server xml reload, and reopen logging available via admin url
***slave startup re-organised so that relays are more independant
***on xml reload, active sources are updated as well
***When max-listeners reached, a HTTP 302 code can be sent to redirect clients to alternative slave hosts.
***authenticated relays, those that match the relay user/pass, bypass the max-listener check

== Wish List ==

As good ideas are never a waste, and for tracking purposes, please list here all the features you're missing in icecast trunk.

Note: please check that the feature you request is not already in trunk before posting !

* WebM streaming
* OggOpus streaming
* PUT method support
* Ponies

[[Category:Xiph-related Software]]

Icecast Server

2011-11-21T19:50:40Z

Basilgohar: Ponies should be spelled correctly (http://en.wiktionary.org/wiki/ponies)

'''Icecast''' is an open source multi-platform streaming server. It supports [[Ogg]] [[Vorbis]], Ogg [[Theora]], and [[MP3]].

== External links ==

* [http://www.icecast.org/ Icecast homepage]
* [http://dir.xiph.org/index.php Stream directory]
* [http://www.nabble.com/Icecast-f2880.html Icecast archive / forum] - an Icecast mailing list archive that combines both user and dev lists. It is hosted by [http://www.nabble.com/ Nabble]. You can search or browse Icecast discussions here.

== Development ==

*trunk http://svn.xiph.org/icecast/trunk/icecast
*kh-branch http://svn.xiph.org/icecast/branches/kh/icecast
**diff to trunk
***fast pre-buffering aka burst-on-connect. State a burst size in bytes to indicate how much should be sent at listener connect.
***mp3 accepts artist and title separately on the url.
***program invocation at stream start and end, per mount based.
***on-demand relays, activated on first listener, disconnected when listenersfalls to 0. Available for master relays as well.
***multiple Ogg codec streaming. Current codecs handled are Theora, Vorbis, Speex, Writ.
***Clients are started at theora key frame if theora is being streamed.
***Added URL and command based listener authentication
***server xml reload, and reopen logging available via admin url
***slave startup re-organised so that relays are more independant
***on xml reload, active sources are updated as well
***When max-listeners reached, a HTTP 302 code can be sent to redirect clients to alternative slave hosts.
***authenticated relays, those that match the relay user/pass, bypass the max-listener check

== Wish List ==

As good ideas are never a waste, and for tracking purposes, please list here all the features you're missing in icecast trunk.

Note: please check that the feature you request is not already in trunk before posting !

* WebM Streaming
* PUT method support
* Ponies

[[Category:Xiph-related Software]]

Icecast Server

2011-11-21T19:47:50Z

Basilgohar: added WebM streaming and PUT method support

'''Icecast''' is an open source multi-platform streaming server. It supports [[Ogg]] [[Vorbis]], Ogg [[Theora]], and [[MP3]].

== External links ==

* [http://www.icecast.org/ Icecast homepage]
* [http://dir.xiph.org/index.php Stream directory]
* [http://www.nabble.com/Icecast-f2880.html Icecast archive / forum] - an Icecast mailing list archive that combines both user and dev lists. It is hosted by [http://www.nabble.com/ Nabble]. You can search or browse Icecast discussions here.

== Development ==

*trunk http://svn.xiph.org/icecast/trunk/icecast
*kh-branch http://svn.xiph.org/icecast/branches/kh/icecast
**diff to trunk
***fast pre-buffering aka burst-on-connect. State a burst size in bytes to indicate how much should be sent at listener connect.
***mp3 accepts artist and title separately on the url.
***program invocation at stream start and end, per mount based.
***on-demand relays, activated on first listener, disconnected when listenersfalls to 0. Available for master relays as well.
***multiple Ogg codec streaming. Current codecs handled are Theora, Vorbis, Speex, Writ.
***Clients are started at theora key frame if theora is being streamed.
***Added URL and command based listener authentication
***server xml reload, and reopen logging available via admin url
***slave startup re-organised so that relays are more independant
***on xml reload, active sources are updated as well
***When max-listeners reached, a HTTP 302 code can be sent to redirect clients to alternative slave hosts.
***authenticated relays, those that match the relay user/pass, bypass the max-listener check

== Wish List ==

As good ideas are never a waste, and for tracking purposes, please list here all the features you're missing in icecast trunk.

Note: please check that the feature you request is not already in trunk before posting !

* Ponnies
* WebM Streaming
* PUT method support

[[Category:Xiph-related Software]]

User:Basilgohar

2010-10-13T03:49:31Z

Basilgohar: Created my personal page

My name is Basil Mohamed Gohar. For more extensive personal information, you can probably find out a lot more by visiting my [http://www.basilgohar.com personal website].
=Introduction=
I am currently (at least, as of 2010) a fulltime web developer who works for a small consulting firm in Columbus, Ohio. I am blessed to be able to work in an environment where the work I need to do can be achieved almost exclusively with free (as in freedom) software. However, my normal work doesn't require me to work extensively with multimedia, as I am primarily writing, extending, and maintaining web applications written in PHP and running atop MySQL. I do the vast majority of my multimedia work on the side or in my free time. It is here that I find great value in projects like [[Xiph]] and its sponsored technologies, such as [[Ogg]], [[Theora]], & [[Vorbis]], which allow me to create & share media without restrictions or hidden costs.
=Projects=
I try my best to incorporate free formats into my projects as much as possible, as well as use them as a platform to share free media and educate others about free formats and the need for them.
==Libre Video==
''Libre Video'' is a fledgling website/community I started to promote awareness about the issues surrounding free formats, software patents, and the hidden costs of intellectual property.
==Adam to Akhira==
"Adam to Akhira" was a promotional event held in Cleveland, Ohio in January, 2010, featuring a multitude of speakers, including a couple from AlMaghrib Institute. I mention this here because, with the permission (and to the great pleasure) of the organizers, I posted the audio & video recordings of the entire days worth of events online. While perhaps not amazing in terms of quality (the audio-only records came out well enough, though), the entire production process was done exclusively with free software.
===How it was produced===
All the videos were recorded using my very cheap MiniDV camcorder. The video was recorded straight to my laptop in interlaced DV format. I edited these source

mwEmbed is used to enable in-browser video playback without the explicit use of proprietary software, even in browsers that do not yet support the <video> tag from HTML5.
=External Links=
*[http://www.basilgohar.com/blog HidayahTech (my blog)]
*[http://www.librevideo.org Libre Video]

GST cookbook

2010-04-16T21:53:48Z

Basilgohar: Added example of capturing v4l2 video, displaying decoded video locally, saving encoded video to file, and streaming encoded video to Icecast server

In addition to being a powerful multimedia infrastructure for applications Gstreamer is also a useful tool for general manipulations of multimedia data. By invoking gst-launch from the command-line with a custom pipeline many useful processing steps are possible.

Gstreamer also usually has good support for Xiph-related formats.

Unfortunately, it can be rather difficult to figure out an appropriate pipeline without a starting point.

Here are some useful examples: 

===Encode a .wav to Vorbis:===
*gst-launch filesrc location="INPUT.wav" ! wavparse ! audioconvert ! vorbisenc ! oggmux ! filesink location="OUTPUT.ogg"

===Dump a Theora video to PNGs:===
*gst-launch filesrc location="INPUT.ogv" ! oggdemux ! theoradec ! ffmpegcolorspace ! pngenc snapshot=false ! multifilesink location="OUTPUT%04d.png"

===Transmux a MKV (containing vorbis and theora) to Ogg:===
*gst-launch filesrc location="INPUT.mkv" ! matroskademux name=d ! video/x-theora ! queue ! theoraparse ! oggmux name=mux ! filesink location="OUTPUT.ogv" d. ! audio/x-vorbis ! queue ! vorbisparse ! queue ! mux.

===Encode a y4m to lossless Dirac in Ogg:===
*gst-launch filesrc location="INPUT.y4m" ! decodebin ! schroenc force-profile=vc2_main rate-control=lossless ! oggmux ! filesink location="OUTPUT.ogv"

===Pull from a windows media stream, transcode to Ogg/Thera+Vorbis and send to a icecast server:===
(requires purchasing fluendo plugins for decoding the encumbered codecs)
*gst-launch uridecodebin uri=mms://SOURCE.SERVER.COM/path name=d ! queue max-size-time=100000000 ! ffmpegcolorspace ! theoraenc bitrate=800 ! oggmux name=mux ! shout2send ip=YOURICECAST.SERVER.COM port=8000 password=YOURPASSWORD mount=/OUTPUTFILENAME.ogv d. ! queue max-size-time=100000000 ! audioconvert ! vorbisenc ! mux.

===Capture video from a webcam, encode to an Ogg Theora file, decode and display on screen, write to a file whose name is the current date+time, and stream to an IceCast server===
*gst-launch-0.10 --eos-on-shutdown v4l2src ! 'video/x-raw-yuv, width=640, height=480' ! videorate ! 'video/x-raw-yuv, framerate=15/1' ! queue max-size-bytes=100000000 max-size-time=0 ! theoraenc bitrate=150 ! oggmux ! tee name=ogged ! queue max-size-bytes=100000000 max-size-time=0 ! oggdemux ! theoradec ! xvimagesink sync=false force-aspect-ratio=true ogged. ! queue max-size-bytes=100000000 max-size-time=0 ! filesink location=`date +%F_%T`.ogv ogged. ! queue max-size-bytes=100000000 max-size-time=0 ! shout2send ip=YOURICECAST.SERVER.COM port=8000 password=YOURPASSWORD mount=/OUTPUTFILENAME.ogv streamname=YOURSTREAMNAME description=YOURDESCRIPTION genre=YOURGENRE url=YOURSTREAMURL ogged.

Notes on testing theora

2010-04-11T16:55:01Z

Basilgohar: Fixed some typos and/or grammatical mistakes & misspellings, as well as Americanizing one instance of "colour"

This page is still in development, and it's something of a rant at this point in time.

== On testing ==

Testing correctly is enormously difficult. I've yet to see a video codec comparison that didn't have at least one material flaw which could be expected to influence the results. This includes my own testing.

Much of this difficulty comes from the fact that we're trying to measure highly dimensional and highly flexible things in a fair manner. Achieving real fairness is often not possible, because the operating space of two codecs doesn't often completely overlap. How you combine these orthogonal dimensions into one metric of "better" or "worse" is necessarily subjective.

Testing is so difficult that experts in testing, codec developers, and academics have all gotten it wrong. If you're not willing to spend the time to take a serious effort to understand exactly what is happening, then you should probably abandon any hope of performing a rigorous test and instead admit that your test is just a casual one which might only be applicable to your own usage.

All that said, I'm tired of seeing the same mistakes repeated over and over again. So I thought I'd make this list so that when you fail, as you inevitably will, you can at least fail in new and interesting ways. --[[User:Gmaxwell|Gmaxwell]] 21:17, 10 April 2010 (UTC)

== Comparable operating modes ==

By default, libtheora provides either a constant QI encode (target quality), a rate-controlled encode (target bitrate), or a two-pass rate controlled encode (target bitrate). The defaults for each are designed for the situations in which they're expected to be used, and ''not'' for consistent codec testing.

Libtheora's one-pass modes are designed for live streaming, which was its original target use case. Thus, they are "zero" latency and use a short keyframe interval. The one-pass rate-controlled mode is also strictly buffer constrained (constant bit-rate over a small buffering window). This hard CBR requirement significantly hurts quality at a given bitrate but it makes streaming more reliable and lower latency. Encoders for many other formats provide this kind of behavior only if you explicitly ask for it. You can relax, but not disable, this constraint in libtheora if it is not useful for your application, or use a target quality if your application does not require particular bitrates. The frequent keyframes also require a lot more bits, but improve loss robustness and startup times.

Libtheora's two-pass mode does not impose a hard CBR constraint by default, and uses a higher keyframe interval. It's also able to look ahead at the stream to aid its rate control decisions. This is similar to the default behavior of many encoders for other formats. You can still request a hard CBR constraint if it is useful for your application.

You should not compare libtheora's one-pass, one frame in/one frame out, hard CBR mode against other codecs' one-pass VBR mode with several seconds of lookahead. Even though they are both one pass, you are comparing holodecks to oranges. You should also explicitly set the keyframe interval to the same value for every codec you use. Every codec has a different default, and even relatively similar values may give statistically significant differences in the results, depending on how well they allow keyframes to align with scene changes.

=== Profiles ===

Some formats offer multiple profiles, and some devices can only decode some profiles and not others. Arguably this means that they aren't a single format, but are instead a collection of related formats under a common name. Theora has only a single profile.

It isn't reasonable to compare Theora's "one setting works everwhere" to the best characteristics of different profiles of a competing encoder.

If you compare Theora against a high complexity profile that restricts what devices that other format can play on, understand that you've added a subjective factor to the comparison. There isn't anything wrong with this, and it can be impossible to avoid, but it should be understood. You cannot simultaneously compare Theora's quality/bitrate performance against the most complex profile another codec offers and compare its device support to the simplest.

=== Encoder speed levels ===

Although it is fairly fast, libtheora's encoder is not extensively optimized for performance. It's basically been made fast enough to encode streaming resolutions on normal hardware. As of now, current development efforts have gone into improving other parts of the codec. There is a lot of low-hanging fruit in this area.

In particular, speed level 2 is provided as an emergency "make it go fast without regard to bitrate" knob. It disables half of the formats' features as a quick measure to make the encoder faster for some real-time applications. The same kind of speed is possible without sacrificing as much quality/bitrate, but no one has had a chance to write code for that yet.

For a comparison you should probably test libtheora at the default speed. It's completely fair to note that other encoders currently offer additional speed knobs that make them much faster than libtheora. It's not really fair to run libtheora at speed level 2 then measure the quality at low bitrates. Unless you want to measure it against MJPEG. The quality/bitrate at speed level 2 is poor. It's supposed to be. It constrains the format to act similar to MJPEG with limited delta frames.

== General points on "objective" measures like PSNR and SSIM ==
Objective measurements like PSNR and SSIM are tools for measuring "quality" which don't require a human to judge the quality, as the computer does it for you. As such, they can enormously reduce the cost of testing.

Unfortunately, much of the art in codec development comes from the fact that computers are not particular good judges of quality. So what a tool that measures PSNR or SSIM isn't measuring quality, it's measuring PSNR or SSIM. Under particular circumstances these metrics can be shown to correlate well with human quality judgements. But "particular circumstances" do not mean 'all circumstances'— it isn't too difficult to construct modified images that get a great objective measure but look like crap, and it is VERY easy to modify in image in a way which is almost imperceptible but which ruins the objective measures. I'm not aware of any study demonstrating that any of the available objective measures are useful across significantly different compression techniques, but if used correctly they are probably usable as a rough yard stick.

The key there being used correctly. There seems to be a problem with using them correctly, since roughly a majority of comparisons seem to screw this up.

'''If you are comparing PSNR or SSIM and the scores for all your test cases do not converge to a similar very high quality at very high bitrates then you are almost certainly doing something wrong.'''

... or you've found some kind of gross bug in the encoder. Depending on the nature of the bug, your objective measure may not predict human results at all.

=== Offsets ===

The primary cause of getting strange and invalid results from PSNR or SSIM is offset handling.

If you shift a video over a couple of pixels a human observer will hardly notice. But your SSIM or PSNR will become very bad.

If you see this kind of behaviour, ''look'' at the results and see if they agree with the numbers. If the PSNR/SSIM says the video looks no better at 10mbit than it did at 1mbit, but to your eyes it clearly does then your test is broken and useless.

Additionally, different formats have the sub-sampled pixels for chroma at different locations. Theora _always_ uses the MPEG-1/MJPEG convention. MPEG2 has another convention which is used by default by most other codecs. Getting the chroma location wrong doesn't produce a ''visible'' harm on most material, but it does screw up PSNR or SSIM. Many media formats, frameworks, and libraries do not even have the ability to specify which subsampling should be used, and thus do whatever that particular programmer expected would be "normal". Most PC player software, even software focused on modern MPEG formats, always decodes as in the MJPEG style, because it is computationally cheaper. Other software pedantically tries to correct for chroma tagged in one format when it is set up to handle a different one, even though these tags are often wrong. Unless you know exactly what your tools are doing, you are at risk of one of them screwing this up. If you apply objective measures in the original YUV space, difference in handling the chroma offset should be limited to damaging the chroma PSNR/SSIM scores, which makes this error more obvious.

If you are planning on comparing your PSNR or SSIM scores with someone else's, you probably shouldn't bother. There is too much inconsistency between tools and too much likelihood of using somewhat different input clips. This is why its important that you publish your input material— without it people have no hope of reproducing your results, and without reproducing your results the public should have zero trust that you didn't make a simple measurement error.

=== Frame drops ===

If Theora runs out of bits in its hard-bitrate constraint or determines that a frame has almost no motion it will output a zero byte packet. Some decoders simply ignore the zero byte packets, and depend on their normal AV sync mechanisms to keep things well timed. However, if you are dumping the output to a file for analysis with a SSIM/PSNR tool, these 'lost' frames will result in a time-offset and frames will be compared with the wrong source frames. This will have somewhere between a big and a huge negative impact on the scores.

'''If your output doesn't have the same number of frames as the input. Stop. You've done something wrong'''.

This case can also cause the "never gets good at high bitrates" behaviour.

=== Levels ===

Objective measures are also very sensitive to changes in luma levels (brightening / darkening). The best way to avoid this case is to measure in the colorspace that the codec worked in, without conversions. '''Do not convert to RGB before using objective metrics on codecs that operate in YUV''', because you will probably screw it up in a non-obvious way.

=== Input files ===

It's obvious to most people attempting testing that you must begin with the same input for all of the codecs under test. It's less obvious that you may run into problems if you begin with RGB input: Current codecs all operate in some YUV colorspace rather than RGB. If you use RGB files for your inputs differences in the conversion from RGB to YUV between different tools may reduce the objective scores for some formats compared to others.

These differences can even create visible quality differences in corner cases.

== General points on subjective testing ==

=== Codec X has "better colors" ===

Modern lossy codecs don't do anything which should grossly change the overall brightness, hue, or saturation of an image. If the colors look different in your comparison, your software is probably mishandling colorspace conversions. Figure out what is broken and try again.