XiphWiki - User contributions [en]

Chapter Extension

2012-10-12T02:51:44Z

Silvia: make chapter spec simpler

Chapters are a means of providing direct and semantically relevant navigation points for a media file. On particular use case are so-called [http://en.wikipedia.org/wiki/Enhanced_podcast "enhanced podcasts"], i.e. audio files with additional chapter markers.

Chapters are typically provided for a recorded file, not for a live resource.

Since chapters are used for navigation - in particular to avoid having to listen to large amounts of undesired content in order to get to desired content - it is important that the information about chapters is available at the start of a media file to be able to be displayed without having to decode the media file. Therefore, we regard chapters as header-style metadata.

Header-style metadata has traditionally been transported in [[VorbisComment]] headers inside Ogg.

== Format ==

We therefore propose an extensions to VorbisComment for transporting chapters. We introduce VorbisComment fields called CHAPTERxxx and CHAPTERxxxNAME with xxx being a number between 000 and 999. (1000 chapters are assumed to be sufficient.)

The value for the CHAPTERxxx field name is the start time of the chapter (Hour:Min:DecimalSeconds). See the [[#Examples|examples]] below.

The value for the CHAPTERxxxNAME field name is just a text string (8 bit clean UTF-8 encoded values, as is required for VorbisComments).

This should basically support the same input format that [http://savvyadmin.com/adding-chapters-to-videos-using-mkv-containers/ Matroska chapters] support, too.

== Examples ==

An example chapter file with two sequential chapters:

CHAPTER001=00:00:00.000
CHAPTER001NAME=Chapter 1
CHAPTER002=00:05:00.000
CHAPTER002NAME=Chapter 2

== Further Fields ==

Other fields can also be used to support enhanced podcasts:

* a field to extend chapters with a url to navigate to while listening to a chapter of a podcast:

CHAPTERxxxURL=http://...

== WebVTT ==

We expect people may also want support for hierarchically structured chapters with subchapters etc. This specification does not support this need. We recommend making use of WebVTT as a chapter format specification to satisfy this need. At a future time we expect a mapping of WebVTT into Ogg to allow for embedding of such files into Ogg, too.

WebVTT is a more modern means of specifying chapters [http://dev.w3.org/html5/webvtt/#webvtt-file-using-chapter-title-text that use cues with chapter titles] and can deal with time overlapping cues.

Chapter Extension

2012-04-11T12:32:53Z

Silvia: added URLs to chapters

Chapters are a means of providing direct and semantically relevant navigation points for a media file. On particular use case are so-called [http://en.wikipedia.org/wiki/Enhanced_podcast "enhanced podcasts"], i.e. audio files with additional chapter markers.

Chapters are typically provided for a recorded file, not for a live resource.

Since chapters are used for navigation - in particular to avoid having to listen to large amounts of undesired content in order to get to desired content - it is important that the information about chapters is available at the start of a media file to be able to be displayed without having to decode the media file. Therefore, we regard chapters as header-style metadata.

Header-style metadata has traditionally been transported in [[VorbisComment]] headers inside Ogg.

== Format ==

We therefore propose an extensions to VorbisComment for transporting chapters. We introduce VorbisComment fields called CHAPTERxxx and CHAPTERxxxNAME with xxx being a number between 000 and 999. (1000 chapters are assumed to be sufficient.)

The value for the CHAPTERxxx field name is the start time of the chapter (Hour:Min:DecimalSeconds), "-->" (U+002D, U+002D, U+003E), and the end time (Hour:Min:DecimalSeconds). See the [[#Examples|examples]] below.

The value for the CHAPTERxxxNAME field name is just a text string (8 bit clean UTF-8 encoded values, as is required for VorbisComments).

This should basically support the same input format that [http://savvyadmin.com/adding-chapters-to-videos-using-mkv-containers/ Matroska chapters] support, too.

== Examples ==

An example chapter file with two sequential chapters:

CHAPTER001=00:00:00.000-->00:05:00.000
CHAPTER001NAME=Chapter 1
CHAPTER002=00:05:00.000-->00:10:00.000
CHAPTER002NAME=Chapter 2

Another example chapter file with a chapter and three hierarchically structured subchapters:

CHAPTER001=00:00:00.000-->00:06:00.000
CHAPTER001NAME=Chapter 1
CHAPTER002=00:00:00.000-->00:02:00.000
CHAPTER002NAME=Chapter 1.1
CHAPTER003=00:00:02.000-->00:04:00.000
CHAPTER003NAME=Chapter 1.2
CHAPTER004=00:00:04.000-->00:06:00.000
CHAPTER004NAME=Chapter 1.3

== Further Fields ==

Other fields can also be used to support enhanced podcasts:

* a field to extend chapters with a url to navigate to while listening a chapter of a podcast:

CHAPTERxxxURL=http://...

== WebVTT ==

A more modern means of specifying chapters is through [http://dev.w3.org/html5/webvtt/#webvtt-file-using-chapter-title-text WebVTT files that use cues with chapter titles].

VorbisComment

2012-03-05T09:49:53Z

Silvia: added chapter extension

VorbisComment is a base-level [[Metadata]] format initially created for use with Ogg [[Vorbis]]. It has since been adopted in the specifications of
[[Ogg]] encapsulations for other Xiph.Org codecs including [[Theora]], [[Speex]] and [[FLAC]].

The use case for VorbisComment is given as:
<blockquote>
... much like someone jotting a quick note on the bottom of a CDR. It should be a little information to remember the disc by and explain it to others; a short, to-the-point text note that need not only be a couple words, but isn't going to be more than a short paragraph.[http://xiph.org/vorbis/doc/v-comment.html]
</blockquote>

VorbisComments are typically used to provide basic information like the title and copyright holder of a work.
As such the scope is similar to that of ID3 tags used with MP3 files.
VorbisComment is widely supported on [[VorbisHardware|portable Ogg Vorbis players]] as well as streaming, editing and playback software.

Although the syntax of VorbisComment is well-specified, various conventions exist for the field names in use.
The goal for this page is to codify best practices and collect proposals for standardization of VorbisComment field names.

VorbisComments are typically encoded as the second packet in a codec stream. When VorbisComments are included in the first (ie. Theora) stream of an Ogg Theora file, they are assumed to cover all streams in the multiplexed group. [http://lists.xiph.org/pipermail/vorbis-dev/2008-December/019676.html]

VorbisComment is the simplest and most widely-supported mechanism for storing metadata with Xiph.Org codecs. For other existing and proposed mechanisms, see [[Metadata]].

== Recommended field names ==

The current [http://xiph.org/vorbis/doc/v-comment.html VorbisComment recommendation] contains a recommended set
of field names for comments.

== Proposed field names ==

Some proposals for extra field names:

* [http://age.hobba.nl/audio/mirroredpages/ogg-tagging.html Ogg Vorbis Comment Field Recommendations]
* [http://gophernet.org/articles/vorbiscomment/ Proposals for extending Ogg Vorbis comments]
* [[Field names]]
* [[Chapter Extension]]

Comments are intended to be free-form, but for the purposes of interoperability, it is helpful to define tag sets for particular applications, and provide some guidelines for machine parsing. Note that some field names have to be non-free-form to achieve machine parsing.

=== Cover art ===

==== METADATA_BLOCK_PICTURE ====
The [http://flac.sourceforge.net/format.html#metadata_block_picture binary FLAC picture structure] is base64 encoded and placed within a VorbisComment with the tag name "METADATA_BLOCK_PICTURE". This is the preferred and recommended way of embedding cover art within VorbisComments. It has the following benefits:

* Easy to use for developers since the identical (or similar) structure is also used by FLAC and MP3.
* The cover art can either be linked or embedded within the stream.
* Common picture file formats are supported (jpg and png).
* A description may be included and the picture type (front cover, back cover...) and image mime type are provided.
* Base64 encoded data is invariant under UTF-8 and a valid UTF-8 string, so obeys the rules for comment data.

Implementations interpreting or writing picture blocks should note the following details:

===== General encoding/decoding =====
* Failure to decode a picture block should not prevent playback of the file (failure to deal with the particularly large packet required by the comment header is a separate problem with the player implementation).
* Base64 encoding is used as in section 4 of [http://www.faqs.org/rfcs/rfc4648.html RFC4648]. We note that line feeds are not allowed and padding characters ('=') are required.
* Applications adding picture blocks should inform users that some applications or hardware may not support them and should provide a method to remove the blocks (this is expected to be trivial for applications capable of adding them).

===== Block handling =====
* The unencoded format is that of the [http://flac.sourceforge.net/format.html#metadata_block_picture FLAC picture block]. The fields are stored in big endian order as in FLAC, picture data is stored according to the relevant standard.
* Picture data should be stored in PNG or JPEG formats or linked separately. It is recommended readers support both PNG and JPEG
* Allowed values for the MIME string are "image/", "image/png", "image/jpeg" and "-->" (the link indicator) and "" (length 0). An empty MIME string indicates type "image/"
* Fields present in the ID3V2.4.0 [http://www.id3.org/id3v2.4.0-frames#line-1085 Attached Picture Frame] (APIC Frame) take the same interpretation as in the ID3V2.4.0 format with the following exceptions (following the FLAC format):
** The description field is UTF-8 (encoded without ID3V2's initial 'encoding byte')
** String fields are not null terminated: their preceding length fields are used instead.

===== Linked images =====
Support for linked images is optional for applications handling picture blocks. When a linked picture is indicated the following rules are observed:
* The picture data is a complete URL indicating the picture to be used, relative URLs are allowed (note relative URLs do not start with a protocol specifier and are retrieved with the same protocol as the file being processed).
* Links are ISO-8859-1 encoded
* Applications MAY retrieve linked images via the file:// protocol.
* Applications MUST obtain user approval if they wish to retrieve images via remote protocols.
* Link targets may become unavailable: applications supporting linked images SHOULD recover gracefully from this and MAY report the absence to the user.
* The type of the linked file is not restricted to JPEG and JFIF and applications MAY support other formats
* If the application does not support linked images, the target is unavailable, not permitted or an unknown format the picture block should be skipped.
* Applications may make links available to users, this is of particular use when links are unsupported or of unsupported type

===== Image dimension fields =====
* The height, width, colour depth and 'number of colours' fields are for purely informational purposes. Applications MUST NOT use them for decoding purposes, but MAY display them to the user and MAY use them to make a decision whether to skip the block (for example if selecting the most appropriate among multiple blocks).
* Applications writing picture blocks MUST set these fields correctly OR set them all to zero.

===== Multiple blocks =====
* Multiple image blocks MAY be included as separate METADATA_BLOCK_PICTURE comments.
* There may only be one each of picture type (APIC type) 1 and 2 in a Vorbis stream.
* Block order is significant for some types and applications should preserve the comment order when reading or writing VorbisComment headers. The block order may be used to determine the order pictures are presented to the user.

===== Playback tests =====
Embedding a picture into the file might break playback of existing players (especially hardware players, software players could be updated easily). A workaround would be to link the picture within the tag. Furthermore users should become informed in some way that embedding a picture COULD cause problems (as stated above).

In order to test if there are playback problems, there are test files available [http://www.audioranger.com/coverart_mk.ogg here] and [http://www.audioranger.com/coverart_im.ogg here]. You're invited to download one of these test files (or both), test playback on your software and hardware players, and report the results here on the wiki.

'''Tested software players'''
* Audacious 1.5.1: no problem
* foobar2000: no problems
* Gnome: built-in preview playback: no problem
* MediaMonkey: no problems
* Media Player Classic (unicode build) 6.4.9.1: no problem
* RoarAudio: no problems (server and client side)
* Rythmbox 0.11.6: no problem
* Totem 2.24.3: no problem
* VLC 0.9.4/0.9.6: doesn't play
** Patch send to VLC to fix this - should get in 1.0.0
* WinAmp: no problems
* Windows Media Player 11: no problem
* XMPlay 3.4.2: no problem
* Nero ShowTime: no problem
* Songbird 1.8.0: no problem, able to show and edit embedded pictures

'''Tested hardware players'''
* Logitech Squeezebox: Supported as of January 2009 (server version 7.3.3)
* Sandisk Sansa Fuze (Firmware 01.01.22): Hangs up when trying to playback the demo file - had to reset the player
** Note: The "Fuze" can play ogg vorbis files which have embedded pictures from "Easytag"
* Cowon iAudio U3 (Firmware 1.29, 4 GB): works
* Cowon D2: no problem (latest Firmware: 2.59, 8GB Version)
* iRiver E100: no problem (latest Firmware: 1.16 G_U, 8GB Version)
* Samsung YP-R1: no problem (latest Firmware: 3.07, 16GB Version)

'''Tested tag editors'''
* Easytag 2.1.6: can open the file to edit the normal tag fields
* MP3Tag 2.42e: can open the file to edit the normal tag fields
* MP3Tag 2.47b: is able to show and edit embedded pictures

'''Tested other software'''
* Total Recorder: capable to work with artwork according to the specification.

==== Unofficial COVERART field (deprecated) ====
There also exists an unofficial, not well supported comment field named "COVERART". It includes a base64-encoded string of the binary picture data (usually a JPEG file, but this could be a different file format too). The disadvantages are that
* no additional information like a description about the cover art or its type (front cover, back cover etc.) is provided,
* the cover art can't be linked
* the base64 string is displayed within many tag editors as plain text because of their missing support for this "COVERART" field
* it may breaks the playback on hardware players because of a large VorbisComment header
The unofficial "COVERART" field is supported for example by such software as AudioShell (http://www.softpointer.com/AudioShell.htm) - read/write, and Total Recorder (http://www.totalrecorder.com/)- only read.

===== Conversion to METADATA_BLOCK_PICTURE =====
Old "COVERART" tags should be converted to the new METADATA_BLOCK_PICTURE tag (see above for its specification). This conversion is straightforward and is suggested to be done the following way:

* Decode the COVERART tag. A program MAY check the signature of the embedded picture in order to determine whether it is an allowed type. Lossless conversion from disallowed types to allowed types MAY be carried out.
* Fill out the FLAC block with the binary picture data. If the MIME type of the picture is unknown or can't be determined, the MIME type "image/" MAY be used instead. Supplying image dimensions, color depth etc. is optional (see specification above).
* In the absence of other information the picture type 'Other' should be used. Applications may want to allow users to select a default type or specify the type to use.
* Encode the new picture block, remove the COVERART tag from the comments and add the METADATA_BLOCK_PICTURE entry.
* If multiple tags are being converted the order of the METADATA_BLOCK_PICTURE tags should be the same as that of the COVERART tags they are replacing.

=== Date and time ===

The goal is to specify '''one''' standard format for describing date and/or time.

==== ISO proposal ====
The date format for any field describing a date must follow the ISO scheme: YYYY-MM-DD, shortened to just YYYY-MM or simply YYYY.

We have been recommending this usage with the DATE tag for some time. It is proposed that the spec be amended to include this information for machinability.

The time format for any field '''except''' track duration must be specified with leading T and ending with a time zone. Schemas with and without dates:
<pre>
YYYY-MM-DDTHH:MM:SS+TS
</pre>
<pre>
THH:MM+TZ
</pre>

=== ENCODER ===

The goal is to attribute encoder software. This value can be used in the future to determine which files can be improved by being re encoded with a newer version.

:'''Comment''': What is lacking from the vendor string present in the spec from the start? All libvorbis and encoder tunings I'm aware of have recorded the encoder version here.

Rationale for not using the vendor string:
* The vendor string is usually used to store the name and version of the underlying codec library
* The idea of ENCODER is to store the name of the user-visible application, for example <tt>ffmpeg2theora</tt>.
* It can be useful for debugging to store the name and version of the calling application.
* The libvorbis API does not let applications override the vendor string.

==== Proposal: Inclusion of URL in ENCODER value ====
The encoder field name must be a unique URL providing both encoder software name and version. If no unique URL address is available were both name and version is available; then the version number can be specified by separating with a space character. For example:

<nowiki>ENCODER=http://flac.sourceforge.net/ 1.2.1</nowiki>

* Note that ffmpeg2theora uses ENCODER, but does not include a url. ''Added by Rillian on September 17, 2007''

==== Proposal: ENCODED_BY ====

I've also seen ENCODED_BY. ''Added by Rillian on September 17, 2007''
: ENCODED_BY is usually the person who did the encoding. This should not be part of the recommendation due to legal problems around deliberate and accidental distribution to third parties. Basically the name of the encoder should not be included to protect encoders from their own egos and possible legal prosecution. ''Added by Aleksandersen on September 20, 2007''

=== Improving license data ===

The goal is to provide a method for proclaiming license and copyright information (basically clarifying ‘distribution rights (if any) and ownership’).

The [http://xiph.org/vorbis/doc/v-comment.html specification document] describes LICENSE and COPYRIGHT fields. But is not clear enough about whether these should be machine-readable.

We should consider working together with Creative Commons to have complementary and interlinked information on the Creative Commons and Xiph wikis. Refer to the [http://wiki.creativecommons.org/Ogg Ogg page] in the Creative Commons wiki.

==== New RIGHTS field name proposal ====
One proposal is to replace the COPYRIGHT and LICENSE field names with RIGHTS. RIGHTS must be a human-readable copyright statement. Basic example:

<nowiki>RIGHTS=Copyright © Recording Company Inc. All distribution rights reserved.</nowiki>

But this is not machine-readable. Adding two complementary field names should do the trick: RIGHTS-DATE, describing the date of copyright; and RIGHTS-URI, providing a method for linking to a license. Software agents can assume that multiple songs uses the sameURIs, such as in the case for Creative Commons. Full example:

<nowiki>RIGHTS=Copyright © 2019 Recording Company Inc. All distribution rights reserved.</nowiki><br />
<nowiki>RIGHTS-DATE=2019-04</nowiki><br />
<nowiki>RIGHTS-URI=http://somewhere.com/license.xhtml</nowiki>

Software such as for multimedia management and playback are encouraged to display the RIGHTS statement as a linked phrase using RIGHTS-URI.

RIGHTS-DATE does not need to be displayed as it is required in the human readable version by international copyright agreements. RIGHTS-DATE can be used to determine when a copyrighted work falls under the public domain and related matters. (''The Beatles''' copyright on their original studio recordings (not the remixes) are soon expiering. So mechanisms such as the RIGHTS-DATE are indeed required in music management and filesharing software!)

To remain machine-readable it would be required to have at most one instance of each RIGHTS field name. All fields would of course remain optional.

The ''Dublin Core Metadata Initiative'' recommends the use of ‘rights’ to describe license and copyright matters. The web feed format Atom 1.0 has implemented a rights element in their specification.

:'''Comment''': The triplet RIGHTS, RIGHTS-DATE, RIGHTS-URI is an example of structured metadata. VorbisComments are inherently unstructured, and this should be respected. Structured metadata belongs in a different stream, such as XML (using [[Metadata#XMLEmbedding|XMLEmbedding]]).

==== Improving existing fields proposal ====
Similar to the DATE tag above, we have generally recommended that a URL uniquely identifying the license be included in the LICENSE field to allow machine identification of the license. This is in agreement with the proposal in the Creative Commons wiki. Since the COPYRIGHT field is a human-readable statement of the copyright, like the proposed RIGHTS tag above, some people include a license url there. Therefore if a url can't be found in a LICENSE tag if any, applications should use one from the COPYRIGHT tag, if any. Contact information for verification, attribution, relicensing, etc. can be obtained from the COPYRIGHT field, but Creative Commons also recommend a separate CONTACT tag for this information. This is reasonable, so we propose it be included.

=== Geo Location fields ===

The existing LOCATION field is meant to carry a human readable location for the recording/creation of the media file.

Having geographical coordinates according to [http://en.wikipedia.org/wiki/World_Geodetic_System WGS84] can be useful as well, especially in a form that can be machine parsed. The agreed format is similar to this [http://en.wikipedia.org/wiki/Geo_(microformat) geo microformat]:

GEO_LOCATION= ''latitude'' ; ''longitude'' [; ''elevation'' ]

where each value is a fixed point decimal number formatted in the C locale with a period (.) for the radix. Values are separated with a ';' and white space is not significant. The elevation is optional.

''latitude'' is the geo latitude location of where the media has been recorded or produced in decimal degrees according to WGS84 (zero at the equator, negative values for southern latitudes) (C double).

''longitude'' is the geo longitude location of where the media has been recorded or produced in decimal degrees according to WGS84 (zero at the prime meridian in Greenwich/UK, negative values for western longitudes). (C double).

''elevation'' is the geo elevation of where the media has been recorded or produced in meters according to WGS84 (zero is average sea level) (C double).

=== Replay Gain ===

The REPLAYGAIN_* fields implement the Replay Gain proposal for storing a track relative volume adjustment, which can be used to "fix" quiet or loud sounding Vorbis or FLAC streams. The set of tags is intended to be machine parsed, and has the following form:
<pre>
REPLAYGAIN_TRACK_GAIN=-7.03 dB
REPLAYGAIN_TRACK_PEAK=1.21822226
REPLAYGAIN_ALBUM_GAIN=-6.37 dB
REPLAYGAIN_ALBUM_PEAK=1.21822226
</pre>

See http://www.replaygain.org/ for detailed information about Replay Gain and how the different values are calculated.

=== Tantalos resource ID ===
Tantalos is a protocol to auto locate and access content in a network scope (normally 'LAN' but can also be bigger like 'company network'). It uses UUIDs to identify resources. There are two groups of those UUIDs: IDs generated using the meta data of the track and IDs generated in some other way (for example random IDs, see UUID specs). The later group may need to be passed with the track. The VCLT Playlist format (see below) uses 'HASH={UUID}id...' with hex-dash format for this (HASH={UUID}e278173d-4d6d-4c66-95ec-4ec85eedc7d1).
--[[User:Ph3-der-loewe|Ph3-der-loewe]] 02:05, 13 December 2011 (PST)

== Other (non-proposed field names) ==
=== VCLT playlist format ===
The VCLT playlist format uses some ''keys'' which look like VorbisComments but they aren't nor they are proposed to become (expect for HASH). This includes the keys STREAMURL, FILENAME, FILEURL, LENGTH, HASH (see above for this one), SIGNALINFO, AUDIOINFO and OFFSET. You can find more infos about those at [https://bts.keep-cool.org/wiki/Specs/VCLT https://bts.keep-cool.org/wiki/Specs/VCLT].

== Implementations ==

* [http://sbooth.org/importers/ OggImporter] – imports Ogg Vorbis and Ogg FLAC files to Spotlight.
* vorbiscomment – a commandline tool, part of [[VorbisTools]].
* [http://www.xiph.org/oggz/ oggz-comment] – a commandline tool, part of [[Oggz]] tool. It can add comments to multitrack and video files.

Chapter Extension

2012-03-05T09:48:48Z

Silvia: started specification

Chapters are a means of providing direct and semantically relevant navigation points for a media file. On particular use case are so-called http://en.wikipedia.org/wiki/Enhanced_podcast "enhanced podcasts"], i.e. audio files with additional chapter markers.

Chapters are typically provided for a recorded file, not for a live resource.

Since chapters are used for navigation - in particular to avoid having to listen to large amounts of undesired content in order to get to desired content - it is important that the information about chapters is available at the start of a media file to be able to be displayed without having to decode the media file. Therefore, we regard chapters as header-style metadata.

Header-style metadata has traditionally been transported in VorbisComment headers inside Ogg.

We therefore propose an extensions to VorbisComment for transporting chapters. We introduce VorbisComment fields called CHAPTERxxx and CHAPTERxxxNAME with xxx being a number between 000 and 999. We assume 1000 chapters to be sufficient.

An example chapter file with two sequential chapters:

CHAPTER001=00:00:00.000-00:05:00.000
CHAPTER001NAME=Chapter 1
CHAPTER002=00:05:00.000-00:10:00.000
CHAPTER002NAME=Chapter 2

Another example chapter file with a chapter and three hierarchically structured subchapters:

CHAPTER001=00:00:00.000-00:06:00.000
CHAPTER001NAME=Chapter 1
CHAPTER002=00:00:00.000-00:02:00.000
CHAPTER002NAME=Chapter 1.1
CHAPTER003=00:00:02.000-00:04:00.000
CHAPTER003NAME=Chapter 1.2
CHAPTER004=00:00:04.000-00:06:00.000
CHAPTER004NAME=Chapter 1.3

This should basically support the same input format that [http://savvyadmin.com/adding-chapters-to-videos-using-mkv-containers/ Matroska chapters] support, too.

A more modern means of specifying chapters is through [http://dev.w3.org/html5/webvtt/#webvtt-file-using-chapter-title-text WebVTT files that use cues with chapter titles].

CMML

2011-08-26T01:18:01Z

Silvia: make Kate a link

= NOTE: CMML is deprecated; Xiph recommends you use [[Kate]] instead =

'''CMML''' stands for <b>Continuous Media Markup Language</b> and is to audio or video what HTML is to text. CMML is essentially a timed text codec. It allows to structure a time-continuously sampled data file by dividing it into temporal sections (so-called <i>clips</i>) and provides these clips with some additional information. This information is HTML-like and is essentially a textual representation of the audio or video file. CMML enables textual searches on these otherwise binary files.

CMML is appropriate for use with all [[Ogg]] media formats, to provide subtitles and timed metadata. This description gives a quick introduction only and explains how to map CMML into Ogg. For full specifications, see [http://www.annodex.net/specifications.html http://www.annodex.net/specifications.html].

== CMML specification ==

Before describing the actual data that goes into a logical Ogg bitstream, we need to understand what the stand-alone "codec" packets contains.

CMML basically consists of:

* a <b>head</b> tag which contains information for the complete audio/video file
* a set of <b>clip</b> tags which each contains information on a temporal section of the file
* for authoring purposes, CMML also allows a <b>stream</b> tag which spcifies the file it describes

An example CMML file looks like this:

<pre>
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<!DOCTYPE cmml SYSTEM "cmml.dtd">

<cmml lang="en" id="simple" granulerate="1000/1">

<stream id="fish" basetime="0">
<import id="videosrc" lang="en" title="Video fish"
granulerate="25/1" contenttype="video/ogg"
src="fish.ogv" start="0" end="360">
<param id="vheight" name="video.height" value="250"/>
<param id="vwidth" name="video.width" value="180"/>
</import>
</stream>

<head>
<title>Types of fish</title>
<meta name="Producer" content="Joe Ordinary"/>
<meta name="DC.Author" content="Joe's friend"/>
</head>

<clip id="intro" start="0">
<a href="http://www.example.com/fish.html">Read more about fish</a>
<desc>This is the introduction to the film Joe made about fish.</desc>
</clip>

<clip id="dolphin" start="npt:3.5" end="npt:5:5.9">
<img src="dolphin.jpg"/>
<desc>Here, Joe caught sight of a dolphin in the ocean.</desc>
<meta name="Subject" content="dolphin"/>
</clip>

<clip id="goldfish" start="npt:5:5.9">
<a href="http://www.example.com/morefish.anx?id=goldfish">More video clips on goldfish.</a>
<img src="http://www.example.com/goldfish.jpg"/>
<desc>Joe has a fishtank at home with many colourful fish. The common goldfish is one of them and Joe's favourite.
Here are some fabulous pictures he has taken of them.</desc>
<meta name="Location" content="Joe's fishtank"/>
<meta name="Subject" content="goldfish"/>
</clip>

</cmml>
</pre>

The head element is a standard head element from html.

Clips contain (amongst others) the following information:

* a name in the <b>id</b> attribute so addressing of the clips is possible, as in http://www.example.com/morefish.anx?id=goldfish (Web server needs to [http://annodex.net/software/mod_annodex/ support] this)
* a <b>start</b> and possibly an <b>end</b> attribute, to tell the clip where it is temporally located
* a <b>title</b> attribute to give it a short description
* <b>meta</b> elements to provide it with structed meta data as name-value pairs
* a <b>img</b> element which links to a picture that represents the content of the clip visually
* a <b>a</b> element which puts a hyperlink to another Web resource into the clip
* a <b>desc</b> element giving a long, free-text description/annotation/transcription for the clip

Most of this information is optional.

== CMML mapping into Ogg ==

When CMML is mapped into an Ogg logical bitstream it needs to be serialised first. XML is a hierarchical file format, so is not generally serialisable. However, CMML has been designed to be serialised easily.

CMML is serialised by having some initial header packets that set up the CMML decoding environment, and contain header type information. The content packets of a CMML logical bitstream then consists of <b>clip</b> tags only. The <b>stream</b> tag is not copied into the CMML bitstream as it controls the authoring only.

All of the CMML bitstream information is text. As it gets encoded into a binary bitstream, an encoding format has to be specified. To simplify things, UTF-8 is defined as the mandatory encoding format for all data in a CMML binary bitstream. Also, the encoding process MUST ensure that newline characters are represented as LF (or "\n" in C) only and replace any new line representations that come as CR LF combinations (or "\r\n" in C) with LF only.

The media mapping for CMML into Ogg is as follows:
* The bos page contains a CMML ident packet.
* The first secondary header packet of CMML contains the xml preamble.
* The second secondary header packet contains the CMML "head" tag.
* The content or data packets for CMML contain the CMML "clip" tags each encoded in their own packet and inserted at the accurate time.
* The eos page contains a packet with an empty clip tag.

=== The CMML ident header packet ===

The CMML logical bitstream starts with an ident header which is encapsulated into the CMML bos page. The ident header contains all information required to identify the CMML bitstream and to set up a CMML decoder. It has the following format:

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1| Byte
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Identifier 'CMML\0\0\0\0' | 0-3
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | 4-7
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Version major | Version minor | 8-11
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Granulerate numerator | 12-15
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | 16-19
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Granulerate denominator | 20-23
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | 24-27
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Granuleshift | 28
+-+-+-+-+-+-+-+-+
| ...

The CMML <i>version</i> as described here is major=2 minor=1.

The <i>granulerate</i> represents the temporal resolution of the logical bitstream in Hz given as a rational number in the same way as the [[OggSkeleton]] fisbone secondary header specifies granulerate. It enables a mapping of granule position of the data pages to time by calculating "granulepos / granulerate".

The default granule rate for CMML is: 1/1000.

The <i>granuleshift</i> is a 1 Byte integer number describing whether to partition the granule_position into two for the CMML logical bitstream, and how many of the lower bits to use for the partitioning. The upper bits then still signify a time-continuous granule position for a directly decodable and presentable data granule. The lower bits allow for specification of the granule position of a previous CMML data packet (i.e. "clip" element), which helps to identify how much backwards seeking is necessary to get to the last and still active "clip" element (of the given track). The granuleshift is therefore the log of the maximum possible clip spacing.

The default granule shift used is 32, which halfs the granule position to allow for the backwards pointer.

=== The CMML secondary header packets ===

The CMML secondary headers are a sequence of two packets that contain the CMML and XML "setup" information:
* one packet with the CMML xml preamble and <b>cmml</b> tag.
* one packet with the CMML <b>head</b> tag.

These packets contain textual, not binary information.

The CMML preamble tags are all single-line tags, such as the xml processing instruction (<?xml...>) and the document type declaration (<!DOCTYPE...>).

The only CMML tag that is not already serialized from a CMML file is the <b>cmml</b> tag, as it encloses all the other content tags. To serialise it, the <b>cmml</b> start tag is transformed into a processing instruction, retaining all its attributes (<?cmml ...>), and the <b>cmml</b> end tag is deleted.

The first CMML secondary header packet has the following format:

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1| Byte
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| <?xml ... | 0-
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| <!DOCTYPE ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| <?cmml ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The second CMML secondary header packet contains the CMML <b>head</b> element with all its attributes and other containing elements and has the following format.

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1| Byte
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| <head ... | 0-
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| </head> |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

=== The CMML data packets ===

The data packets of the CMML bitstream contain the CMML <b>clip</b> elements. Their <b>start</b> and <b>end</b> attributes however only exist for authoring purposes and are not copied into the bitstream (to avoid contradictory information), but are rather represented through the time mapping of the encapsulation format that interleaves CMML data with data from other time-continuous bitstreams. Generally the time mapping is done through some timestamp representation and through the position in the stream.

A <b>clip</b> tag is encoded with all tags (except for the <b>start</b> and <b>end</b> attributes) as a string printed into a clip packet. The <b>clip</b> tag's <b>start</b> attribute tells the encapsulator at what time to insert the clip packet into the bitstream. If an <b>end</b> attribute is present, it leads to the creation of another clip packet, unless another clip packet starts on the same track beforehand. This clip packet contains an "empty" <b>clip</b> tag, i.e. a <b>clip</b> tag without <b>meta</b>, <b>a</b>, <b>img</b> or <b>desc</b> elements and no attribute values except for a copy of the <b>track</b> attribute from the original <b>clip</b> tag.

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1| Byte
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| <clip ... | 0-
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| </clip> |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

== Development ==

Ogg CMML is being supported by the following projects:
* the Ogg Directshow filters: see [http://www.illiminable.com/ogg/ illiminable]
* liboggz: [http://svn.annodex.net/liboggz/ liboggz svn] or [http://annodex.net/software/liboggz/ liboggz]
* libcmml: [http://svn.annodex.net/libcmml/ libcmml svn] or [http://annodex.net/software/libcmml/ libcmml]
* libannodex: [http://svn.annodex.net/libannodex/ libannodex svn] or [http://annodex.net/software/libannodex/ libannodex]
* the Annodex technology: [http://www.annodex.net/ annodex.net] including perl, python, php bindings, a firefox plugin, authoring software etc.

== External links ==

* CMML is described in more detail in the CMML v2.1 specification: [http://svn.annodex.net/standards/ I-D in svn] or [http://annodex.net/specifications.html I-D]

[[Category:Ogg Mappings]]

CMML

2011-08-26T01:17:09Z

Silvia: deprecated CMML

= NOTE: CMML is deprecated; Xiph recommends you use Kate instead =

'''CMML''' stands for <b>Continuous Media Markup Language</b> and is to audio or video what HTML is to text. CMML is essentially a timed text codec. It allows to structure a time-continuously sampled data file by dividing it into temporal sections (so-called <i>clips</i>) and provides these clips with some additional information. This information is HTML-like and is essentially a textual representation of the audio or video file. CMML enables textual searches on these otherwise binary files.

CMML is appropriate for use with all [[Ogg]] media formats, to provide subtitles and timed metadata. This description gives a quick introduction only and explains how to map CMML into Ogg. For full specifications, see [http://www.annodex.net/specifications.html http://www.annodex.net/specifications.html].

== CMML specification ==

Before describing the actual data that goes into a logical Ogg bitstream, we need to understand what the stand-alone "codec" packets contains.

CMML basically consists of:

* a <b>head</b> tag which contains information for the complete audio/video file
* a set of <b>clip</b> tags which each contains information on a temporal section of the file
* for authoring purposes, CMML also allows a <b>stream</b> tag which spcifies the file it describes

An example CMML file looks like this:

<pre>
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<!DOCTYPE cmml SYSTEM "cmml.dtd">

<cmml lang="en" id="simple" granulerate="1000/1">

<stream id="fish" basetime="0">
<import id="videosrc" lang="en" title="Video fish"
granulerate="25/1" contenttype="video/ogg"
src="fish.ogv" start="0" end="360">
<param id="vheight" name="video.height" value="250"/>
<param id="vwidth" name="video.width" value="180"/>
</import>
</stream>

<head>
<title>Types of fish</title>
<meta name="Producer" content="Joe Ordinary"/>
<meta name="DC.Author" content="Joe's friend"/>
</head>

<clip id="intro" start="0">
<a href="http://www.example.com/fish.html">Read more about fish</a>
<desc>This is the introduction to the film Joe made about fish.</desc>
</clip>

<clip id="dolphin" start="npt:3.5" end="npt:5:5.9">
<img src="dolphin.jpg"/>
<desc>Here, Joe caught sight of a dolphin in the ocean.</desc>
<meta name="Subject" content="dolphin"/>
</clip>

<clip id="goldfish" start="npt:5:5.9">
<a href="http://www.example.com/morefish.anx?id=goldfish">More video clips on goldfish.</a>
<img src="http://www.example.com/goldfish.jpg"/>
<desc>Joe has a fishtank at home with many colourful fish. The common goldfish is one of them and Joe's favourite.
Here are some fabulous pictures he has taken of them.</desc>
<meta name="Location" content="Joe's fishtank"/>
<meta name="Subject" content="goldfish"/>
</clip>

</cmml>
</pre>

The head element is a standard head element from html.

Clips contain (amongst others) the following information:

* a name in the <b>id</b> attribute so addressing of the clips is possible, as in http://www.example.com/morefish.anx?id=goldfish (Web server needs to [http://annodex.net/software/mod_annodex/ support] this)
* a <b>start</b> and possibly an <b>end</b> attribute, to tell the clip where it is temporally located
* a <b>title</b> attribute to give it a short description
* <b>meta</b> elements to provide it with structed meta data as name-value pairs
* a <b>img</b> element which links to a picture that represents the content of the clip visually
* a <b>a</b> element which puts a hyperlink to another Web resource into the clip
* a <b>desc</b> element giving a long, free-text description/annotation/transcription for the clip

Most of this information is optional.

== CMML mapping into Ogg ==

When CMML is mapped into an Ogg logical bitstream it needs to be serialised first. XML is a hierarchical file format, so is not generally serialisable. However, CMML has been designed to be serialised easily.

CMML is serialised by having some initial header packets that set up the CMML decoding environment, and contain header type information. The content packets of a CMML logical bitstream then consists of <b>clip</b> tags only. The <b>stream</b> tag is not copied into the CMML bitstream as it controls the authoring only.

All of the CMML bitstream information is text. As it gets encoded into a binary bitstream, an encoding format has to be specified. To simplify things, UTF-8 is defined as the mandatory encoding format for all data in a CMML binary bitstream. Also, the encoding process MUST ensure that newline characters are represented as LF (or "\n" in C) only and replace any new line representations that come as CR LF combinations (or "\r\n" in C) with LF only.

The media mapping for CMML into Ogg is as follows:
* The bos page contains a CMML ident packet.
* The first secondary header packet of CMML contains the xml preamble.
* The second secondary header packet contains the CMML "head" tag.
* The content or data packets for CMML contain the CMML "clip" tags each encoded in their own packet and inserted at the accurate time.
* The eos page contains a packet with an empty clip tag.

=== The CMML ident header packet ===

The CMML logical bitstream starts with an ident header which is encapsulated into the CMML bos page. The ident header contains all information required to identify the CMML bitstream and to set up a CMML decoder. It has the following format:

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1| Byte
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Identifier 'CMML\0\0\0\0' | 0-3
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | 4-7
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Version major | Version minor | 8-11
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Granulerate numerator | 12-15
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | 16-19
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Granulerate denominator | 20-23
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | 24-27
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Granuleshift | 28
+-+-+-+-+-+-+-+-+
| ...

The CMML <i>version</i> as described here is major=2 minor=1.

The <i>granulerate</i> represents the temporal resolution of the logical bitstream in Hz given as a rational number in the same way as the [[OggSkeleton]] fisbone secondary header specifies granulerate. It enables a mapping of granule position of the data pages to time by calculating "granulepos / granulerate".

The default granule rate for CMML is: 1/1000.

The <i>granuleshift</i> is a 1 Byte integer number describing whether to partition the granule_position into two for the CMML logical bitstream, and how many of the lower bits to use for the partitioning. The upper bits then still signify a time-continuous granule position for a directly decodable and presentable data granule. The lower bits allow for specification of the granule position of a previous CMML data packet (i.e. "clip" element), which helps to identify how much backwards seeking is necessary to get to the last and still active "clip" element (of the given track). The granuleshift is therefore the log of the maximum possible clip spacing.

The default granule shift used is 32, which halfs the granule position to allow for the backwards pointer.

=== The CMML secondary header packets ===

The CMML secondary headers are a sequence of two packets that contain the CMML and XML "setup" information:
* one packet with the CMML xml preamble and <b>cmml</b> tag.
* one packet with the CMML <b>head</b> tag.

These packets contain textual, not binary information.

The CMML preamble tags are all single-line tags, such as the xml processing instruction (<?xml...>) and the document type declaration (<!DOCTYPE...>).

The only CMML tag that is not already serialized from a CMML file is the <b>cmml</b> tag, as it encloses all the other content tags. To serialise it, the <b>cmml</b> start tag is transformed into a processing instruction, retaining all its attributes (<?cmml ...>), and the <b>cmml</b> end tag is deleted.

The first CMML secondary header packet has the following format:

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1| Byte
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| <?xml ... | 0-
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| <!DOCTYPE ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| <?cmml ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The second CMML secondary header packet contains the CMML <b>head</b> element with all its attributes and other containing elements and has the following format.

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1| Byte
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| <head ... | 0-
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| </head> |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

=== The CMML data packets ===

The data packets of the CMML bitstream contain the CMML <b>clip</b> elements. Their <b>start</b> and <b>end</b> attributes however only exist for authoring purposes and are not copied into the bitstream (to avoid contradictory information), but are rather represented through the time mapping of the encapsulation format that interleaves CMML data with data from other time-continuous bitstreams. Generally the time mapping is done through some timestamp representation and through the position in the stream.

A <b>clip</b> tag is encoded with all tags (except for the <b>start</b> and <b>end</b> attributes) as a string printed into a clip packet. The <b>clip</b> tag's <b>start</b> attribute tells the encapsulator at what time to insert the clip packet into the bitstream. If an <b>end</b> attribute is present, it leads to the creation of another clip packet, unless another clip packet starts on the same track beforehand. This clip packet contains an "empty" <b>clip</b> tag, i.e. a <b>clip</b> tag without <b>meta</b>, <b>a</b>, <b>img</b> or <b>desc</b> elements and no attribute values except for a copy of the <b>track</b> attribute from the original <b>clip</b> tag.

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1| Byte
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| <clip ... | 0-
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| </clip> |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

== Development ==

Ogg CMML is being supported by the following projects:
* the Ogg Directshow filters: see [http://www.illiminable.com/ogg/ illiminable]
* liboggz: [http://svn.annodex.net/liboggz/ liboggz svn] or [http://annodex.net/software/liboggz/ liboggz]
* libcmml: [http://svn.annodex.net/libcmml/ libcmml svn] or [http://annodex.net/software/libcmml/ libcmml]
* libannodex: [http://svn.annodex.net/libannodex/ libannodex svn] or [http://annodex.net/software/libannodex/ libannodex]
* the Annodex technology: [http://www.annodex.net/ annodex.net] including perl, python, php bindings, a firefox plugin, authoring software etc.

== External links ==

* CMML is described in more detail in the CMML v2.1 specification: [http://svn.annodex.net/standards/ I-D in svn] or [http://annodex.net/specifications.html I-D]

[[Category:Ogg Mappings]]

MediaWiki:Sidebar

2010-11-22T23:20:27Z

Silvia: updated skeleton link

__NOTOC__
__NOEDITSECTION__
<div class="portlet">
<font size="+1">'''[[Main Page]]'''</font>
===<font color="black">Xiph.Org Projects</font>===
===Audio—===
*[[Vorbis]]
*[[FLAC]]
*[[Speex]]
*[[CELT]]
===Video—===
*[[Theora]]
*[[Dirac]]
===Text—===
*[[XSPF]]
*[[CMML]]
*[[Kate]]
===Container—===
*[[Ogg]]
*[[Ogg Skeleton 4|Skeleton]]
===Streaming—===
[[Icecast]]
</div>

Main Page

2010-11-22T02:09:04Z

Silvia: moved to skeleton 4

In an effort to bring open-source ideals to the world of multimedia the [[Xiph.Org Foundation]] develops a multitude of amazing products. This wiki describes our free and open protocols and software.

----

= Demonstrations of Xiph technologies =

Want to hear or see Xiph in action? These projects are using our codecs, formats, or libraries.

* [[VorbisStreams|Vorbis Streams]]: Stations streaming with the [[Vorbis]] codec
* [[Games that use Vorbis]]: Games using the Vorbis codec for music or sound effects
* [[VorbisHardware|Vorbis Hardware]]: Hardware players using the Vorbis codec
* [[VorbisSoftwarePlayers|Vorbis Software Players]]: list of media players with out-of-box support for Vorbis
* [[TheoraHardware|Theora Hardware]]: Hardware using the Theora video codec
* [[TheoraSoftwarePlayers|Theora Software Players]]: list of media players with Theora support
* [[List of Theora videos]]: Sources for video encoded with [[Theora]]

= Projects/Formats =

== Container Formats ==

* [[Ogg]]: Media container. This is our native format and the recommended container for Xiph codecs.
** [[Ogg Skeleton 4]]: Skeleton information on all logical content bitstreams in Ogg.
** [[MIMETypesCodecs|Specification of MIME types and respective codecs parameter]]
* [[SpeexRTP]]: RTP payload format for voice
* [[VorbisRTP]]: RTP payload format for general audio
* [[TheoraRTP]]: RTP payload format for video
* [[XSPF]]: XML Sharable Playlist Format

== Codecs ==

* '''Compressed Audio/Video Codecs:'''
** [[Vorbis]]: Audio codec with a [[Tremor|fixed point decoder]]
** [[Theora]]: Video codec
** [[FLAC]]: Free Lossless Audio Codec
** [[Speex]]: Speech codec
* '''Uncompressed Audio/Video Codecs:'''
** [[OggPCM]]: Audio codec
* '''Timed Text/Metadata Codecs:'''
** [[CMML]]: Continuous Media Markup Language, used for [http://www.annodex.net/ Annodex] and subtitles (xine, vlc, gstreamer, and DirectShow support)
**[[OggKate|Kate]]: new format for lyrics and subtitles

== Software ==

* '''Software for distributing media'''
** [[Icecast]]: Streaming server
** [[Ices]]: Source client for Icecast servers

* '''Libraries'''
** [[OggPlay]]: library for synchronised Xiph media playback
**[[XiphQT]]: Quicktime component to play the main Xiph formats
** [[VorbisCommentEdit]]: Macintosh Framework making it easy to incorporate the editing of [[VorbisComment|Vorbis Comments]]

* '''Other software'''
** [[OggComponent/VorbisComponent]]: Wrappers to integrate Vorbis into Mac OS X (does not yet support encoding)
** [http://xiph.org/paranoia/ cdparanoia]: CDDA extractor/ripper

== Community ==

*[[How to help]]
*[[Spread Open Media]]: project to promote Xiph formats.
**[[MailOgging]]: provides templates for anyone willing to contact a company requesting them to add support for Xiph formats.
*[[People]]: Who's who in Xiph.

== Work in Progress ==
* [[Work In Progress]]: codecs and software still in the research and development stages.
* [[Todo]]: To-do list for various Xiph projects.

= Project management =

* [[AdminProcesses]]: who's in charge of what project
* [[MonthlyMeeting]]: page with information on Xiph's MonthlyMeeting
* [[MailingLists]]: list of Xiph's mailing lists
* [[Bounties]]: list of bounties that you can take to improve Xiph's projects

= Resources for Video and Audio programmers =

* [[Ambisonics]]: page with technical information on Ambisonics
* [[Resources and papers on Audio, Music and Speech|Courses and papers on Audio, Music and Speech]]: page with links to MIT and other universities' content
* [[Oggless]]: for ideas on how to use the different Xiph codecs outside Ogg

= Wiki internal =

* [[Translations]]: What about some translation work
* [[Sandbox]]: Testbed for testing editing skills
* [[XiphWiki:Copyrights]]: License used for all content on the XiphWiki

TheoraSoftwareEncoders

2010-04-22T12:32:36Z

Silvia: added directshow filters - not sure why they were missing

== Multi-platform ==
*[http://www.v2v.cc/~j/ffmpeg2theora/ ffmpeg2theora] a commandline encoder from any format read by ffmpeg to Theora/Vorbis: [http://svn.xiph.org/trunk/ffmpeg2theora/ svn], [http://www.v2v.cc/~j/ffmpeg2theora/download.html major releases], [http://firefogg.org/nightly/ very latest versions of ffmpeg2theora and some more stuff]
* [http://diracvideo.org/download/ffmpeg2dirac/ ffmpeg2dirac] - fork of ffmpeg2theora, can enode into OGG Dirac but also Theora
*[http://www.videolan.org/ VLC Media Player] Can transcode from any source it supports into Ogg/Theora. WARNING: Apparently creates broken Ogg streams.
*[http://gstreamer.freedesktop.org/ GStreamer] GStreamer is a library that allows the construction of graphs of media-handling components, ranging from simple Vorbis playback to complex audio (mixing) and video (non-linear editing) processing.
*[http://sarava.org/theorur/ Theorur] is a GUI for Ogg/Theora streaming (icecast2 system), written using gtk2.
*[http://handbrake.fr Handbrake] is a GUI/CLI free software for ripping/encoding DVD/Files into various containers and formats including theora & vorbis since September 2008.
*[http://firefogg.org/ firefogg] is a Firefox extension that encodes locally and uploads in chunks or when encoding finishes

== Windows ==
*[http://dir.visonair.tv/streamer.php Visonair.tv Ogg Streamer] A Windows application to stream directly from a webcam to an Icecast server.
*[http://www.visonair.tv/ Visonair.tv Virtual Stage] Includes an application to encode to Theora, forces fixed size and encoding parameters though. Registration required.
*[http://www.freewarefiles.com/program_6_227_33306.html GFrontend] GUI Frontend for ffmpeg2theora. An unsupported / discontinued open source project.
*[http://sourceforge.net/projects/theoraconverter/ Theora Converter .NET] A GUI frontend for ffmpeg2theora based on GFrontend. Supports 2 pass encoding with ffmpeg2theora 0.26.
*[http://mediacoder.sourceforge.net/ MediaCoder] Application to encode media files into many target formats, including Theora.
*[http://www.erightsoft.com/ SUPER] General purpose converter application, also serves as a frontend to ffmpeg2theora.
*[http://teejee2008.wordpress.com/ffcoder/ FFCoder], general purpose converter
*[http://en.wikipedia.org/wiki/Theora#Encoding The Wikipedia Theora Page] provides an up to date list of software that can encode Theora.
*[http://www.xiph.org/dshow/Ogg DSF DirectShow Filters for Windows] allows playback and encoding of Ogg Theora/Vorbis in all Windows apps that use the DirectShow framework.

== Linux/BSD ==
*[http://thoggen.net/ Thoggen] is a DVD backup utility ('DVD ripper') for Linux, based on GStreamer and Gtk+.
*[http://freej.org/ Freej] Freej is a realtime video mixer. It can stream Theora and Vorbis live to [http://icecast.org icecast]. Check [http://lab.dyne.org/FreejStreaming here] for more info.
* [http://oggconvert.tristanb.net/ OggConvert] is a small Gnome utility which uses GStreamer to convert (almost) any media file to Vorbis, Theora and Dirac.

== Mac OS X ==
*[http://xiph.org/quicktime XiphQT] allows you to export from Quicktime, or any application supporting Quicktime (i.e. Final Cut Pro), to Theora/Vorbis.
*[http://v2v.cc/~j/SimpleTheoraEncoder/ Simple Theora Encoder] an ffmpeg2theora frontend

== See also==
{{Template:Theora}}

[[Category:Theora]]

SkeletonHeaders

2010-03-24T01:18:47Z

Silvia: removed transparentcolor

{{draft}}

= Ogg Skeleton 3.4 with new Message Headers =

'''DRAFT, last updated 23 March 2010'''

'''This specification is still a work in progress, and does not yet constitute an official Ogg specification.'''

== Adding New Message Headers to Skeleton ==

With the HTML5 video element, Ogg is now a major format on the Web and is being applied to solve use cases it hasn't had to solve before, but was built to allow, see http://www.xiph.org/ogg/doc/oggstream.html.

One particular such use case is dealing with multitrack audio and video, such as in videos with multiple view angles encoded in one, or ones with a sign language video track, an audio description audio track, a caption track and several subtitle tracks in different languages (i.e. several theora, several vorbis and several kate tracks).

While encoding of multitrack files is already possible, it is unclear how such files would be rendered, how tracks would be differentiated and addressed (e.g. from a JavaScript API), etc. Skeleton has been built in a way such that it is extensible with message header fields for this purpose.

On this wiki page, we are collecting such new information fields.

=== Content-type ===

Right now, there is one mandatory message header field for all of the logical bitstreams: the "Content-type" header field, which contains the mime type of the track. The mime types in use here are listed at http://wiki.xiph.org/MIME_Types_and_File_Extensions#Codec_MIME_types.

=== Language ===

Content in a track usually originates from a specific language. This language can be specified in a Language message header field. The code is created according to http://www.w3.org/TR/ltli/ and http://www.rfc-editor.org/rfc/bcp/bcp47.txt.

For audio tracks with speech, the Language would be the language that dominates.

For video tracks, it might be the language that is signed (if it is a sign language video), or the language that is most often represented in scene text.

For text tracks, it is the dominating language in the text, e.g. English or German subtitles.

Examples are: en-US, de-DE, sgn-ase, en-cockney

The Language field will have the dominating language specified as the first language. It is possible to specify less non-dominating languages as a list after the main language.

Example:
Language: en-US, fr

=== Role ===

Role describe what semantic type of content is contained in a track. Every track can only have a single role value, so the most appropriate role has to be chosen. The same role can be used across multiple tracks.

The following list some commonly used roles. Other roles are possible, too, but should only be used/introduced if there is really a need for it.

Text tracks:
* "text/caption" - transcription of all sounds, including speech, for purposes of the hard-of-hearing
* "text/subtitle" - translation of all speech, typically into a different language
* "text/textaudiodesc" - description/transcription of everything that happens in a video as text to be used for the vision-impaired through screen readers or braille
* "text/karaoke" - music lyrics delivered in chunks for singing along
* "text/chapters" - titles for sections of the media that provide a kind of chapter segmentation (similar to DVD chapters)
* "text/tickertext" - text to run as informative text at the bottom of the media display
* "text/lyrics" - transcript of the text used in music media
* "text/metadata" - name-value pairs that are associated with certain sections of the media
* "text/annotation" - free text associated with certain sections of the media
* "text/linguistic" - linguistic markup of the spoken words

Video tracks:
* "video/main" - the main video track
* "video/alternate" - an alternative video track, e.g. different camera angle
* "video/sign" - a sign language video track

Audio tracks:
* "audio/main" - the main audio track
* "audio/alternate" - an alternative audio track, probably linked to an alternate video track
* "audio/dub" - the audio track but with speech in a different language to the original
* "audio/audiodesc" - an audio description recording for the vision-impaired
* "audio/music" - a music track, e.g. when music, speech and sound effects are delivered in different tracks
* "audio/speech" - a speech track, e.g. when music, speech and sound effects are delivered in different tracks
* "audio/sfx" - a sound effects track, e.g. when music, speech and sound effects are delivered in different tracks

Notice how we are re-using the Content-type approach of specifying the main semantic type of the track first. This is necessary, since mime types don't always provide the right main content type (e.g. application/kate is semantically a text format).

There may also be parameters to describe the roles better, such as "video/alternate;angle=nw"

=== Name ===

This field provides the opportunity to associate a free text string with the track to allow direct addressing of the track through its name.

Characters allowed are basically all the characters that are also allowed for XML id fields:

the first character has to be one of:
[A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] |
[#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF]

any following characters can be one of:
[A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] |
[#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF] |
"-" | "." | [0-9] | #xB7 | [#x0300-#x036F] | [#x203F-#x2040]

The name needs to be unique between all the track names, otherwise it is undefined which of the tracks is retrieved when addressing by name.

An example means of addressing the track by name is: track[name="Madonna_singing"]

=== Title ===

A free text field to provide a description of the track content.

Example:
Title: "the French audio track for the movie"

=== Display-hint ===

Media players that do not get informed about how a content author intends a media file to be displayed have no change to display the content "correctly". This is why the Display-hint message header field allows providing of hints on how a certain track should be displayed. A media player can of course decide to ignore these hints.

Currently proposed hints are:

* pip(x,y,w,h) on a video track - picture-in-picture display in relation to the zero coordinates of the display area of the video with x,y providing the origin of the top left corner of the PIP video and w,h the width and height in pixels which are optional. x, y, w, and h can be specified in percentage, thus allowing persistent placement independent of the scaling of the video display.

Examples:
Display-hint: pip(20%,20%)
Display-hint: pip(40,40,690,60)

* mask(img,x,y,w,h) on a video track - use the image given at img url (?) as a video mask to allow the video to appear in shapes other than rectangular. The masking image should be a black shape on a white background. The image is placed at offset x,y and scaled to width and height w and h. x,y,w, and h can be provided in pixels or in percent. Pixels under the white background are made transparent and only pixels under the black shape are retained.

Examples:
Display-hint: mask(http://www.example.com/image.png)
Display-hint: mask(http://www.example.com/image.png,30%,25%)
Display-hint: mask(http://www.example.com/image.png,20,20,400,320)

* transparent(transparency) on a video track - put a transparency of x% (int value between 0 and 100) on the complete video track as it will be rendered on top of other content. This transparency is applied to all pixels in the same way.

Examples:
Display-hint: transparent(25%)
Display-hint: transparent(7%)

=== Track order ===

In many applications it is necessary to walk through all the tracks in a media file and address tracks by an index.

In Ogg, the means to number through the tracks is by the order in which the bos pages of the tracks appear in the Ogg stream. If a file is re-encoded, the order may change, so you can only rely on this for addressing if the file doesn't change.

For example, a video file with the following composition would have the following indexes:
* track[0]: Skeleton BOS
* track[1]: Theora BOS for main video
* track[2]: Vorbis BOS for main audio
* track[3]: Kate BOS for English captions
* track[4]: Kate BOS for German subtitles
* track[5]: Vorbis BOS for audio descriptions
* track[6]: Theora BOS for sign language

This track order is simply to have a means to address tracks through an index in a consistent manner across different media players, such that e.g. JavaScript can always link to the same track reliably across browsers. It has no influence on what should be displayed on top of which other track.

=== Altitude ===

The Altitude (better name?) message header field defines the stack order of the tracks, i.e. which track is displayed further towards the top of the stack and which further down. By default, a "main" track is always displayed bottom-most unless otherwise defined.

The Altitude field takes the same numerical values as the z-index in CSS, unlimited negative and positive numbers.
An element with greater stack order is always in front of an element with a lower stack order.

Example: Altitude: -150

=== Track dependencies ===

It is tempting to introduce dependencies between tracks - to specify things such as:

* track b depends on track a being available (e.g. main audio depending on main video), so always display them together and if you remove a track, remove all depending tracks, too

* track c and d are alternative tracks to track b (e.g. dubs in other languages for main audio), so don't display them together and if you activate one, disable the others

* track a and one of b,c,d one of e,f,g where e depends on b, f depends on c, and g depends on d, make up a presentation profile and should be displayed together (e.g. main video, one of the audio dubs, and their respective captions).

It is not clear yet whether there is an actual need to maintain this information as author-provided hints or whether a media player can itself determine a lot from the other fields, such as role and language.

MPEG has a "groupID" element which allows for tracks to be put into groups of alternative tracks. This feature is, however, not used very often and decisions are being left to the media player.

At this stage, it's probably too early to make a specification for how to encode this in Ogg. The need has not been totally clarified yet.

SkeletonHeaders

2010-03-24T01:02:07Z

Silvia: added descriptions to the roles

{{draft}}

= Ogg Skeleton 3.4 with new Message Headers =

'''DRAFT, last updated 23 March 2010'''

'''This specification is still a work in progress, and does not yet constitute an official Ogg specification.'''

== Adding New Message Headers to Skeleton ==

With the HTML5 video element, Ogg is now a major format on the Web and is being applied to solve use cases it hasn't had to solve before, but was built to allow, see http://www.xiph.org/ogg/doc/oggstream.html.

One particular such use case is dealing with multitrack audio and video, such as in videos with multiple view angles encoded in one, or ones with a sign language video track, an audio description audio track, a caption track and several subtitle tracks in different languages (i.e. several theora, several vorbis and several kate tracks).

While encoding of multitrack files is already possible, it is unclear how such files would be rendered, how tracks would be differentiated and addressed (e.g. from a JavaScript API), etc. Skeleton has been built in a way such that it is extensible with message header fields for this purpose.

On this wiki page, we are collecting such new information fields.

=== Content-type ===

Right now, there is one mandatory message header field for all of the logical bitstreams: the "Content-type" header field, which contains the mime type of the track. The mime types in use here are listed at http://wiki.xiph.org/MIME_Types_and_File_Extensions#Codec_MIME_types.

=== Language ===

Content in a track usually originates from a specific language. This language can be specified in a Language message header field. The code is created according to http://www.w3.org/TR/ltli/ and http://www.rfc-editor.org/rfc/bcp/bcp47.txt.

For audio tracks with speech, the Language would be the language that dominates.

For video tracks, it might be the language that is signed (if it is a sign language video), or the language that is most often represented in scene text.

For text tracks, it is the dominating language in the text, e.g. English or German subtitles.

Examples are: en-US, de-DE, sgn-ase, en-cockney

The Language field will have the dominating language specified as the first language. It is possible to specify less non-dominating languages as a list after the main language.

Example:
Language: en-US, fr

=== Role ===

Role describe what semantic type of content is contained in a track. Every track can only have a single role value, so the most appropriate role has to be chosen. The same role can be used across multiple tracks.

The following list some commonly used roles. Other roles are possible, too, but should only be used/introduced if there is really a need for it.

Text tracks:
* "text/caption" - transcription of all sounds, including speech, for purposes of the hard-of-hearing
* "text/subtitle" - translation of all speech, typically into a different language
* "text/textaudiodesc" - description/transcription of everything that happens in a video as text to be used for the vision-impaired through screen readers or braille
* "text/karaoke" - music lyrics delivered in chunks for singing along
* "text/chapters" - titles for sections of the media that provide a kind of chapter segmentation (similar to DVD chapters)
* "text/tickertext" - text to run as informative text at the bottom of the media display
* "text/lyrics" - transcript of the text used in music media
* "text/metadata" - name-value pairs that are associated with certain sections of the media
* "text/annotation" - free text associated with certain sections of the media
* "text/linguistic" - linguistic markup of the spoken words

Video tracks:
* "video/main" - the main video track
* "video/alternate" - an alternative video track, e.g. different camera angle
* "video/sign" - a sign language video track

Audio tracks:
* "audio/main" - the main audio track
* "audio/alternate" - an alternative audio track, probably linked to an alternate video track
* "audio/dub" - the audio track but with speech in a different language to the original
* "audio/audiodesc" - an audio description recording for the vision-impaired
* "audio/music" - a music track, e.g. when music, speech and sound effects are delivered in different tracks
* "audio/speech" - a speech track, e.g. when music, speech and sound effects are delivered in different tracks
* "audio/sfx" - a sound effects track, e.g. when music, speech and sound effects are delivered in different tracks

Notice how we are re-using the Content-type approach of specifying the main semantic type of the track first. This is necessary, since mime types don't always provide the right main content type (e.g. application/kate is semantically a text format).

There may also be parameters to describe the roles better, such as "video/alternate;angle=nw"

=== Display-hint ===

Media players that do not get informed about how a content author intends a media file to be displayed have no change to display the content "correctly". This is why the Display-hint message header field allows providing of hints on how a certain track should be displayed. A media player can of course decide to ignore these hints.

Currently proposed hints are:

* pip(x,y,w,h) on a video track - picture-in-picture display in relation to the zero coordinates of the display area of the video with x,y providing the origin of the top left corner of the PIP video and w,h the width and height in pixels which are optional. x, y, w, and h can be specified in percentage, thus allowing persistent placement independent of the scaling of the video display.

Examples:
Display-hint: pip(20%,20%)
Display-hint: pip(40,40,690,60)

* mask(img,x,y,w,h) on a video track - use the image given at img url (?) as a video mask to allow the video to appear in shapes other than rectangular. The masking image should be a black shape on a white background. The image is placed at offset x,y and scaled to width and height w and h. x,y,w, and h can be provided in pixels or in percent. Pixels under the white background are made transparent and only pixels under the black shape are retained.

Examples:
Display-hint: mask(http://www.example.com/image.png)
Display-hint: mask(http://www.example.com/image.png,30%,25%)
Display-hint: mask(http://www.example.com/image.png,20,20,400,320)

* transparent(transparency) on a video track - put a transparency of x% (int value between 0 and 100) on the complete video track as it will be rendered on top of other content. This transparency is applied to all pixels in the same way.

Examples:
Display-hint: transparent(25%)
Display-hint: transparent(7%)

* transparentcolor(colorcode) on a video track - turn all pixels of the color identified by the colorcode into transparent pixels.

Examples:
Display-hint: transparentcolor(#454545)
Display-hint: transparentcolor(#777777)

=== Name ===

This field provides the opportunity to associate a free text string with the track to allow direct addressing of the track through its name.

Characters allowed are basically all the characters that are also allowed for XML id fields:

the first character has to be one of:
[A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] |
[#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF]

any following characters can be one of:
[A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] |
[#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF] |
"-" | "." | [0-9] | #xB7 | [#x0300-#x036F] | [#x203F-#x2040]

The name needs to be unique between all the track names, otherwise it is undefined which of the tracks is retrieved when addressing by name.

An example means of addressing the track by name is: track[name="Madonna_singing"]

=== Track order ===

In many applications it is necessary to walk through all the tracks in a media file and address tracks by an index.

In Ogg, the means to number through the tracks is by the order in which the bos pages of the tracks appear in the Ogg stream. If a file is re-encoded, the order may change, so you can only rely on this for addressing if the file doesn't change.

For example, a video file with the following composition would have the following indexes:
* track[0]: Skeleton BOS
* track[1]: Theora BOS for main video
* track[2]: Vorbis BOS for main audio
* track[3]: Kate BOS for English captions
* track[4]: Kate BOS for German subtitles
* track[5]: Vorbis BOS for audio descriptions
* track[6]: Theora BOS for sign language

This track order is simply to have a means to address tracks through an index in a consistent manner across different media players, such that e.g. JavaScript can always link to the same track reliably across browsers. It has no influence on what should be displayed on top of which other track.

=== Altitude ===

The Altitude (better name?) message header field defines the stack order of the tracks, i.e. which track is displayed further towards the top of the stack and which further down. By default, a "main" track is always displayed bottom-most unless otherwise defined.

The Altitude field takes the same numerical values as the z-index in CSS, unlimited negative and positive numbers.
An element with greater stack order is always in front of an element with a lower stack order.

Example: Altitude: -150

=== Track dependencies ===

It is tempting to introduce dependencies between tracks - to specify things such as:

* track b depends on track a being available (e.g. main audio depending on main video), so always display them together and if you remove a track, remove all depending tracks, too

* track c and d are alternative tracks to track b (e.g. dubs in other languages for main audio), so don't display them together and if you activate one, disable the others

* track a and one of b,c,d one of e,f,g where e depends on b, f depends on c, and g depends on d, make up a presentation profile and should be displayed together (e.g. main video, one of the audio dubs, and their respective captions).

It is not clear yet whether there is an actual need to maintain this information as author-provided hints or whether a media player can itself determine a lot from the other fields, such as role and language.

MPEG has a "groupID" element which allows for tracks to be put into groups of alternative tracks. This feature is, however, not used very often and decisions are being left to the media player.

At this stage, it's probably too early to make a specification for how to encode this in Ogg. The need has not been totally clarified yet.

SkeletonHeaders

2010-03-24T00:46:47Z

Silvia: added multi-value to language

{{draft}}

= Ogg Skeleton 3.4 with new Message Headers =

'''DRAFT, last updated 23 March 2010'''

'''This specification is still a work in progress, and does not yet constitute an official Ogg specification.'''

== Adding New Message Headers to Skeleton ==

With the HTML5 video element, Ogg is now a major format on the Web and is being applied to solve use cases it hasn't had to solve before, but was built to allow, see http://www.xiph.org/ogg/doc/oggstream.html.

One particular such use case is dealing with multitrack audio and video, such as in videos with multiple view angles encoded in one, or ones with a sign language video track, an audio description audio track, a caption track and several subtitle tracks in different languages (i.e. several theora, several vorbis and several kate tracks).

While encoding of multitrack files is already possible, it is unclear how such files would be rendered, how tracks would be differentiated and addressed (e.g. from a JavaScript API), etc. Skeleton has been built in a way such that it is extensible with message header fields for this purpose.

On this wiki page, we are collecting such new information fields.

=== Content-type ===

Right now, there is one mandatory message header field for all of the logical bitstreams: the "Content-type" header field, which contains the mime type of the track. The mime types in use here are listed at http://wiki.xiph.org/MIME_Types_and_File_Extensions#Codec_MIME_types.

=== Language ===

Content in a track usually originates from a specific language. This language can be specified in a Language message header field. The code is created according to http://www.w3.org/TR/ltli/ and http://www.rfc-editor.org/rfc/bcp/bcp47.txt.

For audio tracks with speech, the Language would be the language that dominates.

For video tracks, it might be the language that is signed (if it is a sign language video), or the language that is most often represented in scene text.

For text tracks, it is the dominating language in the text, e.g. English or German subtitles.

Examples are: en-US, de-DE, sgn-ase, en-cockney

The Language field will have the dominating language specified as the first language. It is possible to specify less non-dominating languages as a list after the main language.

Example:
Language: en-US, fr

=== Role ===

Role describe what semantic type of content is contained in a track. Every track can only have a single role value, so the most appropriate role has to be chosen. The same role can be used across multiple tracks.

The following list some commonly used roles. Other roles are possible, too, but should only be used/introduced if there is really a need for it.

Text tracks:
* "text/caption"
* "text/subtitle"
* "text/textaudiodesc"
* "text/karaoke"
* "text/chapters"
* "text/tickertext"
* "text/lyrics"
* "text/activeregion"
* "text/metadata"
* "text/annotation"
* "text/transcript"
* "text/linguistic"
* "text/chapters"

Video tracks:
* "video/main"
* "video/alternate" (e.g. different camera angle)
* "video/sign" (for sign language)
* "video/alpha" (a track to alpha blend)

Audio tracks:
* "audio/main"
* "audio/alternate" (probably linked to an alternate video track)
* "audio/dub"
* "audio/audiodesc"
* "audio/music"
* "audio/speech"
* "audio/sfx" (sound effects)

Notice how we are re-using the Content-type approach of specifying the main semantic type of the track first. This is necessary, since mime types don't always provide the right main content type (e.g. application/kate is semantically a text format).

There may also be parameters to describe the roles better, such as "video/alternate;angle=nw"

=== Display-hint ===

Media players that do not get informed about how a content author intends a media file to be displayed have no change to display the content "correctly". This is why the Display-hint message header field allows providing of hints on how a certain track should be displayed. A media player can of course decide to ignore these hints.

Currently proposed hints are:

* pip(x,y,w,h) on a video track - picture-in-picture display in relation to the zero coordinates of the display area of the video with x,y providing the origin of the top left corner of the PIP video and w,h the width and height in pixels which are optional. x, y, w, and h can be specified in percentage, thus allowing persistent placement independent of the scaling of the video display.

Examples:
Display-hint: pip(20%,20%)
Display-hint: pip(40,40,690,60)

* mask(img,x,y,w,h) on a video track - use the image given at img url (?) as a video mask to allow the video to appear in shapes other than rectangular. The masking image should be a black shape on a white background. The image is placed at offset x,y and scaled to width and height w and h. x,y,w, and h can be provided in pixels or in percent. Pixels under the white background are made transparent and only pixels under the black shape are retained.

Examples:
Display-hint: mask(http://www.example.com/image.png)
Display-hint: mask(http://www.example.com/image.png,30%,25%)
Display-hint: mask(http://www.example.com/image.png,20,20,400,320)

* transparent(transparency) on a video track - put a transparency of x% (int value between 0 and 100) on the complete video track as it will be rendered on top of other content. This transparency is applied to all pixels in the same way.

Examples:
Display-hint: transparent(25%)
Display-hint: transparent(7%)

* transparentcolor(colorcode) on a video track - turn all pixels of the color identified by the colorcode into transparent pixels.

Examples:
Display-hint: transparentcolor(#454545)
Display-hint: transparentcolor(#777777)

=== Name ===

This field provides the opportunity to associate a free text string with the track to allow direct addressing of the track through its name.

Characters allowed are basically all the characters that are also allowed for XML id fields:

the first character has to be one of:
[A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] |
[#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF]

any following characters can be one of:
[A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] |
[#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF] |
"-" | "." | [0-9] | #xB7 | [#x0300-#x036F] | [#x203F-#x2040]

The name needs to be unique between all the track names, otherwise it is undefined which of the tracks is retrieved when addressing by name.

An example means of addressing the track by name is: track[name="Madonna_singing"]

=== Track order ===

In many applications it is necessary to walk through all the tracks in a media file and address tracks by an index.

In Ogg, the means to number through the tracks is by the order in which the bos pages of the tracks appear in the Ogg stream. If a file is re-encoded, the order may change, so you can only rely on this for addressing if the file doesn't change.

For example, a video file with the following composition would have the following indexes:
* track[0]: Skeleton BOS
* track[1]: Theora BOS for main video
* track[2]: Vorbis BOS for main audio
* track[3]: Kate BOS for English captions
* track[4]: Kate BOS for German subtitles
* track[5]: Vorbis BOS for audio descriptions
* track[6]: Theora BOS for sign language

This track order is simply to have a means to address tracks through an index in a consistent manner across different media players, such that e.g. JavaScript can always link to the same track reliably across browsers. It has no influence on what should be displayed on top of which other track.

=== Altitude ===

The Altitude (better name?) message header field defines the stack order of the tracks, i.e. which track is displayed further towards the top of the stack and which further down. By default, a "main" track is always displayed bottom-most unless otherwise defined.

The Altitude field takes the same numerical values as the z-index in CSS, unlimited negative and positive numbers.
An element with greater stack order is always in front of an element with a lower stack order.

Example: Altitude: -150

=== Track dependencies ===

It is tempting to introduce dependencies between tracks - to specify things such as:

* track b depends on track a being available (e.g. main audio depending on main video), so always display them together and if you remove a track, remove all depending tracks, too

* track c and d are alternative tracks to track b (e.g. dubs in other languages for main audio), so don't display them together and if you activate one, disable the others

* track a and one of b,c,d one of e,f,g where e depends on b, f depends on c, and g depends on d, make up a presentation profile and should be displayed together (e.g. main video, one of the audio dubs, and their respective captions).

It is not clear yet whether there is an actual need to maintain this information as author-provided hints or whether a media player can itself determine a lot from the other fields, such as role and language.

MPEG has a "groupID" element which allows for tracks to be put into groups of alternative tracks. This feature is, however, not used very often and decisions are being left to the media player.

At this stage, it's probably too early to make a specification for how to encode this in Ogg. The need has not been totally clarified yet.

SkeletonHeaders

2010-03-23T05:45:22Z

Silvia: added intro

{{draft}}

= Ogg Skeleton 3.4 with new Message Headers =

'''DRAFT, last updated 23 March 2010'''

'''This specification is still a work in progress, and does not yet constitute an official Ogg specification.'''

== Adding New Message Headers to Skeleton ==

With the HTML5 video element, Ogg is now a major format on the Web and is being applied to solve use cases it hasn't had to solve before, but was built to allow, see http://www.xiph.org/ogg/doc/oggstream.html.

One particular such use case is dealing with multitrack audio and video, such as in videos with multiple view angles encoded in one, or ones with a sign language video track, an audio description audio track, a caption track and several subtitle tracks in different languages (i.e. several theora, several vorbis and several kate tracks).

While encoding of multitrack files is already possible, it is unclear how such files would be rendered, how tracks would be differentiated and addressed (e.g. from a JavaScript API), etc. Skeleton has been built in a way such that it is extensible with message header fields for this purpose.

On this wiki page, we are collecting such new information fields.

=== Content-type ===

Right now, there is one mandatory message header field for all of the logical bitstreams: the "Content-type" header field, which contains the mime type of the track. The mime types in use here are listed at http://wiki.xiph.org/MIME_Types_and_File_Extensions#Codec_MIME_types.

=== Language ===

Content in a track usually originates from a specific language. This language can be specified in a Language message header field. The code is created according to http://www.w3.org/TR/ltli/ and http://www.rfc-editor.org/rfc/bcp/bcp47.txt.

For audio tracks with speech, the Language would be the language that dominates.

For video tracks, it might be the language that is signed (if it is a sign language video), or the language that is most often represented in scene text.

For text tracks, it is the dominating language in the text, e.g. English or German subtitles.

Examples are: en-US, de-DE, sgn-ase, en-cockney

=== Role ===

Role describe what semantic type of content is contained in a track. Every track can only have a single role value, so the most appropriate role has to be chosen. The same role can be used across multiple tracks.

The following list some commonly used roles. Other roles are possible, too, but should only be used/introduced if there is really a need for it.

Text tracks:
* "text/caption"
* "text/subtitle"
* "text/textaudiodesc"
* "text/karaoke"
* "text/chapters"
* "text/tickertext"
* "text/lyrics"
* "text/activeregion"
* "text/metadata"
* "text/annotation"
* "text/transcript"
* "text/linguistic"
* "text/chapters"

Video tracks:
* "video/main"
* "video/alternate" (e.g. different camera angle)
* "video/sign" (for sign language)
* "video/alpha" (a track to alpha blend)

Audio tracks:
* "audio/main"
* "audio/alternate" (probably linked to an alternate video track)
* "audio/dub"
* "audio/audiodesc"
* "audio/music"
* "audio/speech"
* "audio/sfx" (sound effects)

Notice how we are re-using the Content-type approach of specifying the main semantic type of the track first. This is necessary, since mime types don't always provide the right main content type (e.g. application/kate is semantically a text format).

There may also be parameters to describe the roles better, such as "video/alternate;angle=nw"

=== Display-hint ===

Media players that do not get informed about how a content author intends a media file to be displayed have no change to display the content "correctly". This is why the Display-hint message header field allows providing of hints on how a certain track should be displayed. A media player can of course decide to ignore these hints.

Currently proposed hints are:

* pip(x,y,w,h) on a video track - picture-in-picture display in relation to the zero coordinates of the display area of the video with x,y providing the origin of the top left corner of the PIP video and w,h the width and height in pixels which are optional. x, y, w, and h can be specified in percentage, thus allowing persistent placement independent of the scaling of the video display.

Examples:
Display-hint: pip(20%,20%)
Display-hint: pip(40,40,690,60)

* mask(img,x,y,w,h) on a video track - use the image given at img url (?) as a video mask to allow the video to appear in shapes other than rectangular. The masking image should be a black shape on a white background. The image is placed at offset x,y and scaled to width and height w and h. x,y,w, and h can be provided in pixels or in percent. Pixels under the white background are made transparent and only pixels under the black shape are retained.

Examples:
Display-hint: mask(http://www.example.com/image.png)
Display-hint: mask(http://www.example.com/image.png,30%,25%)
Display-hint: mask(http://www.example.com/image.png,20,20,400,320)

* transparent(transparency) on a video track - put a transparency of x% (int value between 0 and 100) on the complete video track as it will be rendered on top of other content. This transparency is applied to all pixels in the same way.

Examples:
Display-hint: transparent(25%)
Display-hint: transparent(7%)

* transparentcolor(colorcode) on a video track - turn all pixels of the color identified by the colorcode into transparent pixels.

Examples:
Display-hint: transparentcolor(#454545)
Display-hint: transparentcolor(#777777)

=== Name ===

This field provides the opportunity to associate a free text string with the track to allow direct addressing of the track through its name.

Characters allowed are basically all the characters that are also allowed for XML id fields:

the first character has to be one of:
[A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] |
[#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF]

any following characters can be one of:
[A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] |
[#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF] |
"-" | "." | [0-9] | #xB7 | [#x0300-#x036F] | [#x203F-#x2040]

The name needs to be unique between all the track names, otherwise it is undefined which of the tracks is retrieved when addressing by name.

An example means of addressing the track by name is: track[name="Madonna_singing"]

=== Track order ===

In many applications it is necessary to walk through all the tracks in a media file and address tracks by an index.

In Ogg, the means to number through the tracks is by the order in which the bos pages of the tracks appear in the Ogg stream. If a file is re-encoded, the order may change, so you can only rely on this for addressing if the file doesn't change.

For example, a video file with the following composition would have the following indexes:
* track[0]: Skeleton BOS
* track[1]: Theora BOS for main video
* track[2]: Vorbis BOS for main audio
* track[3]: Kate BOS for English captions
* track[4]: Kate BOS for German subtitles
* track[5]: Vorbis BOS for audio descriptions
* track[6]: Theora BOS for sign language

This track order is simply to have a means to address tracks through an index in a consistent manner across different media players, such that e.g. JavaScript can always link to the same track reliably across browsers. It has no influence on what should be displayed on top of which other track.

=== Altitude ===

The Altitude (better name?) message header field defines the stack order of the tracks, i.e. which track is displayed further towards the top of the stack and which further down. By default, a "main" track is always displayed bottom-most unless otherwise defined.

The Altitude field takes the same numerical values as the z-index in CSS, unlimited negative and positive numbers.
An element with greater stack order is always in front of an element with a lower stack order.

Example: Altitude: -150

=== Track dependencies ===

It is tempting to introduce dependencies between tracks - to specify things such as:

* track b depends on track a being available (e.g. main audio depending on main video), so always display them together and if you remove a track, remove all depending tracks, too

* track c and d are alternative tracks to track b (e.g. dubs in other languages for main audio), so don't display them together and if you activate one, disable the others

* track a and one of b,c,d one of e,f,g where e depends on b, f depends on c, and g depends on d, make up a presentation profile and should be displayed together (e.g. main video, one of the audio dubs, and their respective captions).

It is not clear yet whether there is an actual need to maintain this information as author-provided hints or whether a media player can itself determine a lot from the other fields, such as role and language.

MPEG has a "groupID" element which allows for tracks to be put into groups of alternative tracks. This feature is, however, not used very often and decisions are being left to the media player.

At this stage, it's probably too early to make a specification for how to encode this in Ogg. The need has not been totally clarified yet.

SkeletonHeaders

2010-03-23T05:22:13Z

Silvia: /* Display-hint */

== Adding New Required Headers to Skeleton ==

With the HTML5 video element, Ogg is now a major format on the Web and is being applied to solve use cases it hasn't had to solve before, but was built to allow, see http://www.xiph.org/ogg/doc/oggstream.html.

One particular such use case is dealing with multitrack audio and video, such as in videos with multiple view angles encoded in one, or ones with a sign language video track, an audio description audio track, a caption track and several subtitle tracks in different languages (i.e. several theora, several vorbis and several kate tracks).

While encoding of multitrack files is already possible, it is unclear how such files would be rendered, how tracks would be differentiated and addressed (e.g. from a JavaScript API), etc. Skeleton has been built in a way such that it is extensible with message header fields for this purpose.

On this wiki page, we are collecting such new information fields.

=== Content-type ===

Right now, there is one mandatory message header field for all of the logical bitstreams: the "Content-type" header field, which contains the mime type of the track. The mime types in use here are listed at http://wiki.xiph.org/MIME_Types_and_File_Extensions#Codec_MIME_types.

=== Language ===

Content in a track usually originates from a specific language. This language can be specified in a Language message header field. The code is created according to http://www.w3.org/TR/ltli/ and http://www.rfc-editor.org/rfc/bcp/bcp47.txt.

For audio tracks with speech, the Language would be the language that dominates.

For video tracks, it might be the language that is signed (if it is a sign language video), or the language that is most often represented in scene text.

For text tracks, it is the dominating language in the text, e.g. English or German subtitles.

Examples are: en-US, de-DE, sgn-ase, en-cockney

=== Role ===

Role describe what semantic type of content is contained in a track. Every track can only have a single role value, so the most appropriate role has to be chosen. The same role can be used across multiple tracks.

The following list some commonly used roles. Other roles are possible, too, but should only be used/introduced if there is really a need for it.

Text tracks:
* "text/caption"
* "text/subtitle"
* "text/textaudiodesc"
* "text/karaoke"
* "text/chapters"
* "text/tickertext"
* "text/lyrics"
* "text/activeregion"
* "text/metadata"
* "text/annotation"
* "text/transcript"
* "text/linguistic"
* "text/chapters"

Video tracks:
* "video/main"
* "video/alternate" (e.g. different camera angle)
* "video/sign" (for sign language)
* "video/alpha" (a track to alpha blend)

Audio tracks:
* "audio/main"
* "audio/alternate" (probably linked to an alternate video track)
* "audio/dub"
* "audio/audiodesc"
* "audio/music"
* "audio/speech"
* "audio/sfx" (sound effects)

Notice how we are re-using the Content-type approach of specifying the main semantic type of the track first. This is necessary, since mime types don't always provide the right main content type (e.g. application/kate is semantically a text format).

There may also be parameters to describe the roles better, such as "video/alternate;angle=nw"

=== Display-hint ===

Media players that do not get informed about how a content author intends a media file to be displayed have no change to display the content "correctly". This is why the Display-hint message header field allows providing of hints on how a certain track should be displayed. A media player can of course decide to ignore these hints.

Currently proposed hints are:

* pip(x,y,w,h) on a video track - picture-in-picture display in relation to the zero coordinates of the display area of the video with x,y providing the origin of the top left corner of the PIP video and w,h the width and height in pixels which are optional. x, y, w, and h can be specified in percentage, thus allowing persistent placement independent of the scaling of the video display.

Examples:
Display-hint: pip(20%,20%)
Display-hint: pip(40,40,690,60)

* mask(img,x,y,w,h) on a video track - use the image given at img url (?) as a video mask to allow the video to appear in shapes other than rectangular. The masking image should be a black shape on a white background. The image is placed at offset x,y and scaled to width and height w and h. x,y,w, and h can be provided in pixels or in percent. Pixels under the white background are made transparent and only pixels under the black shape are retained.

Examples:
Display-hint: mask(http://www.example.com/image.png)
Display-hint: mask(http://www.example.com/image.png,30%,25%)
Display-hint: mask(http://www.example.com/image.png,20,20,400,320)

* transparent(transparency) on a video track - put a transparency of x% (int value between 0 and 100) on the complete video track as it will be rendered on top of other content. This transparency is applied to all pixels in the same way.

Examples:
Display-hint: transparent(25%)
Display-hint: transparent(7%)

* transparentcolor(colorcode) on a video track - turn all pixels of the color identified by the colorcode into transparent pixels.

Examples:
Display-hint: transparentcolor(#454545)
Display-hint: transparentcolor(#777777)

=== Name ===

This field provides the opportunity to associate a free text string with the track to allow direct addressing of the track through its name.

Characters allowed are basically all the characters that are also allowed for XML id fields:

the first character has to be one of:
[A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] |
[#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF]

any following characters can be one of:
[A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] |
[#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF] |
"-" | "." | [0-9] | #xB7 | [#x0300-#x036F] | [#x203F-#x2040]

The name needs to be unique between all the track names, otherwise it is undefined which of the tracks is retrieved when addressing by name.

An example means of addressing the track by name is: track[name="Madonna_singing"]

=== Track order ===

In many applications it is necessary to walk through all the tracks in a media file and address tracks by an index.

In Ogg, the means to number through the tracks is by the order in which the bos pages of the tracks appear in the Ogg stream. If a file is re-encoded, the order may change, so you can only rely on this for addressing if the file doesn't change.

For example, a video file with the following composition would have the following indexes:
* track[0]: Skeleton BOS
* track[1]: Theora BOS for main video
* track[2]: Vorbis BOS for main audio
* track[3]: Kate BOS for English captions
* track[4]: Kate BOS for German subtitles
* track[5]: Vorbis BOS for audio descriptions
* track[6]: Theora BOS for sign language

This track order is simply to have a means to address tracks through an index in a consistent manner across different media players, such that e.g. JavaScript can always link to the same track reliably across browsers. It has no influence on what should be displayed on top of which other track.

=== Altitude ===

The Altitude (better name?) message header field defines the stack order of the tracks, i.e. which track is displayed further towards the top of the stack and which further down. By default, a "main" track is always displayed bottom-most unless otherwise defined.

The Altitude field takes the same numerical values as the z-index in CSS, unlimited negative and positive numbers.
An element with greater stack order is always in front of an element with a lower stack order.

Example: Altitude: -150

=== Track dependencies ===

It is tempting to introduce dependencies between tracks - to specify things such as:

* track b depends on track a being available (e.g. main audio depending on main video), so always display them together and if you remove a track, remove all depending tracks, too

* track c and d are alternative tracks to track b (e.g. dubs in other languages for main audio), so don't display them together and if you activate one, disable the others

* track a and one of b,c,d one of e,f,g where e depends on b, f depends on c, and g depends on d, make up a presentation profile and should be displayed together (e.g. main video, one of the audio dubs, and their respective captions).

It is not clear yet whether there is an actual need to maintain this information as author-provided hints or whether a media player can itself determine a lot from the other fields, such as role and language.

MPEG has a "groupID" element which allows for tracks to be put into groups of alternative tracks. This feature is, however, not used very often and decisions are being left to the media player.

At this stage, it's probably too early to make a specification for how to encode this in Ogg. The need has not been totally clarified yet.

SkeletonHeaders

2010-03-21T11:25:58Z

Silvia:

== Adding New Required Headers to Skeleton ==

With the HTML5 video element, Ogg is now a major format on the Web and is being applied to solve use cases it hasn't had to solve before, but was built to allow, see http://www.xiph.org/ogg/doc/oggstream.html.

One particular such use case is dealing with multitrack audio and video, such as in videos with multiple view angles encoded in one, or ones with a sign language video track, an audio description audio track, a caption track and several subtitle tracks in different languages (i.e. several theora, several vorbis and several kate tracks).

While encoding of multitrack files is already possible, it is unclear how such files would be rendered, how tracks would be differentiated and addressed (e.g. from a JavaScript API), etc. Skeleton has been built in a way such that it is extensible with message header fields for this purpose.

On this wiki page, we are collecting such new information fields.

=== Content-type ===

Right now, there is one mandatory message header field for all of the logical bitstreams: the "Content-type" header field, which contains the mime type of the track. The mime types in use here are listed at http://wiki.xiph.org/MIME_Types_and_File_Extensions#Codec_MIME_types.

=== Language ===

Content in a track usually originates from a specific language. This language can be specified in a Language message header field. The code is created according to http://www.w3.org/TR/ltli/ and http://www.rfc-editor.org/rfc/bcp/bcp47.txt.

For audio tracks with speech, the Language would be the language that dominates.

For video tracks, it might be the language that is signed (if it is a sign language video), or the language that is most often represented in scene text.

For text tracks, it is the dominating language in the text, e.g. English or German subtitles.

Examples are: en-US, de-DE, sgn-ase, en-cockney

=== Role ===

Role describe what semantic type of content is contained in a track. Every track can only have a single role value, so the most appropriate role has to be chosen. The same role can be used across multiple tracks.

The following list some commonly used roles. Other roles are possible, too, but should only be used/introduced if there is really a need for it.

Text tracks:
* "text/caption"
* "text/subtitle"
* "text/textaudiodesc"
* "text/karaoke"
* "text/chapters"
* "text/tickertext"
* "text/lyrics"
* "text/activeregion"
* "text/metadata"
* "text/annotation"
* "text/transcript"
* "text/linguistic"
* "text/chapters"

Video tracks:
* "video/main"
* "video/alternate" (e.g. different camera angle)
* "video/sign" (for sign language)
* "video/alpha" (a track to alpha blend)

Audio tracks:
* "audio/main"
* "audio/alternate" (probably linked to an alternate video track)
* "audio/dub"
* "audio/audiodesc"
* "audio/music"
* "audio/speech"
* "audio/sfx" (sound effects)

Notice how we are re-using the Content-type approach of specifying the main semantic type of the track first. This is necessary, since mime types don't always provide the right main content type (e.g. application/kate is semantically a text format).

There may also be parameters to describe the roles better, such as "video/alternate;angle=nw"

=== Display-hint ===

Media players that do not get informed about how a content author intends a media file to be displayed have no change to display the content "correctly". This is why the Display-hint message header field allows providing of hints on how a certain track should be displayed. A media player can of course decide to ignore these hints.

Currently available hints are:

* pip(x,y,w,h) on a video track - picture-in-picture display in relation to the zero coordinates of the display area of the video with x,y providing the origin of the top left corner of the PIP video and w,h the width and height in pixels which are optional. x, y, w, and h can be specified in percentage, thus allowing persistent placement independent of the scaling of the video display.

Examples:
Display-hint: pip(20%,20%)
Display-hint: pip(40,40,690,60)

* mask(img,x,y,w,h) on a video track - use the image given at img url (?) as a video mask to allow the video to appear in shapes other than rectangular. The masking image should be a black shape on a white background. The image is placed at offset x,y and scaled to width and height w and h. x,y,w, and h can be provided in pixels or in percent. Pixels under the white background are made transparent and only pixels under the black shape are retained.

Examples:
Display-hint: mask(http://www.example.com/image.png)
Display-hint: mask(http://www.example.com/image.png,30%,25%)
Display-hint: mask(http://www.example.com/image.png,20,20,400,320)

* transparent(transparency) on a video track - put a transparency of x% (int value between 0 and 100) on the complete video track as it will be rendered on top of other content. This transparency is applied to all pixels in the same way.

Examples:
Display-hint: transparent(25%)
Display-hint: transparent(7%)

* transparentcolor(colorcode) on a video track - turn all pixels of the color identified by the colorcode into transparent pixels.

Examples:
Display-hint: transparentcolor(#454545)
Display-hint: transparentcolor(#777777)

=== Name ===

This field provides the opportunity to associate a free text string with the track to allow direct addressing of the track through its name.

Characters allowed are basically all the characters that are also allowed for XML id fields:

the first character has to be one of:
[A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] |
[#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF]

any following characters can be one of:
[A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] |
[#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF] |
"-" | "." | [0-9] | #xB7 | [#x0300-#x036F] | [#x203F-#x2040]

The name needs to be unique between all the track names, otherwise it is undefined which of the tracks is retrieved when addressing by name.

An example means of addressing the track by name is: track[name="Madonna_singing"]

=== Track order ===

In many applications it is necessary to walk through all the tracks in a media file and address tracks by an index.

In Ogg, the means to number through the tracks is by the order in which the bos pages of the tracks appear in the Ogg stream. If a file is re-encoded, the order may change, so you can only rely on this for addressing if the file doesn't change.

For example, a video file with the following composition would have the following indexes:
* track[0]: Skeleton BOS
* track[1]: Theora BOS for main video
* track[2]: Vorbis BOS for main audio
* track[3]: Kate BOS for English captions
* track[4]: Kate BOS for German subtitles
* track[5]: Vorbis BOS for audio descriptions
* track[6]: Theora BOS for sign language

This track order is simply to have a means to address tracks through an index in a consistent manner across different media players, such that e.g. JavaScript can always link to the same track reliably across browsers. It has no influence on what should be displayed on top of which other track.

=== Altitude ===

The Altitude (better name?) message header field defines the stack order of the tracks, i.e. which track is displayed further towards the top of the stack and which further down. By default, a "main" track is always displayed bottom-most unless otherwise defined.

The Altitude field takes the same numerical values as the z-index in CSS, unlimited negative and positive numbers.
An element with greater stack order is always in front of an element with a lower stack order.

Example: Altitude: -150

=== Track dependencies ===

It is tempting to introduce dependencies between tracks - to specify things such as:

* track b depends on track a being available (e.g. main audio depending on main video), so always display them together and if you remove a track, remove all depending tracks, too

* track c and d are alternative tracks to track b (e.g. dubs in other languages for main audio), so don't display them together and if you activate one, disable the others

* track a and one of b,c,d one of e,f,g where e depends on b, f depends on c, and g depends on d, make up a presentation profile and should be displayed together (e.g. main video, one of the audio dubs, and their respective captions).

It is not clear yet whether there is an actual need to maintain this information as author-provided hints or whether a media player can itself determine a lot from the other fields, such as role and language.

MPEG has a "groupID" element which allows for tracks to be put into groups of alternative tracks. This feature is, however, not used very often and decisions are being left to the media player.

At this stage, it's probably too early to make a specification for how to encode this in Ogg. The need has not been totally clarified yet.

SkeletonHeaders

2010-03-21T00:51:40Z

Silvia: clarifications

== Adding Required Headers to Skeleton ==

With the HTML5 video element, Ogg is now a major format on the Web and is being applied to solve use cases it hasn't had to solve before, but was built to allow, see http://www.xiph.org/ogg/doc/oggstream.html.

One particular such use case is dealing with multitrack audio and video, such as in videos with multiple view angles encoded in one, or ones with a sign language video track, an audio description audio track, a caption track and several subtitle tracks in different languages (i.e. several theora, several vorbis and several kate tracks).

While encoding of multitrack files is already possible, it is unclear how such files would be rendered, how tracks would be differentiated and addressed (e.g. from a JavaScript API), etc. Skeleton has been built in a way such that it is extensible with message header fields for this purpose.

On this wiki page, we are collecting such new information fields.

=== Content-type ===

Right now, there is one mandatory message header field for all of the logical bitstreams: the "Content-type" header field, which contains the mime type of the track. The mime types in use here are listed at http://wiki.xiph.org/MIME_Types_and_File_Extensions#Codec_MIME_types.

=== Language ===

Content in a track usually originates from a specific language. This language can be specified in a Language message header field. The code is created according to http://www.w3.org/TR/ltli/ and http://www.rfc-editor.org/rfc/bcp/bcp47.txt.

For audio tracks with speech, the Language would be the language that dominates.

For video tracks, it might be the language that is signed (if it is a sign language video), or the language that is most often represented in scene text.

For text tracks, it is the dominating language in the text, e.g. English or German subtitles.

Examples are: en-US, de-DE, sgn-ase, en-cockney

=== Role ===

Role describe what semantic type of content is contained in a track. Every track can only have a single role value, so the most appropriate role has to be chosen. The same role can be used across multiple tracks.

The following list some commonly used roles. Other roles are possible, too, but should only be used/introduced if there is really a need for it.

Text tracks:
* "text/caption"
* "text/subtitle"
* "text/textaudiodesc"
* "text/karaoke"
* "text/chapters"
* "text/tickertext"
* "text/lyrics"
* "text/activeregion"
* "text/metadata"
* "text/annotation"
* "text/transcript"
* "text/linguistic"
* "text/chapters"

Video tracks:
* "video/main"
* "video/alternate" (e.g. different camera angle)
* "video/sign" (for sign language)
* "video/alpha" (a track to alpha blend)

Audio tracks:
* "audio/main"
* "audio/alternate" (probably linked to an alternate video track)
* "audio/dub"
* "audio/audiodesc"
* "audio/music"
* "audio/speech"
* "audio/sfx" (sound effects)

Notice how we are re-using the Content-type approach of specifying the main semantic type of the track first. This is necessary, since mime types don't always provide the right main content type (e.g. application/kate is semantically a text format).

There may also be parameters to describe the roles better, such as "video/alternate;angle=nw"

=== Display-hint ===

Media players that do not get informed about how a content author intends a media file to be displayed have no change to display the content "correctly". This is why the Display-hint message header field allows providing of hints on how a certain track should be displayed. A media player can of course decide to ignore these hints.

Currently available hints are:

* pip(x,y,w,h) on a video track - picture-in-picture display in relation to the zero coordinates of the display area of the video with x,y providing the origin of the top left corner of the PIP video and w,h the width and height in pixels which are optional. x, y, w, and h can be specified in percentage, thus allowing persistent placement independent of the scaling of the video display.

Examples:
Display-hint: pip(20%,20%)
Display-hint: pip(40,40,690,60)

* mask(img,x,y,w,h) on a video track - use the image given at img url (?) as a video mask to allow the video to appear in shapes other than rectangular. The masking image should be a black shape on a white background. The image is placed at offset x,y and scaled to width and height w and h. x,y,w, and h can be provided in pixels or in percent. Pixels under the white background are made transparent and only pixels under the black shape are retained.

Examples:
Display-hint: mask(http://www.example.com/image.png)
Display-hint: mask(http://www.example.com/image.png,30%,25%)
Display-hint: mask(http://www.example.com/image.png,20,20,400,320)

* transparent(transparency) on a video track - put a transparency of x% (int value between 0 and 100) on the complete video track as it will be rendered on top of other content. This transparency is applied to all pixels in the same way.

Examples:
Display-hint: transparent(25%)
Display-hint: transparent(7%)

* transparentcolor(colorcode) on a video track - turn all pixels of the color identified by the colorcode into transparent pixels.

Examples:
Display-hint: transparentcolor(#454545)
Display-hint: transparentcolor(#777777)

=== Name ===

This field provides the opportunity to associate a free text string with the track to allow direct addressing of the track through its name.

Characters allowed are basically all the characters that are also allowed for XML id fields:

the first character has to be one of:
[A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] |
[#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF]

any following characters can be one of:
[A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] |
[#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF] |
"-" | "." | [0-9] | #xB7 | [#x0300-#x036F] | [#x203F-#x2040]

The name needs to be unique between all the track names, otherwise it is undefined which of the tracks is retrieved when addressing by name.

An example means of addressing the track by name is: track[name="Madonna_singing"]

=== Track order ===

In many applications it is necessary to walk through all the tracks in a media file and address tracks by an index.

In Ogg, the means to number through the tracks is by the order in which the bos pages of the tracks appear in the Ogg stream. If a file is re-encoded, the order may change, so you can only rely on this for addressing if the file doesn't change.

For example, a video file with the following composition would have the following indexes:
* track[0]: Skeleton BOS
* track[1]: Theora BOS for main video
* track[2]: Vorbis BOS for main audio
* track[3]: Kate BOS for English captions
* track[4]: Kate BOS for German subtitles
* track[5]: Vorbis BOS for audio descriptions
* track[6]: Theora BOS for sign language

This track order is simply to have a means to address tracks through an index in a consistent manner across different media players, such that e.g. JavaScript can always link to the same track reliably across browsers. It has no influence on what should be displayed on top of which other track.

=== Altitude ===

The Altitude (better name?) message header field defines the stack order of the tracks, i.e. which track is displayed further towards the top of the stack and which further down. By default, a "main" track is always displayed bottom-most unless otherwise defined.

The Altitude field takes the same numerical values as the z-index in CSS, unlimited negative and positive numbers.
An element with greater stack order is always in front of an element with a lower stack order.

Example: Altitude: -150

=== Track dependencies ===

It is tempting to introduce dependencies between tracks - to specify things such as:

* track b depends on track a being available (e.g. main audio depending on main video), so always display them together and if you remove a track, remove all depending tracks, too

* track c and d are alternative tracks to track b (e.g. dubs in other languages for main audio), so don't display them together and if you activate one, disable the others

* track a and one of b,c,d one of e,f,g where e depends on b, f depends on c, and g depends on d, make up a presentation profile and should be displayed together (e.g. main video, one of the audio dubs, and their respective captions).

It is not clear yet whether there is an actual need to maintain this information as author-provided hints or whether a media player can itself determine a lot from the other fields, such as role and language.

MPEG has a "groupID" element which allows for tracks to be put into groups of alternative tracks. This feature is, however, not used very often and decisions are being left to the media player.

At this stage, it's probably too early to make a specification for how to encode this in Ogg. The need has not been totally clarified yet.

SkeletonHeaders

2010-03-21T00:39:01Z

Silvia: turn it into actual percentages

== Adding Required Headers to Skeleton ==

With the HTML5 video element, Ogg is now a major format on the Web and is being applied to solve use cases it hasn't had to solve before, but was built to allow, see http://www.xiph.org/ogg/doc/oggstream.html.

One particular such use case is dealing with multitrack audio and video, such as in videos with multiple view angles encoded in one, or ones with a sign language video track, an audio description audio track, a caption track and several subtitle tracks in different languages (i.e. several theora, several vorbis and several kate tracks).

While encoding of multitrack files is already possible, it is unclear how such files would be rendered, how tracks would be differentiated and addressed (e.g. from a JavaScript API), etc. Skeleton has been built in a way such that it is extensible with message header fields for this purpose.

On this wiki page, we are collecting such new information fields.

=== Content-type ===

Right now, there is one mandatory message header field for all of the logical bitstreams: the "Content-type" header field, which contains the mime type of the track. The mime types in use here are listed at http://wiki.xiph.org/MIME_Types_and_File_Extensions#Codec_MIME_types.

=== Language ===

Content in a track usually originates from a specific language. This language can be specified in a Language message header field. The code is created according to http://www.w3.org/TR/ltli/ and http://www.rfc-editor.org/rfc/bcp/bcp47.txt.

For audio tracks with speech, the Language would be the language that dominates.

For video tracks, it might be the language that is signed (if it is a sign language video), or the language that is most often represented in scene text.

For text tracks, it is the dominating language in the text, e.g. English or German subtitles.

Examples are: en-US, de-DE, sgn-ase, en-cockney

=== Role ===

Role describe what semantic type of content is contained in a track. Every track can only have a single role value, so the most appropriate role has to be chosen. The same role can be used across multiple tracks.

The following list some commonly used roles. Other roles are possible, too, but should only be used/introduced if there is really a need for it.

Text tracks:
* "text/caption"
* "text/subtitle"
* "text/textaudiodesc"
* "text/karaoke"
* "text/chapters"
* "text/tickertext"
* "text/lyrics"
* "text/activeregion"
* "text/metadata"
* "text/annotation"
* "text/transcript"
* "text/linguistic"
* "text/chapters"

Video tracks:
* "video/main"
* "video/alternate" (e.g. different camera angle)
* "video/sign" (for sign language)
* "video/alpha" (a track to alpha blend)

Audio tracks:
* "audio/main"
* "audio/alternate" (probably linked to an alternate video track)
* "audio/dub"
* "audio/audiodesc"
* "audio/music"
* "audio/speech"
* "audio/sfx" (sound effects)

Notice how we are re-using the Content-type approach of specifying the main semantic type of the track first. This is necessary, since mime types don't always provide the right main content type (e.g. application/kate is semantically a text format).

There may also be parameters to describe the roles better, such as "video/alternate;angle=nw"

=== Display-hint ===

Media players that do not get informed about how a content author intends a media file to be displayed have no change to display the content "correctly". This is why the Display-hint message header field allows providing of hints on how a certain track should be displayed. A media player can of course decide to ignore these hints.

Currently available hints are:

* pip(x,y,w,h) on a video track - picture-in-picture display in relation to the "main" video track with x,y providing the origin of the top left corner of the PIP video and w,h the width and height which are optional

Examples:
Display-hint: pip(20,20)
Display-hint: pip(40,40,690,60)

* mask(img,x,y,w,h) on a video track - use the image given at img url (?) as a video mask to allow the video to appear in shapes other than rectangular. The masking image should be a black shape on a white background. The image is placed at offset x,y and scaled to width and height w and h. Pixels under the white background are made transparent and only pixels under the black shape are retained.

Examples:
Display-hint: mask(http://www.example.com/image.png)
Display-hint: mask(http://www.example.com/image.png,20,20,400,320)

* transparent(transparency) on a video track - put a transparency of x% (int value between 0 and 100) on the complete video track as it will be rendered on top of other content.

Examples:
Display-hint: transparent(25)
Display-hint: transparent(7)

* transparentcolor(colorcode) on a video track - turn the color identified by the colorcode into transparent pixels.

Examples:
Display-hint: transparentcolor(#454545)
Display-hint: transparentcolor(#777777)

=== Name ===

This field provides the opportunity to associate a free text string with the track to allow direct addressing of the track through its name.

Characters allowed are basically all the characters that are also allowed for XML id fields:

the first character has to be one of:
[A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] |
[#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF]

any following characters can be one of:
[A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] |
[#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF] |
"-" | "." | [0-9] | #xB7 | [#x0300-#x036F] | [#x203F-#x2040]

The name needs to be unique between all the track names, otherwise it is undefined which of the tracks is retrieved when addressing by name.

An example means of addressing the track by name is: track[name="Madonna_singing"]

=== Track order ===

In many applications it is necessary to walk through all the tracks in a media file and address tracks by an index.

In Ogg, the means to number through the tracks is by the order in which the bos pages of the tracks appear in the Ogg stream. If a file is re-encoded, the order may change, so you can only rely on this for addressing if the file doesn't change.

For example, a video file with the following composition would have the following indexes:
* track[0]: Skeleton BOS
* track[1]: Theora BOS for main video
* track[2]: Vorbis BOS for main audio
* track[3]: Kate BOS for English captions
* track[4]: Kate BOS for German subtitles
* track[5]: Vorbis BOS for audio descriptions
* track[6]: Theora BOS for sign language

This track order is simply to have a means to address tracks through an index in a consistent manner across different media players, such that e.g. JavaScript can always link to the same track reliably across browsers. It has no influence on what should be displayed on top of which other track.

=== Altitude ===

The Altitude (better name?) message header field defines the stack order of the tracks, i.e. which track is displayed further towards the top of the stack and which further down. By default, a "main" track is always displayed bottom-most unless otherwise defined.

The Altitude field takes the same numerical values as the z-index in CSS, unlimited negative and positive numbers.
An element with greater stack order is always in front of an element with a lower stack order.

Example: Altitude: -150

=== Track dependencies ===

It is tempting to introduce dependencies between tracks - to specify things such as:

* track b depends on track a being available (e.g. main audio depending on main video), so always display them together and if you remove a track, remove all depending tracks, too

* track c and d are alternative tracks to track b (e.g. dubs in other languages for main audio), so don't display them together and if you activate one, disable the others

* track a and one of b,c,d one of e,f,g where e depends on b, f depends on c, and g depends on d, make up a presentation profile and should be displayed together (e.g. main video, one of the audio dubs, and their respective captions).

It is not clear yet whether there is an actual need to maintain this information as author-provided hints or whether a media player can itself determine a lot from the other fields, such as role and language.

MPEG has a "groupID" element which allows for tracks to be put into groups of alternative tracks. This feature is, however, not used very often and decisions are being left to the media player.

At this stage, it's probably too early to make a specification for how to encode this in Ogg. The need has not been totally clarified yet.

SkeletonHeaders

2010-03-21T00:29:27Z

Silvia: added dependencies section

== Adding Required Headers to Skeleton ==

With the HTML5 video element, Ogg is now a major format on the Web and is being applied to solve use cases it hasn't had to solve before, but was built to allow, see http://www.xiph.org/ogg/doc/oggstream.html.

One particular such use case is dealing with multitrack audio and video, such as in videos with multiple view angles encoded in one, or ones with a sign language video track, an audio description audio track, a caption track and several subtitle tracks in different languages (i.e. several theora, several vorbis and several kate tracks).

While encoding of multitrack files is already possible, it is unclear how such files would be rendered, how tracks would be differentiated and addressed (e.g. from a JavaScript API), etc. Skeleton has been built in a way such that it is extensible with message header fields for this purpose.

On this wiki page, we are collecting such new information fields.

=== Content-type ===

Right now, there is one mandatory message header field for all of the logical bitstreams: the "Content-type" header field, which contains the mime type of the track. The mime types in use here are listed at http://wiki.xiph.org/MIME_Types_and_File_Extensions#Codec_MIME_types.

=== Language ===

Content in a track usually originates from a specific language. This language can be specified in a Language message header field. The code is created according to http://www.w3.org/TR/ltli/ and http://www.rfc-editor.org/rfc/bcp/bcp47.txt.

For audio tracks with speech, the Language would be the language that dominates.

For video tracks, it might be the language that is signed (if it is a sign language video), or the language that is most often represented in scene text.

For text tracks, it is the dominating language in the text, e.g. English or German subtitles.

Examples are: en-US, de-DE, sgn-ase, en-cockney

=== Role ===

Role describe what semantic type of content is contained in a track. Every track can only have a single role value, so the most appropriate role has to be chosen. The same role can be used across multiple tracks.

The following list some commonly used roles. Other roles are possible, too, but should only be used/introduced if there is really a need for it.

Text tracks:
* "text/caption"
* "text/subtitle"
* "text/textaudiodesc"
* "text/karaoke"
* "text/chapters"
* "text/tickertext"
* "text/lyrics"
* "text/activeregion"
* "text/metadata"
* "text/annotation"
* "text/transcript"
* "text/linguistic"
* "text/chapters"

Video tracks:
* "video/main"
* "video/alternate" (e.g. different camera angle)
* "video/sign" (for sign language)
* "video/alpha" (a track to alpha blend)

Audio tracks:
* "audio/main"
* "audio/alternate" (probably linked to an alternate video track)
* "audio/dub"
* "audio/audiodesc"
* "audio/music"
* "audio/speech"
* "audio/sfx" (sound effects)

Notice how we are re-using the Content-type approach of specifying the main semantic type of the track first. This is necessary, since mime types don't always provide the right main content type (e.g. application/kate is semantically a text format).

There may also be parameters to describe the roles better, such as "video/alternate;angle=nw"

=== Display-hint ===

Media players that do not get informed about how a content author intends a media file to be displayed have no change to display the content "correctly". This is why the Display-hint message header field allows providing of hints on how a certain track should be displayed. A media player can of course decide to ignore these hints.

Currently available hints are:

* pip(x,y,w,h) on a video track - picture-in-picture display in relation to the "main" video track with x,y providing the origin of the top left corner of the PIP video and w,h the width and height which are optional

Examples:
Display-hint: pip(20,20)
Display-hint: pip(40,40,690,60)

* mask(img,x,y,w,h) on a video track - use the image given at img url (?) as a video mask to allow the video to appear in shapes other than rectangular. The masking image should be a black shape on a white background. The image is placed at offset x,y and scaled to width and height w and h. Pixels under the white background are made transparent and only pixels under the black shape are retained.

Examples:
Display-hint: mask(http://www.example.com/image.png)
Display-hint: mask(http://www.example.com/image.png,20,20,400,320)

* transparent(transparency) on a video track - put a transparency of x% on the complete video track as it will be rendered on top of other content.

Examples:
Display-hint: transparent(0.25)
Display-hint: transparent(0.7)

* transparentcolor(colorcode) on a video track - turn the color identified by the colorcode into transparent pixels.

Examples:
Display-hint: transparentcolor(#454545)
Display-hint: transparentcolor(#777777)

=== Name ===

This field provides the opportunity to associate a free text string with the track to allow direct addressing of the track through its name.

Characters allowed are basically all the characters that are also allowed for XML id fields:

the first character has to be one of:
[A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] |
[#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF]

any following characters can be one of:
[A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] |
[#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF] |
"-" | "." | [0-9] | #xB7 | [#x0300-#x036F] | [#x203F-#x2040]

The name needs to be unique between all the track names, otherwise it is undefined which of the tracks is retrieved when addressing by name.

An example means of addressing the track by name is: track[name="Madonna_singing"]

=== Track order ===

In many applications it is necessary to walk through all the tracks in a media file and address tracks by an index.

In Ogg, the means to number through the tracks is by the order in which the bos pages of the tracks appear in the Ogg stream. If a file is re-encoded, the order may change, so you can only rely on this for addressing if the file doesn't change.

For example, a video file with the following composition would have the following indexes:
* track[0]: Skeleton BOS
* track[1]: Theora BOS for main video
* track[2]: Vorbis BOS for main audio
* track[3]: Kate BOS for English captions
* track[4]: Kate BOS for German subtitles
* track[5]: Vorbis BOS for audio descriptions
* track[6]: Theora BOS for sign language

This track order is simply to have a means to address tracks through an index in a consistent manner across different media players, such that e.g. JavaScript can always link to the same track reliably across browsers. It has no influence on what should be displayed on top of which other track.

=== Altitude ===

The Altitude (better name?) message header field defines the stack order of the tracks, i.e. which track is displayed further towards the top of the stack and which further down. By default, a "main" track is always displayed bottom-most unless otherwise defined.

The Altitude field takes the same numerical values as the z-index in CSS, unlimited negative and positive numbers.
An element with greater stack order is always in front of an element with a lower stack order.

Example: Altitude: -150

=== Track dependencies ===

It is tempting to introduce dependencies between tracks - to specify things such as:

* track b depends on track a being available (e.g. main audio depending on main video), so always display them together and if you remove a track, remove all depending tracks, too

* track c and d are alternative tracks to track b (e.g. dubs in other languages for main audio), so don't display them together and if you activate one, disable the others

* track a and one of b,c,d one of e,f,g where e depends on b, f depends on c, and g depends on d, make up a presentation profile and should be displayed together (e.g. main video, one of the audio dubs, and their respective captions).

It is not clear yet whether there is an actual need to maintain this information as author-provided hints or whether a media player can itself determine a lot from the other fields, such as role and language.

MPEG has a "groupID" element which allows for tracks to be put into groups of alternative tracks. This feature is, however, not used very often and decisions are being left to the media player.

At this stage, it's probably too early to make a specification for how to encode this in Ogg. The need has not been totally clarified yet.

SkeletonHeaders

2010-03-20T13:47:10Z

Silvia: added Altitude

== Adding Required Headers to Skeleton ==

With the HTML5 video element, Ogg is now a major format on the Web and is being applied to solve use cases it hasn't had to solve before, but was built to allow, see http://www.xiph.org/ogg/doc/oggstream.html.

One particular such use case is dealing with multitrack audio and video, such as in videos with multiple view angles encoded in one, or ones with a sign language video track, an audio description audio track, a caption track and several subtitle tracks in different languages (i.e. several theora, several vorbis and several kate tracks).

While encoding of multitrack files is already possible, it is unclear how such files would be rendered, how tracks would be differentiated and addressed (e.g. from a JavaScript API), etc. Skeleton has been built in a way such that it is extensible with message header fields for this purpose.

On this wiki page, we are collecting such new information fields.

=== Content-type ===

Right now, there is one mandatory message header field for all of the logical bitstreams: the "Content-type" header field, which contains the mime type of the track. The mime types in use here are listed at http://wiki.xiph.org/MIME_Types_and_File_Extensions#Codec_MIME_types.

=== Language ===

Content in a track usually originates from a specific language. This language can be specified in a Language message header field. The code is created according to http://www.w3.org/TR/ltli/ and http://www.rfc-editor.org/rfc/bcp/bcp47.txt.

For audio tracks with speech, the Language would be the language that dominates.

For video tracks, it might be the language that is signed (if it is a sign language video), or the language that is most often represented in scene text.

For text tracks, it is the dominating language in the text, e.g. English or German subtitles.

Examples are: en-US, de-DE, sgn-ase, en-cockney

=== Role ===

Role describe what semantic type of content is contained in a track. Every track can only have a single role value, so the most appropriate role has to be chosen. The same role can be used across multiple tracks.

The following list some commonly used roles. Other roles are possible, too, but should only be used/introduced if there is really a need for it.

Text tracks:
* "text/caption"
* "text/subtitle"
* "text/textaudiodesc"
* "text/karaoke"
* "text/chapters"
* "text/tickertext"
* "text/lyrics"
* "text/activeregion"
* "text/metadata"
* "text/annotation"
* "text/transcript"
* "text/linguistic"
* "text/chapters"

Video tracks:
* "video/main"
* "video/alternate" (e.g. different camera angle)
* "video/sign" (for sign language)
* "video/alpha" (a track to alpha blend)

Audio tracks:
* "audio/main"
* "audio/alternate" (probably linked to an alternate video track)
* "audio/dub"
* "audio/audiodesc"
* "audio/music"
* "audio/speech"
* "audio/sfx" (sound effects)

Notice how we are re-using the Content-type approach of specifying the main semantic type of the track first. This is necessary, since mime types don't always provide the right main content type (e.g. application/kate is semantically a text format).

There may also be parameters to describe the roles better, such as "video/alternate;angle=nw"

=== Display-hint ===

Media players that do not get informed about how a content author intends a media file to be displayed have no change to display the content "correctly". This is why the Display-hint message header field allows providing of hints on how a certain track should be displayed. A media player can of course decide to ignore these hints.

Currently available hints are:

* pip(x,y,w,h) on a video track - picture-in-picture display in relation to the "main" video track with x,y providing the origin of the top left corner of the PIP video and w,h the width and height which are optional

Examples:
Display-hint: pip(20,20)
Display-hint: pip(40,40,690,60)

* mask(img,x,y,w,h) on a video track - use the image given at img url (?) as a video mask to allow the video to appear in shapes other than rectangular. The masking image should be a black shape on a white background. The image is placed at offset x,y and scaled to width and height w and h. Pixels under the white background are made transparent and only pixels under the black shape are retained.

Examples:
Display-hint: mask(http://www.example.com/image.png)
Display-hint: mask(http://www.example.com/image.png,20,20,400,320)

* transparent(transparency) on a video track - put a transparency of x% on the complete video track as it will be rendered on top of other content.

Examples:
Display-hint: transparent(0.25)
Display-hint: transparent(0.7)

* transparentcolor(colorcode) on a video track - turn the color identified by the colorcode into transparent pixels.

Examples:
Display-hint: transparentcolor(#454545)
Display-hint: transparentcolor(#777777)

=== Name ===

This field provides the opportunity to associate a free text string with the track to allow direct addressing of the track through its name.

Characters allowed are basically all the characters that are also allowed for XML id fields:

the first character has to be one of:
[A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] |
[#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF]

any following characters can be one of:
[A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] |
[#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF] |
"-" | "." | [0-9] | #xB7 | [#x0300-#x036F] | [#x203F-#x2040]

The name needs to be unique between all the track names, otherwise it is undefined which of the tracks is retrieved when addressing by name.

An example means of addressing the track by name is: track[name="Madonna_singing"]

=== Track order ===

In many applications it is necessary to walk through all the tracks in a media file and address tracks by an index.

In Ogg, the means to number through the tracks is by the order in which the bos pages of the tracks appear in the Ogg stream. If a file is re-encoded, the order may change, so you can only rely on this for addressing if the file doesn't change.

For example, a video file with the following composition would have the following indexes:
* track[0]: Skeleton BOS
* track[1]: Theora BOS for main video
* track[2]: Vorbis BOS for main audio
* track[3]: Kate BOS for English captions
* track[4]: Kate BOS for German subtitles
* track[5]: Vorbis BOS for audio descriptions
* track[6]: Theora BOS for sign language

This track order is simply to have a means to address tracks through an index in a consistent manner across different media players, such that e.g. JavaScript can always link to the same track reliably across browsers. It has no influence on what should be displayed on top of which other track.

=== Altitude ===

The Altitude message header field defines the stack order of the tracks, i.e. which track is displayed further towards the top of the stack and which further down. By default, a "main" track is always displayed bottom-most unless otherwise defined.

The Altitude field takes the same numerical values as the z-index in CSS, unlimited negative and positive numbers.
An element with greater stack order is always in front of an element with a lower stack order.

Example: Altitude: -150

=== Track dependencies ===

SkeletonHeaders

2010-03-20T11:32:47Z

Silvia: started on display hints

SkeletonHeaders

2010-03-20T10:33:54Z

Silvia: added roles section

SkeletonHeaders

2010-03-20T08:37:36Z

Silvia: added more details

SkeletonHeaders

2010-03-20T07:48:54Z

Silvia: started page

MIME Types and File Extensions

2010-03-20T07:06:15Z

Silvia: /* Codec MIME types */

STATUS: Work on RFCs and tools is in process to reflect these policies. More details are [http://wiki.xiph.org/index.php/MIMETypesCodecs here], which also include a specification of the codecs parameter of the MIME tyes. Use the correct file extensions straight away.

DISCLAIMER: currently, application/ogg, video/ogg, audio/ogg and audio/vorbis are registered MIME types. Registration for the others will be undertaken. During this process, the "x-" versions of these unregistered MIME types may be used.

IMPLEMENTATION recommendations and patches: see [[MIME-Migration]].

== .ogx - application/ogg ==

* Ogg Multiplex Profile (anything in [[Ogg]])
* can contain any logical bitstreams multiplexed together in an ogg container
* will replace the .ogg extension from RFC 3534 http://www.ietf.org/rfc/rfc3534.txt
* random multitrack files MUST contain a [[Skeleton]] track to identify all containing logical bitstreams
* apps that identify a logical bitstream which they cannot decode SHOULD ignore it but MAY still decode the ones they can
* thus, e.g. an annodex file can gracefully degrade to .ogx if an app cannot decode [[CMML]] and/or [[Skeleton]]
* USE: application/ogg has been registered, so can be used immediately

== .ogv - video/ogg ==

* Ogg Video Profile (a/v in Ogg container)
* apps supporting .oga, .ogv SHOULD support decoding from muxed Ogg streams
* covers e.g. [[Theora]], Theora + Vorbis, Theora + Speex, Theora + FLAC, [[Dirac]] + Vorbis, [[OggMNG|MNG]] + FLAC, [[OggUVS]] inside Ogg
* This list is not exhaustive (for example, [[Dirac]] + FLAC is acceptable too)
* SHOULD contain a Skeleton track and/or MAY contain a CMML logical bitstream.

== .oga - audio/ogg ==

* Ogg Audio Profile (audio in Ogg container)
* Applications supporting .oga, .ogv SHOULD support decoding from muxed Ogg streams
* Covers Ogg [[FLAC]], [[Ghost]], and [[OggPCM]]
* Although they share the same MIME type, Vorbis and Speex use different file extensions.
* SHOULD contain a Skeleton logical bitstream.
* Vorbis and Speex may use .oga, but it is not the prefered method of distributing these files because of backwards-compatibility issues.

== .ogg - audio/ogg ==

* Ogg Vorbis I Profile
* .ogg applies now for Vorbis I files only
* .ogg has more recently also been used for Ogg FLAC and for Theora, too — these uses are deprecated now in favor of .oga and .ogv respectively
* has been defined in RFC 3534 http://www.ietf.org/rfc/rfc3534.txt for application/ogg, so rfc 3534 will be re-defined

RATIONALE: .ogg has traditionally been used for Vorbis I files, in particular in HW players, hence it is kept for backwards-compatibility

== .spx - audio/ogg ==

* Ogg Speex Profile
* .spx has traditionally been used for Speex files within Ogg and should be considered for backwards-compatibility

== .flac - audio/flac ==

* FLAC in native encapsulation format

== .anx - application/annodex ==

* Profile for multiplexed Ogg that includes a skeleton track and at least one CMML logical bitstream
* apps that identify a logical bitstream which they cannot decode SHOULD ignore it but MAY still decode the ones they can
* apps that come across an annodex file and cannot decode CMML and/or Skeleton, but can deal with the others SHOULD gracefully degrade by ignoring these

== .axa - audio/annodex ==

* Profile for audio in Annodex
* covers e.g. [[Vorbis]], [[Speex]], [[FLAC]], [[Ghost]], [[OggPCM]] inside Ogg with Skeleton and CMML

== .axv - video/annodex ==

* Profile for video in Annodex
* covers e.g. [[Theora]], Theora + Vorbis, Theora + Speex, Theora + FLAC, [[Dirac]] + Vorbis, [[OggMNG|MNG]] + FLAC, [[OggUVS]] inside Ogg with Skeleton and CMML

== .xspf - application/xspf+xml ==

* Profile for XSPF
* Covers [[XSPF]], while being used through XML
* Does not cover [[JSPF]], which is XSPF but on JSON

== Ogg Kate files - application/kate ==

* Binary representation of Kate encapsulated in Ogg
* may have a skeleton
* can be used to identify the mime type of the track itself (e.g. in skeleton)
* uses .ogx extension when in a file by itself
* is subdued by the dominant mime type if in a audio or video file to become audio/ogg or video/ogg

== Codec MIME types ==

Codecs need their own MIME types for streaming in RTP and to be used in multitrack ogg files using skeleton:

* audio/vorbis for Vorbis without container
* video/theora for Theora without container
* audio/speex for Speex without container
* audio/flac for FLAC without and in native container
* text/cmml for CMML without container
* application/kate for the textual representation of Kate (.kate files)

Ogg Index

2010-03-20T06:37:26Z

Silvia: typo - duplicate field

{{draft}}

= Ogg Skeleton 3.3 with Keyframe Index =

'''DRAFT, last updated 27 January 2010'''

'''This specification is still a work in progress, and does not yet constitute an official Ogg track format.'''

== Overview ==

Seeking in an Ogg file is typically implemented as a bisection search
over the pages in the file. The Ogg physical bitstream is bisected and
the next Ogg page's end-time is extracted. The bisection continues until
it reaches an Ogg page with an end-time close enough to the seek target
time. However in media containing streams which have keyframes and
interframes, such as Theora streams, your bisection search won't
necessarily terminate at a keyframe. Thus if you begin decoding after your
first bisection terminates, you're likely to only get partial incomplete
frames, with "visual artifacts", until you decode up to the next keyframe.
So to eliminate these visual artifacts, after the first bisection
terminates, you must extract the keyframe's timestamp from the last Theora
page's granulepos, and seek again back to the start of the keyframe and
decode forward until you reach the frame at the seek target.

This is further complicated by the fact that packets often span multiple
Ogg pages, and that Ogg pages from different streams can be interleaved
between spanning packets.

The bisection method above works fine for seeking in local files, but
for seeking in files served over the Internet via HTTP, each bisection
or non sequential read can trigger a new HTTP request, which can have
very high latency, making seeking very slow.

== Seeking with an index ==

The Skeleton 3.3 bitstream attempts to alleviate this problem, by
providing an index of periodic keyframes for every content stream in an
Ogg segment. Note that the Skeleton 3.3 track only holds data for the
segment or "link" in which it resides. So if two Ogg files are concatenated
together ("chained"), the Skeleton 3.3's keyframe indexes in the first Ogg
segment (the first "link" in the "chain") do not contain information
about the keyframes in the second Ogg segment (the second link in the chain).

Each content track has a separate index, which is stored in its own
packet in the Skeleton 3.3 track. The index for streams without the
concept of a keyframe, such as Vorbis streams, can instead record the
time position at periodic intervals, which achieves the same result.
When this document refers to keyframes, it also implicitly refers to these
independent periodic samples from keyframe-less streams.

All the Skeleton 3.3 track's pages appear in the header pages of the Ogg
segment. This means the all the keyframe indexes are immediately
available once the header packets have been read when playing the media
over a network connection.

For every content stream in an Ogg segment, the Ogg index bitstream
provides seek algorithms with an ordered table of "key points". A key
point is intrinsically associated with exactly one stream, and stores the
offset of the page on which it starts, o, as well as the presentation time
of the keyframe t, as a fraction of seconds. This specifies that in order
to render the stream at presentation time t, the last page which lies before
all information required to render the keyframe at presentation time t begins
exactly at byte offset o, as offset from the beginning of the Ogg segment.
The offset is exactly the first byte of the page, so if you seek to a
keypoint's offset and don't find the beginning of a page there, you can
assume that the Ogg segment has been modified since the index was constructed,
and that the index is now invalid and should not be used. The time t is the
keyframe's presentation time corresponding to the granulepos, and is
represented as a fraction in seconds. Note that if a stream requires any
preroll, this will be accounted for in the time stored in the keypoint.

The Skeleton 3.3 track contains one index for each content stream in the
file. To seek in an Ogg file which contains keyframe indexes, first
construct the set which contains every active streams' last keypoint which
has time less than or equal to the seek target time. Then from that set
of key points, select the key point with the smallest byte offset. You then
verify that there's a page found at exactly that offset, and if so, you can
begin decoding. If the first keyframe you encounter has a time equal to
that stored in the keypoint, you have made the optimal seek, and can safely
continue to decode up to the seek target time. You are guaranteed to pass
keyframes on all streams with time less than or equal to your seek target
time while decoding up to the seek target. However if the first keyframe
you encounter after decoding does not have the same presentation time as
is stored in the keypoint, you then the index is invalid (possibly the file
has been changed without updating the index) and you must either fallback
to a bisection search, or keep decoding if you've landed "close enough"
to the seek target.

Be aware that you cannot assume that any or all Ogg files will contain
keyframe indexes, so when implementing Ogg seeking, you must gracefully
fall-back to a bisection search or other seek algorithm when the index
is not present, or when it is invalid.

The Skeleton 3.3 BOS packet also stores meta data about the segment in
which it resides. It stores the timestamps of the first and last samples
in the segment. This also allows you to determine the duration of the
indexed Ogg media without having to decode the start and end of the
Ogg segment to calculate the difference (which is the duration).

The Skeleton 3.3 BOS packet also contains the length of the indexed segment
in bytes. This is so that if the seek target is outside of the indexed range,
you can immediately move to the next/previous segment and either seek using
that segment's index, or narrow the bisection window if that segment has no
index. You can also use the segement length to verify if the index is valid.
If the contents of the segment have changed, it's highly likely that the
length of the segment has changed as well. When you load the segment's
header pages, you should check the length of the physical segment, and if it
doesn't match that stored in the Skeleton header packet, you know the index
is out of date and not safe to use.

The Skeleton 3.3 BOS packet also contains the offset of the first non header
page in the Ogg segment. This means that if you wish to delay loading of an
index for whatever reason, you can skip forward to that offset, and start
decoding from that offset forwards.

When using the index to seek, you must verify that the index is still
correct. You can consider the index invalid if any of the following are true:

# The segment length stored in the Skeleton BOS packet doesn't match the length of the physical segment, or
# after a seek to a keypoint's offset, you don't land exactly on a page boundary, or
# the first keyframe decoded after seeking to a keypoint's offset doesn't have the same presentation time as stored in the index.

You should also always check the Skeleton version header field
to ensure your decoder correctly knows how to parse the Skeleton track.

Be aware that a keyframe index may not index all keyframes in the Ogg segment,
it may only index periodic keyframes instead.

== Format Specification ==

Unless otherwise specified, all integers and fields in the bitstream are
encoded with the least significant bit coming first in each byte.
Integers and fields comprising of more than one byte are encoded least
significant byte first (i.e. little endian byte order).

The Skeleton 3.3 track is intended to be backwards compatible with the
Skeleton 3.0 specification, available at
http://www.xiph.org/ogg/doc/skeleton.html . Unless specified
differently here, it is safe to assume that anything specified for a
Skeleton 3.0 track holds for a Skeleton 3.3 track.

As per the Skeleton 3.0 track, a segment containing a Skeleton 3.3 track
must begin with a '''Skeleton 3.3 fishead BOS packet''' on a page by itself, with the
following format:

# Identifier: 8 bytes, "fishead\0".
# Version major: 2 Byte unsigned integer denoting the major version (3)
# Version minor: 2 Byte unsigned integer denoting the minor version (1)
# Presentationtime numerator: 8 Byte signed integer
# Presentationtime denominator: 8 Byte signed integer
# Basetime numerator: 8 Byte signed integer
# Basetime denominator: 8 Byte signed integer
# UTC [ISO8601]: a 20 Byte string containing a UTC time
# '''[NEW]''' First-sample-time numerator: 8 byte signed integer representing the numerator for the presentation time of the first sample in the media. Note that samples between the first-sample-time and the Presentationtime are supposed to be skipped during playback.
# '''[NEW]''' First-sample-time denominator: 8 byte signed integer, with value 0 if the timestamp is unknown. Decoders should always ensure that the denominator is not 0 before using it as a divisor!
# '''[NEW]''' Last-sample-time numerator: 8 byte signed integer representing the end time of the last sample in the segment.
# '''[NEW]''' Last-sample-time denominator: 8 byte signed integer, with value 0 if the timestamp is unknown. Decoders should always ensure that the denominator is not 0 before using it as a divisor!
# '''[NEW]''' The length of the segment, in bytes: 8 byte unsigned integer, 0 if unknown.
# '''[NEW]''' The offset of the first non-header page, in bytes: 8 byte unsigned integer, 0 if unknown.

The first-sample-time and last-sample-time are rational numbers, in units
of seconds. If the denominator is 0 for the first-sample-time or the
last-sample-time, then that value was unable to be determined at indexing
time, and is unknown. The duration of the Ogg segment can be calculated by
subtracting the first-sample-time from the last-sample-time.

In '''Skeleton 3.3 the "fisbone" packets remain unchanged from Skeleton
3.0''', and will still follow after the other streams' BOS pages and
secondary header pages.

Before the Skeleton EOS page in the segment header pages come the
Skeleton 3.3 keyframe index packets. There should be one index packet for
each content stream in the Ogg segment, but index packets are not required
for a Skeleton 3.3 track to be considered valid. Each keypoint in the index
is stored in a "keypoint", which in turn stores an offset, checksum, and
timestamp. In order to save space, the offsets and timestamps are stored as
deltas, and then variable byte-encoded. The offset and timestamp deltas
store the difference between the keypoint's offset and timestamp from the
previous keypoint's offset and timestamp. So to calculate the page offset
of a keypoint you must sum the offset deltas of up to and including the
keypoint in the index.

The variable byte encoded integers are encoded using 7 bits per byte to
store the integer's bits, and the high bit is set in the last byte used
to encode the integer. The bits and bytes are in little endian byte order.
For example, the integer 7843, or <tt>0001 1110 1010 0011</tt> in binary, would be
stored as two bytes: <tt>0xBD 0x23</tt>, or <tt>1011 1101 0010 0011</tt> in binary.

Each '''Skeleton 3.3 keyframe index packet''' contains the following:

# Identifier 6 bytes: "index\0"
# The serialno of the stream this index applies to, as a 4 byte field.
# The number of keypoints in this index packet, 'n' as a 8 byte unsigned integer. This can be 0.
# The keypoint presentation time denominator, as an 8 byte signed integer.
# 'n' key points, each of which contain, in the following order:
## the keyframe's page's byte offset delta, as a variable byte encoded integer. This is the number of bytes that this keypoint is after the preceeding keypoint's offset, or from the start of the segment if this is the first keypoint. The keypoint's page start is therefore the sum of the byte-offset-deltas of all the keypoints which come before it.
## the presentation time numerator delta, of the first key frame which starts on the page at the keypoint's offset, as a variable byte encoded integer. This is the difference from the previous keypoint's timestamp numerator. The keypoint's timestamp numerator is therefore the sum of all the timestamp numerator deltas up to and including the keypoint's. Divide the timestamp numerator sum by the timestamp denominator stored earlier in the index packet to determine the presentation time of the keyframe in seconds.

Note that a keypoint always represents the first key frame on a page. If an
Ogg page contains two or more keyframes, the index's key point *must* refer
to the first keyframe on that page, not any subsequent keyframes on that page.

The key points are stored in increasing order by offset (and thus by
presentation time as well).

The byte offsets stored in keypoints are relative to the start of the Ogg
bitstream segment. So if you have a physical Ogg bitstream made up of two
chained Oggs, the offsets in the second Ogg segment's bitstream's index
are relative to the beginning of the second Ogg in the chain, not the first.
Also note that if a physical Ogg bitstream is made up of chained Oggs, the
presence of an index in one segment does not imply that there will be an
index in any other segment.

The exact number of keyframes used to construct key points in the index
is up to the indexer, but to limit the index size, we recommend
including at most one key point per every 64KB of data, or every 2000ms,
whichever is least frequent.

As per the Skeleton 3.0 track, '''the last packet in the Skeleton 3.3 track
is an empty EOS packet'''.

== Software Prototype ==

For a prototype indexer, see [http://github.com/cpearce/OggIndex OggIndex]. Also included there is a program OggIndexValid, which can verify that Theora and Vorbis indexes are valid. If you're implementing your own indexer, or going to be modifying existing indexes, always verify that your modified indexes are valid as per OggIndexValid!

Recent [http://firefogg.org/nightly/ ffmpeg2theora nightlies] will also include a keyframe index in the Skeleton
3.3 track if you specify the command line option <tt>--seek-index</tt>.

To see how indexes improves network seeking performance, you can download a development
version of Firefox which can take advantage of indexes here:

http://pearce.org.nz/video/firefox-indexed-seek-linux.tar.bz2

http://pearce.org.nz/video/firefox-indexed-seek-macosx.dmg

http://pearce.org.nz/video/firefox-indexed-seek-win32.zip

If you already have a Firefox instance running, you'll need to either close your running
Firefox instance before starting the index-capable Firefox, or start the index-capable
Firefox with the <tt>--no-remote</tt> command line parameter.

To compare the network performance of indexed versus non-indexed seeking, point the
index-capable Firefox here:

http://pearce.org.nz/video/indexed-seek-demo.html

IDABC Questionnaire 2009

2009-11-15T23:46:59Z

Silvia: clarified consensus seeking process

<strong>This is a draft document. A work in progress. A scratchpad for ideas. It should not be widely circulated in this form.</strong>

= Context =
We received [http://lists.xiph.org/pipermail/theora/2009-November/002996.html an e-mail] from a consultant studying the suitability of Theora for use in "eGovernment", on behalf of the [http://ec.europa.eu/idabc/ IDABC], an EU governmental agency responsible for "Interoperability" with an emphasis on open source. The investigation is in the context of [http://ec.europa.eu/idabc/en/document/7728 European Interoperability Framework], about which there has been [http://www.computerworlduk.com/community/blogs/index.cfm?entryid=2620&blogid=14&pn=1 some real controversy].

The method of assessment is the Common Assessment Method for Standards and Specifications, including the questions below.

= CAMSS Questions =
== Part 4: Market Criteria ==

This group of Market criteria analyses the formal specification in the scope of its market environment, and more precisely it examines the implementations of the formal specification and the market players. This implies identifying to which extent the formal specification benefits from market support and wide adoption, what are its level of maturity and its capacity of reusability.

Market support is evaluated through an analysis of how many products implementing the formal specification exist, what their market share is and who their end-users are. The quality and the completeness (in case of partitioning) of the implementations of the formal specification can also be analysed. Availability of existing or planned mechanisms to assess conformity of implementations to the standard or to the specification could also be identified. The existence of at least one reference implementation (i.e.: mentioning a recognized certification process) - and of which one is an open source implementation - can also be relevant to the assessment. Wide adoption can also be assessed across domains (i.e.: public and private sectors), in an open environment, and/or in a similar field (i.e.: best practices).

A formal specification is mature if it has been in use and development for long enough that most of its initial problems have been overcome and its underlying technology is well understood and well defined. Maturity is also assessed by identifying if all aspects of the formal specification are considered as validated by usage, (i.e.: if the formal specification is partitioned), and if the reported issues have been solved and documented.

Reusability of a formal specification is enabled if it includes guidelines for its implementation in a given context. The identification of successful implementations of the standard or specification should focus on good practices in a similar field. Its incompatibility with related standards or specifications should also be taken into account.

The ideas behind the Market Criteria can also be expressed in the form of the following questions:

=== Market support ===
* Does the standard have strong support in the marketplace?
: Yes. For example, among web browsers, support for Xiph's Ogg, Theora, and Vorbis standards is now included by default in Mozilla Firefox, Google Chrome, and the latest versions of Opera, representing hundreds of millions of installed users just in this market alone. Further, a QuickTime component exists which enables use of Xiph's Ogg, Theora, and Vorbis standards in all Mac OS X applications that make use of the QuickTime framework - which includes Safari/Webkit, iMovie, QuickTime, and many others. On Windows, DirectShow filters exist which also enable all Windows applications that use the DirectShow framework to use Xiph's Ogg, Theora, and Vorbis standards.
* What products exist for this formal specification ?
: Theora is a video codec, and as such the required products are encoders, decoders, and transmission systems. All three types of products are widely available for Theora.
* How many implementations of the formal specification are there?
: Xiph does not require implementors to acquire any license before implementing the specification. Therefore, we do not have a definitive count of the number of implementations. In addition to the reference implementation, which has been ported to most modern platforms and highly optimized for x86 and ARM CPUs and TI C64x+ DSPs, we are aware of a number of independent, conformant or mostly-conformant implementations. These include two C decoders (ffmpeg and QTheora), a Java decoder (Jheora), a C# decoder, an FPGA decoder, and an FPGA encoder.
* Are there products from different suppliers in the market that implement this formal specification ?
: Yes. Corporations such as Atari, Canonical, DailyMotion, Elphel, Fluendo, Google, Mozilla, Novell, Opera, Red Hat, Sun Microsystems, Ubisoft, and countless others have supplied products with an implementation of the Theora standard.
* Are there many products readily available from a variety of suppliers?
: Yes. Theora has been deployed in embedded devices, security cameras, video games, video conferencing systems, web browsers, home theater systems, and many other products. A complete, legal, open-source reference implementation can also be downloaded free of charge, including components for all major media frameworks (DirectShow, gstreamer, and Quicktime), giving the plethora of applications which use these frameworks the ability to use the codec.
* What is the market share of the products implementing the formal specification, versus other implementations of competing formal specifications ?
: Theora playback is extremely widely available, covering virtually the entire market of personal computers. Theora is also increasingly available in mobile and embedded devices. Since we do not require licensing for products that implement the specification, we do not have market share numbers that can be compared with competing formal specifications. Because implementations are readily available and free, Theora is included in many products that support multiple codecs, and is sometimes the only video codec included in free software products.
* Who are the end-users of these products implementing the formal specification?
: The end users are television viewers, video gamers, web surfers, movie makers, business people, video distribution services, and anyone else who interacts with moving pictures.

=== Maturity ===
* Are there any existing or planned mechanisms to assess conformity of the implementations of the formal specification?
: Yes. In addition to a continuous peer review process, we maintain a suite of [http://v2v.cc/~j/theora_testsuite/ test vectors] that allow implementors to assess decoder conformity. We also provide free online developer support and testing for those attempting to make a conforming implementation. An [http://validator.xiph.org/ online validation service] is available.
* Is there a reference implementation (i.e.: mentioning a recognized certification process)?
: Yes. Xiph maintains a reference implementation called [http://downloads.xiph.org/releases/theora/ libtheora]. In addition to serving as a reference, libtheora is also highly optimized to achieve the maximum possible speed, accuracy, reliability, efficiency, and video quality. As a result, many implementors of Theora adopt the reference implementation.
* Is there an open source implementation?
: Yes. libtheora is made available under a completely permissive BSD-like license. Its open-source nature also contributes to its quality as a reference implementation, as implementors are welcome to contribute their improvements to the reference. There are also several other open source implementations.
* Does the formal specification show wide adoption?
** across different domains? (I.e.: public and private)
: Yes. In addition to the private companies mentioned in the previous section, Theora has also been specified as the sole format supported by non-profit institutions such as Wikipedia, currently the 6th largest website in the world, or as one of a small number of preferred formats supported by other public organizations, such as the Norwegian government.
** in an open environment?
: Yes. On open/free operating systems such as those distributed by Novell/SuSE, Canonical, and Red Hat, Theora is the primary default video codec.
** in a similar field? (i.e.: can best practices be identified?)
* Has the formal specification been in use and development long enough that most of its initial problems have been overcome?
: Yes. Theora was derived from VP3, which was originally released in May 2000. The Theora specification was completed in 2004. Theora has now been used in a wide variety of applications, on the full spectrum of computing devices.
* Is the underlying technology of the standard well-understood? (e.g., a reference model is well defined, appropriate concepts of the technology are in widespread use, the technology may have been in use for many years, a formal mathematical model is defined, etc.)
: Yes. The underlying technology has been in use for nearly a decade, and most of the concepts have been in widespread use for even longer.
* Is the formal specification based upon technology that has not been well-defined and may be relatively new?
: No. The formal specification is based on technology from the On2 VP3 codec, which is substantially similar to simple block-transform codecs like H.261. This class of codecs is extremely well understood, and has been actively in use for over 20 years.
* Has the formal specification been revised? (Yes/No, Nof)
: Yes. The specification of the encoder is continuously revised based on user feedback to improve clarity and accuracy. The specification of the decoding part has been stable for years.
* Is the formal specification under the auspices of an architectural board? (Yes/No)
: No. Although officially maintained by the Xiph.Org Foundation, anyone is free to join this organization, and one need not even be a member to make contributions. However, the core developers will review contributions and make sure they do not contradict the general architecture and they work well with the existing code and the test cases.
* Is the formal specification partitioned in its functionality? (Yes/No)
: No. Theora is very deliberately not partitioned, to avoid the confusion created by a "standard" composed of many incompatible "profiles". The Theora standard does not have any optional components. A compliant Theora decoder can correctly process any Theora stream.
** To what extent does each partition participate to its overall functionality? (NN%)
: N/A.
** To what extent is each partition implemented? (NN%) (cf market adoption)
: N/A.

=== Re-usability ===
* Does the formal specification provide guidelines for its implementation in a given organisation?
: Yes. For example, [http://theora.org/doc/Theora.pdf the Theora specification] provides "non-normative" advice and explanation for implementors of Theora decoders and encoders, including example algorithms for implementing required mathematical transforms. Xiph also maintains [http://wiki.xiph.org/Main_Page a documentation base] for implementors who desire more guidelines beyond the specification itself.
* Can other cases where similar systems implement the formal specification be considered as successful implementations and good practices?
: Xiph's standards have successfully been implemented by many organisations in a wide variety of environments. We maintain (non-exhaustive) [http://wiki.xiph.org/TheoraSoftwarePlayers lists] of products which implement Theora support, many of them open source, so that others may use them as a reference when preparing their own products. A particularly well known, independent, but interoperable implementation is provided by the FFmpeg open source project.
* Is its compatibility with related formal specification documented?
: Yes. For example, [http://theora.org/doc/Theora.pdf the Theora specification] also documents the use of Theora within the [http://www.ietf.org/rfc/rfc3533.txt standard Ogg encapsulation format], and the [http://svn.xiph.org/trunk/theora/doc/draft-ietf-avt-rtp-theora-00.txt TheoraRTP draft specification] explains how to transmit Theora using the [http://tools.ietf.org/html/rfc3550 RTP standard]. In addition, the specification documents Theora's compatibility with ITU-R B.470, ITU-R B.601, ITU-R B.709, SMPTE-170M, [http://tools.ietf.org/html/rfc2044 UTF-8], ISO 10646, and [http://www.xiph.org/vorbis/doc/Vorbis_I_spec.pdf Ogg Vorbis].

== Part 5: Standardisation Criteria ==
From Idabc-camss

Note: Throughout this section, “Organisation” refers to the standardisation/fora/consortia body in charge of the formal specification.

Significant characteristics of the way the organisation operates are for example the way it gives the possibility to stakeholders to influence the evolution of the formal specification, or which conditions it attaches to the use of the formal specification or its implementation. Moreover, it is important to know how the formal specification is defined, supported, and made available, as well as how interaction with stakeholders is managed by the organisation during these steps. Governance of interoperability testing with other formal specifications is also indicative.

The standardisation criteria analyses therefore the following elements:

=== Availability of Documentation ===
The availability of documentation criteria is linked to cost and online availability. Access to all preliminary results documentation can be online, online for members only, offline, offline, for members only or not available. Access can be free or for a fee (which fee?).
: Every Xiph standard is permanently available online to everyone at no cost. For example, we invite everyone to download [http://theora.org/doc/Theora.pdf the most up-to-date copy of the Theora specification], and [http://xiph.org/vorbis/doc/Vorbis_I_spec.html the latest revision of Vorbis]. All previous revisions are available from Xiph's [http://svn.xiph.org/ revision control system].

=== Intellectual Property Right ===
The Intellectual Property Rights evaluation criteria relates to the ability for implementers to use the formal specification in products without legal or financial implications. The IPR policy of the organisation is therefore evaluated according to:
* the availability of the IPR or copyright policies of the organisation (available on-line or off-line, or not available);
: The reference implementations of each codec include all necessary IPR and copyright licenses for that codec, including all documentation, and are freely available to everyone.
* the organisation’s governance to disclose any IPR from any contributor (ex-ante, online, offline, for free for all, for a fee for all, for members only, not available);
: Xiph does not require the identification of specific patents that may be required to implement a standard, however it does require an open-source compatible, royalty free license from a contributor for any such patents they may own before the corresponding technology can be included in a standard. These license are made available online, for free, to all parties.
* the level of IPR set "mandatory" by the organisation (no patent, royalty free patent, patent and RAND with limited liability , patent and classic RAND, patent with explicit licensing, patent with defensive licensing, or none);
: All standards, specifications, and software published by the Xiph.Org Foundation are required to have "open-source compatible" IPR. This means that a contribution must either be entirely clear of any known patents, or any patents that read upon the contribution must be available under a transferable, irrevocable public nonassertion agreement to all people everywhere. For example, see [http://svn.xiph.org/trunk/theora/LICENSE our On2 patent nonassertion warrant]. Other common "royalty free" patent licenses are either not transferable, revocable under certain conditions (such as patent infringement litigation against the originating party), or otherwise impose restrictions that would prevent distribution under common [http://www.opensource.org/ OSI]-approved licenses. These would not be acceptable.
* the level of IPR "recommended" by the organisation (no patent, royalty free patent, patent and RAND with limited liability, patent and classic RAND, patent with explicit licensing, patent with defensive licensing, or none). [Note: RAND (Reasonable and Non Discriminatory License) is based on a "fairness" concept. Companies agree that if they receive any patents on technologies that become essential to the standard then they agree to allow other groups attempting to implement the standard to use these patents and they agree that the charges for the patents shall be reasonable. "RAND with limited availability" is a version of RAND where the "reasonable charges" have an upper limit.]
: Xiph's recommended IPR requirements are the same as our mandatory requirements.

=== Accessibility ===

The accessibility evaluation criteria describe the importance of equal and safe accessibility by the users of implementations of formal specifications. This aspect can be related to safety (physical safety and conformance safety) and accessibility of physical impaired people (design for all).

Focus is made particularly on accessibility and conformance safety. Conformance testing is testing to determine whether a system meets some specified formal specification. The result can be results from a test suite. Conformance validation is when the conformance test uniquely qualifies a given implementation as conformant or not. Conformance certification is a process that provides a public and easily visible "stamp of approval" that an implementation of a standard validates as conformant.

The following questions allow an assessment of accessibility and conformance safety:
* Does a mechanism that ensures disability support by a formal specification exist? (Y/N)
: Yes. Xiph ensures support for users with disabilities by providing specifications for accessible technologies independent of the codec itself. Notably, the Xiph [http://wiki.xiph.org/OggKate OggKate] codec for time-aligned text and image content provides support for subtitles for internationalisation, captions for the hearing-impaired, and textual audio descriptions for the visually impaired. Further, Ogg supports multiple tracks of audio and video content in one container, such that sign language tracks and audio descriptions can be included into one file. For this to work, Xiph have defined [http://wiki.xiph.org/Ogg_Skeleton Skeleton] which holds metadata about each track encapsulated within the one Ogg file. When Theora is transmitted or stored in an Ogg container, it is automatically compatible with these accessibility measures.
* Is conformance governance always part of a standard? (Y/N)
: No. Xiph does not normally provide a formal conformance testing process as part of a standard.
* Is a conformance test offered to implementers? (Y/N)
: Yes. Xiph maintains a suite of [http://v2v.cc/~j/theora_testsuite/ test vectors] that can be used by implementors to confirm basic conformance. Also, Xiph's [http://validator.xiph.org online validation service] is a freely available service that can be used by anyone to check conformance.
* Is conformance validation available to implementers? (Y/N)
: Yes. Informal conformance testing is available to implementors upon request, and Xiph has provided such testing for a number of implementations in the past. The oggz tools contain a validation program called oggz-validate which implementers have made massive use of in the past.
* Is conformance certification available? (Y/N)
: Yes. Xiph does not require certification, but maintains the right to withhold the use of our trademarks from implementors that act in bad faith. Implementors may, however, request explicit permission to use our trademarks with a conforming implementation.
* Is localisation of a formal specification possible? (Y/N)
: Yes. We welcome anyone who wishes to translate Xiph specifications into other languages. We have no policy requiring that the normative specification be written in English.

=== Interoperability governance ===
The interoperability governance evaluation criteria relates to how interoperability is identified and maintained between interoperable formal specifications. In order to do this, the organisation may provide governance for:
* open identification in formal specifications,
* open negotiation in formal specifications,
* open selection in formal specifications.

=== Meeting and consultation ===
The meeting and consultation evaluation criteria relates to the process of defining a formal specification. As formal specifications are usually defined by committees, and these committees normally consist of members of the organisation, this criteria studies how to become a member and which are the financial barriers for this, as well as how are non-members able to have an influence on the process of defining the formal specification. It analyses:
* if the organisation is open to all types of companies and organisations and to individuals;
: Yes. Xiph welcomes representatives from all companies and organizations.
* if the standardisation process may specifically allow participation of members with limited abilities when relevant;
: Yes. Standardization occurs almost entirely in internet communications channels, allowing participants with disabilities to engage fully in the standards development process. We also encourage nonexperts and students to assist us as they can, and to learn about Xiph technologies by participating in the standards development process.
* if meetings are open to all members;
: Xiph meetings are open to everyone. We charge no fee for and place no restrictions on attendance or participation. For example, anyone interested in contributing to the Theora specification may join [http://lists.xiph.org/pipermail/theora-dev/ the Theora development mailing list].
* if all can participate in the formal specification creation process;
: Yes. All people are welcome to participate in the specification creation process. No dues or fees are required to participate
* if non-members can participate in the formal specification creation process.
: Yes. Xiph does not maintain an explicit list of members, and no one is excluded from contributing to specifications as they are developed.

=== Consensus ===
Consensus is decision making primarily with regard to the approval of formal specifications and review with interest groups (non-members). The consensus evaluation criterion is evaluated with the following questions:
* Does the organisation have a stated objective of reaching consensus when making decisions on standards?
: There is no explicitly stated objective of reaching consensus. However, when new contributions are made, the key specification developers will be able to veto on the introduction of a new feature. Generally, differences are discussed openly and new features are adapted until they fit the overall architecture of the standards, at which stage they are introduced into the specification, standards and software.
* If consensus is not reached, can the standard be approved? (answers are: cannot be approved but referred back to working group/committee, approved with 75% majority, approved with 66% majority, approved with 51% majority, can be decided by a "director" or similar in the organisation).
: The standard can be approved without consensus via the decision of a "director" or similar.
* Is there a formal process for external review of standard proposals by interest groups (nonmembers)?
: Since anyone may participate in the development process and make proposals, there is no need for a separate formal process to include proposals by nonmembers.

=== Due Process ===
The due process evaluation criteria relates to the level of respect of each member of the organisation with regard to its rights. More specifically, it must be assured that if a member believes an error has been made in the process of defining a formal specification, it must be possible to appeal this to an independent, higher instance. The question is therefore: can a member formally appeal or raise objections to a procedure or to a technical specification to an independent, higher instance?

: Yes. Even if a member fails an appeal within the organization, because all of the technology Xiph standardizes is open and freely implementable, they are always free to develop their own, competing version. Such competing versions may even still be eligible for standardization under the Xiph umbrella.

=== Changes to the formal specification ===
The suggested changes made to a formal specification need to be presented, evaluated and approved in the same way as the formal specification was first defined. This criteria therefore applies the above criteria to the changes made to the formal specification(availability of documentation, Intellectual Property Right, accessibility, interoperability governance, meeting and consultation, consensus, due process).

: The exact same process is used for revisions to the standard as was used for the original development of the standard, and thus the answers to all of the above questions remain the same.

=== Support ===
It is critical that the organisation takes responsibility for the formal specification throughout its life span. This can be done in several ways such as for example a regular periodic review of the formal specification. The support criteria relates to the level of commitment the organisation has taken to support the formal specification throughout its life:
* does the organisation provide support until removal of the published formal specification from public domain (Including this process?
: Xiph.Org standards are never removed from the public domain. Xiph endeavors to provide support for as long as the standard remains in use.
* does the organisation make the formal specification still available even when in non-maintenance mode?
: Yes. All Xiph.Org standards are freely licensed and will always be available.
* does the organisation add new features and keep the formal specification up-to-date?
: Yes. Xiph maintains its ecosystem of standards on a continuous basis.
* does the organisation rectify problems identified in initial implementations?
: Yes. Xiph maintains [https://trac.xiph.org/report a problem reporting system] that is open to the public, and invites everyone to submit suggestions for improvements. Improvements are made both to the standards documents and to the reference implementations.
* does the organisation only create the formal specification?
: No. Xiph also produces high-quality reusable reference implementations of its standards, released under an open license.

<strong>This is a draft document. A work in progress. A scratchpad for ideas. It should not be widely circulated in this form.</strong>

IDABC Questionnaire 2009

2009-11-15T23:42:22Z

Silvia: clarification on a11y

<strong>This is a draft document. A work in progress. A scratchpad for ideas. It should not be widely circulated in this form.</strong>

= Context =
We received [http://lists.xiph.org/pipermail/theora/2009-November/002996.html an e-mail] from a consultant studying the suitability of Theora for use in "eGovernment", on behalf of the [http://ec.europa.eu/idabc/ IDABC], an EU governmental agency responsible for "Interoperability" with an emphasis on open source. The investigation is in the context of [http://ec.europa.eu/idabc/en/document/7728 European Interoperability Framework], about which there has been [http://www.computerworlduk.com/community/blogs/index.cfm?entryid=2620&blogid=14&pn=1 some real controversy].

The method of assessment is the Common Assessment Method for Standards and Specifications, including the questions below.

= CAMSS Questions =
== Part 4: Market Criteria ==

This group of Market criteria analyses the formal specification in the scope of its market environment, and more precisely it examines the implementations of the formal specification and the market players. This implies identifying to which extent the formal specification benefits from market support and wide adoption, what are its level of maturity and its capacity of reusability.

Market support is evaluated through an analysis of how many products implementing the formal specification exist, what their market share is and who their end-users are. The quality and the completeness (in case of partitioning) of the implementations of the formal specification can also be analysed. Availability of existing or planned mechanisms to assess conformity of implementations to the standard or to the specification could also be identified. The existence of at least one reference implementation (i.e.: mentioning a recognized certification process) - and of which one is an open source implementation - can also be relevant to the assessment. Wide adoption can also be assessed across domains (i.e.: public and private sectors), in an open environment, and/or in a similar field (i.e.: best practices).

A formal specification is mature if it has been in use and development for long enough that most of its initial problems have been overcome and its underlying technology is well understood and well defined. Maturity is also assessed by identifying if all aspects of the formal specification are considered as validated by usage, (i.e.: if the formal specification is partitioned), and if the reported issues have been solved and documented.

Reusability of a formal specification is enabled if it includes guidelines for its implementation in a given context. The identification of successful implementations of the standard or specification should focus on good practices in a similar field. Its incompatibility with related standards or specifications should also be taken into account.

The ideas behind the Market Criteria can also be expressed in the form of the following questions:

=== Market support ===
* Does the standard have strong support in the marketplace?
: Yes. For example, among web browsers, support for Xiph's Ogg, Theora, and Vorbis standards is now included by default in Mozilla Firefox, Google Chrome, and the latest versions of Opera, representing hundreds of millions of installed users just in this market alone. Further, a QuickTime component exists which enables use of Xiph's Ogg, Theora, and Vorbis standards in all Mac OS X applications that make use of the QuickTime framework - which includes Safari/Webkit, iMovie, QuickTime, and many others. On Windows, DirectShow filters exist which also enable all Windows applications that use the DirectShow framework to use Xiph's Ogg, Theora, and Vorbis standards.
* What products exist for this formal specification ?
: Theora is a video codec, and as such the required products are encoders, decoders, and transmission systems. All three types of products are widely available for Theora.
* How many implementations of the formal specification are there?
: Xiph does not require implementors to acquire any license before implementing the specification. Therefore, we do not have a definitive count of the number of implementations. In addition to the reference implementation, which has been ported to most modern platforms and highly optimized for x86 and ARM CPUs and TI C64x+ DSPs, we are aware of a number of independent, conformant or mostly-conformant implementations. These include two C decoders (ffmpeg and QTheora), a Java decoder (Jheora), a C# decoder, an FPGA decoder, and an FPGA encoder.
* Are there products from different suppliers in the market that implement this formal specification ?
: Yes. Corporations such as Atari, Canonical, DailyMotion, Elphel, Fluendo, Google, Mozilla, Novell, Opera, Red Hat, Sun Microsystems, Ubisoft, and countless others have supplied products with an implementation of the Theora standard.
* Are there many products readily available from a variety of suppliers?
: Yes. Theora has been deployed in embedded devices, security cameras, video games, video conferencing systems, web browsers, home theater systems, and many other products. A complete, legal, open-source reference implementation can also be downloaded free of charge, including components for all major media frameworks (DirectShow, gstreamer, and Quicktime), giving the plethora of applications which use these frameworks the ability to use the codec.
* What is the market share of the products implementing the formal specification, versus other implementations of competing formal specifications ?
: Theora playback is extremely widely available, covering virtually the entire market of personal computers. Theora is also increasingly available in mobile and embedded devices. Since we do not require licensing for products that implement the specification, we do not have market share numbers that can be compared with competing formal specifications. Because implementations are readily available and free, Theora is included in many products that support multiple codecs, and is sometimes the only video codec included in free software products.
* Who are the end-users of these products implementing the formal specification?
: The end users are television viewers, video gamers, web surfers, movie makers, business people, video distribution services, and anyone else who interacts with moving pictures.

=== Maturity ===
* Are there any existing or planned mechanisms to assess conformity of the implementations of the formal specification?
: Yes. In addition to a continuous peer review process, we maintain a suite of [http://v2v.cc/~j/theora_testsuite/ test vectors] that allow implementors to assess decoder conformity. We also provide free online developer support and testing for those attempting to make a conforming implementation. An [http://validator.xiph.org/ online validation service] is available.
* Is there a reference implementation (i.e.: mentioning a recognized certification process)?
: Yes. Xiph maintains a reference implementation called [http://downloads.xiph.org/releases/theora/ libtheora]. In addition to serving as a reference, libtheora is also highly optimized to achieve the maximum possible speed, accuracy, reliability, efficiency, and video quality. As a result, many implementors of Theora adopt the reference implementation.
* Is there an open source implementation?
: Yes. libtheora is made available under a completely permissive BSD-like license. Its open-source nature also contributes to its quality as a reference implementation, as implementors are welcome to contribute their improvements to the reference. There are also several other open source implementations.
* Does the formal specification show wide adoption?
** across different domains? (I.e.: public and private)
: Yes. In addition to the private companies mentioned in the previous section, Theora has also been specified as the sole format supported by non-profit institutions such as Wikipedia, currently the 6th largest website in the world, or as one of a small number of preferred formats supported by other public organizations, such as the Norwegian government.
** in an open environment?
: Yes. On open/free operating systems such as those distributed by Novell/SuSE, Canonical, and Red Hat, Theora is the primary default video codec.
** in a similar field? (i.e.: can best practices be identified?)
* Has the formal specification been in use and development long enough that most of its initial problems have been overcome?
: Yes. Theora was derived from VP3, which was originally released in May 2000. The Theora specification was completed in 2004. Theora has now been used in a wide variety of applications, on the full spectrum of computing devices.
* Is the underlying technology of the standard well-understood? (e.g., a reference model is well defined, appropriate concepts of the technology are in widespread use, the technology may have been in use for many years, a formal mathematical model is defined, etc.)
: Yes. The underlying technology has been in use for nearly a decade, and most of the concepts have been in widespread use for even longer.
* Is the formal specification based upon technology that has not been well-defined and may be relatively new?
: No. The formal specification is based on technology from the On2 VP3 codec, which is substantially similar to simple block-transform codecs like H.261. This class of codecs is extremely well understood, and has been actively in use for over 20 years.
* Has the formal specification been revised? (Yes/No, Nof)
: Yes. The specification of the encoder is continuously revised based on user feedback to improve clarity and accuracy. The specification of the decoding part has been stable for years.
* Is the formal specification under the auspices of an architectural board? (Yes/No)
: No. Although officially maintained by the Xiph.Org Foundation, anyone is free to join this organization, and one need not even be a member to make contributions. However, the core developers will review contributions and make sure they do not contradict the general architecture and they work well with the existing code and the test cases.
* Is the formal specification partitioned in its functionality? (Yes/No)
: No. Theora is very deliberately not partitioned, to avoid the confusion created by a "standard" composed of many incompatible "profiles". The Theora standard does not have any optional components. A compliant Theora decoder can correctly process any Theora stream.
** To what extent does each partition participate to its overall functionality? (NN%)
: N/A.
** To what extent is each partition implemented? (NN%) (cf market adoption)
: N/A.

=== Re-usability ===
* Does the formal specification provide guidelines for its implementation in a given organisation?
: Yes. For example, [http://theora.org/doc/Theora.pdf the Theora specification] provides "non-normative" advice and explanation for implementors of Theora decoders and encoders, including example algorithms for implementing required mathematical transforms. Xiph also maintains [http://wiki.xiph.org/Main_Page a documentation base] for implementors who desire more guidelines beyond the specification itself.
* Can other cases where similar systems implement the formal specification be considered as successful implementations and good practices?
: Xiph's standards have successfully been implemented by many organisations in a wide variety of environments. We maintain (non-exhaustive) [http://wiki.xiph.org/TheoraSoftwarePlayers lists] of products which implement Theora support, many of them open source, so that others may use them as a reference when preparing their own products. A particularly well known, independent, but interoperable implementation is provided by the FFmpeg open source project.
* Is its compatibility with related formal specification documented?
: Yes. For example, [http://theora.org/doc/Theora.pdf the Theora specification] also documents the use of Theora within the [http://www.ietf.org/rfc/rfc3533.txt standard Ogg encapsulation format], and the [http://svn.xiph.org/trunk/theora/doc/draft-ietf-avt-rtp-theora-00.txt TheoraRTP draft specification] explains how to transmit Theora using the [http://tools.ietf.org/html/rfc3550 RTP standard]. In addition, the specification documents Theora's compatibility with ITU-R B.470, ITU-R B.601, ITU-R B.709, SMPTE-170M, [http://tools.ietf.org/html/rfc2044 UTF-8], ISO 10646, and [http://www.xiph.org/vorbis/doc/Vorbis_I_spec.pdf Ogg Vorbis].

== Part 5: Standardisation Criteria ==
From Idabc-camss

Note: Throughout this section, “Organisation” refers to the standardisation/fora/consortia body in charge of the formal specification.

Significant characteristics of the way the organisation operates are for example the way it gives the possibility to stakeholders to influence the evolution of the formal specification, or which conditions it attaches to the use of the formal specification or its implementation. Moreover, it is important to know how the formal specification is defined, supported, and made available, as well as how interaction with stakeholders is managed by the organisation during these steps. Governance of interoperability testing with other formal specifications is also indicative.

The standardisation criteria analyses therefore the following elements:

=== Availability of Documentation ===
The availability of documentation criteria is linked to cost and online availability. Access to all preliminary results documentation can be online, online for members only, offline, offline, for members only or not available. Access can be free or for a fee (which fee?).
: Every Xiph standard is permanently available online to everyone at no cost. For example, we invite everyone to download [http://theora.org/doc/Theora.pdf the most up-to-date copy of the Theora specification], and [http://xiph.org/vorbis/doc/Vorbis_I_spec.html the latest revision of Vorbis]. All previous revisions are available from Xiph's [http://svn.xiph.org/ revision control system].

=== Intellectual Property Right ===
The Intellectual Property Rights evaluation criteria relates to the ability for implementers to use the formal specification in products without legal or financial implications. The IPR policy of the organisation is therefore evaluated according to:
* the availability of the IPR or copyright policies of the organisation (available on-line or off-line, or not available);
: The reference implementations of each codec include all necessary IPR and copyright licenses for that codec, including all documentation, and are freely available to everyone.
* the organisation’s governance to disclose any IPR from any contributor (ex-ante, online, offline, for free for all, for a fee for all, for members only, not available);
: Xiph does not require the identification of specific patents that may be required to implement a standard, however it does require an open-source compatible, royalty free license from a contributor for any such patents they may own before the corresponding technology can be included in a standard. These license are made available online, for free, to all parties.
* the level of IPR set "mandatory" by the organisation (no patent, royalty free patent, patent and RAND with limited liability , patent and classic RAND, patent with explicit licensing, patent with defensive licensing, or none);
: All standards, specifications, and software published by the Xiph.Org Foundation are required to have "open-source compatible" IPR. This means that a contribution must either be entirely clear of any known patents, or any patents that read upon the contribution must be available under a transferable, irrevocable public nonassertion agreement to all people everywhere. For example, see [http://svn.xiph.org/trunk/theora/LICENSE our On2 patent nonassertion warrant]. Other common "royalty free" patent licenses are either not transferable, revocable under certain conditions (such as patent infringement litigation against the originating party), or otherwise impose restrictions that would prevent distribution under common [http://www.opensource.org/ OSI]-approved licenses. These would not be acceptable.
* the level of IPR "recommended" by the organisation (no patent, royalty free patent, patent and RAND with limited liability, patent and classic RAND, patent with explicit licensing, patent with defensive licensing, or none). [Note: RAND (Reasonable and Non Discriminatory License) is based on a "fairness" concept. Companies agree that if they receive any patents on technologies that become essential to the standard then they agree to allow other groups attempting to implement the standard to use these patents and they agree that the charges for the patents shall be reasonable. "RAND with limited availability" is a version of RAND where the "reasonable charges" have an upper limit.]
: Xiph's recommended IPR requirements are the same as our mandatory requirements.

=== Accessibility ===

The accessibility evaluation criteria describe the importance of equal and safe accessibility by the users of implementations of formal specifications. This aspect can be related to safety (physical safety and conformance safety) and accessibility of physical impaired people (design for all).

Focus is made particularly on accessibility and conformance safety. Conformance testing is testing to determine whether a system meets some specified formal specification. The result can be results from a test suite. Conformance validation is when the conformance test uniquely qualifies a given implementation as conformant or not. Conformance certification is a process that provides a public and easily visible "stamp of approval" that an implementation of a standard validates as conformant.

The following questions allow an assessment of accessibility and conformance safety:
* Does a mechanism that ensures disability support by a formal specification exist? (Y/N)
: Yes. Xiph ensures support for users with disabilities by providing specifications for accessible technologies independent of the codec itself. Notably, the Xiph [http://wiki.xiph.org/OggKate OggKate] codec for time-aligned text and image content provides support for subtitles for internationalisation, captions for the hearing-impaired, and textual audio descriptions for the visually impaired. Further, Ogg supports multiple tracks of audio and video content in one container, such that sign language tracks and audio descriptions can be included into one file. For this to work, Xiph have defined [http://wiki.xiph.org/Ogg_Skeleton Skeleton] which holds metadata about each track encapsulated within the one Ogg file. When Theora is transmitted or stored in an Ogg container, it is automatically compatible with these accessibility measures.
* Is conformance governance always part of a standard? (Y/N)
: No. Xiph does not normally provide a formal conformance testing process as part of a standard.
* Is a conformance test offered to implementers? (Y/N)
: Yes. Xiph maintains a suite of [http://v2v.cc/~j/theora_testsuite/ test vectors] that can be used by implementors to confirm basic conformance. Also, Xiph's [http://validator.xiph.org online validation service] is a freely available service that can be used by anyone to check conformance.
* Is conformance validation available to implementers? (Y/N)
: Yes. Informal conformance testing is available to implementors upon request, and Xiph has provided such testing for a number of implementations in the past. The oggz tools contain a validation program called oggz-validate which implementers have made massive use of in the past.
* Is conformance certification available? (Y/N)
: Yes. Xiph does not require certification, but maintains the right to withhold the use of our trademarks from implementors that act in bad faith. Implementors may, however, request explicit permission to use our trademarks with a conforming implementation.
* Is localisation of a formal specification possible? (Y/N)
: Yes. We welcome anyone who wishes to translate Xiph specifications into other languages. We have no policy requiring that the normative specification be written in English.

=== Interoperability governance ===
The interoperability governance evaluation criteria relates to how interoperability is identified and maintained between interoperable formal specifications. In order to do this, the organisation may provide governance for:
* open identification in formal specifications,
* open negotiation in formal specifications,
* open selection in formal specifications.

=== Meeting and consultation ===
The meeting and consultation evaluation criteria relates to the process of defining a formal specification. As formal specifications are usually defined by committees, and these committees normally consist of members of the organisation, this criteria studies how to become a member and which are the financial barriers for this, as well as how are non-members able to have an influence on the process of defining the formal specification. It analyses:
* if the organisation is open to all types of companies and organisations and to individuals;
: Yes. Xiph welcomes representatives from all companies and organizations.
* if the standardisation process may specifically allow participation of members with limited abilities when relevant;
: Yes. Standardization occurs almost entirely in internet communications channels, allowing participants with disabilities to engage fully in the standards development process. We also encourage nonexperts and students to assist us as they can, and to learn about Xiph technologies by participating in the standards development process.
* if meetings are open to all members;
: Xiph meetings are open to everyone. We charge no fee for and place no restrictions on attendance or participation. For example, anyone interested in contributing to the Theora specification may join [http://lists.xiph.org/pipermail/theora-dev/ the Theora development mailing list].
* if all can participate in the formal specification creation process;
: Yes. All people are welcome to participate in the specification creation process. No dues or fees are required to participate
* if non-members can participate in the formal specification creation process.
: Yes. Xiph does not maintain an explicit list of members, and no one is excluded from contributing to specifications as they are developed.

=== Consensus ===
Consensus is decision making primarily with regard to the approval of formal specifications and review with interest groups (non-members). The consensus evaluation criterion is evaluated with the following questions:
* Does the organisation have a stated objective of reaching consensus when making decisions on standards?
: There is no explicitly stated objective of reaching consensus.
* If consensus is not reached, can the standard be approved? (answers are: cannot be approved but referred back to working group/committee, approved with 75% majority, approved with 66% majority, approved with 51% majority, can be decided by a "director" or similar in the organisation).
: The standard can be approved without consensus via the decision of a "director" or similar.
* Is there a formal process for external review of standard proposals by interest groups (nonmembers)?
: Since anyone may participate in the development process and make proposals, there is no need for a separate formal process to include proposals by nonmembers.

=== Due Process ===
The due process evaluation criteria relates to the level of respect of each member of the organisation with regard to its rights. More specifically, it must be assured that if a member believes an error has been made in the process of defining a formal specification, it must be possible to appeal this to an independent, higher instance. The question is therefore: can a member formally appeal or raise objections to a procedure or to a technical specification to an independent, higher instance?

: Yes. Even if a member fails an appeal within the organization, because all of the technology Xiph standardizes is open and freely implementable, they are always free to develop their own, competing version. Such competing versions may even still be eligible for standardization under the Xiph umbrella.

=== Changes to the formal specification ===
The suggested changes made to a formal specification need to be presented, evaluated and approved in the same way as the formal specification was first defined. This criteria therefore applies the above criteria to the changes made to the formal specification(availability of documentation, Intellectual Property Right, accessibility, interoperability governance, meeting and consultation, consensus, due process).

: The exact same process is used for revisions to the standard as was used for the original development of the standard, and thus the answers to all of the above questions remain the same.

=== Support ===
It is critical that the organisation takes responsibility for the formal specification throughout its life span. This can be done in several ways such as for example a regular periodic review of the formal specification. The support criteria relates to the level of commitment the organisation has taken to support the formal specification throughout its life:
* does the organisation provide support until removal of the published formal specification from public domain (Including this process?
: Xiph.Org standards are never removed from the public domain. Xiph endeavors to provide support for as long as the standard remains in use.
* does the organisation make the formal specification still available even when in non-maintenance mode?
: Yes. All Xiph.Org standards are freely licensed and will always be available.
* does the organisation add new features and keep the formal specification up-to-date?
: Yes. Xiph maintains its ecosystem of standards on a continuous basis.
* does the organisation rectify problems identified in initial implementations?
: Yes. Xiph maintains [https://trac.xiph.org/report a problem reporting system] that is open to the public, and invites everyone to submit suggestions for improvements. Improvements are made both to the standards documents and to the reference implementations.
* does the organisation only create the formal specification?
: No. Xiph also produces high-quality reusable reference implementations of its standards, released under an open license.

<strong>This is a draft document. A work in progress. A scratchpad for ideas. It should not be widely circulated in this form.</strong>

IDABC Questionnaire 2009

2009-11-15T23:32:10Z

Silvia: some word smithing to clarify "standards" of Xiph

<strong>This is a draft document. A work in progress. A scratchpad for ideas. It should not be widely circulated in this form.</strong>

= Context =
We received [http://lists.xiph.org/pipermail/theora/2009-November/002996.html an e-mail] from a consultant studying the suitability of Theora for use in "eGovernment", on behalf of the [http://ec.europa.eu/idabc/ IDABC], an EU governmental agency responsible for "Interoperability" with an emphasis on open source. The investigation is in the context of [http://ec.europa.eu/idabc/en/document/7728 European Interoperability Framework], about which there has been [http://www.computerworlduk.com/community/blogs/index.cfm?entryid=2620&blogid=14&pn=1 some real controversy].

The method of assessment is the Common Assessment Method for Standards and Specifications, including the questions below.

= CAMSS Questions =
== Part 4: Market Criteria ==

This group of Market criteria analyses the formal specification in the scope of its market environment, and more precisely it examines the implementations of the formal specification and the market players. This implies identifying to which extent the formal specification benefits from market support and wide adoption, what are its level of maturity and its capacity of reusability.

Market support is evaluated through an analysis of how many products implementing the formal specification exist, what their market share is and who their end-users are. The quality and the completeness (in case of partitioning) of the implementations of the formal specification can also be analysed. Availability of existing or planned mechanisms to assess conformity of implementations to the standard or to the specification could also be identified. The existence of at least one reference implementation (i.e.: mentioning a recognized certification process) - and of which one is an open source implementation - can also be relevant to the assessment. Wide adoption can also be assessed across domains (i.e.: public and private sectors), in an open environment, and/or in a similar field (i.e.: best practices).

A formal specification is mature if it has been in use and development for long enough that most of its initial problems have been overcome and its underlying technology is well understood and well defined. Maturity is also assessed by identifying if all aspects of the formal specification are considered as validated by usage, (i.e.: if the formal specification is partitioned), and if the reported issues have been solved and documented.

Reusability of a formal specification is enabled if it includes guidelines for its implementation in a given context. The identification of successful implementations of the standard or specification should focus on good practices in a similar field. Its incompatibility with related standards or specifications should also be taken into account.

The ideas behind the Market Criteria can also be expressed in the form of the following questions:

=== Market support ===
* Does the standard have strong support in the marketplace?
: Yes. For example, among web browsers, support for Xiph's Ogg, Theora, and Vorbis standards is now included by default in Mozilla Firefox, Google Chrome, and the latest versions of Opera, representing hundreds of millions of installed users just in this market alone. Further, a QuickTime component exists which enables use of Xiph's Ogg, Theora, and Vorbis standards in all Mac OS X applications that make use of the QuickTime framework - which includes Safari/Webkit, iMovie, QuickTime, and many others. On Windows, DirectShow filters exist which also enable all Windows applications that use the DirectShow framework to use Xiph's Ogg, Theora, and Vorbis standards.
* What products exist for this formal specification ?
: Theora is a video codec, and as such the required products are encoders, decoders, and transmission systems. All three types of products are widely available for Theora.
* How many implementations of the formal specification are there?
: Xiph does not require implementors to acquire any license before implementing the specification. Therefore, we do not have a definitive count of the number of implementations. In addition to the reference implementation, which has been ported to most modern platforms and highly optimized for x86 and ARM CPUs and TI C64x+ DSPs, we are aware of a number of independent, conformant or mostly-conformant implementations. These include two C decoders (ffmpeg and QTheora), a Java decoder (Jheora), a C# decoder, an FPGA decoder, and an FPGA encoder.
* Are there products from different suppliers in the market that implement this formal specification ?
: Yes. Corporations such as Atari, Canonical, DailyMotion, Elphel, Fluendo, Google, Mozilla, Novell, Opera, Red Hat, Sun Microsystems, Ubisoft, and countless others have supplied products with an implementation of the Theora standard.
* Are there many products readily available from a variety of suppliers?
: Yes. Theora has been deployed in embedded devices, security cameras, video games, video conferencing systems, web browsers, home theater systems, and many other products. A complete, legal, open-source reference implementation can also be downloaded free of charge, including components for all major media frameworks (DirectShow, gstreamer, and Quicktime), giving the plethora of applications which use these frameworks the ability to use the codec.
* What is the market share of the products implementing the formal specification, versus other implementations of competing formal specifications ?
: Theora playback is extremely widely available, covering virtually the entire market of personal computers. Theora is also increasingly available in mobile and embedded devices. Since we do not require licensing for products that implement the specification, we do not have market share numbers that can be compared with competing formal specifications. Because implementations are readily available and free, Theora is included in many products that support multiple codecs, and is sometimes the only video codec included in free software products.
* Who are the end-users of these products implementing the formal specification?
: The end users are television viewers, video gamers, web surfers, movie makers, business people, video distribution services, and anyone else who interacts with moving pictures.

=== Maturity ===
* Are there any existing or planned mechanisms to assess conformity of the implementations of the formal specification?
: Yes. In addition to a continuous peer review process, we maintain a suite of [http://v2v.cc/~j/theora_testsuite/ test vectors] that allow implementors to assess decoder conformity. We also provide free online developer support and testing for those attempting to make a conforming implementation. An [http://validator.xiph.org/ online validation service] is available.
* Is there a reference implementation (i.e.: mentioning a recognized certification process)?
: Yes. Xiph maintains a reference implementation called [http://downloads.xiph.org/releases/theora/ libtheora]. In addition to serving as a reference, libtheora is also highly optimized to achieve the maximum possible speed, accuracy, reliability, efficiency, and video quality. As a result, many implementors of Theora adopt the reference implementation.
* Is there an open source implementation?
: Yes. libtheora is made available under a completely permissive BSD-like license. Its open-source nature also contributes to its quality as a reference implementation, as implementors are welcome to contribute their improvements to the reference. There are also several other open source implementations.
* Does the formal specification show wide adoption?
** across different domains? (I.e.: public and private)
: Yes. In addition to the private companies mentioned in the previous section, Theora has also been specified as the sole format supported by non-profit institutions such as Wikipedia, currently the 6th largest website in the world, or as one of a small number of preferred formats supported by other public organizations, such as the Norwegian government.
** in an open environment?
: Yes. On open/free operating systems such as those distributed by Novell/SuSE, Canonical, and Red Hat, Theora is the primary default video codec.
** in a similar field? (i.e.: can best practices be identified?)
* Has the formal specification been in use and development long enough that most of its initial problems have been overcome?
: Yes. Theora was derived from VP3, which was originally released in May 2000. The Theora specification was completed in 2004. Theora has now been used in a wide variety of applications, on the full spectrum of computing devices.
* Is the underlying technology of the standard well-understood? (e.g., a reference model is well defined, appropriate concepts of the technology are in widespread use, the technology may have been in use for many years, a formal mathematical model is defined, etc.)
: Yes. The underlying technology has been in use for nearly a decade, and most of the concepts have been in widespread use for even longer.
* Is the formal specification based upon technology that has not been well-defined and may be relatively new?
: No. The formal specification is based on technology from the On2 VP3 codec, which is substantially similar to simple block-transform codecs like H.261. This class of codecs is extremely well understood, and has been actively in use for over 20 years.
* Has the formal specification been revised? (Yes/No, Nof)
: Yes. The specification of the encoder is continuously revised based on user feedback to improve clarity and accuracy. The specification of the decoding part has been stable for years.
* Is the formal specification under the auspices of an architectural board? (Yes/No)
: No. Although officially maintained by the Xiph.Org Foundation, anyone is free to join this organization, and one need not even be a member to make contributions. However, the core developers will review contributions and make sure they do not contradict the general architecture and they work well with the existing code and the test cases.
* Is the formal specification partitioned in its functionality? (Yes/No)
: No. Theora is very deliberately not partitioned, to avoid the confusion created by a "standard" composed of many incompatible "profiles". The Theora standard does not have any optional components. A compliant Theora decoder can correctly process any Theora stream.
** To what extent does each partition participate to its overall functionality? (NN%)
: N/A.
** To what extent is each partition implemented? (NN%) (cf market adoption)
: N/A.

=== Re-usability ===
* Does the formal specification provide guidelines for its implementation in a given organisation?
: Yes. For example, [http://theora.org/doc/Theora.pdf the Theora specification] provides "non-normative" advice and explanation for implementors of Theora decoders and encoders, including example algorithms for implementing required mathematical transforms. Xiph also maintains [http://wiki.xiph.org/Main_Page a documentation base] for implementors who desire more guidelines beyond the specification itself.
* Can other cases where similar systems implement the formal specification be considered as successful implementations and good practices?
: Xiph's standards have successfully been implemented by many organisations in a wide variety of environments. We maintain (non-exhaustive) [http://wiki.xiph.org/TheoraSoftwarePlayers lists] of products which implement Theora support, many of them open source, so that others may use them as a reference when preparing their own products. A particularly well known, independent, but interoperable implementation is provided by the FFmpeg open source project.
* Is its compatibility with related formal specification documented?
: Yes. For example, [http://theora.org/doc/Theora.pdf the Theora specification] also documents the use of Theora within the [http://www.ietf.org/rfc/rfc3533.txt standard Ogg encapsulation format], and the [http://svn.xiph.org/trunk/theora/doc/draft-ietf-avt-rtp-theora-00.txt TheoraRTP draft specification] explains how to transmit Theora using the [http://tools.ietf.org/html/rfc3550 RTP standard]. In addition, the specification documents Theora's compatibility with ITU-R B.470, ITU-R B.601, ITU-R B.709, SMPTE-170M, [http://tools.ietf.org/html/rfc2044 UTF-8], ISO 10646, and [http://www.xiph.org/vorbis/doc/Vorbis_I_spec.pdf Ogg Vorbis].

== Part 5: Standardisation Criteria ==
From Idabc-camss

Note: Throughout this section, “Organisation” refers to the standardisation/fora/consortia body in charge of the formal specification.

Significant characteristics of the way the organisation operates are for example the way it gives the possibility to stakeholders to influence the evolution of the formal specification, or which conditions it attaches to the use of the formal specification or its implementation. Moreover, it is important to know how the formal specification is defined, supported, and made available, as well as how interaction with stakeholders is managed by the organisation during these steps. Governance of interoperability testing with other formal specifications is also indicative.

The standardisation criteria analyses therefore the following elements:

=== Availability of Documentation ===
The availability of documentation criteria is linked to cost and online availability. Access to all preliminary results documentation can be online, online for members only, offline, offline, for members only or not available. Access can be free or for a fee (which fee?).
: Every Xiph standard is permanently available online to everyone at no cost. For example, we invite everyone to download [http://theora.org/doc/Theora.pdf the most up-to-date copy of the Theora specification], and [http://xiph.org/vorbis/doc/Vorbis_I_spec.html the latest revision of Vorbis]. All previous revisions are available from Xiph's [http://svn.xiph.org/ revision control system].

=== Intellectual Property Right ===
The Intellectual Property Rights evaluation criteria relates to the ability for implementers to use the formal specification in products without legal or financial implications. The IPR policy of the organisation is therefore evaluated according to:
* the availability of the IPR or copyright policies of the organisation (available on-line or off-line, or not available);
: The reference implementations of each codec include all necessary IPR and copyright licenses for that codec, including all documentation, and are freely available to everyone.
* the organisation’s governance to disclose any IPR from any contributor (ex-ante, online, offline, for free for all, for a fee for all, for members only, not available);
: Xiph does not require the identification of specific patents that may be required to implement a standard, however it does require an open-source compatible, royalty free license from a contributor for any such patents they may own before the corresponding technology can be included in a standard. These license are made available online, for free, to all parties.
* the level of IPR set "mandatory" by the organisation (no patent, royalty free patent, patent and RAND with limited liability , patent and classic RAND, patent with explicit licensing, patent with defensive licensing, or none);
: All standards, specifications, and software published by the Xiph.Org Foundation are required to have "open-source compatible" IPR. This means that a contribution must either be entirely clear of any known patents, or any patents that read upon the contribution must be available under a transferable, irrevocable public nonassertion agreement to all people everywhere. For example, see [http://svn.xiph.org/trunk/theora/LICENSE our On2 patent nonassertion warrant]. Other common "royalty free" patent licenses are either not transferable, revocable under certain conditions (such as patent infringement litigation against the originating party), or otherwise impose restrictions that would prevent distribution under common [http://www.opensource.org/ OSI]-approved licenses. These would not be acceptable.
* the level of IPR "recommended" by the organisation (no patent, royalty free patent, patent and RAND with limited liability, patent and classic RAND, patent with explicit licensing, patent with defensive licensing, or none). [Note: RAND (Reasonable and Non Discriminatory License) is based on a "fairness" concept. Companies agree that if they receive any patents on technologies that become essential to the standard then they agree to allow other groups attempting to implement the standard to use these patents and they agree that the charges for the patents shall be reasonable. "RAND with limited availability" is a version of RAND where the "reasonable charges" have an upper limit.]
: Xiph's recommended IPR requirements are the same as our mandatory requirements.

=== Accessibility ===

The accessibility evaluation criteria describe the importance of equal and safe accessibility by the users of implementations of formal specifications. This aspect can be related to safety (physical safety and conformance safety) and accessibility of physical impaired people (design for all).

Focus is made particularly on accessibility and conformance safety. Conformance testing is testing to determine whether a system meets some specified formal specification. The result can be results from a test suite. Conformance validation is when the conformance test uniquely qualifies a given implementation as conformant or not. Conformance certification is a process that provides a public and easily visible "stamp of approval" that an implementation of a standard validates as conformant.

The following questions allow an assessment of accessibility and conformance safety:
* Does a mechanism that ensures disability support by a formal specification exist? (Y/N)
: Yes. Xiph ensures support for users with disabilities by providing specifications for accessible technologies independent of the codec itself. Notable Xiph specifications include [http://wiki.xiph.org/OggKate OggKate] and [http://wiki.xiph.org/index.php/CMML CMML], which provide subtitles for the hearing-impaired, as well as [http://wiki.xiph.org/Ogg_Skeleton Skeleton], which can specify scene description audio tracks for the visually impaired. When Theora is transmitted or stored in an Ogg container, it is automatically compatible with these accessibility measures.
* Is conformance governance always part of a standard? (Y/N)
: No. Xiph does not normally provide a formal conformance testing process as part of a standard.
* Is a conformance test offered to implementers? (Y/N)
: Yes. Xiph maintains a suite of [http://v2v.cc/~j/theora_testsuite/ test vectors] that can be used by implementors to confirm basic conformance.
* Is conformance validation available to implementers? (Y/N)
: Yes. Informal conformance testing is available to implementors upon request, and Xiph has provided such testing for a number of implementations in the past.
* Is conformance certification available? (Y/N)
: Yes. Xiph does not require certification, but maintains the right to withhold the use of our trademarks from implementors that act in bad faith. Implementors may, however, request explicit permission to use our trademarks with a conforming implementation.
* Is localisation of a formal specification possible? (Y/N)
: Yes. We welcome anyone who wishes to translate Xiph specifications into other languages. We have no policy requiring that the normative specification be written in English.

=== Interoperability governance ===
The interoperability governance evaluation criteria relates to how interoperability is identified and maintained between interoperable formal specifications. In order to do this, the organisation may provide governance for:
* open identification in formal specifications,
* open negotiation in formal specifications,
* open selection in formal specifications.

=== Meeting and consultation ===
The meeting and consultation evaluation criteria relates to the process of defining a formal specification. As formal specifications are usually defined by committees, and these committees normally consist of members of the organisation, this criteria studies how to become a member and which are the financial barriers for this, as well as how are non-members able to have an influence on the process of defining the formal specification. It analyses:
* if the organisation is open to all types of companies and organisations and to individuals;
: Yes. Xiph welcomes representatives from all companies and organizations.
* if the standardisation process may specifically allow participation of members with limited abilities when relevant;
: Yes. Standardization occurs almost entirely in internet communications channels, allowing participants with disabilities to engage fully in the standards development process. We also encourage nonexperts and students to assist us as they can, and to learn about Xiph technologies by participating in the standards development process.
* if meetings are open to all members;
: Xiph meetings are open to everyone. We charge no fee for and place no restrictions on attendance or participation. For example, anyone interested in contributing to the Theora specification may join [http://lists.xiph.org/pipermail/theora-dev/ the Theora development mailing list].
* if all can participate in the formal specification creation process;
: Yes. All people are welcome to participate in the specification creation process. No dues or fees are required to participate
* if non-members can participate in the formal specification creation process.
: Yes. Xiph does not maintain an explicit list of members, and no one is excluded from contributing to specifications as they are developed.

=== Consensus ===
Consensus is decision making primarily with regard to the approval of formal specifications and review with interest groups (non-members). The consensus evaluation criterion is evaluated with the following questions:
* Does the organisation have a stated objective of reaching consensus when making decisions on standards?
: There is no explicitly stated objective of reaching consensus.
* If consensus is not reached, can the standard be approved? (answers are: cannot be approved but referred back to working group/committee, approved with 75% majority, approved with 66% majority, approved with 51% majority, can be decided by a "director" or similar in the organisation).
: The standard can be approved without consensus via the decision of a "director" or similar.
* Is there a formal process for external review of standard proposals by interest groups (nonmembers)?
: Since anyone may participate in the development process and make proposals, there is no need for a separate formal process to include proposals by nonmembers.

=== Due Process ===
The due process evaluation criteria relates to the level of respect of each member of the organisation with regard to its rights. More specifically, it must be assured that if a member believes an error has been made in the process of defining a formal specification, it must be possible to appeal this to an independent, higher instance. The question is therefore: can a member formally appeal or raise objections to a procedure or to a technical specification to an independent, higher instance?

: Yes. Even if a member fails an appeal within the organization, because all of the technology Xiph standardizes is open and freely implementable, they are always free to develop their own, competing version. Such competing versions may even still be eligible for standardization under the Xiph umbrella.

=== Changes to the formal specification ===
The suggested changes made to a formal specification need to be presented, evaluated and approved in the same way as the formal specification was first defined. This criteria therefore applies the above criteria to the changes made to the formal specification(availability of documentation, Intellectual Property Right, accessibility, interoperability governance, meeting and consultation, consensus, due process).

: The exact same process is used for revisions to the standard as was used for the original development of the standard, and thus the answers to all of the above questions remain the same.

=== Support ===
It is critical that the organisation takes responsibility for the formal specification throughout its life span. This can be done in several ways such as for example a regular periodic review of the formal specification. The support criteria relates to the level of commitment the organisation has taken to support the formal specification throughout its life:
* does the organisation provide support until removal of the published formal specification from public domain (Including this process?
: Xiph.Org standards are never removed from the public domain. Xiph endeavors to provide support for as long as the standard remains in use.
* does the organisation make the formal specification still available even when in non-maintenance mode?
: Yes. All Xiph.Org standards are freely licensed and will always be available.
* does the organisation add new features and keep the formal specification up-to-date?
: Yes. Xiph maintains its ecosystem of standards on a continuous basis.
* does the organisation rectify problems identified in initial implementations?
: Yes. Xiph maintains [https://trac.xiph.org/report a problem reporting system] that is open to the public, and invites everyone to submit suggestions for improvements. Improvements are made both to the standards documents and to the reference implementations.
* does the organisation only create the formal specification?
: No. Xiph also produces high-quality reusable reference implementations of its standards, released under an open license.

<strong>This is a draft document. A work in progress. A scratchpad for ideas. It should not be widely circulated in this form.</strong>

IDABC Questionnaire 2009

2009-11-15T23:20:53Z

Silvia: ups, type

<strong>This is a draft document. A work in progress. A scratchpad for ideas. It should not be widely circulated in this form.</strong>

= Context =
We received [http://lists.xiph.org/pipermail/theora/2009-November/002996.html an e-mail] from a consultant studying the suitability of Theora for use in "eGovernment", on behalf of the [http://ec.europa.eu/idabc/ IDABC], an EU governmental agency responsible for "Interoperability" with an emphasis on open source. The investigation is in the context of [http://ec.europa.eu/idabc/en/document/7728 European Interoperability Framework], about which there has been [http://www.computerworlduk.com/community/blogs/index.cfm?entryid=2620&blogid=14&pn=1 some real controversy].

The method of assessment is the Common Assessment Method for Standards and Specifications, including the questions below.

= CAMSS Questions =
== Part 4: Market Criteria ==

This group of Market criteria analyses the formal specification in the scope of its market environment, and more precisely it examines the implementations of the formal specification and the market players. This implies identifying to which extent the formal specification benefits from market support and wide adoption, what are its level of maturity and its capacity of reusability.

Market support is evaluated through an analysis of how many products implementing the formal specification exist, what their market share is and who their end-users are. The quality and the completeness (in case of partitioning) of the implementations of the formal specification can also be analysed. Availability of existing or planned mechanisms to assess conformity of implementations to the standard or to the specification could also be identified. The existence of at least one reference implementation (i.e.: mentioning a recognized certification process) - and of which one is an open source implementation - can also be relevant to the assessment. Wide adoption can also be assessed across domains (i.e.: public and private sectors), in an open environment, and/or in a similar field (i.e.: best practices).

A formal specification is mature if it has been in use and development for long enough that most of its initial problems have been overcome and its underlying technology is well understood and well defined. Maturity is also assessed by identifying if all aspects of the formal specification are considered as validated by usage, (i.e.: if the formal specification is partitioned), and if the reported issues have been solved and documented.

Reusability of a formal specification is enabled if it includes guidelines for its implementation in a given context. The identification of successful implementations of the standard or specification should focus on good practices in a similar field. Its incompatibility with related standards or specifications should also be taken into account.

The ideas behind the Market Criteria can also be expressed in the form of the following questions:

=== Market support ===
* Does the standard have strong support in the marketplace?
: Yes. For example, among web browsers, support for Xiph's Ogg, Theora, and Vorbis standards is now included by default in Mozilla Firefox, Google Chrome, and the latest versions of Opera, representing hundreds of millions of installed users just in this market alone. Further, a QuickTime component exists which enables use of Xiph's Ogg, Theora, and Vorbis standards in all Mac OS X applications that make use of the QuickTime framework - which includes Safari/Webkit, iMovie, QuickTime, and many others. On Windows, DirectShow filters exist which also enable all Windows applications that use the DirectShow framework to use Xiph's Ogg, Theora, and Vorbis standards.
* What products exist for this formal specification ?
: Theora is a video codec, and as such the required products are encoders, decoders, and transmission systems. All three types of products are widely available for Theora.
* How many implementations of the formal specification are there?
: Xiph does not require implementors to acquire any license before implementing the specification. Therefore, we do not have a definitive count of the number of implementations. In addition to the reference implementation, which has been ported to most modern platforms and highly optimized for x86 and ARM CPUs and TI C64x+ DSPs, we are aware of a number of independent, conformant or mostly-conformant implementations. These include two C decoders (ffmpeg and QTheora), a Java decoder (Jheora), a C# decoder, an FPGA decoder, and an FPGA encoder.
* Are there products from different suppliers in the market that implement this formal specification ?
: Yes. Corporations such as Atari, Canonical, DailyMotion, Elphel, Fluendo, Google, Mozilla, Novell, Opera, Red Hat, Sun Microsystems, Ubisoft, and countless others have supplied products with an implementation of the Theora standard.
* Are there many products readily available from a variety of suppliers?
: Yes. Theora has been deployed in embedded devices, security cameras, video games, video conferencing systems, web browsers, home theater systems, and many other products. A complete, legal, open-source reference implementation can also be downloaded free of charge, including components for all major media frameworks (DirectShow, gstreamer, and Quicktime), giving the plethora of applications which use these frameworks the ability to use the codec.
* What is the market share of the products implementing the formal specification, versus other implementations of competing formal specifications ?
: Theora playback is extremely widely available, covering virtually the entire market of personal computers. Theora is also increasingly available in mobile and embedded devices. Since we do not require licensing for products that implement the specification, we do not have market share numbers that can be compared with competing formal specifications. Because implementations are readily available and free, Theora is included in many products that support multiple codecs, and is sometimes the only video codec included in free software products.
* Who are the end-users of these products implementing the formal specification?
: The end users are television viewers, video gamers, web surfers, movie makers, business people, video distribution services, and anyone else who interacts with moving pictures.

=== Maturity ===
* Are there any existing or planned mechanisms to assess conformity of the implementations of the formal specification?
: Yes. In addition to a continuous peer review process, we maintain a suite of [http://v2v.cc/~j/theora_testsuite/ test vectors] that allow implementors to assess decoder conformity. We also provide free online developer support and testing for those attempting to make a conforming implementation. An [http://validator.xiph.org/ online validation service] is available.
* Is there a reference implementation (i.e.: mentioning a recognized certification process)?
: Yes. Xiph maintains a reference implementation called [http://downloads.xiph.org/releases/theora/ libtheora]. In addition to serving as a reference, libtheora is also highly optimized to achieve the maximum possible speed, accuracy, reliability, efficiency, and video quality. As a result, many implementors of Theora adopt the reference implementation.
* Is there an open source implementation?
: Yes. libtheora is made available under a completely permissive BSD-like license. Its open-source nature also contributes to its quality as a reference implementation, as implementors are welcome to contribute their improvements to the reference. There are also several other open source implementations.
* Does the formal specification show wide adoption?
** across different domains? (I.e.: public and private)
: Yes. In addition to the private companies mentioned in the previous section, Theora has also been specified as the sole format supported by non-profit institutions such as Wikipedia, currently the 6th largest website in the world, or as one of a small number of preferred formats supported by other public organizations, such as the Norwegian government.
** in an open environment?
: Yes. On open/free operating systems such as those distributed by Novell/SuSE, Canonical, and Red Hat, Theora is the primary default video codec.
** in a similar field? (i.e.: can best practices be identified?)
* Has the formal specification been in use and development long enough that most of its initial problems have been overcome?
: Yes. Theora was derived from VP3, which was originally released in May 2000. The Theora specification was completed in 2004. Theora has now been used in a wide variety of applications, on the full spectrum of computing devices.
* Is the underlying technology of the standard well-understood? (e.g., a reference model is well defined, appropriate concepts of the technology are in widespread use, the technology may have been in use for many years, a formal mathematical model is defined, etc.)
: Yes. The underlying technology has been in use for nearly a decade, and most of the concepts have been in widespread use for even longer.
* Is the formal specification based upon technology that has not been well-defined and may be relatively new?
: No. The formal specification is based on technology from the On2 VP3 codec, which is substantially similar to simple block-transform codecs like H.261. This class of codecs is extremely well understood, and has been actively in use for over 20 years.
* Has the formal specification been revised? (Yes/No, Nof)
: Yes. The specification of the encoder is continuously revised based on user feedback to improve clarity and accuracy. The specification of the decoding part has been stable for years.
* Is the formal specification under the auspices of an architectural board? (Yes/No)
: No. Although officially maintained by the Xiph.Org Foundation, anyone is free to join this organization, and one need not even be a member to make contributions. However, the core developers will review contributions and make sure they do not contradict the general architecture and they work well with the existing code and the test cases.
* Is the formal specification partitioned in its functionality? (Yes/No)
: No. Theora is very deliberately not partitioned, to avoid the confusion created by a "standard" composed of many incompatible "profiles". The Theora standard does not have any optional components. A compliant Theora decoder can correctly process any Theora stream.
** To what extent does each partition participate to its overall functionality? (NN%)
: N/A.
** To what extent is each partition implemented? (NN%) (cf market adoption)
: N/A.

=== Re-usability ===
* Does the formal specification provide guidelines for its implementation in a given organisation?
: Yes. For example, [http://theora.org/doc/Theora.pdf the Theora specification] provides "non-normative" advice and explanation for implementors of Theora decoders and encoders, including example algorithms for implementing required mathematical transforms. Xiph also maintains [http://wiki.xiph.org/Main_Page a documentation base] for implementors who desire more guidelines beyond the specification itself.
* Can other cases where similar systems implement the formal specification be considered as successful implementations and good practices?
: Xiph's standards have successfully been implemented by many organisations in a wide variety of environments. We maintain (non-exhaustive) [http://wiki.xiph.org/TheoraSoftwarePlayers lists] of products which implement Theora support, many of them open source, so that others may use them as a reference when preparing their own products. A particularly well known, independent, but interoperable implementation is provided by the FFmpeg open source project.
* Is its compatibility with related formal specification documented?
: Yes. For example, [http://theora.org/doc/Theora.pdf the Theora specification] also documents the use of Theora within the [http://www.ietf.org/rfc/rfc3533.txt standard Ogg encapsulation format], and the [http://svn.xiph.org/trunk/theora/doc/draft-ietf-avt-rtp-theora-00.txt TheoraRTP draft specification] explains how to transmit Theora using the [http://tools.ietf.org/html/rfc3550 RTP standard]. In addition, the specification documents Theora's compatibility with ITU-R B.470, ITU-R B.601, ITU-R B.709, SMPTE-170M, [http://tools.ietf.org/html/rfc2044 UTF-8], ISO 10646, and [http://www.xiph.org/vorbis/doc/Vorbis_I_spec.pdf Ogg Vorbis].

== Part 5: Standardisation Criteria ==
From Idabc-camss

Note: Throughout this section, “Organisation” refers to the standardisation/fora/consortia body in charge of the formal specification.

Significant characteristics of the way the organisation operates are for example the way it gives the possibility to stakeholders to influence the evolution of the formal specification, or which conditions it attaches to the use of the formal specification or its implementation. Moreover, it is important to know how the formal specification is defined, supported, and made available, as well as how interaction with stakeholders is managed by the organisation during these steps. Governance of interoperability testing with other formal specifications is also indicative.

The standardisation criteria analyses therefore the following elements:

=== Availability of Documentation ===
The availability of documentation criteria is linked to cost and online availability. Access to all preliminary results documentation can be online, online for members only, offline, offline, for members only or not available. Access can be free or for a fee (which fee?).
: Every Xiph standard is permanently available online to everyone at no cost. For example, we invite everyone to download [http://theora.org/doc/Theora.pdf the most up-to-date copy of the Theora specification], and [http://xiph.org/vorbis/doc/Vorbis_I_spec.html the latest revision of Vorbis]. All previous revisions are available from Xiph's [http://svn.xiph.org/ revision control system].

=== Intellectual Property Right ===
The Intellectual Property Rights evaluation criteria relates to the ability for implementers to use the formal specification in products without legal or financial implications. The IPR policy of the organisation is therefore evaluated according to:
* the availability of the IPR or copyright policies of the organisation (available on-line or off-line, or not available);
: The reference implementations of each codec include all necessary IPR and copyright licenses for that codec, including all documentation, and are freely available to everyone.
* the organisation’s governance to disclose any IPR from any contributor (ex-ante, online, offline, for free for all, for a fee for all, for members only, not available);
: Xiph does not require the identification of specific patents that may be required to implement a standard, however it does require an open-source compatible, royalty free license from a contributor for any such patents they may own before the corresponding technology can be included in a standard. These license are made available online, for free, to all parties.
* the level of IPR set "mandatory" by the organisation (no patent, royalty free patent, patent and RAND with limited liability , patent and classic RAND, patent with explicit licensing, patent with defensive licensing, or none);
: All standards published by the Xiph.Org Foundation are required to have "open-source compatible" IPR. This means that a standard must either be entirely clear of any known patents, or any patents that read upon the standard must be available under a transferable, irrevocable public nonassertion agreement to all people everywhere. For example, see [http://svn.xiph.org/trunk/theora/LICENSE our On2 patent nonassertion warrant]. Other common "royalty free" patent licenses are either not transferable, revocable under certain conditions (such as patent infringement litigation against the originating party), or otherwise impose restrictions that would prevent distribution under common [http://www.opensource.org/ OSI]-approved licenses. These would not be acceptable.
* the level of IPR "recommended" by the organisation (no patent, royalty free patent, patent and RAND with limited liability, patent and classic RAND, patent with explicit licensing, patent with defensive licensing, or none). [Note: RAND (Reasonable and Non Discriminatory License) is based on a "fairness" concept. Companies agree that if they receive any patents on technologies that become essential to the standard then they agree to allow other groups attempting to implement the standard to use these patents and they agree that the charges for the patents shall be reasonable. "RAND with limited availability" is a version of RAND where the "reasonable charges" have an upper limit.]
: Xiph's recommended IPR requirements are the same as our mandatory requirements.

=== Accessibility ===

The accessibility evaluation criteria describe the importance of equal and safe accessibility by the users of implementations of formal specifications. This aspect can be related to safety (physical safety and conformance safety) and accessibility of physical impaired people (design for all).

Focus is made particularly on accessibility and conformance safety. Conformance testing is testing to determine whether a system meets some specified formal specification. The result can be results from a test suite. Conformance validation is when the conformance test uniquely qualifies a given implementation as conformant or not. Conformance certification is a process that provides a public and easily visible "stamp of approval" that an implementation of a standard validates as conformant.

The following questions allow an assessment of accessibility and conformance safety:
* Does a mechanism that ensures disability support by a formal specification exist? (Y/N)
: Yes. Xiph ensures support for users with disabilities by providing specifications for accessible technologies independent of the codec itself. Notable Xiph specifications include [http://wiki.xiph.org/OggKate OggKate] and [http://wiki.xiph.org/index.php/CMML CMML], which provide subtitles for the hearing-impaired, as well as [http://wiki.xiph.org/Ogg_Skeleton Skeleton], which can specify scene description audio tracks for the visually impaired. When Theora is transmitted or stored in an Ogg container, it is automatically compatible with these accessibility measures.
* Is conformance governance always part of a standard? (Y/N)
: No. Xiph does not normally provide a formal conformance testing process as part of a standard.
* Is a conformance test offered to implementers? (Y/N)
: Yes. Xiph maintains a suite of [http://v2v.cc/~j/theora_testsuite/ test vectors] that can be used by implementors to confirm basic conformance.
* Is conformance validation available to implementers? (Y/N)
: Yes. Informal conformance testing is available to implementors upon request, and Xiph has provided such testing for a number of implementations in the past.
* Is conformance certification available? (Y/N)
: Yes. Xiph does not require certification, but maintains the right to withhold the use of our trademarks from implementors that act in bad faith. Implementors may, however, request explicit permission to use our trademarks with a conforming implementation.
* Is localisation of a formal specification possible? (Y/N)
: Yes. We welcome anyone who wishes to translate Xiph specifications into other languages. We have no policy requiring that the normative specification be written in English.

=== Interoperability governance ===
The interoperability governance evaluation criteria relates to how interoperability is identified and maintained between interoperable formal specifications. In order to do this, the organisation may provide governance for:
* open identification in formal specifications,
* open negotiation in formal specifications,
* open selection in formal specifications.

=== Meeting and consultation ===
The meeting and consultation evaluation criteria relates to the process of defining a formal specification. As formal specifications are usually defined by committees, and these committees normally consist of members of the organisation, this criteria studies how to become a member and which are the financial barriers for this, as well as how are non-members able to have an influence on the process of defining the formal specification. It analyses:
* if the organisation is open to all types of companies and organisations and to individuals;
: Yes. Xiph welcomes representatives from all companies and organizations.
* if the standardisation process may specifically allow participation of members with limited abilities when relevant;
: Yes. Standardization occurs almost entirely in internet communications channels, allowing participants with disabilities to engage fully in the standards development process. We also encourage nonexperts and students to assist us as they can, and to learn about Xiph technologies by participating in the standards development process.
* if meetings are open to all members;
: Xiph meetings are open to everyone. We charge no fee for and place no restrictions on attendance or participation. For example, anyone interested in contributing to the Theora specification may join [http://lists.xiph.org/pipermail/theora-dev/ the Theora development mailing list].
* if all can participate in the formal specification creation process;
: Yes. All people are welcome to participate in the specification creation process. No dues or fees are required to participate
* if non-members can participate in the formal specification creation process.
: Yes. Xiph does not maintain an explicit list of members, and no one is excluded from contributing to specifications as they are developed.

=== Consensus ===
Consensus is decision making primarily with regard to the approval of formal specifications and review with interest groups (non-members). The consensus evaluation criterion is evaluated with the following questions:
* Does the organisation have a stated objective of reaching consensus when making decisions on standards?
: There is no explicitly stated objective of reaching consensus.
* If consensus is not reached, can the standard be approved? (answers are: cannot be approved but referred back to working group/committee, approved with 75% majority, approved with 66% majority, approved with 51% majority, can be decided by a "director" or similar in the organisation).
: The standard can be approved without consensus via the decision of a "director" or similar.
* Is there a formal process for external review of standard proposals by interest groups (nonmembers)?
: Since anyone may participate in the development process and make proposals, there is no need for a separate formal process to include proposals by nonmembers.

=== Due Process ===
The due process evaluation criteria relates to the level of respect of each member of the organisation with regard to its rights. More specifically, it must be assured that if a member believes an error has been made in the process of defining a formal specification, it must be possible to appeal this to an independent, higher instance. The question is therefore: can a member formally appeal or raise objections to a procedure or to a technical specification to an independent, higher instance?

: Yes. Even if a member fails an appeal within the organization, because all of the technology Xiph standardizes is open and freely implementable, they are always free to develop their own, competing version. Such competing versions may even still be eligible for standardization under the Xiph umbrella.

=== Changes to the formal specification ===
The suggested changes made to a formal specification need to be presented, evaluated and approved in the same way as the formal specification was first defined. This criteria therefore applies the above criteria to the changes made to the formal specification(availability of documentation, Intellectual Property Right, accessibility, interoperability governance, meeting and consultation, consensus, due process).

: The exact same process is used for revisions to the standard as was used for the original development of the standard, and thus the answers to all of the above questions remain the same.

=== Support ===
It is critical that the organisation takes responsibility for the formal specification throughout its life span. This can be done in several ways such as for example a regular periodic review of the formal specification. The support criteria relates to the level of commitment the organisation has taken to support the formal specification throughout its life:
* does the organisation provide support until removal of the published formal specification from public domain (Including this process?
: Xiph.Org standards are never removed from the public domain. Xiph endeavors to provide support for as long as the standard remains in use.
* does the organisation make the formal specification still available even when in non-maintenance mode?
: Yes. All Xiph.Org standards are freely licensed and will always be available.
* does the organisation add new features and keep the formal specification up-to-date?
: Yes. Xiph maintains its ecosystem of standards on a continuous basis.
* does the organisation rectify problems identified in initial implementations?
: Yes. Xiph maintains [https://trac.xiph.org/report a problem reporting system] that is open to the public, and invites everyone to submit suggestions for improvements. Improvements are made both to the standards documents and to the reference implementations.
* does the organisation only create the formal specification?
: No. Xiph also produces high-quality reusable reference implementations of its standards, released under an open license.

<strong>This is a draft document. A work in progress. A scratchpad for ideas. It should not be widely circulated in this form.</strong>

IDABC Questionnaire 2009

2009-11-15T23:20:15Z

Silvia: added ffmpeg independent implementation

<strong>This is a draft document. A work in progress. A scratchpad for ideas. It should not be widely circulated in this form.</strong>

= Context =
We received [http://lists.xiph.org/pipermail/theora/2009-November/002996.html an e-mail] from a consultant studying the suitability of Theora for use in "eGovernment", on behalf of the [http://ec.europa.eu/idabc/ IDABC], an EU governmental agency responsible for "Interoperability" with an emphasis on open source. The investigation is in the context of [http://ec.europa.eu/idabc/en/document/7728 European Interoperability Framework], about which there has been [http://www.computerworlduk.com/community/blogs/index.cfm?entryid=2620&blogid=14&pn=1 some real controversy].

The method of assessment is the Common Assessment Method for Standards and Specifications, including the questions below.

= CAMSS Questions =
== Part 4: Market Criteria ==

This group of Market criteria analyses the formal specification in the scope of its market environment, and more precisely it examines the implementations of the formal specification and the market players. This implies identifying to which extent the formal specification benefits from market support and wide adoption, what are its level of maturity and its capacity of reusability.

Market support is evaluated through an analysis of how many products implementing the formal specification exist, what their market share is and who their end-users are. The quality and the completeness (in case of partitioning) of the implementations of the formal specification can also be analysed. Availability of existing or planned mechanisms to assess conformity of implementations to the standard or to the specification could also be identified. The existence of at least one reference implementation (i.e.: mentioning a recognized certification process) - and of which one is an open source implementation - can also be relevant to the assessment. Wide adoption can also be assessed across domains (i.e.: public and private sectors), in an open environment, and/or in a similar field (i.e.: best practices).

A formal specification is mature if it has been in use and development for long enough that most of its initial problems have been overcome and its underlying technology is well understood and well defined. Maturity is also assessed by identifying if all aspects of the formal specification are considered as validated by usage, (i.e.: if the formal specification is partitioned), and if the reported issues have been solved and documented.

Reusability of a formal specification is enabled if it includes guidelines for its implementation in a given context. The identification of successful implementations of the standard or specification should focus on good practices in a similar field. Its incompatibility with related standards or specifications should also be taken into account.

The ideas behind the Market Criteria can also be expressed in the form of the following questions:

=== Market support ===
* Does the standard have strong support in the marketplace?
: Yes. For example, among web browsers, support for Xiph's Ogg, Theora, and Vorbis standards is now included by default in Mozilla Firefox, Google Chrome, and the latest versions of Opera, representing hundreds of millions of installed users just in this market alone. Further, a QuickTime component exists which enables use of Xiph's Ogg, Theora, and Vorbis standards in all Mac OS X applications that make use of the QuickTime framework - which includes Safari/Webkit, iMovie, QuickTime, and many others. On Windows, DirectShow filters exist which also enable all Windows applications that use the DirectShow framework to use Xiph's Ogg, Theora, and Vorbis standards.
* What products exist for this formal specification ?
: Theora is a video codec, and as such the required products are encoders, decoders, and transmission systems. All three types of products are widely available for Theora.
* How many implementations of the formal specification are there?
: Xiph does not require implementors to acquire any license before implementing the specification. Therefore, we do not have a definitive count of the number of implementations. In addition to the reference implementation, which has been ported to most modern platforms and highly optimized for x86 and ARM CPUs and TI C64x+ DSPs, we are aware of a number of independent, conformant or mostly-conformant implementations. These include two C decoders (ffmpeg and QTheora), a Java decoder (Jheora), a C# decoder, an FPGA decoder, and an FPGA encoder.
* Are there products from different suppliers in the market that implement this formal specification ?
: Yes. Corporations such as Atari, Canonical, DailyMotion, Elphel, Fluendo, Google, Mozilla, Novell, Opera, Red Hat, Sun Microsystems, Ubisoft, and countless others have supplied products with an implementation of the Theora standard.
* Are there many products readily available from a variety of suppliers?
: Yes. Theora has been deployed in embedded devices, security cameras, video games, video conferencing systems, web browsers, home theater systems, and many other products. A complete, legal, open-source reference implementation can also be downloaded free of charge, including components for all major media frameworks (DirectShow, gstreamer, and Quicktime), giving the plethora of applications which use these frameworks the ability to use the codec.
* What is the market share of the products implementing the formal specification, versus other implementations of competing formal specifications ?
: Theora playback is extremely widely available, covering virtually the entire market of personal computers. Theora is also increasingly available in mobile and embedded devices. Since we do not require licensing for products that implement the specification, we do not have market share numbers that can be compared with competing formal specifications. Because implementations are readily available and free, Theora is included in many products that support multiple codecs, and is sometimes the only video codec included in free software products.
* Who are the end-users of these products implementing the formal specification?
: The end users are television viewers, video gamers, web surfers, movie makers, business people, video distribution services, and anyone else who interacts with moving pictures.

=== Maturity ===
* Are there any existing or planned mechanisms to assess conformity of the implementations of the formal specification?
: Yes. In addition to a continuous peer review process, we maintain a suite of [http://v2v.cc/~j/theora_testsuite/ test vectors] that allow implementors to assess decoder conformity. We also provide free online developer support and testing for those attempting to make a conforming implementation. An [http://validator.xiph.org/ online validation service] is available.
* Is there a reference implementation (i.e.: mentioning a recognized certification process)?
: Yes. Xiph maintains a reference implementation called [http://downloads.xiph.org/releases/theora/ libtheora]. In addition to serving as a reference, libtheora is also highly optimized to achieve the maximum possible speed, accuracy, reliability, efficiency, and video quality. As a result, many implementors of Theora adopt the reference implementation.
* Is there an open source implementation?
: Yes. libtheora is made available under a completely permissive BSD-like license. Its open-source nature also contributes to its quality as a reference implementation, as implementors are welcome to contribute their improvements to the reference. There are also several other open source implementations.
* Does the formal specification show wide adoption?
** across different domains? (I.e.: public and private)
: Yes. In addition to the private companies mentioned in the previous section, Theora has also been specified as the sole format supported by non-profit institutions such as Wikipedia, currently the 6th largest website in the world, or as one of a small number of preferred formats supported by other public organizations, such as the Norwegian government.
** in an open environment?
: Yes. On open/free operating systems such as those distributed by Novell/SuSE, Canonical, and Red Hat, Theora is the primary default video codec.
** in a similar field? (i.e.: can best practices be identified?)
* Has the formal specification been in use and development long enough that most of its initial problems have been overcome?
: Yes. Theora was derived from VP3, which was originally released in May 2000. The Theora specification was completed in 2004. Theora has now been used in a wide variety of applications, on the full spectrum of computing devices.
* Is the underlying technology of the standard well-understood? (e.g., a reference model is well defined, appropriate concepts of the technology are in widespread use, the technology may have been in use for many years, a formal mathematical model is defined, etc.)
: Yes. The underlying technology has been in use for nearly a decade, and most of the concepts have been in widespread use for even longer.
* Is the formal specification based upon technology that has not been well-defined and may be relatively new?
: No. The formal specification is based on technology from the On2 VP3 codec, which is substantially similar to simple block-transform codecs like H.261. This class of codecs is extremely well understood, and has been actively in use for over 20 years.
* Has the formal specification been revised? (Yes/No, Nof)
: Yes. The specification of the encoder is continuously revised based on user feedback to improve clarity and accuracy. The specification of the decoding part has been stable for years.
* Is the formal specification under the auspices of an architectural board? (Yes/No)
: No. Although officially maintained by the Xiph.Org Foundation, anyone is free to join this organization, and one need not even be a member to make contributions. However, the core developers will review contributions and make sure they do not contradict the general architecture and they work well with the existing code and the test cases.
* Is the formal specification partitioned in its functionality? (Yes/No)
: No. Theora is very deliberately not partitioned, to avoid the confusion created by a "standard" composed of many incompatible "profiles". The Theora standard does not have any optional components. A compliant Theora decoder can correctly process any Theora stream.
** To what extent does each partition participate to its overall functionality? (NN%)
: N/A.
** To what extent is each partition implemented? (NN%) (cf market adoption)
: N/A.

=== Re-usability ===
* Does the formal specification provide guidelines for its implementation in a given organisation?
: Yes. For example, [http://theora.org/doc/Theora.pdf the Theora specification] provides "non-normative" advice and explanation for implementors of Theora decoders and encoders, including example algorithms for implementing required mathematical transforms. Xiph also maintains [http://wiki.xiph.org/Main_Page a documentation base] for implementors who desire more guidelines beyond the specification itself.
* Can other cases where similar systems implement the formal specification be considered as successful implementations and good practices?
: Xiph's standards have successfully been implemented by many organisations in a wide variety of environments. We maintain (non-exhaustive) [http://wiki.xiph.org/TheoraSoftwarePlayers lists] of products which implement Theora support, many of them open source, so that others may use them as a reference when preparing their own products. A particularly well known, independent, but interoperable implementation is provided by the ffmpeg open source project.
* Is its compatibility with related formal specification documented?
: Yes. For example, [http://theora.org/doc/Theora.pdf the Theora specification] also documents the use of Theora within the [http://www.ietf.org/rfc/rfc3533.txt standard Ogg encapsulation format], and the [http://svn.xiph.org/trunk/theora/doc/draft-ietf-avt-rtp-theora-00.txt TheoraRTP draft specification] explains how to transmit Theora using the [http://tools.ietf.org/html/rfc3550 RTP standard]. In addition, the specification documents Theora's compatibility with ITU-R B.470, ITU-R B.601, ITU-R B.709, SMPTE-170M, [http://tools.ietf.org/html/rfc2044 UTF-8], ISO 10646, and [http://www.xiph.org/vorbis/doc/Vorbis_I_spec.pdf Ogg Vorbis].

== Part 5: Standardisation Criteria ==
From Idabc-camss

Note: Throughout this section, “Organisation” refers to the standardisation/fora/consortia body in charge of the formal specification.

Significant characteristics of the way the organisation operates are for example the way it gives the possibility to stakeholders to influence the evolution of the formal specification, or which conditions it attaches to the use of the formal specification or its implementation. Moreover, it is important to know how the formal specification is defined, supported, and made available, as well as how interaction with stakeholders is managed by the organisation during these steps. Governance of interoperability testing with other formal specifications is also indicative.

The standardisation criteria analyses therefore the following elements:

=== Availability of Documentation ===
The availability of documentation criteria is linked to cost and online availability. Access to all preliminary results documentation can be online, online for members only, offline, offline, for members only or not available. Access can be free or for a fee (which fee?).
: Every Xiph standard is permanently available online to everyone at no cost. For example, we invite everyone to download [http://theora.org/doc/Theora.pdf the most up-to-date copy of the Theora specification], and [http://xiph.org/vorbis/doc/Vorbis_I_spec.html the latest revision of Vorbis]. All previous revisions are available from Xiph's [http://svn.xiph.org/ revision control system].

=== Intellectual Property Right ===
The Intellectual Property Rights evaluation criteria relates to the ability for implementers to use the formal specification in products without legal or financial implications. The IPR policy of the organisation is therefore evaluated according to:
* the availability of the IPR or copyright policies of the organisation (available on-line or off-line, or not available);
: The reference implementations of each codec include all necessary IPR and copyright licenses for that codec, including all documentation, and are freely available to everyone.
* the organisation’s governance to disclose any IPR from any contributor (ex-ante, online, offline, for free for all, for a fee for all, for members only, not available);
: Xiph does not require the identification of specific patents that may be required to implement a standard, however it does require an open-source compatible, royalty free license from a contributor for any such patents they may own before the corresponding technology can be included in a standard. These license are made available online, for free, to all parties.
* the level of IPR set "mandatory" by the organisation (no patent, royalty free patent, patent and RAND with limited liability , patent and classic RAND, patent with explicit licensing, patent with defensive licensing, or none);
: All standards published by the Xiph.Org Foundation are required to have "open-source compatible" IPR. This means that a standard must either be entirely clear of any known patents, or any patents that read upon the standard must be available under a transferable, irrevocable public nonassertion agreement to all people everywhere. For example, see [http://svn.xiph.org/trunk/theora/LICENSE our On2 patent nonassertion warrant]. Other common "royalty free" patent licenses are either not transferable, revocable under certain conditions (such as patent infringement litigation against the originating party), or otherwise impose restrictions that would prevent distribution under common [http://www.opensource.org/ OSI]-approved licenses. These would not be acceptable.
* the level of IPR "recommended" by the organisation (no patent, royalty free patent, patent and RAND with limited liability, patent and classic RAND, patent with explicit licensing, patent with defensive licensing, or none). [Note: RAND (Reasonable and Non Discriminatory License) is based on a "fairness" concept. Companies agree that if they receive any patents on technologies that become essential to the standard then they agree to allow other groups attempting to implement the standard to use these patents and they agree that the charges for the patents shall be reasonable. "RAND with limited availability" is a version of RAND where the "reasonable charges" have an upper limit.]
: Xiph's recommended IPR requirements are the same as our mandatory requirements.

=== Accessibility ===

The accessibility evaluation criteria describe the importance of equal and safe accessibility by the users of implementations of formal specifications. This aspect can be related to safety (physical safety and conformance safety) and accessibility of physical impaired people (design for all).

Focus is made particularly on accessibility and conformance safety. Conformance testing is testing to determine whether a system meets some specified formal specification. The result can be results from a test suite. Conformance validation is when the conformance test uniquely qualifies a given implementation as conformant or not. Conformance certification is a process that provides a public and easily visible "stamp of approval" that an implementation of a standard validates as conformant.

The following questions allow an assessment of accessibility and conformance safety:
* Does a mechanism that ensures disability support by a formal specification exist? (Y/N)
: Yes. Xiph ensures support for users with disabilities by providing specifications for accessible technologies independent of the codec itself. Notable Xiph specifications include [http://wiki.xiph.org/OggKate OggKate] and [http://wiki.xiph.org/index.php/CMML CMML], which provide subtitles for the hearing-impaired, as well as [http://wiki.xiph.org/Ogg_Skeleton Skeleton], which can specify scene description audio tracks for the visually impaired. When Theora is transmitted or stored in an Ogg container, it is automatically compatible with these accessibility measures.
* Is conformance governance always part of a standard? (Y/N)
: No. Xiph does not normally provide a formal conformance testing process as part of a standard.
* Is a conformance test offered to implementers? (Y/N)
: Yes. Xiph maintains a suite of [http://v2v.cc/~j/theora_testsuite/ test vectors] that can be used by implementors to confirm basic conformance.
* Is conformance validation available to implementers? (Y/N)
: Yes. Informal conformance testing is available to implementors upon request, and Xiph has provided such testing for a number of implementations in the past.
* Is conformance certification available? (Y/N)
: Yes. Xiph does not require certification, but maintains the right to withhold the use of our trademarks from implementors that act in bad faith. Implementors may, however, request explicit permission to use our trademarks with a conforming implementation.
* Is localisation of a formal specification possible? (Y/N)
: Yes. We welcome anyone who wishes to translate Xiph specifications into other languages. We have no policy requiring that the normative specification be written in English.

=== Interoperability governance ===
The interoperability governance evaluation criteria relates to how interoperability is identified and maintained between interoperable formal specifications. In order to do this, the organisation may provide governance for:
* open identification in formal specifications,
* open negotiation in formal specifications,
* open selection in formal specifications.

=== Meeting and consultation ===
The meeting and consultation evaluation criteria relates to the process of defining a formal specification. As formal specifications are usually defined by committees, and these committees normally consist of members of the organisation, this criteria studies how to become a member and which are the financial barriers for this, as well as how are non-members able to have an influence on the process of defining the formal specification. It analyses:
* if the organisation is open to all types of companies and organisations and to individuals;
: Yes. Xiph welcomes representatives from all companies and organizations.
* if the standardisation process may specifically allow participation of members with limited abilities when relevant;
: Yes. Standardization occurs almost entirely in internet communications channels, allowing participants with disabilities to engage fully in the standards development process. We also encourage nonexperts and students to assist us as they can, and to learn about Xiph technologies by participating in the standards development process.
* if meetings are open to all members;
: Xiph meetings are open to everyone. We charge no fee for and place no restrictions on attendance or participation. For example, anyone interested in contributing to the Theora specification may join [http://lists.xiph.org/pipermail/theora-dev/ the Theora development mailing list].
* if all can participate in the formal specification creation process;
: Yes. All people are welcome to participate in the specification creation process. No dues or fees are required to participate
* if non-members can participate in the formal specification creation process.
: Yes. Xiph does not maintain an explicit list of members, and no one is excluded from contributing to specifications as they are developed.

=== Consensus ===
Consensus is decision making primarily with regard to the approval of formal specifications and review with interest groups (non-members). The consensus evaluation criterion is evaluated with the following questions:
* Does the organisation have a stated objective of reaching consensus when making decisions on standards?
: There is no explicitly stated objective of reaching consensus.
* If consensus is not reached, can the standard be approved? (answers are: cannot be approved but referred back to working group/committee, approved with 75% majority, approved with 66% majority, approved with 51% majority, can be decided by a "director" or similar in the organisation).
: The standard can be approved without consensus via the decision of a "director" or similar.
* Is there a formal process for external review of standard proposals by interest groups (nonmembers)?
: Since anyone may participate in the development process and make proposals, there is no need for a separate formal process to include proposals by nonmembers.

=== Due Process ===
The due process evaluation criteria relates to the level of respect of each member of the organisation with regard to its rights. More specifically, it must be assured that if a member believes an error has been made in the process of defining a formal specification, it must be possible to appeal this to an independent, higher instance. The question is therefore: can a member formally appeal or raise objections to a procedure or to a technical specification to an independent, higher instance?

: Yes. Even if a member fails an appeal within the organization, because all of the technology Xiph standardizes is open and freely implementable, they are always free to develop their own, competing version. Such competing versions may even still be eligible for standardization under the Xiph umbrella.

=== Changes to the formal specification ===
The suggested changes made to a formal specification need to be presented, evaluated and approved in the same way as the formal specification was first defined. This criteria therefore applies the above criteria to the changes made to the formal specification(availability of documentation, Intellectual Property Right, accessibility, interoperability governance, meeting and consultation, consensus, due process).

: The exact same process is used for revisions to the standard as was used for the original development of the standard, and thus the answers to all of the above questions remain the same.

=== Support ===
It is critical that the organisation takes responsibility for the formal specification throughout its life span. This can be done in several ways such as for example a regular periodic review of the formal specification. The support criteria relates to the level of commitment the organisation has taken to support the formal specification throughout its life:
* does the organisation provide support until removal of the published formal specification from public domain (Including this process?
: Xiph.Org standards are never removed from the public domain. Xiph endeavors to provide support for as long as the standard remains in use.
* does the organisation make the formal specification still available even when in non-maintenance mode?
: Yes. All Xiph.Org standards are freely licensed and will always be available.
* does the organisation add new features and keep the formal specification up-to-date?
: Yes. Xiph maintains its ecosystem of standards on a continuous basis.
* does the organisation rectify problems identified in initial implementations?
: Yes. Xiph maintains [https://trac.xiph.org/report a problem reporting system] that is open to the public, and invites everyone to submit suggestions for improvements. Improvements are made both to the standards documents and to the reference implementations.
* does the organisation only create the formal specification?
: No. Xiph also produces high-quality reusable reference implementations of its standards, released under an open license.

<strong>This is a draft document. A work in progress. A scratchpad for ideas. It should not be widely circulated in this form.</strong>

IDABC Questionnaire 2009

2009-11-15T23:17:41Z

Silvia: added validation service, architecture consistency, and decoder stability

<strong>This is a draft document. A work in progress. A scratchpad for ideas. It should not be widely circulated in this form.</strong>

= Context =
We received [http://lists.xiph.org/pipermail/theora/2009-November/002996.html an e-mail] from a consultant studying the suitability of Theora for use in "eGovernment", on behalf of the [http://ec.europa.eu/idabc/ IDABC], an EU governmental agency responsible for "Interoperability" with an emphasis on open source. The investigation is in the context of [http://ec.europa.eu/idabc/en/document/7728 European Interoperability Framework], about which there has been [http://www.computerworlduk.com/community/blogs/index.cfm?entryid=2620&blogid=14&pn=1 some real controversy].

The method of assessment is the Common Assessment Method for Standards and Specifications, including the questions below.

= CAMSS Questions =
== Part 4: Market Criteria ==

This group of Market criteria analyses the formal specification in the scope of its market environment, and more precisely it examines the implementations of the formal specification and the market players. This implies identifying to which extent the formal specification benefits from market support and wide adoption, what are its level of maturity and its capacity of reusability.

Market support is evaluated through an analysis of how many products implementing the formal specification exist, what their market share is and who their end-users are. The quality and the completeness (in case of partitioning) of the implementations of the formal specification can also be analysed. Availability of existing or planned mechanisms to assess conformity of implementations to the standard or to the specification could also be identified. The existence of at least one reference implementation (i.e.: mentioning a recognized certification process) - and of which one is an open source implementation - can also be relevant to the assessment. Wide adoption can also be assessed across domains (i.e.: public and private sectors), in an open environment, and/or in a similar field (i.e.: best practices).

A formal specification is mature if it has been in use and development for long enough that most of its initial problems have been overcome and its underlying technology is well understood and well defined. Maturity is also assessed by identifying if all aspects of the formal specification are considered as validated by usage, (i.e.: if the formal specification is partitioned), and if the reported issues have been solved and documented.

Reusability of a formal specification is enabled if it includes guidelines for its implementation in a given context. The identification of successful implementations of the standard or specification should focus on good practices in a similar field. Its incompatibility with related standards or specifications should also be taken into account.

The ideas behind the Market Criteria can also be expressed in the form of the following questions:

=== Market support ===
* Does the standard have strong support in the marketplace?
: Yes. For example, among web browsers, support for Xiph's Ogg, Theora, and Vorbis standards is now included by default in Mozilla Firefox, Google Chrome, and the latest versions of Opera, representing hundreds of millions of installed users just in this market alone. Further, a QuickTime component exists which enables use of Xiph's Ogg, Theora, and Vorbis standards in all Mac OS X applications that make use of the QuickTime framework - which includes Safari/Webkit, iMovie, QuickTime, and many others. On Windows, DirectShow filters exist which also enable all Windows applications that use the DirectShow framework to use Xiph's Ogg, Theora, and Vorbis standards.
* What products exist for this formal specification ?
: Theora is a video codec, and as such the required products are encoders, decoders, and transmission systems. All three types of products are widely available for Theora.
* How many implementations of the formal specification are there?
: Xiph does not require implementors to acquire any license before implementing the specification. Therefore, we do not have a definitive count of the number of implementations. In addition to the reference implementation, which has been ported to most modern platforms and highly optimized for x86 and ARM CPUs and TI C64x+ DSPs, we are aware of a number of independent, conformant or mostly-conformant implementations. These include two C decoders (ffmpeg and QTheora), a Java decoder (Jheora), a C# decoder, an FPGA decoder, and an FPGA encoder.
* Are there products from different suppliers in the market that implement this formal specification ?
: Yes. Corporations such as Atari, Canonical, DailyMotion, Elphel, Fluendo, Google, Mozilla, Novell, Opera, Red Hat, Sun Microsystems, Ubisoft, and countless others have supplied products with an implementation of the Theora standard.
* Are there many products readily available from a variety of suppliers?
: Yes. Theora has been deployed in embedded devices, security cameras, video games, video conferencing systems, web browsers, home theater systems, and many other products. A complete, legal, open-source reference implementation can also be downloaded free of charge, including components for all major media frameworks (DirectShow, gstreamer, and Quicktime), giving the plethora of applications which use these frameworks the ability to use the codec.
* What is the market share of the products implementing the formal specification, versus other implementations of competing formal specifications ?
: Theora playback is extremely widely available, covering virtually the entire market of personal computers. Theora is also increasingly available in mobile and embedded devices. Since we do not require licensing for products that implement the specification, we do not have market share numbers that can be compared with competing formal specifications. Because implementations are readily available and free, Theora is included in many products that support multiple codecs, and is sometimes the only video codec included in free software products.
* Who are the end-users of these products implementing the formal specification?
: The end users are television viewers, video gamers, web surfers, movie makers, business people, video distribution services, and anyone else who interacts with moving pictures.

=== Maturity ===
* Are there any existing or planned mechanisms to assess conformity of the implementations of the formal specification?
: Yes. In addition to a continuous peer review process, we maintain a suite of [http://v2v.cc/~j/theora_testsuite/ test vectors] that allow implementors to assess decoder conformity. We also provide free online developer support and testing for those attempting to make a conforming implementation. An [http://validator.xiph.org/ online validation service] is available.
* Is there a reference implementation (i.e.: mentioning a recognized certification process)?
: Yes. Xiph maintains a reference implementation called [http://downloads.xiph.org/releases/theora/ libtheora]. In addition to serving as a reference, libtheora is also highly optimized to achieve the maximum possible speed, accuracy, reliability, efficiency, and video quality. As a result, many implementors of Theora adopt the reference implementation.
* Is there an open source implementation?
: Yes. libtheora is made available under a completely permissive BSD-like license. Its open-source nature also contributes to its quality as a reference implementation, as implementors are welcome to contribute their improvements to the reference. There are also several other open source implementations.
* Does the formal specification show wide adoption?
** across different domains? (I.e.: public and private)
: Yes. In addition to the private companies mentioned in the previous section, Theora has also been specified as the sole format supported by non-profit institutions such as Wikipedia, currently the 6th largest website in the world, or as one of a small number of preferred formats supported by other public organizations, such as the Norwegian government.
** in an open environment?
: Yes. On open/free operating systems such as those distributed by Novell/SuSE, Canonical, and Red Hat, Theora is the primary default video codec.
** in a similar field? (i.e.: can best practices be identified?)
* Has the formal specification been in use and development long enough that most of its initial problems have been overcome?
: Yes. Theora was derived from VP3, which was originally released in May 2000. The Theora specification was completed in 2004. Theora has now been used in a wide variety of applications, on the full spectrum of computing devices.
* Is the underlying technology of the standard well-understood? (e.g., a reference model is well defined, appropriate concepts of the technology are in widespread use, the technology may have been in use for many years, a formal mathematical model is defined, etc.)
: Yes. The underlying technology has been in use for nearly a decade, and most of the concepts have been in widespread use for even longer.
* Is the formal specification based upon technology that has not been well-defined and may be relatively new?
: No. The formal specification is based on technology from the On2 VP3 codec, which is substantially similar to simple block-transform codecs like H.261. This class of codecs is extremely well understood, and has been actively in use for over 20 years.
* Has the formal specification been revised? (Yes/No, Nof)
: Yes. The specification of the encoder is continuously revised based on user feedback to improve clarity and accuracy. The specification of the decoding part has been stable for years.
* Is the formal specification under the auspices of an architectural board? (Yes/No)
: No. Although officially maintained by the Xiph.Org Foundation, anyone is free to join this organization, and one need not even be a member to make contributions. However, the core developers will review contributions and make sure they do not contradict the general architecture and they work well with the existing code and the test cases.
* Is the formal specification partitioned in its functionality? (Yes/No)
: No. Theora is very deliberately not partitioned, to avoid the confusion created by a "standard" composed of many incompatible "profiles". The Theora standard does not have any optional components. A compliant Theora decoder can correctly process any Theora stream.
** To what extent does each partition participate to its overall functionality? (NN%)
: N/A.
** To what extent is each partition implemented? (NN%) (cf market adoption)
: N/A.

=== Re-usability ===
* Does the formal specification provide guidelines for its implementation in a given organisation?
: Yes. For example, [http://theora.org/doc/Theora.pdf the Theora specification] provides "non-normative" advice and explanation for implementors of Theora decoders and encoders, including example algorithms for implementing required mathematical transforms. Xiph also maintains [http://wiki.xiph.org/Main_Page a documentation base] for implementors who desire more guidelines beyond the specification itself.
* Can other cases where similar systems implement the formal specification be considered as successful implementations and good practices?
: Xiph's standards have successfully been implemented by many organisations in a wide variety of environments. We maintain (non-exhaustive) [http://wiki.xiph.org/TheoraSoftwarePlayers lists] of products which implement Theora support, many of them open source, so that others may use them as a reference when preparing their own products.
* Is its compatibility with related formal specification documented?
: Yes. For example, [http://theora.org/doc/Theora.pdf the Theora specification] also documents the use of Theora within the [http://www.ietf.org/rfc/rfc3533.txt standard Ogg encapsulation format], and the [http://svn.xiph.org/trunk/theora/doc/draft-ietf-avt-rtp-theora-00.txt TheoraRTP draft specification] explains how to transmit Theora using the [http://tools.ietf.org/html/rfc3550 RTP standard]. In addition, the specification documents Theora's compatibility with ITU-R B.470, ITU-R B.601, ITU-R B.709, SMPTE-170M, [http://tools.ietf.org/html/rfc2044 UTF-8], ISO 10646, and [http://www.xiph.org/vorbis/doc/Vorbis_I_spec.pdf Ogg Vorbis].

== Part 5: Standardisation Criteria ==
From Idabc-camss

Note: Throughout this section, “Organisation” refers to the standardisation/fora/consortia body in charge of the formal specification.

Significant characteristics of the way the organisation operates are for example the way it gives the possibility to stakeholders to influence the evolution of the formal specification, or which conditions it attaches to the use of the formal specification or its implementation. Moreover, it is important to know how the formal specification is defined, supported, and made available, as well as how interaction with stakeholders is managed by the organisation during these steps. Governance of interoperability testing with other formal specifications is also indicative.

The standardisation criteria analyses therefore the following elements:

=== Availability of Documentation ===
The availability of documentation criteria is linked to cost and online availability. Access to all preliminary results documentation can be online, online for members only, offline, offline, for members only or not available. Access can be free or for a fee (which fee?).
: Every Xiph standard is permanently available online to everyone at no cost. For example, we invite everyone to download [http://theora.org/doc/Theora.pdf the most up-to-date copy of the Theora specification], and [http://xiph.org/vorbis/doc/Vorbis_I_spec.html the latest revision of Vorbis]. All previous revisions are available from Xiph's [http://svn.xiph.org/ revision control system].

=== Intellectual Property Right ===
The Intellectual Property Rights evaluation criteria relates to the ability for implementers to use the formal specification in products without legal or financial implications. The IPR policy of the organisation is therefore evaluated according to:
* the availability of the IPR or copyright policies of the organisation (available on-line or off-line, or not available);
: The reference implementations of each codec include all necessary IPR and copyright licenses for that codec, including all documentation, and are freely available to everyone.
* the organisation’s governance to disclose any IPR from any contributor (ex-ante, online, offline, for free for all, for a fee for all, for members only, not available);
: Xiph does not require the identification of specific patents that may be required to implement a standard, however it does require an open-source compatible, royalty free license from a contributor for any such patents they may own before the corresponding technology can be included in a standard. These license are made available online, for free, to all parties.
* the level of IPR set "mandatory" by the organisation (no patent, royalty free patent, patent and RAND with limited liability , patent and classic RAND, patent with explicit licensing, patent with defensive licensing, or none);
: All standards published by the Xiph.Org Foundation are required to have "open-source compatible" IPR. This means that a standard must either be entirely clear of any known patents, or any patents that read upon the standard must be available under a transferable, irrevocable public nonassertion agreement to all people everywhere. For example, see [http://svn.xiph.org/trunk/theora/LICENSE our On2 patent nonassertion warrant]. Other common "royalty free" patent licenses are either not transferable, revocable under certain conditions (such as patent infringement litigation against the originating party), or otherwise impose restrictions that would prevent distribution under common [http://www.opensource.org/ OSI]-approved licenses. These would not be acceptable.
* the level of IPR "recommended" by the organisation (no patent, royalty free patent, patent and RAND with limited liability, patent and classic RAND, patent with explicit licensing, patent with defensive licensing, or none). [Note: RAND (Reasonable and Non Discriminatory License) is based on a "fairness" concept. Companies agree that if they receive any patents on technologies that become essential to the standard then they agree to allow other groups attempting to implement the standard to use these patents and they agree that the charges for the patents shall be reasonable. "RAND with limited availability" is a version of RAND where the "reasonable charges" have an upper limit.]
: Xiph's recommended IPR requirements are the same as our mandatory requirements.

=== Accessibility ===

The accessibility evaluation criteria describe the importance of equal and safe accessibility by the users of implementations of formal specifications. This aspect can be related to safety (physical safety and conformance safety) and accessibility of physical impaired people (design for all).

Focus is made particularly on accessibility and conformance safety. Conformance testing is testing to determine whether a system meets some specified formal specification. The result can be results from a test suite. Conformance validation is when the conformance test uniquely qualifies a given implementation as conformant or not. Conformance certification is a process that provides a public and easily visible "stamp of approval" that an implementation of a standard validates as conformant.

The following questions allow an assessment of accessibility and conformance safety:
* Does a mechanism that ensures disability support by a formal specification exist? (Y/N)
: Yes. Xiph ensures support for users with disabilities by providing specifications for accessible technologies independent of the codec itself. Notable Xiph specifications include [http://wiki.xiph.org/OggKate OggKate] and [http://wiki.xiph.org/index.php/CMML CMML], which provide subtitles for the hearing-impaired, as well as [http://wiki.xiph.org/Ogg_Skeleton Skeleton], which can specify scene description audio tracks for the visually impaired. When Theora is transmitted or stored in an Ogg container, it is automatically compatible with these accessibility measures.
* Is conformance governance always part of a standard? (Y/N)
: No. Xiph does not normally provide a formal conformance testing process as part of a standard.
* Is a conformance test offered to implementers? (Y/N)
: Yes. Xiph maintains a suite of [http://v2v.cc/~j/theora_testsuite/ test vectors] that can be used by implementors to confirm basic conformance.
* Is conformance validation available to implementers? (Y/N)
: Yes. Informal conformance testing is available to implementors upon request, and Xiph has provided such testing for a number of implementations in the past.
* Is conformance certification available? (Y/N)
: Yes. Xiph does not require certification, but maintains the right to withhold the use of our trademarks from implementors that act in bad faith. Implementors may, however, request explicit permission to use our trademarks with a conforming implementation.
* Is localisation of a formal specification possible? (Y/N)
: Yes. We welcome anyone who wishes to translate Xiph specifications into other languages. We have no policy requiring that the normative specification be written in English.

=== Interoperability governance ===
The interoperability governance evaluation criteria relates to how interoperability is identified and maintained between interoperable formal specifications. In order to do this, the organisation may provide governance for:
* open identification in formal specifications,
* open negotiation in formal specifications,
* open selection in formal specifications.

=== Meeting and consultation ===
The meeting and consultation evaluation criteria relates to the process of defining a formal specification. As formal specifications are usually defined by committees, and these committees normally consist of members of the organisation, this criteria studies how to become a member and which are the financial barriers for this, as well as how are non-members able to have an influence on the process of defining the formal specification. It analyses:
* if the organisation is open to all types of companies and organisations and to individuals;
: Yes. Xiph welcomes representatives from all companies and organizations.
* if the standardisation process may specifically allow participation of members with limited abilities when relevant;
: Yes. Standardization occurs almost entirely in internet communications channels, allowing participants with disabilities to engage fully in the standards development process. We also encourage nonexperts and students to assist us as they can, and to learn about Xiph technologies by participating in the standards development process.
* if meetings are open to all members;
: Xiph meetings are open to everyone. We charge no fee for and place no restrictions on attendance or participation. For example, anyone interested in contributing to the Theora specification may join [http://lists.xiph.org/pipermail/theora-dev/ the Theora development mailing list].
* if all can participate in the formal specification creation process;
: Yes. All people are welcome to participate in the specification creation process. No dues or fees are required to participate
* if non-members can participate in the formal specification creation process.
: Yes. Xiph does not maintain an explicit list of members, and no one is excluded from contributing to specifications as they are developed.

=== Consensus ===
Consensus is decision making primarily with regard to the approval of formal specifications and review with interest groups (non-members). The consensus evaluation criterion is evaluated with the following questions:
* Does the organisation have a stated objective of reaching consensus when making decisions on standards?
: There is no explicitly stated objective of reaching consensus.
* If consensus is not reached, can the standard be approved? (answers are: cannot be approved but referred back to working group/committee, approved with 75% majority, approved with 66% majority, approved with 51% majority, can be decided by a "director" or similar in the organisation).
: The standard can be approved without consensus via the decision of a "director" or similar.
* Is there a formal process for external review of standard proposals by interest groups (nonmembers)?
: Since anyone may participate in the development process and make proposals, there is no need for a separate formal process to include proposals by nonmembers.

=== Due Process ===
The due process evaluation criteria relates to the level of respect of each member of the organisation with regard to its rights. More specifically, it must be assured that if a member believes an error has been made in the process of defining a formal specification, it must be possible to appeal this to an independent, higher instance. The question is therefore: can a member formally appeal or raise objections to a procedure or to a technical specification to an independent, higher instance?

: Yes. Even if a member fails an appeal within the organization, because all of the technology Xiph standardizes is open and freely implementable, they are always free to develop their own, competing version. Such competing versions may even still be eligible for standardization under the Xiph umbrella.

=== Changes to the formal specification ===
The suggested changes made to a formal specification need to be presented, evaluated and approved in the same way as the formal specification was first defined. This criteria therefore applies the above criteria to the changes made to the formal specification(availability of documentation, Intellectual Property Right, accessibility, interoperability governance, meeting and consultation, consensus, due process).

: The exact same process is used for revisions to the standard as was used for the original development of the standard, and thus the answers to all of the above questions remain the same.

=== Support ===
It is critical that the organisation takes responsibility for the formal specification throughout its life span. This can be done in several ways such as for example a regular periodic review of the formal specification. The support criteria relates to the level of commitment the organisation has taken to support the formal specification throughout its life:
* does the organisation provide support until removal of the published formal specification from public domain (Including this process?
: Xiph.Org standards are never removed from the public domain. Xiph endeavors to provide support for as long as the standard remains in use.
* does the organisation make the formal specification still available even when in non-maintenance mode?
: Yes. All Xiph.Org standards are freely licensed and will always be available.
* does the organisation add new features and keep the formal specification up-to-date?
: Yes. Xiph maintains its ecosystem of standards on a continuous basis.
* does the organisation rectify problems identified in initial implementations?
: Yes. Xiph maintains [https://trac.xiph.org/report a problem reporting system] that is open to the public, and invites everyone to submit suggestions for improvements. Improvements are made both to the standards documents and to the reference implementations.
* does the organisation only create the formal specification?
: No. Xiph also produces high-quality reusable reference implementations of its standards, released under an open license.

<strong>This is a draft document. A work in progress. A scratchpad for ideas. It should not be widely circulated in this form.</strong>

IDABC Questionnaire 2009

2009-11-15T23:07:46Z

Silvia: /* Market support */

<strong>This is a draft document. A work in progress. A scratchpad for ideas. It should not be widely circulated in this form.</strong>

= Context =
We received [http://lists.xiph.org/pipermail/theora/2009-November/002996.html an e-mail] from a consultant studying the suitability of Theora for use in "eGovernment", on behalf of the [http://ec.europa.eu/idabc/ IDABC], an EU governmental agency responsible for "Interoperability" with an emphasis on open source. The investigation is in the context of [http://ec.europa.eu/idabc/en/document/7728 European Interoperability Framework], about which there has been [http://www.computerworlduk.com/community/blogs/index.cfm?entryid=2620&blogid=14&pn=1 some real controversy].

The method of assessment is the Common Assessment Method for Standards and Specifications, including the questions below.

= CAMSS Questions =
== Part 4: Market Criteria ==

This group of Market criteria analyses the formal specification in the scope of its market environment, and more precisely it examines the implementations of the formal specification and the market players. This implies identifying to which extent the formal specification benefits from market support and wide adoption, what are its level of maturity and its capacity of reusability.

Market support is evaluated through an analysis of how many products implementing the formal specification exist, what their market share is and who their end-users are. The quality and the completeness (in case of partitioning) of the implementations of the formal specification can also be analysed. Availability of existing or planned mechanisms to assess conformity of implementations to the standard or to the specification could also be identified. The existence of at least one reference implementation (i.e.: mentioning a recognized certification process) - and of which one is an open source implementation - can also be relevant to the assessment. Wide adoption can also be assessed across domains (i.e.: public and private sectors), in an open environment, and/or in a similar field (i.e.: best practices).

A formal specification is mature if it has been in use and development for long enough that most of its initial problems have been overcome and its underlying technology is well understood and well defined. Maturity is also assessed by identifying if all aspects of the formal specification are considered as validated by usage, (i.e.: if the formal specification is partitioned), and if the reported issues have been solved and documented.

Reusability of a formal specification is enabled if it includes guidelines for its implementation in a given context. The identification of successful implementations of the standard or specification should focus on good practices in a similar field. Its incompatibility with related standards or specifications should also be taken into account.

The ideas behind the Market Criteria can also be expressed in the form of the following questions:

=== Market support ===
* Does the standard have strong support in the marketplace?
: Yes. For example, among web browsers, support for Xiph's Ogg, Theora, and Vorbis standards is now included by default in Mozilla Firefox, Google Chrome, and the latest versions of Opera, representing hundreds of millions of installed users just in this market alone. Further, a QuickTime component exists which enables use of Xiph's Ogg, Theora, and Vorbis standards in all Mac OS X applications that make use of the QuickTime framework - which includes Safari/Webkit, iMovie, QuickTime, and many others. On Windows, DirectShow filters exist which also enable all Windows applications that use the DirectShow framework to use Xiph's Ogg, Theora, and Vorbis standards.
* What products exist for this formal specification ?
: Theora is a video codec, and as such the required products are encoders, decoders, and transmission systems. All three types of products are widely available for Theora.
* How many implementations of the formal specification are there?
: Xiph does not require implementors to acquire any license before implementing the specification. Therefore, we do not have a definitive count of the number of implementations. In addition to the reference implementation, which has been ported to most modern platforms and highly optimized for x86 and ARM CPUs and TI C64x+ DSPs, we are aware of a number of independent, conformant or mostly-conformant implementations. These include two C decoders (ffmpeg and QTheora), a Java decoder (Jheora), a C# decoder, an FPGA decoder, and an FPGA encoder.
* Are there products from different suppliers in the market that implement this formal specification ?
: Yes. Corporations such as Atari, Canonical, DailyMotion, Elphel, Fluendo, Google, Mozilla, Novell, Opera, Red Hat, Sun Microsystems, Ubisoft, and countless others have supplied products with an implementation of the Theora standard.
* Are there many products readily available from a variety of suppliers?
: Yes. Theora has been deployed in embedded devices, security cameras, video games, video conferencing systems, web browsers, home theater systems, and many other products. A complete, legal, open-source reference implementation can also be downloaded free of charge, including components for all major media frameworks (DirectShow, gstreamer, and Quicktime), giving the plethora of applications which use these frameworks the ability to use the codec.
* What is the market share of the products implementing the formal specification, versus other implementations of competing formal specifications ?
: Theora playback is extremely widely available, covering virtually the entire market of personal computers. Theora is also increasingly available in mobile and embedded devices. Since we do not require licensing for products that implement the specification, we do not have market share numbers that can be compared with competing formal specifications. Because implementations are readily available and free, Theora is included in many products that support multiple codecs, and is sometimes the only video codec included in free software products.
* Who are the end-users of these products implementing the formal specification?
: The end users are television viewers, video gamers, web surfers, movie makers, business people, video distribution services, and anyone else who interacts with moving pictures.

=== Maturity ===
* Are there any existing or planned mechanisms to assess conformity of the implementations of the formal specification?
: Yes. In addition to a continuous peer review process, we maintain a suite of [http://v2v.cc/~j/theora_testsuite/ test vectors] that allow implementors to assess decoder conformity. We also provide free online developer support and testing for those attempting to make a conforming implementation.
* Is there a reference implementation (i.e.: mentioning a recognized certification process)?
: Yes. Xiph maintains a reference implementation called [http://downloads.xiph.org/releases/theora/ libtheora]. In addition to serving as a reference, libtheora is also highly optimized to achieve the maximum possible speed, accuracy, reliability, efficiency, and video quality. As a result, many implementors of Theora adopt the reference implementation.
* Is there an open source implementation?
: Yes. libtheora is made available under a completely permissive BSD-like license. Its open-source nature also contributes to its quality as a reference implementation, as implementors are welcome to contribute their improvements to the reference. There are also several other open source implementations.
* Does the formal specification show wide adoption?
** across different domains? (I.e.: public and private)
: Yes. In addition to the private companies mentioned in the previous section, Theora has also been specified as the sole format supported by non-profit institutions such as Wikipedia, currently the 6th largest website in the world, or as one of a small number of preferred formats supported by other public organizations, such as the Norwegian government.
** in an open environment?
: Yes. On open/free operating systems such as those distributed by Novell/SuSE, Canonical, and Red Hat, Theora is the primary default video codec.
** in a similar field? (i.e.: can best practices be identified?)
* Has the formal specification been in use and development long enough that most of its initial problems have been overcome?
: Yes. Theora was derived from VP3, which was originally released in May 2000. The Theora specification was completed in 2004. Theora has now been used in a wide variety of applications, on the full spectrum of computing devices.
* Is the underlying technology of the standard well-understood? (e.g., a reference model is welldefined, appropriate concepts of the technology are in widespread use, the technology may have been in use for many years, a formal mathematical model is defined, etc.)
: Yes. The underlying technology has been in use for nearly a decade, and most of the concepts have been in widespread use for even longer.
* Is the formal specification based upon technology that has not been well-defined and may be relatively new?
: No. The formal specification is based on technology from the On2 VP3 codec, which is substantially similar to simple block-transform codecs like H.261. This class of codecs is extremely well understood, and has been actively in use for over 20 years.
* Has the formal specification been revised? (Yes/No, Nof)
: Yes. The specification is continuously revised based on user feedback to improve clarity and accuracy.
* Is the formal specification under the auspices of an architectural board? (Yes/No)
: No. Although officially maintained by the Xiph.Org Foundation, anyone is free to join this organization, and one need not even be a member to make contributions.
* Is the formal specification partitioned in its functionality? (Yes/No)
: No. Theora is very deliberately not partitioned, to avoid the confusion created by a "standard" composed of many incompatible "profiles". The Theora standard does not have any optional components. A compliant Theora decoder can correctly process any Theora stream.
** To what extent does each partition participate to its overall functionality? (NN%)
: N/A.
** To what extent is each partition implemented? (NN%) (cf market adoption)
: N/A.

=== Re-usability ===
* Does the formal specification provide guidelines for its implementation in a given organisation?
: Yes. For example, [http://theora.org/doc/Theora.pdf the Theora specification] provides "non-normative" advice and explanation for implementors of Theora decoders and encoders, including example algorithms for implementing required mathematical transforms. Xiph also maintains [http://wiki.xiph.org/Main_Page a documentation base] for implementors who desire more guidelines beyond the specification itself.
* Can other cases where similar systems implement the formal specification be considered as successful implementations and good practices?
: Xiph's standards have successfully been implemented by many organisations in a wide variety of environments. We maintain (non-exhaustive) [http://wiki.xiph.org/TheoraSoftwarePlayers lists] of products which implement Theora support, many of them open source, so that others may use them as a reference when preparing their own products.
* Is its compatibility with related formal specification documented?
: Yes. For example, [http://theora.org/doc/Theora.pdf the Theora specification] also documents the use of Theora within the [http://www.ietf.org/rfc/rfc3533.txt standard Ogg encapsulation format], and the [http://svn.xiph.org/trunk/theora/doc/draft-ietf-avt-rtp-theora-00.txt TheoraRTP draft specification] explains how to transmit Theora using the [http://tools.ietf.org/html/rfc3550 RTP standard]. In addition, the specification documents Theora's compatibility with ITU-R B.470, ITU-R B.601, ITU-R B.709, SMPTE-170M, [http://tools.ietf.org/html/rfc2044 UTF-8], ISO 10646, and [http://www.xiph.org/vorbis/doc/Vorbis_I_spec.pdf Ogg Vorbis].

== Part 5: Standardisation Criteria ==
From Idabc-camss

Note: Throughout this section, “Organisation” refers to the standardisation/fora/consortia body in charge of the formal specification.

Significant characteristics of the way the organisation operates are for example the way it gives the possibility to stakeholders to influence the evolution of the formal specification, or which conditions it attaches to the use of the formal specification or its implementation. Moreover, it is important to know how the formal specification is defined, supported, and made available, as well as how interaction with stakeholders is managed by the organisation during these steps. Governance of interoperability testing with other formal specifications is also indicative.

The standardisation criteria analyses therefore the following elements:

=== Availability of Documentation ===
The availability of documentation criteria is linked to cost and online availability. Access to all preliminary results documentation can be online, online for members only, offline, offline, for members only or not available. Access can be free or for a fee (which fee?).
: Every Xiph standard is permanently available online to everyone at no cost. For example, we invite everyone to download [http://theora.org/doc/Theora.pdf the most up-to-date copy of the Theora specification], and [http://xiph.org/vorbis/doc/Vorbis_I_spec.html the latest revision of Vorbis]. All previous revisions are available from Xiph's [http://svn.xiph.org/ revision control system].

=== Intellectual Property Right ===
The Intellectual Property Rights evaluation criteria relates to the ability for implementers to use the formal specification in products without legal or financial implications. The IPR policy of the organisation is therefore evaluated according to:
* the availability of the IPR or copyright policies of the organisation (available on-line or off-line, or not available);
: The reference implementations of each codec include all necessary IPR and copyright licenses for that codec, including all documentation, and are freely available to everyone.
* the organisation’s governance to disclose any IPR from any contributor (ex-ante, online, offline, for free for all, for a fee for all, for members only, not available);
: Xiph does not require the identification of specific patents that may be required to implement a standard, however it does require an open-source compatible, royalty free license from a contributor for any such patents they may own before the corresponding technology can be included in a standard. These license are made available online, for free, to all parties.
* the level of IPR set "mandatory" by the organisation (no patent, royalty free patent, patent and RAND with limited liability , patent and classic RAND, patent with explicit licensing, patent with defensive licensing, or none);
: All standards published by the Xiph.Org Foundation are required to have "open-source compatible" IPR. This means that a standard must either be entirely clear of any known patents, or any patents that read upon the standard must be available under a transferable, irrevocable public nonassertion agreement to all people everywhere. For example, see [http://svn.xiph.org/trunk/theora/LICENSE our On2 patent nonassertion warrant]. Other common "royalty free" patent licenses are either not transferable, revocable under certain conditions (such as patent infringement litigation against the originating party), or otherwise impose restrictions that would prevent distribution under common [http://www.opensource.org/ OSI]-approved licenses. These would not be acceptable.
* the level of IPR "recommended" by the organisation (no patent, royalty free patent, patent and RAND with limited liability, patent and classic RAND, patent with explicit licensing, patent with defensive licensing, or none). [Note: RAND (Reasonable and Non Discriminatory License) is based on a "fairness" concept. Companies agree that if they receive any patents on technologies that become essential to the standard then they agree to allow other groups attempting to implement the standard to use these patents and they agree that the charges for the patents shall be reasonable. "RAND with limited availability" is a version of RAND where the "reasonable charges" have an upper limit.]
: Xiph's recommended IPR requirements are the same as our mandatory requirements.

=== Accessibility ===

The accessibility evaluation criteria describe the importance of equal and safe accessibility by the users of implementations of formal specifications. This aspect can be related to safety (physical safety and conformance safety) and accessibility of physical impaired people (design for all).

Focus is made particularly on accessibility and conformance safety. Conformance testing is testing to determine whether a system meets some specified formal specification. The result can be results from a test suite. Conformance validation is when the conformance test uniquely qualifies a given implementation as conformant or not. Conformance certification is a process that provides a public and easily visible "stamp of approval" that an implementation of a standard validates as conformant.

The following questions allow an assessment of accessibility and conformance safety:
* Does a mechanism that ensures disability support by a formal specification exist? (Y/N)
: Yes. Xiph ensures support for users with disabilities by providing specifications for accessible technologies independent of the codec itself. Notable Xiph specifications include [http://wiki.xiph.org/OggKate OggKate] and [http://wiki.xiph.org/index.php/CMML CMML], which provide subtitles for the hearing-impaired, as well as [http://wiki.xiph.org/Ogg_Skeleton Skeleton], which can specify scene description audio tracks for the visually impaired. When Theora is transmitted or stored in an Ogg container, it is automatically compatible with these accessibility measures.
* Is conformance governance always part of a standard? (Y/N)
: No. Xiph does not normally provide a formal conformance testing process as part of a standard.
* Is a conformance test offered to implementers? (Y/N)
: Yes. Xiph maintains a suite of [http://v2v.cc/~j/theora_testsuite/ test vectors] that can be used by implementors to confirm basic conformance.
* Is conformance validation available to implementers? (Y/N)
: Yes. Informal conformance testing is available to implementors upon request, and Xiph has provided such testing for a number of implementations in the past.
* Is conformance certification available? (Y/N)
: Yes. Xiph does not require certification, but maintains the right to withhold the use of our trademarks from implementors that act in bad faith. Implementors may, however, request explicit permission to use our trademarks with a conforming implementation.
* Is localisation of a formal specification possible? (Y/N)
: Yes. We welcome anyone who wishes to translate Xiph specifications into other languages. We have no policy requiring that the normative specification be written in English.

=== Interoperability governance ===
The interoperability governance evaluation criteria relates to how interoperability is identified and maintained between interoperable formal specifications. In order to do this, the organisation may provide governance for:
* open identification in formal specifications,
* open negotiation in formal specifications,
* open selection in formal specifications.

=== Meeting and consultation ===
The meeting and consultation evaluation criteria relates to the process of defining a formal specification. As formal specifications are usually defined by committees, and these committees normally consist of members of the organisation, this criteria studies how to become a member and which are the financial barriers for this, as well as how are non-members able to have an influence on the process of defining the formal specification. It analyses:
* if the organisation is open to all types of companies and organisations and to individuals;
: Yes. Xiph welcomes representatives from all companies and organizations.
* if the standardisation process may specifically allow participation of members with limited abilities when relevant;
: Yes. Standardization occurs almost entirely in internet communications channels, allowing participants with disabilities to engage fully in the standards development process. We also encourage nonexperts and students to assist us as they can, and to learn about Xiph technologies by participating in the standards development process.
* if meetings are open to all members;
: Xiph meetings are open to everyone. We charge no fee for and place no restrictions on attendance or participation. For example, anyone interested in contributing to the Theora specification may join [http://lists.xiph.org/pipermail/theora-dev/ the Theora development mailing list].
* if all can participate in the formal specification creation process;
: Yes. All people are welcome to participate in the specification creation process. No dues or fees are required to participate
* if non-members can participate in the formal specification creation process.
: Yes. Xiph does not maintain an explicit list of members, and no one is excluded from contributing to specifications as they are developed.

=== Consensus ===
Consensus is decision making primarily with regard to the approval of formal specifications and review with interest groups (non-members). The consensus evaluation criterion is evaluated with the following questions:
* Does the organisation have a stated objective of reaching consensus when making decisions on standards?
: There is no explicitly stated objective of reaching consensus.
* If consensus is not reached, can the standard be approved? (answers are: cannot be approved but referred back to working group/committee, approved with 75% majority, approved with 66% majority, approved with 51% majority, can be decided by a "director" or similar in the organisation).
: The standard can be approved without consensus via the decision of a "director" or similar.
* Is there a formal process for external review of standard proposals by interest groups (nonmembers)?
: Since anyone may participate in the development process and make proposals, there is no need for a separate formal process to include proposals by nonmembers.

=== Due Process ===
The due process evaluation criteria relates to the level of respect of each member of the organisation with regard to its rights. More specifically, it must be assured that if a member believes an error has been made in the process of defining a formal specification, it must be possible to appeal this to an independent, higher instance. The question is therefore: can a member formally appeal or raise objections to a procedure or to a technical specification to an independent, higher instance?

: Yes. Even if a member fails an appeal within the organization, because all of the technology Xiph standardizes is open and freely implementable, they are always free to develop their own, competing version. Such competing versions may even still be eligible for standardization under the Xiph umbrella.

=== Changes to the formal specification ===
The suggested changes made to a formal specification need to be presented, evaluated and approved in the same way as the formal specification was first defined. This criteria therefore applies the above criteria to the changes made to the formal specification(availability of documentation, Intellectual Property Right, accessibility, interoperability governance, meeting and consultation, consensus, due process).

: The exact same process is used for revisions to the standard as was used for the original development of the standard, and thus the answers to all of the above questions remain the same.

=== Support ===
It is critical that the organisation takes responsibility for the formal specification throughout its life span. This can be done in several ways such as for example a regular periodic review of the formal specification. The support criteria relates to the level of commitment the organisation has taken to support the formal specification throughout its life:
* does the organisation provide support until removal of the published formal specification from public domain (Including this process?
: Xiph.Org standards are never removed from the public domain. Xiph endeavors to provide support for as long as the standard remains in use.
* does the organisation make the formal specification still available even when in non-maintenance mode?
: Yes. All Xiph.Org standards are freely licensed and will always be available.
* does the organisation add new features and keep the formal specification up-to-date?
: Yes. Xiph maintains its ecosystem of standards on a continuous basis.
* does the organisation rectify problems identified in initial implementations?
: Yes. Xiph maintains [https://trac.xiph.org/report a problem reporting system] that is open to the public, and invites everyone to submit suggestions for improvements. Improvements are made both to the standards documents and to the reference implementations.
* does the organisation only create the formal specification?
: No. Xiph also produces high-quality reusable reference implementations of its standards, released under an open license.

<strong>This is a draft document. A work in progress. A scratchpad for ideas. It should not be widely circulated in this form.</strong>

Work In Progress

2009-10-11T08:58:33Z

Silvia: added Ogg Index

* '''General Usage:'''
** [[Ogg_Index]]: Introducting index headers into Ogg
** [[Metadata]]: Various types of Ogg metadata including the [[M3F]] (Multimedia Metadata Format) and [[XMLEmbedding]]
** [[MIME_Types_and_File_Extensions]]: MIME Types and file extensions for Ogg multimedia files
** [[Subtle]]: Subtitling tool for professional use that intends to support most subtitle formats including CMML and OggKate
** [[OggText]]: A generic media mapping for (discontinuous) text codecs into Ogg
** [[ROE]]: A description format for describing the tracks and languages etc. of an Ogg multitrack composition

* '''Compressed Codecs:'''
** [[Theora]] 1.1 "thusnelda": improving compression efficiency while keeping compatibility, see [[Theora11Todo]]
** [[OggDirac]]: The "next-generation" wavelet based video codec, lossy or lossless
** [[OggCELT]]: A low-latency audio codec
** [[OggMNG]]: A mapping for encapsulating the MNG animation format in Ogg

* '''Uncompressed Codecs:'''
** [[OggKate]]: A codec for karaoke and text encapsulation in Ogg
** [[OggPCM]]: Uncompressed PCM audio, currently being implemented
** [[OggSpots]]: A mapping for encapsulating timed images in Ogg
** [[OggUVS]]: Uncompressed RGB and YUV video

* '''Abandonware''' (nobody working on those as far as we know)
** [[Ghost]]: A "next-generation" audio codec (vapourware so far -- don't hold your breath)
** [[Oggless]]: Embedding Xiph codecs like Vorbis in containers other than Ogg
** [[IceShare]]: P2P content distribution
** [[OggPCM_Draft1]]: Original uncompressed PCM audio proposal
** [[OggRGB]]: Original uncompressed RGB video proposal
** [[OggWrit]]: Text phrase codec (e.g. subtitles)
** [[OggYUV]]: Original uncompressed YUV video proposal

[[Category:Developers stuff]]

MIME Types and File Extensions

2009-10-04T10:50:29Z

Silvia: /* Ogg Kate files - application/kate */

STATUS: Work on RFCs and tools is in process to reflect these policies. More details are [http://wiki.xiph.org/index.php/MIMETypesCodecs here], which also include a specification of the codecs parameter of the MIME tyes. Use the correct file extensions straight away.

DISCLAIMER: currently, application/ogg, video/ogg, audio/ogg and audio/vorbis are registered MIME types. Registration for the others will be undertaken. During this process, the "x-" versions of these unregistered MIME types may be used.

IMPLEMENTATION recommendations and patches: see [[MIME-Migration]].

== .ogx - application/ogg ==

* Ogg Multiplex Profile (anything in [[Ogg]])
* can contain any logical bitstreams multiplexed together in an ogg container
* will replace the .ogg extension from RFC 3534 http://www.ietf.org/rfc/rfc3534.txt
* random multitrack files MUST contain a [[Skeleton]] track to identify all containing logical bitstreams
* apps that identify a logical bitstream which they cannot decode SHOULD ignore it but MAY still decode the ones they can
* thus, e.g. an annodex file can gracefully degrade to .ogx if an app cannot decode [[CMML]] and/or [[Skeleton]]
* USE: application/ogg has been registered, so can be used immediately

== .ogv - video/ogg ==

* Ogg Video Profile (a/v in Ogg container)
* apps supporting .oga, .ogv SHOULD support decoding from muxed Ogg streams
* covers e.g. [[Theora]], Theora + Vorbis, Theora + Speex, Theora + FLAC, [[Dirac]] + Vorbis, [[OggMNG|MNG]] + FLAC, [[OggUVS]] inside Ogg
* This list is not exhaustive (for example, [[Dirac]] + FLAC is acceptable too)
* SHOULD contain a Skeleton track and/or MAY contain a CMML logical bitstream.

== .oga - audio/ogg ==

* Ogg Audio Profile (audio in Ogg container)
* Applications supporting .oga, .ogv SHOULD support decoding from muxed Ogg streams
* Covers Ogg [[FLAC]], [[Ghost]], and [[OggPCM]]
* Although they share the same MIME type, Vorbis and Speex use different file extensions.
* SHOULD contain a Skeleton logical bitstream.
* Vorbis and Speex may use .oga, but it is not the prefered method of distributing these files because of backwards-compatibility issues.

== .ogg - audio/ogg ==

* Ogg Vorbis I Profile
* .ogg applies now for Vorbis I files only
* .ogg has more recently also been used for Ogg FLAC and for Theora, too — these uses are deprecated now in favor of .oga and .ogv respectively
* has been defined in RFC 3534 http://www.ietf.org/rfc/rfc3534.txt for application/ogg, so rfc 3534 will be re-defined

RATIONALE: .ogg has traditionally been used for Vorbis I files, in particular in HW players, hence it is kept for backwards-compatibility

== .spx - audio/ogg ==

* Ogg Speex Profile
* .spx has traditionally been used for Speex files within Ogg and should be considered for backwards-compatibility

== .flac - audio/flac ==

* FLAC in native encapsulation format

== .anx - application/annodex ==

* Profile for multiplexed Ogg that includes a skeleton track and at least one CMML logical bitstream
* apps that identify a logical bitstream which they cannot decode SHOULD ignore it but MAY still decode the ones they can
* apps that come across an annodex file and cannot decode CMML and/or Skeleton, but can deal with the others SHOULD gracefully degrade by ignoring these

== .axa - audio/annodex ==

* Profile for audio in Annodex
* covers e.g. [[Vorbis]], [[Speex]], [[FLAC]], [[Ghost]], [[OggPCM]] inside Ogg with Skeleton and CMML

== .axv - video/annodex ==

* Profile for video in Annodex
* covers e.g. [[Theora]], Theora + Vorbis, Theora + Speex, Theora + FLAC, [[Dirac]] + Vorbis, [[OggMNG|MNG]] + FLAC, [[OggUVS]] inside Ogg with Skeleton and CMML

== .xspf - application/xspf+xml ==

* Profile for XSPF
* Covers [[XSPF]], while being used through XML
* Does not cover [[JSPF]], which is XSPF but on JSON

== Ogg Kate files - application/kate ==

* Binary representation of Kate encapsulated in Ogg
* may have a skeleton
* can be used to identify the mime type of the track itself (e.g. in skeleton)
* uses .ogx extension when in a file by itself
* is subdued by the dominant mime type if in a audio or video file to become audio/ogg or video/ogg

== Codec MIME types ==

Codecs need their own MIME types for streaming in RTP and to be used in multitrack ogg files using skeleton:

* audio/vorbis for Vorbis without container
* video/theora for Theora without container
* audio/speex for Speex without container
* audio/flac for FLAC without and in native container
* text/cmml for CMML without container
* text/kate for the textual representation of Kate (.kate files)

MIME Types and File Extensions

2009-10-04T10:49:55Z

Silvia: /* Ogg Kate files - application/kate */

STATUS: Work on RFCs and tools is in process to reflect these policies. More details are [http://wiki.xiph.org/index.php/MIMETypesCodecs here], which also include a specification of the codecs parameter of the MIME tyes. Use the correct file extensions straight away.

DISCLAIMER: currently, application/ogg, video/ogg, audio/ogg and audio/vorbis are registered MIME types. Registration for the others will be undertaken. During this process, the "x-" versions of these unregistered MIME types may be used.

IMPLEMENTATION recommendations and patches: see [[MIME-Migration]].

== .ogx - application/ogg ==

* Ogg Multiplex Profile (anything in [[Ogg]])
* can contain any logical bitstreams multiplexed together in an ogg container
* will replace the .ogg extension from RFC 3534 http://www.ietf.org/rfc/rfc3534.txt
* random multitrack files MUST contain a [[Skeleton]] track to identify all containing logical bitstreams
* apps that identify a logical bitstream which they cannot decode SHOULD ignore it but MAY still decode the ones they can
* thus, e.g. an annodex file can gracefully degrade to .ogx if an app cannot decode [[CMML]] and/or [[Skeleton]]
* USE: application/ogg has been registered, so can be used immediately

== .ogv - video/ogg ==

* Ogg Video Profile (a/v in Ogg container)
* apps supporting .oga, .ogv SHOULD support decoding from muxed Ogg streams
* covers e.g. [[Theora]], Theora + Vorbis, Theora + Speex, Theora + FLAC, [[Dirac]] + Vorbis, [[OggMNG|MNG]] + FLAC, [[OggUVS]] inside Ogg
* This list is not exhaustive (for example, [[Dirac]] + FLAC is acceptable too)
* SHOULD contain a Skeleton track and/or MAY contain a CMML logical bitstream.

== .oga - audio/ogg ==

* Ogg Audio Profile (audio in Ogg container)
* Applications supporting .oga, .ogv SHOULD support decoding from muxed Ogg streams
* Covers Ogg [[FLAC]], [[Ghost]], and [[OggPCM]]
* Although they share the same MIME type, Vorbis and Speex use different file extensions.
* SHOULD contain a Skeleton logical bitstream.
* Vorbis and Speex may use .oga, but it is not the prefered method of distributing these files because of backwards-compatibility issues.

== .ogg - audio/ogg ==

* Ogg Vorbis I Profile
* .ogg applies now for Vorbis I files only
* .ogg has more recently also been used for Ogg FLAC and for Theora, too — these uses are deprecated now in favor of .oga and .ogv respectively
* has been defined in RFC 3534 http://www.ietf.org/rfc/rfc3534.txt for application/ogg, so rfc 3534 will be re-defined

RATIONALE: .ogg has traditionally been used for Vorbis I files, in particular in HW players, hence it is kept for backwards-compatibility

== .spx - audio/ogg ==

* Ogg Speex Profile
* .spx has traditionally been used for Speex files within Ogg and should be considered for backwards-compatibility

== .flac - audio/flac ==

* FLAC in native encapsulation format

== .anx - application/annodex ==

* Profile for multiplexed Ogg that includes a skeleton track and at least one CMML logical bitstream
* apps that identify a logical bitstream which they cannot decode SHOULD ignore it but MAY still decode the ones they can
* apps that come across an annodex file and cannot decode CMML and/or Skeleton, but can deal with the others SHOULD gracefully degrade by ignoring these

== .axa - audio/annodex ==

* Profile for audio in Annodex
* covers e.g. [[Vorbis]], [[Speex]], [[FLAC]], [[Ghost]], [[OggPCM]] inside Ogg with Skeleton and CMML

== .axv - video/annodex ==

* Profile for video in Annodex
* covers e.g. [[Theora]], Theora + Vorbis, Theora + Speex, Theora + FLAC, [[Dirac]] + Vorbis, [[OggMNG|MNG]] + FLAC, [[OggUVS]] inside Ogg with Skeleton and CMML

== .xspf - application/xspf+xml ==

* Profile for XSPF
* Covers [[XSPF]], while being used through XML
* Does not cover [[JSPF]], which is XSPF but on JSON

== Ogg Kate files - application/kate ==

* Binary representation of Kate encapsulated in Ogg
* may have a skeleton
* can be used to identify the mime type of the track itself (e.g. in skeleton)
* uses .ogx extension when in a file by itself
* is subdued by the dominant mime type if in a audio or video file to become audio/vorbis or video/theora

== Codec MIME types ==

Codecs need their own MIME types for streaming in RTP and to be used in multitrack ogg files using skeleton:

* audio/vorbis for Vorbis without container
* video/theora for Theora without container
* audio/speex for Speex without container
* audio/flac for FLAC without and in native container
* text/cmml for CMML without container
* text/kate for the textual representation of Kate (.kate files)

MIME Types and File Extensions

2009-10-04T10:47:18Z

Silvia: added kate

STATUS: Work on RFCs and tools is in process to reflect these policies. More details are [http://wiki.xiph.org/index.php/MIMETypesCodecs here], which also include a specification of the codecs parameter of the MIME tyes. Use the correct file extensions straight away.

DISCLAIMER: currently, application/ogg, video/ogg, audio/ogg and audio/vorbis are registered MIME types. Registration for the others will be undertaken. During this process, the "x-" versions of these unregistered MIME types may be used.

IMPLEMENTATION recommendations and patches: see [[MIME-Migration]].

== .ogx - application/ogg ==

* Ogg Multiplex Profile (anything in [[Ogg]])
* can contain any logical bitstreams multiplexed together in an ogg container
* will replace the .ogg extension from RFC 3534 http://www.ietf.org/rfc/rfc3534.txt
* random multitrack files MUST contain a [[Skeleton]] track to identify all containing logical bitstreams
* apps that identify a logical bitstream which they cannot decode SHOULD ignore it but MAY still decode the ones they can
* thus, e.g. an annodex file can gracefully degrade to .ogx if an app cannot decode [[CMML]] and/or [[Skeleton]]
* USE: application/ogg has been registered, so can be used immediately

== .ogv - video/ogg ==

* Ogg Video Profile (a/v in Ogg container)
* apps supporting .oga, .ogv SHOULD support decoding from muxed Ogg streams
* covers e.g. [[Theora]], Theora + Vorbis, Theora + Speex, Theora + FLAC, [[Dirac]] + Vorbis, [[OggMNG|MNG]] + FLAC, [[OggUVS]] inside Ogg
* This list is not exhaustive (for example, [[Dirac]] + FLAC is acceptable too)
* SHOULD contain a Skeleton track and/or MAY contain a CMML logical bitstream.

== .oga - audio/ogg ==

* Ogg Audio Profile (audio in Ogg container)
* Applications supporting .oga, .ogv SHOULD support decoding from muxed Ogg streams
* Covers Ogg [[FLAC]], [[Ghost]], and [[OggPCM]]
* Although they share the same MIME type, Vorbis and Speex use different file extensions.
* SHOULD contain a Skeleton logical bitstream.
* Vorbis and Speex may use .oga, but it is not the prefered method of distributing these files because of backwards-compatibility issues.

== .ogg - audio/ogg ==

* Ogg Vorbis I Profile
* .ogg applies now for Vorbis I files only
* .ogg has more recently also been used for Ogg FLAC and for Theora, too — these uses are deprecated now in favor of .oga and .ogv respectively
* has been defined in RFC 3534 http://www.ietf.org/rfc/rfc3534.txt for application/ogg, so rfc 3534 will be re-defined

RATIONALE: .ogg has traditionally been used for Vorbis I files, in particular in HW players, hence it is kept for backwards-compatibility

== .spx - audio/ogg ==

* Ogg Speex Profile
* .spx has traditionally been used for Speex files within Ogg and should be considered for backwards-compatibility

== .flac - audio/flac ==

* FLAC in native encapsulation format

== .anx - application/annodex ==

* Profile for multiplexed Ogg that includes a skeleton track and at least one CMML logical bitstream
* apps that identify a logical bitstream which they cannot decode SHOULD ignore it but MAY still decode the ones they can
* apps that come across an annodex file and cannot decode CMML and/or Skeleton, but can deal with the others SHOULD gracefully degrade by ignoring these

== .axa - audio/annodex ==

* Profile for audio in Annodex
* covers e.g. [[Vorbis]], [[Speex]], [[FLAC]], [[Ghost]], [[OggPCM]] inside Ogg with Skeleton and CMML

== .axv - video/annodex ==

* Profile for video in Annodex
* covers e.g. [[Theora]], Theora + Vorbis, Theora + Speex, Theora + FLAC, [[Dirac]] + Vorbis, [[OggMNG|MNG]] + FLAC, [[OggUVS]] inside Ogg with Skeleton and CMML

== .xspf - application/xspf+xml ==

* Profile for XSPF
* Covers [[XSPF]], while being used through XML
* Does not cover [[JSPF]], which is XSPF but on JSON

== Ogg Kate files - application/kate ==

* Binary representation of Kate encapsulated in Ogg
* may have a skeleton
* can be used to identify the mime type of the track itself (e.g. in skeleton)
* uses .ogx extension when in a file by itself
* is subdued by the dominant mime type if in a audio or video file

== Codec MIME types ==

Codecs need their own MIME types for streaming in RTP and to be used in multitrack ogg files using skeleton:

* audio/vorbis for Vorbis without container
* video/theora for Theora without container
* audio/speex for Speex without container
* audio/flac for FLAC without and in native container
* text/cmml for CMML without container
* text/kate for the textual representation of Kate (.kate files)

Work In Progress

2009-08-25T14:05:57Z

Silvia: added ROE

* '''General Usage:'''
** [[Metadata]]: Various types of Ogg metadata including the [[M3F]] (Multimedia Metadata Format) and [[XMLEmbedding]]
** [[MIME_Types_and_File_Extensions]]: MIME Types and file extensions for Ogg multimedia files
** [[Subtle]]: Subtitling tool for professional use that intends to support most subtitle formats including CMML and OggKate
** [[OggText]]: A generic media mapping for (discontinuous) text codecs into Ogg
** [[ROE]]: A description format for describing the tracks and languages etc. of an Ogg multitrack composition

* '''Compressed Codecs:'''
** [[Theora]] 1.1 "thusnelda": improving compression efficiency while keeping compatibility, see [[Theora11Todo]]
** [[OggDirac]]: The "next-generation" wavelet based video codec, lossy or lossless
** [[OggCELT]]: A low-latency audio codec
** [[OggMNG]]: A mapping for encapsulating the MNG animation format in Ogg

* '''Uncompressed Codecs:'''
** [[OggKate]]: A codec for karaoke and text encapsulation in Ogg
** [[OggPCM]]: New Uncompressed PCM audio, currently being implemented (formerly Draft2)
** [[OggSpots]]: A mapping for encapsulating timed images in Ogg
** [[OggUVS]]: Uncompressed RGB and YUV video, under active development (preferred to OggRGB and OggYUV).

* '''Abandonware''' (nobody working on those as far as we know)
** [[Ghost]]: A "next-generation" audio codec (vapourware so far -- don't hold your breath)
** [[Oggless]]: Embedding Xiph codecs like Vorbis in containers other than Ogg
** [[IceShare]]: P2P content distribution
** [[OggPCM_Draft1]]: Original uncompressed PCM audio proposal
** [[OggRGB]]: Original uncompressed RGB video proposal
** [[OggWrit]]: Text phrase codec (e.g. subtitles)
** [[OggYUV]]: Original uncompressed YUV video proposal

[[Category:Developers stuff]]

Timed Divs HTML

2009-06-21T22:59:46Z

Silvia: /* Direct linking on a HTML5 page */

{{draft}}

= Introduction =

This page specifies a subclass of HTML documents that is a time-aligned text format for audio-visual content. We call the format "timed divs within HTML" or TDHT. It is intended to be used only in a World Wide Web context i.e. everywhere that Web browser functionality is available. Use cases for the format are subtitles, captions, annotations and other time aligned text as listed at http://wiki.xiph.org/index.php/OggText#Categories_of_Text_Codecs .

TDHT may be similar to W3C TimedText DFXP in many respects, but in comparison to DFXP it does not re-invent HTML, CSS and effects, but rather uses existing HTML, CSS and javascript for these. The purpose of DFXP is to create a web-independent exchange format for timed text, which is why it cannot directly be specified as a subpart of HTML.

TDHT in contrast is HTML with a minimum number of changes. TDHT is parsable by any HTML parser. It works with CSS and javascript. No new functionality has to be defined for TDHT.

= File Extension =

Files in this format are to be of text/html mime type since they are valid html files, apart from some extra attributes.

Files in this format should have a file extension of .tdht to separate them from plain html files.

= The TDHT format changes from HTML =

TDHT files are time-aligned text. This means there is a time association with blocks of text and there is time-based seeking functionality on those blocks of text.

Here is an example TDHT file for subtitles:

<pre>
<html>
<head>
<title>Desperate Housewives - Season 5, Episode 6</title>
</head>
<body>
<div start="00:00:00.070" end="00:00:02.270">
<p>Previously on...</p>
</div>
<div start="00:00:02.280" end="00:00:04.270">
<p>We had an agreement to keep things casual.</p>
</div>
<div start="00:00:04.280" end="00:00:06.660">
<p>Susan made her feelings clear.</p>
</div>
<div start="00:00:06.800" end="00:00:10.100">
<p>So if I was with another woman, that wouldn't bother you? No, it wouldn't.</p>
</div>
</body>
</html>
</pre>

The differences of TDHT from HTML are described using [http://www.w3.org/TR/html401/ HTML4.01], but the changes apply the same to [http://www.whatwg.org/specs/web-apps/current-work/ HTML5], which doesn't have a normative schema.

The following changes to HTML are made for TDHT:

== 1. The body element ==

In HTML4.01, the [http://www.w3.org/TR/html401/struct/global.html#h-7.5 body element] is defined as follows:

<pre>
<!ELEMENT BODY O O (%block;|SCRIPT)+ +(INS|DEL) -- document body -->
<!ATTLIST BODY
%attrs; -- %coreattrs, %i18n, %events --
onload %Script; #IMPLIED -- the document has been loaded --
onunload %Script; #IMPLIED -- the document has been removed --
>
</pre>

In TDHT1.0 we restrict body to just contain a sequence of div tags:

<pre>
<!ELEMENT BODY O O (DIV)+ -- document body -->
<!ATTLIST BODY
%attrs; -- %coreattrs, %i18n, %events --
onload %Script; #IMPLIED -- the document has been loaded --
onunload %Script; #IMPLIED -- the document has been removed --
>
</pre>

Any tags inside the body tag that are non-conformant to this specification (such as regular html tags that are allowed inside body) must be ignored for TDHT.

The div tags, however, can contain anything that HTML div tags can contain, thus enabling a very flexible, but time-aligned text model.

== 2. The div element ==

In HTML, the [http://www.w3.org/TR/html401/struct/global.html#h-7.5.4 div element] is defined as follows:

<pre>
<!ELEMENT DIV - - (%flow;)* -- generic language/style container -->
<!ATTLIST DIV
%attrs; -- %coreattrs, %i18n, %events --
>
</pre>

In TDHT1.0 we extend it with start and end time attributes:

<pre>
<!ELEMENT DIV - - (%flow;)* -- generic language/style container -->
<!ATTLIST DIV
%attrs; -- %coreattrs, %i18n, %events --
start %Time; #IMPLIED -- start time
end %Time; #IMPLIED -- end time
>
</pre>

The Time entity represents a valid time string accroding to HTML5: http://www.whatwg.org/specs/web-apps/current-work/#valid-time-string . The end time string must be larger than the start time string, otherwise the div element does not exist for any duration and can never turn active.

<div> elements in a TDHT file should be ordered by start time to simplify parsing. Inside Ogg or when rendered, they will be ordered by start time.

= Rendering in a Web Browser =

A TDHT file is meant to be associated with a audio or video file and rendered in a Web browser in sync with the audio or video file.

The TDHT file's div elements are not rendered into an existing HTML page, but rather a TDHT file creates its own [http://www.whatwg.org/specs/web-apps/current-work/#the-iframe-element iframe-like] new nested browsing context. It is linked to the parent HTML page through an itext element that is inserted as a child of the video element. Creation of a nested browsing context is important because a TDHT file can come from a different URI than the Web page and thus for security reasons and for general base URI computations a nested browsing context is the better approach with the DOM nodes of the hosting page and the DOM nodes of the TDHT document in different owner documents. That way, the hosting document has the security origin of its own URL and the TDHT document has the security origin of its URL.

The rendering and CSS view port are either by default the rectangle occupied by the given <video> or <audio> tag, or an area provided for by the hosting HTML page through the itext element's properties. The zoom factor of the iframe must be set to such a value that the width of the view port established by the itext frame is equally wide in CSS px as the video frame is wide in codec pixels. (Example: If the video encodes a frame that is 240 pixels wide but is displayed at 480 CSS px wide, the zoom factor of the itext frame should be 200% so that the box that on the outsize measures 480 px seems like a box of 240 px from within the itext frame.)

The itext frame is by default transparent.

A TDHT file can get to a browser either as a external resource, or as part of audio or video resource (in particular inside Ogg, see below). Parsing in these two cases is slightly different for the browser.

For the external TDHT file case:
The TDHT file is parsed using the HTML5 parsing algorithm in its normal mode into a non-rendered DOM. To render a div, the children of the div would be cloned into the body of the rendering shell document (replacing possible previous children of body).

For the Ogg-internal TDHT case:
To multiplex an external TDHT file into Ogg, each div with its innerHTML would be placed into a data packet and the head data in to an Ogg header. For decoding, the rendering shell document is set up and the head tag is included from the Ogg headers. To render a packet, the div and its innerHTML would be added to the innerHTML of the body element of the rendering shell document as they come. This will use the HTML fragment parser.

As the browser plays the video, it must render the TDHT <div> tags in sync. As the start time of a <div> tag is reached, the <div> tag is made activate, and it is made inactive as the <div> tag's end time is reached. If no start time is given, the start is assumed to be 0, and if no end time is given, end is assumed to be infinity.

An "active" <div> tag may be a <div> tag that is being displayed ("display: block") in contrast to an "inactive" <div> tag, which may not be displayed ("display: none"). For some text formats however the difference between "active" and "inactive" may be a background colour or the display location on screen or some other mechanism. The default should be between "block" and "none", but changeable through CSS.

As the browser has parsed the TDHT file or its consitutent <div> tags, it is expected to keep the structure in memory. When seeking happens on the video, it can then decide upon which <div> tags are supposed to be active at the seek time and display these. [There is a discussion to be had here about the effect this has on the DOM. Different selectors may apply to a caption depending on whether the video was played back all the way there or seeking skipped over data to get there. It was suggested that inactive captions should be removed from the DOM, so there's always a well-defined small unambiguous DOM to match selectors against. However, this may for example not be desirable on some text display formats.]

= Encapsulation into Ogg =

The [http://wiki.xiph.org/index.php/OggText OggText] specification is used to encapsulate a TDHT file into Ogg.

The codec-specific header data for the OggText ident header is the <head>..</head> part of the TDHT file. The complete <head> tag including all its subtags is encoded into the ident header unchanged.

The <div> elements with all their inner HTML are the data packets of the TDHT text codec and are thus encapsulated into the data packets as text codec data. A complete <div> including all its subtags is encoded into one data packet each.

= Direct linking on a HTML5 page =

Often, subtitles and other time-aligned text files are not actually provided inside a video stream (e.g. inside Ogg), but are referenced as a separate partner resource to a video.

To allow association of such files with a <video> or <audio> element, we propose the following approach:

<pre>
<video i="video" src="http://example.com/video.ogv" controls>
<itext id="caption1" category="CC" lang="en/us" src="caption.srt" style=""></itext>
<itext id="caption2" category="CC" lang="de/de" src="caption.tdht" style=""></itext>
<itext id="subtitle1" category="SUB" lang="de/de" src="german.dfxp" style=""></itext>
<itext id="subtitle2" category="SUB" lang="jp" src="japanese.smil" style="></itext>
<itext id="subtitle3" category="SUB" lang="fr" src="translation_webservice/fr/caption.srt" style=""></itext>
</video>
</pre>

Notice the second set of closed captions being a TDHT file.

The id tag is simply a unique identifier for the tag.
The category is from [http://wiki.xiph.org/index.php/OggText#Categories_of_Text_Codecs Ogg text categories].
The lang contains a natural language according to [http://en.wikipedia.org/wiki/Language_code language codes].
The src element contains the actual file URI that we are after.
The style element allows to attach styling to marked-up import files.

The <itext> element would act like an <iframe> element and create the nested browsing context described earlier. It has been renamed from earlier mentions of this approach from <text> to <itext> to avoid name clashes with e.g. SVG.

The user agent would then provide an interface such as:

interface MediaItextElement : HTMLElement {
attribute DOMString src;
attribute DOMString category;
attribute DOMString lang;
attribute DOMString id;
attribute DOMString style;
};

In javascript there will need to be additional functions such as:

getItext (): returns an array of time-aligned text elements
addItext({src,category,lang,style,name}): adds a time-aligned text element to a <video> or <audio> element
enable(itextElement): activates display of an itext file
disable(itextElement) : deactivates display of an itext file
delay(itextElement, seconds) : delays the itext file in relation to the video file by a positive or negative number of seconds

Timed Divs HTML

2009-06-21T22:27:50Z

Silvia: /* Direct linking on a HTML5 page */

{{draft}}

= Introduction =

This page specifies a subclass of HTML documents that is a time-aligned text format for audio-visual content. We call the format "timed divs within HTML" or TDHT. It is intended to be used only in a World Wide Web context i.e. everywhere that Web browser functionality is available. Use cases for the format are subtitles, captions, annotations and other time aligned text as listed at http://wiki.xiph.org/index.php/OggText#Categories_of_Text_Codecs .

TDHT may be similar to W3C TimedText DFXP in many respects, but in comparison to DFXP it does not re-invent HTML, CSS and effects, but rather uses existing HTML, CSS and javascript for these. The purpose of DFXP is to create a web-independent exchange format for timed text, which is why it cannot directly be specified as a subpart of HTML.

TDHT in contrast is HTML with a minimum number of changes. TDHT is parsable by any HTML parser. It works with CSS and javascript. No new functionality has to be defined for TDHT.

= File Extension =

Files in this format are to be of text/html mime type since they are valid html files, apart from some extra attributes.

Files in this format should have a file extension of .tdht to separate them from plain html files.

= The TDHT format changes from HTML =

TDHT files are time-aligned text. This means there is a time association with blocks of text and there is time-based seeking functionality on those blocks of text.

Here is an example TDHT file for subtitles:

<pre>
<html>
<head>
<title>Desperate Housewives - Season 5, Episode 6</title>
</head>
<body>
<div start="00:00:00.070" end="00:00:02.270">
<p>Previously on...</p>
</div>
<div start="00:00:02.280" end="00:00:04.270">
<p>We had an agreement to keep things casual.</p>
</div>
<div start="00:00:04.280" end="00:00:06.660">
<p>Susan made her feelings clear.</p>
</div>
<div start="00:00:06.800" end="00:00:10.100">
<p>So if I was with another woman, that wouldn't bother you? No, it wouldn't.</p>
</div>
</body>
</html>
</pre>

The differences of TDHT from HTML are described using [http://www.w3.org/TR/html401/ HTML4.01], but the changes apply the same to [http://www.whatwg.org/specs/web-apps/current-work/ HTML5], which doesn't have a normative schema.

The following changes to HTML are made for TDHT:

== 1. The body element ==

In HTML4.01, the [http://www.w3.org/TR/html401/struct/global.html#h-7.5 body element] is defined as follows:

<pre>
<!ELEMENT BODY O O (%block;|SCRIPT)+ +(INS|DEL) -- document body -->
<!ATTLIST BODY
%attrs; -- %coreattrs, %i18n, %events --
onload %Script; #IMPLIED -- the document has been loaded --
onunload %Script; #IMPLIED -- the document has been removed --
>
</pre>

In TDHT1.0 we restrict body to just contain a sequence of div tags:

<pre>
<!ELEMENT BODY O O (DIV)+ -- document body -->
<!ATTLIST BODY
%attrs; -- %coreattrs, %i18n, %events --
onload %Script; #IMPLIED -- the document has been loaded --
onunload %Script; #IMPLIED -- the document has been removed --
>
</pre>

Any tags inside the body tag that are non-conformant to this specification (such as regular html tags that are allowed inside body) must be ignored for TDHT.

The div tags, however, can contain anything that HTML div tags can contain, thus enabling a very flexible, but time-aligned text model.

== 2. The div element ==

In HTML, the [http://www.w3.org/TR/html401/struct/global.html#h-7.5.4 div element] is defined as follows:

<pre>
<!ELEMENT DIV - - (%flow;)* -- generic language/style container -->
<!ATTLIST DIV
%attrs; -- %coreattrs, %i18n, %events --
>
</pre>

In TDHT1.0 we extend it with start and end time attributes:

<pre>
<!ELEMENT DIV - - (%flow;)* -- generic language/style container -->
<!ATTLIST DIV
%attrs; -- %coreattrs, %i18n, %events --
start %Time; #IMPLIED -- start time
end %Time; #IMPLIED -- end time
>
</pre>

The Time entity represents a valid time string accroding to HTML5: http://www.whatwg.org/specs/web-apps/current-work/#valid-time-string . The end time string must be larger than the start time string, otherwise the div element does not exist for any duration and can never turn active.

<div> elements in a TDHT file should be ordered by start time to simplify parsing. Inside Ogg or when rendered, they will be ordered by start time.

= Rendering in a Web Browser =

A TDHT file is meant to be associated with a audio or video file and rendered in a Web browser in sync with the audio or video file.

The TDHT file's div elements are not rendered into an existing HTML page, but rather a TDHT file creates its own [http://www.whatwg.org/specs/web-apps/current-work/#the-iframe-element iframe-like] new nested browsing context. It is linked to the parent HTML page through an itext element that is inserted as a child of the video element. Creation of a nested browsing context is important because a TDHT file can come from a different URI than the Web page and thus for security reasons and for general base URI computations a nested browsing context is the better approach with the DOM nodes of the hosting page and the DOM nodes of the TDHT document in different owner documents. That way, the hosting document has the security origin of its own URL and the TDHT document has the security origin of its URL.

The rendering and CSS view port are either by default the rectangle occupied by the given <video> or <audio> tag, or an area provided for by the hosting HTML page through the itext element's properties. The zoom factor of the iframe must be set to such a value that the width of the view port established by the itext frame is equally wide in CSS px as the video frame is wide in codec pixels. (Example: If the video encodes a frame that is 240 pixels wide but is displayed at 480 CSS px wide, the zoom factor of the itext frame should be 200% so that the box that on the outsize measures 480 px seems like a box of 240 px from within the itext frame.)

The itext frame is by default transparent.

A TDHT file can get to a browser either as a external resource, or as part of audio or video resource (in particular inside Ogg, see below). Parsing in these two cases is slightly different for the browser.

For the external TDHT file case:
The TDHT file is parsed using the HTML5 parsing algorithm in its normal mode into a non-rendered DOM. To render a div, the children of the div would be cloned into the body of the rendering shell document (replacing possible previous children of body).

For the Ogg-internal TDHT case:
To multiplex an external TDHT file into Ogg, each div with its innerHTML would be placed into a data packet and the head data in to an Ogg header. For decoding, the rendering shell document is set up and the head tag is included from the Ogg headers. To render a packet, the div and its innerHTML would be added to the innerHTML of the body element of the rendering shell document as they come. This will use the HTML fragment parser.

As the browser plays the video, it must render the TDHT <div> tags in sync. As the start time of a <div> tag is reached, the <div> tag is made activate, and it is made inactive as the <div> tag's end time is reached. If no start time is given, the start is assumed to be 0, and if no end time is given, end is assumed to be infinity.

An "active" <div> tag may be a <div> tag that is being displayed ("display: block") in contrast to an "inactive" <div> tag, which may not be displayed ("display: none"). For some text formats however the difference between "active" and "inactive" may be a background colour or the display location on screen or some other mechanism. The default should be between "block" and "none", but changeable through CSS.

As the browser has parsed the TDHT file or its consitutent <div> tags, it is expected to keep the structure in memory. When seeking happens on the video, it can then decide upon which <div> tags are supposed to be active at the seek time and display these. [There is a discussion to be had here about the effect this has on the DOM. Different selectors may apply to a caption depending on whether the video was played back all the way there or seeking skipped over data to get there. It was suggested that inactive captions should be removed from the DOM, so there's always a well-defined small unambiguous DOM to match selectors against. However, this may for example not be desirable on some text display formats.]

= Encapsulation into Ogg =

The [http://wiki.xiph.org/index.php/OggText OggText] specification is used to encapsulate a TDHT file into Ogg.

The codec-specific header data for the OggText ident header is the <head>..</head> part of the TDHT file. The complete <head> tag including all its subtags is encoded into the ident header unchanged.

The <div> elements with all their inner HTML are the data packets of the TDHT text codec and are thus encapsulated into the data packets as text codec data. A complete <div> including all its subtags is encoded into one data packet each.

= Direct linking on a HTML5 page =

Often, subtitles and other time-aligned text files are not actually provided inside a video stream (e.g. inside Ogg), but are referenced as a separate partner resource to a video.

To allow association of such files with a <video> or <audio> element, we propose the following approach:

<pre>
<video i="video" src="http://example.com/video.ogv" controls>
<itext id="caption1" category="CC" lang="en/us" src="caption.srt" style=""></itext>
<itext id="caption2" category="CC" lang="de/de" src="caption.tdht" style=""></itext>
<itext id="subtitle1" category="SUB" lang="de/de" src="german.dfxp" style=""></itext>
<itext id="subtitle2" category="SUB" lang="jp" src="japanese.smil" style="></itext>
<itext id="subtitle3" category="SUB" lang="fr" src="translation_webservice/fr/caption.srt" style=""></itext>
</video>
</pre>

Notice the second set of closed captions being a TDHT file.

The id tag is simply a unique identifier for the tag.
The category is from [[http://wiki.xiph.org/index.php/OggText#Categories_of_Text_Codecs Ogg text categories]].
The lang contains a natural language according to [[http://en.wikipedia.org/wiki/Language_code | language codes]].
The src element contains the actual file URI that we are after.
The style element allows to attach styling to marked-up import files.

The <itext> element would act like an <iframe> element and create the nested browsing context described earlier. It has been renamed from earlier mentions of this approach from <text> to <itext> to avoid name clashes with e.g. SVG.

The user agent would then provide an interface such as:

interface MediaItextElement : HTMLElement {
attribute DOMString src;
attribute DOMString category;
attribute DOMString lang;
attribute DOMString id;
attribute DOMString style;
};

In javascript there will need to be additional functions such as:

getItext (): returns an array of time-aligned text elements
addItext({src,category,lang,style,name}): adds a time-aligned text element to a <video> or <audio> element
enable(itextElement): activates display of an itext file
disable(itextElement) : deactivates display of an itext file
delay(itextElement, seconds) : delays the itext file in relation to the video file by a positive or negative number of seconds

MIMETypesCodecs

2009-06-04T11:35:12Z

Silvia: added file extension sentence

== Specification of MIME types and respective codecs parameter ==

Also includes a specification of the recommended file extensions to use with Ogg.

=== MIME Types ===

The following MIME types are now officially registered with IANA and specified with the IETF as [http://www.ietf.org/rfc/rfc5334.txt RFC 5334]:

* video/ogg - for video (with audio) encapsulated in Ogg
** recommends a Skeleton logical bitstrem
** .ogv file extension
** Macintosh File Type Code: OggV

* audio/ogg - for audio encapsulated in Ogg
** recommends a Skeleton logical bitstrem
** .oga file extension, .ogg for Vorbis I, .spx for Speex
** Macintosh File Type Code: OggA

* application/ogg - for complex, multitrack, multiplexed files encapsulated in Ogg
** requires a Skeleton logical bitstream
** .ogx file extension
** Macintosh File Type Code: OggX

[[MIME_Types_and_File_Extensions|Other MIME types]] are still in the process.

=== Codecs Parameter ===

[http://www.rfc-editor.org/rfc/rfc4281.txt Typically], MIME types of media encapsulation formats use the optional "codecs" parameter to specify which codes are being used in a particular file.

Codecs encapsulated in Ogg require a text identifier at the beginning of the first header page to identify the encapsulated codecs. The following table contains the identifiers for existing Xiph codecs and the codecs parameter names used for */ogg MIME types (in alphabetical order):

{| class="codecstable" border="1"
|-
! Codecs Parameter Name
! Codec Type
! Codec Identifier
(decimal, hex, octal)
! Version Field (if available)
|-
| [http://svn.annodex.net/liboggz/trunk/src/liboggz/oggz_auto.h celt]
| audio
| char[0,8]: <tt>'CELT\ \ \ \ '</tt>
hex: <tt>'0x43 0x45 0x4c 0x54 0x20 0x20 0x20 0x20'</tt>

oct: <tt>'0103 0105 0114 0124 0040 0040 0040 0040'</tt>
| char[28,4]: version id
|-
| [http://svn.annodex.net/liboggz/trunk/src/liboggz/oggz_auto.h cmml]
| text
| char[0,8]: <tt>'CMML\0\0\0\0'</tt>
hex: <tt>'0x43 0x4d 0x4d 0x4c 0x00 0x00 0x00 0x00'</tt>

oct: <tt>'0103 0115 0115 0114 0000 0000 0000 0000'</tt>
| char[8,2]: major version number,
char[10,2]: minor version number
|-
| [http://wiki.xiph.org/index.php/OggDirac dirac]
| video
| char[0,5]: <tt>'BBCD\0'</tt>
hex: <tt>'0x42 0x42 0x43 0x44 00'</tt>

oct: <tt>'0102 0102 0103 0104 0000'</tt>
| ??
|-
| [http://flac.sourceforge.net/ogg_mapping.html flac]
| audio
| char[0,5]: <tt>'\177FLAC'</tt>
hex: <tt>'0x7F 0x46 0x4C 0x41 0x43'</tt>

oct: <tt>'0177 0106 0114 0101 0103'</tt>
| char[5,1]: binary major version number,
char[6,1]: binary minor version number of mapping
|-
| [[OggMNG|jng]]
| video
| char[0,8]: <tt>'\213JNG\r\n\032\n'</tt>
hex: <tt>'0x8b 0x4a 0x4e 0x47 0x0D 0x0A 0x1A 0x0A'</tt>

oct: <tt>'0213 0112 0116 0107 0015 0012 0032 0012'</tt>
| ??
|-
| [[OggKate|kate]]
| text
| char[0,8]: <tt>'\x80kate\0\0\0'</tt>
hex: <tt>'0x80 0x6b 0x61 0x74 0x65 0x00 0x00 0x00'</tt>

oct: <tt>'0200 0153 0141 0164 0145 0000 0000 0000'</tt>
| char[9,1]: major version number,
char[10,1]: minor version number
|-
| [http://lists.xiph.org/pipermail/vorbis-dev/2001-August/004501.html midi]
| text
| char[0,8]: <tt>'OggMIDI\0'</tt>
hex: <tt>'0x4f 0x67 0x67 0x4d 0x49 0x44 0x49 0x00'</tt>

oct: <tt>'0117 0147 0147 0115 0111 0104 0111 0000'</tt>
| char[8,1]: version field
|-
| [[OggMNG|mng]]
| video
| char[0,8]: <tt>'\212MNG\r\n\032\n'</tt>
hex: <tt>'0x8a 0x4d 0x4e 0x47 0x0D 0x0A 0x1A 0x0A'</tt>

oct: <tt>'0212 0115 0116 0107 0015 0012 0032 0012'</tt>
| ??
|-
| [[OggPCM|pcm]]
| audio
| char[0,8]: <tt>'PCM\ \ \ \ \ '</tt>
hex: <tt>'0x53 0x70 0x65 0x65 0x78 0x20 0x20 0x20'</tt>

oct: <tt>'0123 0160 0145 0145 0170 0040 0040 0040'</tt>
| char[8,2]: version major field,
char[10,2]: version minor field
|-
| [[OggMNG|png]]
| video
| char[0,8]: <tt>'\211PNG\r\n\032\n'</tt>
hex: <tt>'0x89 0x50 0x4e 0x47 0x0D 0x0A 0x1A 0x0A'</tt>

oct: <tt>'0211 0120 0116 0107 0015 0012 0032 0012'</tt>
| ??
|-
| [http://svn.annodex.net/liboggz/trunk/src/liboggz/oggz_auto.h speex]
| audio
| char[0,8]: <tt>'Speex\ \ \ '</tt>
hex: <tt>'0x53 0x70 0x65 0x65 0x78 0x20 0x20 0x20'</tt>

oct: <tt>'0123 0160 0145 0145 0170 0040 0040 0040'</tt>
| char[28,4]: version id
|-
| [http://svn.annodex.net/liboggz/trunk/src/liboggz/oggz_auto.h theora]
| video
| char[0,7]: <tt>'\x80theora'</tt>
hex: <tt>'0x80 0x74 0x68 0x65 0x6f 0x72 0x61'</tt>

oct: <tt>'0180 0164 0150 0145 0157 0162 0141'</tt>
| char[7,1]: major version number,
char[8,1]: minor version number,

char[9,1]: version revision number
|-
| [http://svn.annodex.net/liboggz/trunk/src/liboggz/oggz_auto.h vorbis]
| audio
| char[0,7]: <tt>'\x01vorbis'</tt>
hex: <tt>'0x01 0x76 0x6f 0x72 0x62 0x69 0x73'</tt>

oct: <tt>'0001 0166 0157 0162 0142 0151 0163'</tt>
| char[7,4]: version field
|-
| [[OggYUV4MPEG|yuv4mpeg]]
| video
| char[0,8]: <tt>'YUV4MPEG'</tt>
hex: <tt>'0x59 0x55 0x56 0x34 0x4d 0x50 0x45 0x47'</tt>

oct: <tt>'0131 0125 0126 0064 0115 0120 0105 0107'</tt>
| char[8,1] = '2' (0x32) for yuv4mpeg format version 2
|}

The "char[x,y]" fields mean here: start at byte number x (counting from 0) for a length of y bytes.

[[Category:Ogg]]

MIMETypesCodecs

2009-06-04T11:33:25Z

Silvia: added audio; moved the ogx section down

== Specification of MIME types and respective codecs parameter ==

=== MIME Types ===

The following MIME types are now officially registered with IANA and specified with the IETF as [http://www.ietf.org/rfc/rfc5334.txt RFC 5334]:

* video/ogg - for video (with audio) encapsulated in Ogg
** recommends a Skeleton logical bitstrem
** .ogv file extension
** Macintosh File Type Code: OggV

* audio/ogg - for audio encapsulated in Ogg
** recommends a Skeleton logical bitstrem
** .oga file extension, .ogg for Vorbis I, .spx for Speex
** Macintosh File Type Code: OggA

* application/ogg - for complex, multitrack, multiplexed files encapsulated in Ogg
** requires a Skeleton logical bitstream
** .ogx file extension
** Macintosh File Type Code: OggX

[[MIME_Types_and_File_Extensions|Other MIME types]] are still in the process.

=== Codecs Parameter ===

[http://www.rfc-editor.org/rfc/rfc4281.txt Typically], MIME types of media encapsulation formats use the optional "codecs" parameter to specify which codes are being used in a particular file.

Codecs encapsulated in Ogg require a text identifier at the beginning of the first header page to identify the encapsulated codecs. The following table contains the identifiers for existing Xiph codecs and the codecs parameter names used for */ogg MIME types (in alphabetical order):

{| class="codecstable" border="1"
|-
! Codecs Parameter Name
! Codec Type
! Codec Identifier
(decimal, hex, octal)
! Version Field (if available)
|-
| [http://svn.annodex.net/liboggz/trunk/src/liboggz/oggz_auto.h celt]
| audio
| char[0,8]: <tt>'CELT\ \ \ \ '</tt>
hex: <tt>'0x43 0x45 0x4c 0x54 0x20 0x20 0x20 0x20'</tt>

oct: <tt>'0103 0105 0114 0124 0040 0040 0040 0040'</tt>
| char[28,4]: version id
|-
| [http://svn.annodex.net/liboggz/trunk/src/liboggz/oggz_auto.h cmml]
| text
| char[0,8]: <tt>'CMML\0\0\0\0'</tt>
hex: <tt>'0x43 0x4d 0x4d 0x4c 0x00 0x00 0x00 0x00'</tt>

oct: <tt>'0103 0115 0115 0114 0000 0000 0000 0000'</tt>
| char[8,2]: major version number,
char[10,2]: minor version number
|-
| [http://wiki.xiph.org/index.php/OggDirac dirac]
| video
| char[0,5]: <tt>'BBCD\0'</tt>
hex: <tt>'0x42 0x42 0x43 0x44 00'</tt>

oct: <tt>'0102 0102 0103 0104 0000'</tt>
| ??
|-
| [http://flac.sourceforge.net/ogg_mapping.html flac]
| audio
| char[0,5]: <tt>'\177FLAC'</tt>
hex: <tt>'0x7F 0x46 0x4C 0x41 0x43'</tt>

oct: <tt>'0177 0106 0114 0101 0103'</tt>
| char[5,1]: binary major version number,
char[6,1]: binary minor version number of mapping
|-
| [[OggMNG|jng]]
| video
| char[0,8]: <tt>'\213JNG\r\n\032\n'</tt>
hex: <tt>'0x8b 0x4a 0x4e 0x47 0x0D 0x0A 0x1A 0x0A'</tt>

oct: <tt>'0213 0112 0116 0107 0015 0012 0032 0012'</tt>
| ??
|-
| [[OggKate|kate]]
| text
| char[0,8]: <tt>'\x80kate\0\0\0'</tt>
hex: <tt>'0x80 0x6b 0x61 0x74 0x65 0x00 0x00 0x00'</tt>

oct: <tt>'0200 0153 0141 0164 0145 0000 0000 0000'</tt>
| char[9,1]: major version number,
char[10,1]: minor version number
|-
| [http://lists.xiph.org/pipermail/vorbis-dev/2001-August/004501.html midi]
| text
| char[0,8]: <tt>'OggMIDI\0'</tt>
hex: <tt>'0x4f 0x67 0x67 0x4d 0x49 0x44 0x49 0x00'</tt>

oct: <tt>'0117 0147 0147 0115 0111 0104 0111 0000'</tt>
| char[8,1]: version field
|-
| [[OggMNG|mng]]
| video
| char[0,8]: <tt>'\212MNG\r\n\032\n'</tt>
hex: <tt>'0x8a 0x4d 0x4e 0x47 0x0D 0x0A 0x1A 0x0A'</tt>

oct: <tt>'0212 0115 0116 0107 0015 0012 0032 0012'</tt>
| ??
|-
| [[OggPCM|pcm]]
| audio
| char[0,8]: <tt>'PCM\ \ \ \ \ '</tt>
hex: <tt>'0x53 0x70 0x65 0x65 0x78 0x20 0x20 0x20'</tt>

oct: <tt>'0123 0160 0145 0145 0170 0040 0040 0040'</tt>
| char[8,2]: version major field,
char[10,2]: version minor field
|-
| [[OggMNG|png]]
| video
| char[0,8]: <tt>'\211PNG\r\n\032\n'</tt>
hex: <tt>'0x89 0x50 0x4e 0x47 0x0D 0x0A 0x1A 0x0A'</tt>

oct: <tt>'0211 0120 0116 0107 0015 0012 0032 0012'</tt>
| ??
|-
| [http://svn.annodex.net/liboggz/trunk/src/liboggz/oggz_auto.h speex]
| audio
| char[0,8]: <tt>'Speex\ \ \ '</tt>
hex: <tt>'0x53 0x70 0x65 0x65 0x78 0x20 0x20 0x20'</tt>

oct: <tt>'0123 0160 0145 0145 0170 0040 0040 0040'</tt>
| char[28,4]: version id
|-
| [http://svn.annodex.net/liboggz/trunk/src/liboggz/oggz_auto.h theora]
| video
| char[0,7]: <tt>'\x80theora'</tt>
hex: <tt>'0x80 0x74 0x68 0x65 0x6f 0x72 0x61'</tt>

oct: <tt>'0180 0164 0150 0145 0157 0162 0141'</tt>
| char[7,1]: major version number,
char[8,1]: minor version number,

char[9,1]: version revision number
|-
| [http://svn.annodex.net/liboggz/trunk/src/liboggz/oggz_auto.h vorbis]
| audio
| char[0,7]: <tt>'\x01vorbis'</tt>
hex: <tt>'0x01 0x76 0x6f 0x72 0x62 0x69 0x73'</tt>

oct: <tt>'0001 0166 0157 0162 0142 0151 0163'</tt>
| char[7,4]: version field
|-
| [[OggYUV4MPEG|yuv4mpeg]]
| video
| char[0,8]: <tt>'YUV4MPEG'</tt>
hex: <tt>'0x59 0x55 0x56 0x34 0x4d 0x50 0x45 0x47'</tt>

oct: <tt>'0131 0125 0126 0064 0115 0120 0105 0107'</tt>
| char[8,1] = '2' (0x32) for yuv4mpeg format version 2
|}

The "char[x,y]" fields mean here: start at byte number x (counting from 0) for a length of y bytes.

[[Category:Ogg]]

ROE

2009-04-13T06:28:44Z

Silvia:

Rich Open multitrack media Exposition (ROE)

= Overview =

ROE (Rich Open multitrack media Exposition) is a way of describing the relationships between tracks of media in a stream. It is used to group tracks which have similar purpose and to identify alternatives.

= Usage =

== Authoring ==
One use of ROE is to author a multi-track audio-visual stream from multiple input files. In this document, we present a description of how to use ROE to author multi-track Ogg files.

== Dynamic Web Requests ==
Another use of ROE is in a Web client-server scenario. The Web server uses ROE as a means of representing the different tracks that are available for a multi-track Web resource. A Web client may not require all available tracks to present the resource to the user. It may decide to request the ROE representation first and then request only a subset of tracks from the server, e.g. only the English soundtrack. Or it may directly request particular tracks only. The server will use the request from the client to dynamically compose a multi-track stream with the requested tracks and mandatory tracks and serve this to satisfy the resource request.

== ROE in Use ==
A draft version of the spec is in use in the mediaWiki extension [http://metavid.org/w/index.php/MetaVidWiki MetaVidWiki]. This runs on the site [http://metavid.org metavid.org] and is used for remote embeding. In [http://metavid-mike.blogspot.com/ this blog] for example all the clips refrence a single roe file to expose multiple video tracks and text transcripts. [http://metavid.org/w/index.php?title=Special:MvExportStream&feed_format=roe&stream_name=House_proceeding_06-09-08_01&t=0%3A01%3A38%2F0%3A10%3A00 Sample ROE output] from metavid

= The ROE model =

Here we describe two representations of ROE: that of ROE XML, and that of ROE in Ogg Skeleton. Each representation is capable of entirely encoding the relationships of the ROE model, such that it is possible to losslessly convert between them.

= ROE XML =

ROE XML is a XML markup language that describes a hierarchical serialization of the ROE model.

A ROE XML file is an instance document of the [http://svn.annodex.net/standards/roe/roe_1_0.xsd ROE XML schema].

It is composed of a <head> tag followed by a <body> tag.

== Head Element ==

=== Head Tags ===
The <head> tag is optional and may optionally contain:

* a <title> tag to provide a textual description for the multi-track stream,
* a set of <link> tags that provide an alternative representation of the multi-track stream, e.g. as a html document,
* a <img> tag to provide a representative thumbnail for the multi-track stream,
* a set of <meta> tags that provide structured name-value annotations of the multi-track stream,
* a <base> tag to provide a base URI for resources referred to in the ROE file, and
* a set of <profile> tags that allows description of so-called track profiles.

The <title>, <link>, <meta>, and <base> tags are taken out of [http://www.w3.org/TR/xhtml1-schema/ XHTML] and serve the same purpose as they serve there.

=== Track Profiles ===
A track profile is a combination of tracks that is pre-defined within the ROE file and can be accessed by Web clients or authoring applications directly. Examples of such profiles are the Director's cut, or the Australian version.

A profile defines a list of references to the tracks of a media resource and possibly a selection from the alternative media sources of the track, to use for a particular pre-defined profile of the resource.

To that end, the profile element has a subelement called "partial" which contains the ID of a selected track and potentially the ID of a selected alternate media source for the track.

An example profile is:

<profile name="director's cut">
<partial track="v" select="v1" />
<partial track="a" />
</profile>

The <head> tag essentially separates the profiles from the core document structure being provided in the <body> element.

== Body Element ==

The <body> tag consists of a sequence of <track> elements that each describe a logical media track.

=== The Track Tag ===
A media track may consist of one of:

* a media source, such as a audio, video, or text stream described in a <mediaSource> tag,
* a sequence of media sources described in a <seq> tag with start and end times, or
* a set of alternate media sources described in a <switch> tag, only one of which can be selected.

The <track> element contains a mandatory "provides" attribute, which introduces a virtual label such as "commentary", "video", "audio", "textoverlay", "closedcaption", "logo", or "scoreboard". The track provides that kind of content.

=== The Switch Tag ===
The <switch> tag provides a choice between alternates, distinguished for a specific reason. The reason is given in the "distinction" attribute of the <switch> tag.

Inside a <switch> tag, the choices can be specified through the following means:

* directly as a <mediaSource>,
* as a sequence of media sources in a <seq> element, or
* as the outcome of another <switch> tag.

Example <switch> element:

<switch distinction="language" default="a3">
<switch id="a1" distinction="bitrate" default="a1b1">
<mediaSource id="a1b1" lang="en" content-type="audio/vorbis" src="http://example.com/lang1b1.oga" />
<mediaSource id="a1b2" lang="en" content-type="audio/vorbis" src="http://example.com/lang1b2.oga" />
</switch>
<mediaSource id="a2" lang="de" content-type="audio/vorbis" src="http://example.com/lang2.oga" />
<seq id="a3">
<mediaSource id="a3a" lang="fr" content-type="audio/vorbis" src="http://example.com/lang3a.oga" />
<mediaSource id="a3b" lang="fr" content-type="audio/vorbis" src="http://example.com/lang3b.oga" />
</seq>
</switch>

In this example, we have a choice between three languages: en, de and fr.
The English language track also comes in two different bitrates.
The French language track comes in two different files that should be played in sequence

=== Inline XML files ===
Some media source elements are XML documents themselves. These can be represented inline in a ROE file. The purpose of this is to contain all or some the annotation information of a media resource inside one XML file. Thus, the "inline" attribute can have the values "false", "partial" or "full".

An example inline XML file is the use of CMML inside a ROE track:

<track id="t1" provides="caption">
<mediaSource id="c" src="http://example.com/cmml1.cmml" inline="partial" content-type="text/cmml" >
<cmml role="caption" xmlns:cmml="http://www.annodex.org/spec/cmml/cmml40">
<cmml:head>
<cmml:title>random 1</cmml:title>
</cmml:head>
<cmml:clip start="t1" end="t2">
<cmml:body>
<html:p><html:span>rillian:</html:span>FOMS rocks</html:p>
</cmml:body>
</cmml:clip>
</cmml>
</mediaSource>
</track>

== An example ROE XML file ==

Putting it all together, here is an example of a ROE XML file:

<?xml version="1.0"?>
<xs:schema targetNamespace="http://www.xiph.org/roe1.0"
xmlns:xs="http://www.w3.org/2001/XMLS
xmlns:html="http://www.w3.org/1999/xhtml"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<ROE>
<head>
<link id="html_linkback" rel="alternate" type="text/html" href="http://example.com/full_video.html"/>
<img id="stream_thumb" src="http://example.com/full_video.jpg"/>
<title>Example video</title>
<profile name="director's cut">
<partial track="v" select="v1" />
<partial track="a" />
</profile>
</head>
<body>
<track id="v" provides="video">
<switch distinction="angle" default="v1">
<mediaSource id="v1" content-type="video/theora" src="http://example.com/angle1.ogv?track=v1&t=t1/t2" />
<mediaSource id="v2" content-type="video/theora" src="http://example.com/angle2.ogv" />
</switch>
</track>
<track id="a" provides="audio">
<switch distinction="Content-Language" default="a3">
<switch id="a1" distinction="bitrate" default="a1b1">
<mediaSource id="a1b1" lang="en" content-type="audio/vorbis" src="http://example.com/lang1b1.oga" />
<mediaSource id="a1b2" lang="en" content-type="audio/vorbis" src="http://example.com/lang1b2.oga" />
</switch>
<mediaSource id="a2" lang="de" content-type="audio/vorbis" src="http://example.com/lang2.oga" />
<seq id="a3">
<mediaSource id="a3a" lang="fr" content-type="audio/vorbis" src="http://example.com/lang3a.oga" />
<mediaSource id="a3b" lang="fr" content-type="audio/vorbis" src="http://example.com/lang3b.org" />
</seq>
</switch>
</track>
<track id="t" provides="text overlay">
<switch distinction="Content-Language" default="t1">
<mediaSource id="t1" lang="en" content-type="text/cmml" src="http://example.com/transcript1.cmml" />
<mediaSource id="t2" lang="de" content-type="text/cmml" src="http://example.com/transcript2.cmml" />
<mediaSource id="t3" lang="fr" content-type="text/cmml" src="http://example.com/transcript3.cmml" />
</switch>
</track>
<track id="l" provides="logo" default="O1">
<seq>
<mediaSource id="O1" content-type="application/ogg" src="http://example.com/mng.ogx?track=1" />
<mediaSource id="O2" content-type="application/ogg" src="http://example.com/mng.ogx?track=2" />
</seq>
</track>
</body>
</ROE>

= Representation in Skeleton =

When the relationships described by ROE are written into an Ogg stream, they are encoded using the message header fields of Ogg Skeleton fisbones for each track. One of the primary design goals for fisbone headers is to minimize the need for global information to be stored in a stream. Each track's fisbone contains headers describing only itself and its relationship to other tracks in the stream. This allows tracks to be inserted or removed at the Ogg level without needing to modify any data in individual headers.

== Relationships ==

Relationships between tracks are given by the following headers:

=== Provides ===

''Provides'' introduces a virtual label such as "commentary", which this track provides. Many tracks may provide the same such label, and as long as one is present then a dependency on that label can be satisfied.

=== Depends ===

This declares that it is not valid to include this track in a stream unless the track it depends on is present. An example use of this might be the generic captioning of sound effects for the deaf, which may not make sense unless the captioning of speech (in an appropriate language) is also rendered.
''Depends'' refers to either a virtual label provided by another track, or an explicit track ID.

When removing a track from a file, any other tracks dependent on it must also be removed.

=== Recommends ===

''Recommends'' refers to either a virtual label provided by another track, or an explicit track ID.

=== Suggests ===

''Suggests'' refers to either a virtual label provided by another track, or an explicit track ID.

=== Conflicts ===

''Conflicts'' refers to either a virtual label provided by another track, or an explicit track ID.

== Serving Suggestions ==

=== Disposition ===

= HTTP-style message headers for client-server negotiation =

ROE

2009-04-13T06:28:14Z

Silvia: added title

= Rich Open multitrack media Exposition (ROE) =

= Overview =

ROE (Rich Open multitrack media Exposition) is a way of describing the relationships between tracks of media in a stream. It is used to group tracks which have similar purpose and to identify alternatives.

= Usage =

== Authoring ==
One use of ROE is to author a multi-track audio-visual stream from multiple input files. In this document, we present a description of how to use ROE to author multi-track Ogg files.

== Dynamic Web Requests ==
Another use of ROE is in a Web client-server scenario. The Web server uses ROE as a means of representing the different tracks that are available for a multi-track Web resource. A Web client may not require all available tracks to present the resource to the user. It may decide to request the ROE representation first and then request only a subset of tracks from the server, e.g. only the English soundtrack. Or it may directly request particular tracks only. The server will use the request from the client to dynamically compose a multi-track stream with the requested tracks and mandatory tracks and serve this to satisfy the resource request.

== ROE in Use ==
A draft version of the spec is in use in the mediaWiki extension [http://metavid.org/w/index.php/MetaVidWiki MetaVidWiki]. This runs on the site [http://metavid.org metavid.org] and is used for remote embeding. In [http://metavid-mike.blogspot.com/ this blog] for example all the clips refrence a single roe file to expose multiple video tracks and text transcripts. [http://metavid.org/w/index.php?title=Special:MvExportStream&feed_format=roe&stream_name=House_proceeding_06-09-08_01&t=0%3A01%3A38%2F0%3A10%3A00 Sample ROE output] from metavid

= The ROE model =

Here we describe two representations of ROE: that of ROE XML, and that of ROE in Ogg Skeleton. Each representation is capable of entirely encoding the relationships of the ROE model, such that it is possible to losslessly convert between them.

= ROE XML =

ROE XML is a XML markup language that describes a hierarchical serialization of the ROE model.

A ROE XML file is an instance document of the [http://svn.annodex.net/standards/roe/roe_1_0.xsd ROE XML schema].

It is composed of a <head> tag followed by a <body> tag.

== Head Element ==

=== Head Tags ===
The <head> tag is optional and may optionally contain:

* a <title> tag to provide a textual description for the multi-track stream,
* a set of <link> tags that provide an alternative representation of the multi-track stream, e.g. as a html document,
* a <img> tag to provide a representative thumbnail for the multi-track stream,
* a set of <meta> tags that provide structured name-value annotations of the multi-track stream,
* a <base> tag to provide a base URI for resources referred to in the ROE file, and
* a set of <profile> tags that allows description of so-called track profiles.

The <title>, <link>, <meta>, and <base> tags are taken out of [http://www.w3.org/TR/xhtml1-schema/ XHTML] and serve the same purpose as they serve there.

=== Track Profiles ===
A track profile is a combination of tracks that is pre-defined within the ROE file and can be accessed by Web clients or authoring applications directly. Examples of such profiles are the Director's cut, or the Australian version.

A profile defines a list of references to the tracks of a media resource and possibly a selection from the alternative media sources of the track, to use for a particular pre-defined profile of the resource.

To that end, the profile element has a subelement called "partial" which contains the ID of a selected track and potentially the ID of a selected alternate media source for the track.

An example profile is:

<profile name="director's cut">
<partial track="v" select="v1" />
<partial track="a" />
</profile>

The <head> tag essentially separates the profiles from the core document structure being provided in the <body> element.

== Body Element ==

The <body> tag consists of a sequence of <track> elements that each describe a logical media track.

=== The Track Tag ===
A media track may consist of one of:

* a media source, such as a audio, video, or text stream described in a <mediaSource> tag,
* a sequence of media sources described in a <seq> tag with start and end times, or
* a set of alternate media sources described in a <switch> tag, only one of which can be selected.

The <track> element contains a mandatory "provides" attribute, which introduces a virtual label such as "commentary", "video", "audio", "textoverlay", "closedcaption", "logo", or "scoreboard". The track provides that kind of content.

=== The Switch Tag ===
The <switch> tag provides a choice between alternates, distinguished for a specific reason. The reason is given in the "distinction" attribute of the <switch> tag.

Inside a <switch> tag, the choices can be specified through the following means:

* directly as a <mediaSource>,
* as a sequence of media sources in a <seq> element, or
* as the outcome of another <switch> tag.

Example <switch> element:

<switch distinction="language" default="a3">
<switch id="a1" distinction="bitrate" default="a1b1">
<mediaSource id="a1b1" lang="en" content-type="audio/vorbis" src="http://example.com/lang1b1.oga" />
<mediaSource id="a1b2" lang="en" content-type="audio/vorbis" src="http://example.com/lang1b2.oga" />
</switch>
<mediaSource id="a2" lang="de" content-type="audio/vorbis" src="http://example.com/lang2.oga" />
<seq id="a3">
<mediaSource id="a3a" lang="fr" content-type="audio/vorbis" src="http://example.com/lang3a.oga" />
<mediaSource id="a3b" lang="fr" content-type="audio/vorbis" src="http://example.com/lang3b.oga" />
</seq>
</switch>

In this example, we have a choice between three languages: en, de and fr.
The English language track also comes in two different bitrates.
The French language track comes in two different files that should be played in sequence

=== Inline XML files ===
Some media source elements are XML documents themselves. These can be represented inline in a ROE file. The purpose of this is to contain all or some the annotation information of a media resource inside one XML file. Thus, the "inline" attribute can have the values "false", "partial" or "full".

An example inline XML file is the use of CMML inside a ROE track:

<track id="t1" provides="caption">
<mediaSource id="c" src="http://example.com/cmml1.cmml" inline="partial" content-type="text/cmml" >
<cmml role="caption" xmlns:cmml="http://www.annodex.org/spec/cmml/cmml40">
<cmml:head>
<cmml:title>random 1</cmml:title>
</cmml:head>
<cmml:clip start="t1" end="t2">
<cmml:body>
<html:p><html:span>rillian:</html:span>FOMS rocks</html:p>
</cmml:body>
</cmml:clip>
</cmml>
</mediaSource>
</track>

== An example ROE XML file ==

Putting it all together, here is an example of a ROE XML file:

<?xml version="1.0"?>
<xs:schema targetNamespace="http://www.xiph.org/roe1.0"
xmlns:xs="http://www.w3.org/2001/XMLS
xmlns:html="http://www.w3.org/1999/xhtml"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<ROE>
<head>
<link id="html_linkback" rel="alternate" type="text/html" href="http://example.com/full_video.html"/>
<img id="stream_thumb" src="http://example.com/full_video.jpg"/>
<title>Example video</title>
<profile name="director's cut">
<partial track="v" select="v1" />
<partial track="a" />
</profile>
</head>
<body>
<track id="v" provides="video">
<switch distinction="angle" default="v1">
<mediaSource id="v1" content-type="video/theora" src="http://example.com/angle1.ogv?track=v1&t=t1/t2" />
<mediaSource id="v2" content-type="video/theora" src="http://example.com/angle2.ogv" />
</switch>
</track>
<track id="a" provides="audio">
<switch distinction="Content-Language" default="a3">
<switch id="a1" distinction="bitrate" default="a1b1">
<mediaSource id="a1b1" lang="en" content-type="audio/vorbis" src="http://example.com/lang1b1.oga" />
<mediaSource id="a1b2" lang="en" content-type="audio/vorbis" src="http://example.com/lang1b2.oga" />
</switch>
<mediaSource id="a2" lang="de" content-type="audio/vorbis" src="http://example.com/lang2.oga" />
<seq id="a3">
<mediaSource id="a3a" lang="fr" content-type="audio/vorbis" src="http://example.com/lang3a.oga" />
<mediaSource id="a3b" lang="fr" content-type="audio/vorbis" src="http://example.com/lang3b.org" />
</seq>
</switch>
</track>
<track id="t" provides="text overlay">
<switch distinction="Content-Language" default="t1">
<mediaSource id="t1" lang="en" content-type="text/cmml" src="http://example.com/transcript1.cmml" />
<mediaSource id="t2" lang="de" content-type="text/cmml" src="http://example.com/transcript2.cmml" />
<mediaSource id="t3" lang="fr" content-type="text/cmml" src="http://example.com/transcript3.cmml" />
</switch>
</track>
<track id="l" provides="logo" default="O1">
<seq>
<mediaSource id="O1" content-type="application/ogg" src="http://example.com/mng.ogx?track=1" />
<mediaSource id="O2" content-type="application/ogg" src="http://example.com/mng.ogx?track=2" />
</seq>
</track>
</body>
</ROE>

= Representation in Skeleton =

When the relationships described by ROE are written into an Ogg stream, they are encoded using the message header fields of Ogg Skeleton fisbones for each track. One of the primary design goals for fisbone headers is to minimize the need for global information to be stored in a stream. Each track's fisbone contains headers describing only itself and its relationship to other tracks in the stream. This allows tracks to be inserted or removed at the Ogg level without needing to modify any data in individual headers.

== Relationships ==

Relationships between tracks are given by the following headers:

=== Provides ===

''Provides'' introduces a virtual label such as "commentary", which this track provides. Many tracks may provide the same such label, and as long as one is present then a dependency on that label can be satisfied.

=== Depends ===

This declares that it is not valid to include this track in a stream unless the track it depends on is present. An example use of this might be the generic captioning of sound effects for the deaf, which may not make sense unless the captioning of speech (in an appropriate language) is also rendered.
''Depends'' refers to either a virtual label provided by another track, or an explicit track ID.

When removing a track from a file, any other tracks dependent on it must also be removed.

=== Recommends ===

''Recommends'' refers to either a virtual label provided by another track, or an explicit track ID.

=== Suggests ===

''Suggests'' refers to either a virtual label provided by another track, or an explicit track ID.

=== Conflicts ===

''Conflicts'' refers to either a virtual label provided by another track, or an explicit track ID.

== Serving Suggestions ==

=== Disposition ===

= HTTP-style message headers for client-server negotiation =

Summer of Code 2009

2009-03-26T07:58:50Z

Silvia: added a libsydneyaudio project

This is our ideas page for [http://code.google.com/soc/ Google Summer of Code 2009] projects with [http://xiph.org Xiph.org] and [http://annodex.org/ Annodex]. The two projects participate jointly this year under Xiph's name.

'''Students''' please use the template at [[Summer of Code Applications]] when applying for a GSoC position.

'''Mentors''' please visit [[Summer of Code Mentoring]] and help us prepare our application as a mentoring organization.

== General Ideas ==

=== Codecs ===

* OpenMAX IL components for Ogg codecs
* Get skeleton patches upstream so players stop choking on it.
* Portable listening application for codec MOS/MUSHRA comparisons (Win32, MacOS, Linux; FF3.1 web application?).

=== Web Video ===

* Proof of concept liboggplay-based media patch for Google's Chrome browser.
* Metavid improvements
* Kate to HTML & CSS overlay library in javascript.
* Firefox extension to record locally and stream to icecast.
* Firefox extension to support RTP for conferencing.
** also consider applying [https://wiki.mozilla.org/Community:SummerOfCode09#Firefox under Mozilla SOC org].
* mod_duration apache module to generate X-Content-Duration headers for Ogg files. ''(We should get people to use '''oggz_chop''' instead ... oggz_chop provides this functionality and more, is already relatively lightweight.)'' Perhaps this project could be more focused on packaging oggz_chop for other web servers like lightpd and maybe a fastCGI version and or maybe push for an mod_ogg to be adopted upstream in apache to improve distribution.

== Detailed Project Descriptions ==

These ideas were suggested by various members of the developer community as projects that would be beneficial and which we feel we can mentor. Students should feel free to select one of these, develop a variation, or propose their own ideas. Here, ideally.

=== Proof of Concept liboggplay (html5 video) support in Chromium Browser ===

This project would focus on integrating support for liboggplay into chrome. This project would only need to be a proof of concept with the end result being some frames decoded in the browser. We have some direct contacts with people on the Chromium project in Google, but would expect the student mostly to work through the Xiph on Chromium online communities.

[http://code.google.com/chromium/ Chromium Home Page]

=== Metavid related projects ===

see [http://metavid.org/wiki/Summer_of_Code_2009 full page on metavid.org]
* Improve transcript import / export system:
** Wiki to SRT
** SRT to Wiki
** CMML to Wiki
** Extend oggz_chop or other tool for exporting transcript encapsulated in the ogg file.

=== Javascript Library for Subtitles, Captions and other time-aligned text ===

The main focus of the project is on enabling video accessibility for Ogg in Firefox.

==== Problem / Intro ====

Captions, subtitles and other categories of time-aligned text are starting to become relevant to HTML5. In Ogg, we currently encapsulate such data in OggKate and can use SRT or Kate as input formats. Display of OggKate is currently supported in VLC and there are patches for various other media players. We now want to enable Web browsers to also deal with these time-aligned text tracks in those Web Browsers that support the HTML5 video tag.

==== Solution / Task ====

There is a proof of concept patch for Firefox 3.1 (now called 3.5) and liboggplay through which Firefox is capable of decoding Ogg Kate tracks and either overlay them onto the video, or handing the raw text to the browser (eg, for text to speech). However, there is no display of OggKate in Firefox 3.5 using HTML5. This can be fixed through the creation of a javascript library that can deal with Kate output and convert it to HTML and CSS. Example libraries exists for displaying SRT for HTML5 video, but they will need to be extended to Kate in this project.

This project includes the creation of example files for different types of time-aligned text. These are then encapsulated into Ogg through Kate encoding. Firefox 3.5 with the applied OggKate patch can decode these files and hand the textual data to the Web browser. It will be necessary to extend liboggplay to pass non textual Kate data (eg, styling, etc) to the browser, as currently the only two ways of dealing with a Kate track is to render it, or pass raw text, ignoring extra styling information. This could be part of the project, or done before the GSoC projects begins. The browser receives the text and styling information, and a javascript library implemented by the student will take care of the display. This will include an implementation of default display mechanisms for the different types of time-aligned text that we decide to deal with.

==== Requirements ====

The project requires a student with experience in javascript development, HTML and CSS, but also with some understanding of C for liboggplay and libkate, and of C++ for Firefox. The student will learn how to deal with Ogg and Ogg tracks, including Ogg Kate. He/she will also get some insight into Firefox development. He/she will work with the developer of Ogg Kate and the video accessibility expert of Xiph, as well as having access to the whole Xiph community including the core developer of Ogg support in Firefox.

The project is adaptable to the qualifications of the student - it may consist in simply implementing a tool-chain for handling srt through OggKate, or it may go much further and include richer forms or time-aligned text such as audio annotations, Karaoke, ticker text, clickable text etc.

==== Mentors ====

* Silvia Pfeiffer (nessy / ginger)
* ogg.k.ogg.k

=== libsydneyaudio ===

libsydneyaudio is a powerful, but easy to use cross-platform API for PCM audio capture and playback. It abstracts away all hardware-related complexity. It thus sits right on top of other audio device interfaces on the platforms, such as ALSA, pulseaudio, OSS on Linux, etc. libsydneyaudio is in use in conjunction with liboggplay in Firefox to provide cross-platform audio support.

===== Problem/Intro =====

libsydneyaudio has several bugs registered toward itself, see https://trac.annodex.net/report/12 . Much of the challenge comes from an incomplete implementation of win32 support of audio, but other platforms have issues too.

==== Solution / Task ====

The student will attack the bugs and make sure that libsydneyaudio works across all major platforms. There are programs available for testing. Once all bugs are fixed, there may be further new features to implement.

==== Requirements ====

The student should bring along a keen interest in audio decoding. A basic understanding of win32 audio/video interfaces, experience in programming C or C++, and cross-platform development would be very helpful, but not necessary.

=== OpenMAX IL components for Ogg codecs ===

OpenMAX is a set of low-level C APIs for media codecs. It is specified by the Khronos Group (who also co-ordinate standards like OpenGL and OpenAL) and is used by many mobile devices, in platforms like [http://www.maemo.org/ Maemo] and [http://source.android.com/ Android]. As we'd like to encourage the use of free codecs on mobile and embedded devices, we want to develop a set of components using our codec libraries.

==== Project goals ====

This project would implement free codec support for the lowest of the three OpenMAX layers, OpenMAX IL (Integration Layer). This is an interface to multimedia codecs implemented in hardware or software. It does not provide any interfaces for synchronized capture or playback of video and audio -- typically this is handled by higher OpenMAX layers, or by a framework like GStreamer.

Developing software OpenMAX IL components will allow application developers to implement Ogg support ahead of hardware support. It would also give hardware manufacturers a set of specific, well-defined goals for implementing Ogg support, with the understanding that the hardware components, when shipped with these software control APIs, will work in a variety of open source applications with minimal modifications.

==== Implementation ====

Your project proposal should cover a reasonable portion of these steps:

* Implement generic Ogg mux/demux components (instead of single Ogg Vorbis component)
* Implement IL components for each codec (Theora, Dirac, Speex, CELT, FLAC)
* Implement GStreamer OpenMAX plugins for each codec

For details, including the motivation for this project and links to related projects, see
[http://blog.kfish.org/2009/02/is-openmax-important-for-free-software.html Is OpenMAX important for Free Software?]

==== Mentors ====

* Conrad Parker (kfish)

=== XSPF-related projects ===

==== XSPF import and export for Songbird ====

===== Problem/Intro =====
* Songbird cannot read XSPF playlists
* Songbird cannot write XSPF playlists

===== Solution/Task =====
* Extend the development line of Songbird by XSPF read and write support.
* Read support should be able to tolerate most above-XML-level errors so users don't get frustrated with XSPF. That's the short version :-)
* Communication with upstream will be needed
* Solution should use whatever solution upstream is likely to accept as a patch. If you get them to accept a libxspf-based solution it's C++, otherwise probably JavaScript.

===== Mentors =====
* Sebastian Pipping (sping)

==== Python library / Online validator refactoring ====

===== Problem/Intro =====
* The [http://validator.xspf.org/ Online XSPF Validator]'s [https://trac.xiph.org/browser/websites/validator.xspf.org code] (Python >=2.4) is a procedural spaghetti mix of logic and presentation.
* There is no Python XSPF library around.

===== Solution/Task =====
* Refactor the current validator code and separate it into a OOP XSPF reading library/API
* Adapt the validator to use former library
* Separate presentation and logic in validator code possibly involving a popular light-wight LGPLv3-compatible Python framework of your choice

===== Mentors =====
* Sebastian Pipping (sping)

=== CELT-related projects ===

==== Conference bridge using CELT ====

When a conference takes place, the voice from all participants is often decoded, mixed together, and re-encoded. The goal of this project is to do better. We would like to have only partial decoding and re-encoding of CELT and reuse as much as possible from the already-encoded streams. This not only decreases the CPU load but also improves quality. The project is specific to the [http://www.celt-codec.org CELT] codec, and during the course of the project, the student will learn the internals of the CELT codec.

==== Reference SIP client for CELT ====

The RTP profile for CELT is currently being written and to ensure that SIP clients with CELT support are really compliant, we would need a reference client that is 100% compliant with the RTP profile. This would provide:
* A reference to test for compatibility
* Some reference C code to copy directly

==See Also==
*[[Summer of Code 2008]]
*[[Summer of Code 2007]]
*[[Summer of Code 2006]]

Summer of Code 2009

2009-03-21T13:27:51Z

Silvia: introduced some structure to the project

This is our ideas page for [http://code.google.com/soc/ Google Summer of Code 2009] projects with [http://xiph.org Xiph.org] and [http://annodex.org/ Annodex]. The two projects participate jointly this year under Xiph's name.

'''Students''' please use the template at [[Summer of Code Applications]] when applying for a GSoC position.

'''Mentors''' please visit [[Summer of Code Mentoring]] and help us prepare our application as a mentoring organization.

== General Ideas ==

=== Codecs ===

* OpenMAX IL components for Ogg codecs
* Get skeleton patches upstream so players stop choking on it.
* Portable listening application for codec MOS/MUSHRA comparisons (Win32, MacOS, Linux; FF3.1 web application?).
* Conference bridge using CELT.
* Reference SIP client for CELT.

=== Web Video ===

* Proof of concept liboggplay-based media patch for Google's Chrome browser.
* Metavid improvements
* Kate to HTML & CSS overlay library in javascript.
* Firefox extension to record locally and stream to icecast.
* Firefox extension to support RTP for conferencing.
** also consider applying [https://wiki.mozilla.org/Community:SummerOfCode09#Firefox under Mozilla SOC org].
* mod_duration apache module to generate X-Content-Duration headers for Ogg files. ''(We should get people to use '''oggz_chop''' instead ... oggz_chop provides this functionality and more, is already relatively lightweight.)'' Perhaps this project could be more focused on packaging oggz_chop for other web servers like lightpd and maybe a fastCGI version and or maybe push for an mod_ogg to be adopted upstream in apache to improve distribution.

== Detailed Project Descriptions ==

These ideas were suggested by various members of the developer community as projects that would be beneficial and which we feel we can mentor. Students should feel free to select one of these, develop a variation, or propose their own ideas. Here, ideally.

=== Proof of Concept liboggplay (html5 video) support in Chromium Browser ===

This project would focus on integrating support for liboggplay into chrome. This project would only need to be a proof of concept with the end result being some frames decoded in the browser. We have some direct contacts with people on the Chromium project in Google, but would expect the student mostly to work through the Xiph on Chromium online communities.

[http://code.google.com/chromium/ Chromium Home Page]

=== Metavid related projects ===

see [http://metavid.org/wiki/Summer_of_Code_2009 full page on metavid.org]
* Improve transcript import / export system:
** Wiki to SRT
** SRT to Wiki
** CMML to Wiki
** Extend oggz_chop or other tool for exporting transcript encapsulated in the ogg file.

=== Javascript Library for Subtitles, Captions and other time-aligned text ===

The main focus of the project is on enabling video accessibility for Ogg in Firefox.

==== Problem / Intro ====

Captions, subtitles and other categories of time-aligned text are starting to become relevant to HTML5. In Ogg, we currently encapsulate such data in OggKate and can use SRT or Kate as input formats. Display of OggKate is currently supported in VLC and there are patches for various other media players. We now want to enable Web browsers to also deal with these time-aligned text tracks in those Web Browsers that support the HTML5 video tag.

==== Solution / Task ====

There is a proof of concept patch for Firefox 3.1 (now called 3.5) and liboggplay through which Firefox is capable of decoding Ogg Kate tracks and either overlay them onto the video, or handing the raw text to the browser (eg, for text to speech). However, there is no display of OggKate in Firefox 3.5 using HTML5. This can be fixed through the creation of a javascript library that can deal with Kate output and convert it to HTML and CSS. Example libraries exists for displaying SRT for HTML5 video, but they will need to be extended to Kate in this project.

This project includes the creation of example files for different types of time-aligned text. These are then encapsulated into Ogg through Kate encoding. Firefox 3.5 with the applied OggKate patch can decode these files and hand the textual data to the Web browser. It will be necessary to extend liboggplay to pass non textual Kate data (eg, styling, etc) to the browser, as currently the only two ways of dealing with a Kate track is to render it, or pass raw text, ignoring extra styling information. This could be part of the project, or done before the GSoC projects begins. The browser receives the text and styling information, and a javascript library implemented by the student will take care of the display. This will include an implementation of default display mechanisms for the different types of time-aligned text that we decide to deal with.

==== Requirements ====

The project requires a student with experience in javascript development, HTML and CSS, but also with some understanding of C for liboggplay and libkate, and of C++ for Firefox. The student will learn how to deal with Ogg and Ogg tracks, including Ogg Kate. He/she will also get some insight into Firefox development. He/she will work with the developer of Ogg Kate and the video accessibility expert of Xiph, as well as having access to the whole Xiph community including the core developer of Ogg support in Firefox.

The project is adaptable to the qualifications of the student - it may consist in simply implementing a tool-chain for handling srt through OggKate, or it may go much further and include richer forms or time-aligned text such as audio annotations, Karaoke, ticker text, clickable text etc.

==== Mentors ====

* Silvia Pfeiffer (nessy / ginger)
* ogg.k.ogg.k

=== OpenMAX IL components for Ogg codecs ===

OpenMAX is a set of low-level C APIs for media codecs. It is used by many mobile devices, in platforms like [http://www.maemo.org/ Maemo] and [http://source.android.com/ Android]. As we'd like to encourage the use of free codecs on mobile and embedded devices, we want to develop a set of components using our codec libraries.

For details, including the motivation for this project and links to related projects, see
[http://blog.kfish.org/2009/02/is-openmax-important-for-free-software.html Is OpenMAX important for Free Software?]

=== XSPF-related projects ===

==== XSPF import and export for Songbird ====

===== Problem/Intro =====
* Songbird cannot read XSPF playlists
* Songbird cannot write XSPF playlists

===== Solution/Task =====
* Extend the development line of Songbird by XSPF read and write support.
* Read support should be able to tolerate most above-XML-level errors so users don't get frustrated with XSPF. That's the short version :-)
* Communication with upstream will be needed
* Solution should use whatever solution upstream is likely to accept as a patch. If you get them to accept a libxspf-based solution it's C++, otherwise probably JavaScript.

===== Mentors =====
* Sebastian Pipping (sping)

==== Python library / Online validator refactoring ====

===== Problem/Intro =====
* The [http://validator.xspf.org/ Online XSPF Validator]'s [https://trac.xiph.org/browser/websites/validator.xspf.org code] (Python >=2.4) is a procedural spaghetti mix of logic and presentation.
* There is no Python XSPF library around.

===== Solution/Task =====
* Refactor the current validator code and separate it into a OOP XSPF reading library/API
* Adapt the validator to use former library
* Separate presentation and logic in validator code possibly involving a popular light-wight LGPLv3-compatible Python framework of your choice

===== Mentors =====
* Sebastian Pipping (sping)

==See Also==
*[[Summer of Code 2008]]
*[[Summer of Code 2007]]
*[[Summer of Code 2006]]

Summer of Code 2009

2009-03-12T09:54:52Z

Silvia: minor improvements

Summer of Code 2009

2009-03-12T00:57:41Z

Silvia: added C++ requirement

Summer of Code 2009

2009-03-12T00:45:27Z

Silvia: extended the description of the project :)

Summer of Code 2009

2009-03-12T00:32:03Z

Silvia: /* Proof of Concept liboggplay (html5 video) support in Chromium Browser */

Summer of Code 2009

2009-02-28T07:46:23Z

Silvia: /* Detailed Project Descriptions */

Summer of Code 2009

2009-02-28T07:36:16Z

Silvia: /* General Ideas */