== Ogg mapping for Opus ==

The IETF Opus codec is a low-latency audio codec optimized for both voice and general-purpose audio. See [http://tools.ietf.org/html/draft-ietf-codec-opus the spec] for technical details.

Almost everything about this codec is either fixed or dynamically switchable, so the usual id and setup header parameters in the header packets of an Ogg encapsulation aren't useful. In particular, bitrate, frame size, mono/stereo, and coding modes are all dynamically switchable from packet to packet. A one-byte header on each data packet defines the parameters for that particular packet.

Remaining parameters we need to signal are:

* magic number for stream identification
* comment/metadata tags

Additionally there's been a desire to support some kind of channel bonding for surround, and some kind of option signalling for "Opus Custom", in particular the granulerate.

=== Draft spec ===

Granulepos is the count of decodeable samples at a fixed rate of 48 kHz.

Two headers: id, comment

==== Id header ====

- Magic signature: "OpusHead" (64 bits)
- Version number (8 bits): zero for this spec
- Channel count 'c' (8 bits unsigned): MUST be > 0
- Pre-skip (16 bits unsigned)
- Input sample rate (32 bits, little endian): informational only
- Output gain (16 bits, little endian, signed Q7.8 in dB) to apply when decoding
- Channel mapping family (8 bits)
-- 0 = one stream, RTP order, 1 = channels in vorbis spec order, 2..254 reserved (treat as 255), 255 = no defined channel meaning
If channel mapping family > 0
- Stream count 'N' (8 bits unsigned): MUST be > 0
- Two-channel stream count 'M' (8 bits unsigned): MUST satisfy M <= N, M+N <= 255
- Channel mapping (8*c bits)
-- one stream index (8 bits unsigned) per channel (255 means silent throughout the file)

Some discussion is in order.

* '''Magic signature'''
The magic signature "OpusHead" allows codec identification and is human readable. Starting with 'Op' helps distinguish it from data packets, as this is an invalid TOC sequence.

* '''Version'''
The version number must always be zero for this version of the encapsulation spec. We do not plan to revise the spec, but this also acts as a null terminator for the signature bytes and helps align the rest of the fields.

* '''Channel count''' 'c'
The number of channels byte specifies the number of output channels (1...255) for this Ogg Opus stream.

* '''Pre-skip'''
This is the number of samples (at 48 kHz) to discard from the decoder output before starting playback.

The purpose of pre-skip is to allow a time-segment of an existing Opus stream to be saved as an independent Ogg file, with single-sample time granularity, without re-encoding. Opus is an asymptotically convergent predictive codec, so the decoded contents of each frame depend on the recent history of decoder inputs. Pre-skip can be used to provide sufficient history to the decoder so that it has already converged before the stream's output begins.

Because more than one page can be needed for re-convergence the Vorbis scheme for signaling pre-skip is not used for Opus.

The granule corresponding to the end time of an Ogg Opus page can be determined by subtracting the pre-skip from the page's granpos value. For example, if the page's granpos is 49970, and the preskip is 11971, then last sample decoded from the page is sample 47999, i.e. the last sample from the first second of absolute time.

When constructing cropped Ogg Opus streams, we recommend a pre-skip of at least '''FIXME''' samples to ensure complete convergence.

* '''Input sample rate'''
This is ''not'' the sample rate to use playback of the encoded data.

Opus has a handful of coding modes, with internal sample rates of 8, 12, 16, 24, and 48 kHz. Each packet in the stream may have a different internal sample rate. Regardless of the internal sample rate, the reference decoder supports decoding any stream to any of these sample rates. The original sample rate of the encoder input is not preserved by the lossy compression.

An Ogg Opus player SHOULD select the playback sample rate according to the following procedure:
** If the hardware supports 48 kHz playback, decode at 48 kHz
** else if the hardware's highest available sample rate is a supported rate, decode at this sample rate
** else if the hardware's highest available sample rate is less than 48 kHz, decode at the next higher supported rate and resample
** else decode at 48 kHz and resample.

However, the Ogg mapping allows the encoder to pass the sample rate of the original input stream as metadata. This may be useful when the user requires the output sample rate to match the input sample rate. For example, a non-player decoder writing PCM format to disk might choose to resample the output audio back to the original input rate to reduce surprise to the user, who might reasonably expect to get back a file with the same sample rate as the one they fed to the encoder.

A value of zero indicates 'unspecified'. Implementations which do something with this field should take care to behave sanely if given crazy values (e.g. don't
actually upsample the output to 10MHz) and encoders should write the actual input rate or zero.

* '''Output gain'''
This is a gain to be applied by the decoder. Virtually all players and media frameworks should apply it by default. If a player chooses to apply any volume adjustment or gain modification, such as the R128_TRACK_GAIN or a user-facing volume knob, the adjustment MUST be applied ''in addition'' to this output gain in order to achieve playback at the desired volume.

An encoder SHOULD set the output gain to zero, and instead apply any gain prior to encoding, when this is possible and does not conflict with the user's wishes. The output gain should only be nonzero when the gain is adjusted after encoding, or when the user wishes to adjust the gain for playback while preserving the ability to recover the original signal amplitude.

Note that although the output gain has enormous range (+/- 128 dB, enough to amplify inaudible sounds to the threshold of physical pain), most applications can only reasonably use a small portion of this range around zero. The large range serves in part to ensure that gain can always be losslessly transferred between OpusHead and R128_TRACK_GAIN (see below) without saturating.

* '''Channel mapping family'''
This byte indicates the order and semantic meaning of the various channels encoded in each Opus packet.

Each possible value of this byte indicates a ''mapping family'', which defines a set of allowed numbers of channels, and the ordered set of channel names for each allowed number of channels. Currently there are three defined mapping families, although more may be added:

** Family 0 (RTP mapping)
*** Allowed numbers of channels: 1 or 2
*** 1 channel: monophonic (mono)
*** 2 channels: stereo (left, right)
*** '''Special mapping''': this channel mapping value also indicates that the contents consists of a single Opus stream that is stereo if and only if c==2, with stream index 0 mapped to channel 0, and (if stereo) stream index 1 mapped to channel 1. When the channel mapping byte has this value, no further fields are present in OpusHead.
** Family 1 ([http://www.xiph.org/vorbis/doc/Vorbis_I_spec.html#x1-800004.3.9 Vorbis mapping])
*** Allowed numbers of channels: 1 ... 8
*** Channel meanings depend on the number of channels, see the Vorbis mapping for details.
** Family 255 (no defined channel meaning)
*** Allowed numbers of channels: 1...255
*** Channels are unidentified. General-purpose players SHOULD NOT attempt to play these streams, and offline decoders MAY deinterleave the output into separate PCM files, one per channel. Decoders SHOULD NOT produce output for channels mapped to stream index 255 (pure silence) unless they have no other way to indicate the index of non-silent channels.

The remaining channel mapping families (2...254) are reserved. A decoder encountering a reserved mapping byte should act as though the mapping byte is 255.

An Ogg Opus player MUST play any Ogg Opus stream with a channel mapping family of 0 or 1, even if the number of channels does not match the physically connected audio hardware. Players SHOULD perform channel mixing to increase or reduce the number of channels as needed.

* '''Stream count''' 'N'
This field indicates the total number of streams so the decoder can correctly parse the packed Opus packets inside the Ogg packet.

For channel mapping family 0, this value defaults to 1, and is not coded.

A multi-channel Opus file is composed of one or more individual Opus streams, each of which produce one or two channels of decoded data. Each Ogg packet contains one Opus packet from each stream. The first N-1 Opus packets are packed using the self-delimiting framing from Appendix B of the Opus specification. The remaining Opus packet is packed using the regular, undelimited framing from Section 3 of the Opus specification. All the Opus packets in a single Ogg packet are constrained to produce the same number of decoded samples.

* '''Two-channel stream count''' 'M'
Describes the number of streams whose decoders should be configured to produce two channels. This must be no larger than the number of total streams.

For channel mapping family 0, this value defaults to c-1 (i.e., 0 for mono and 1 for stereo), and is not coded.

Each packet in an Opus stream has an internal channel count of 1 or 2, which can change from packet to packet. This is selected by the encoder depending on the bitrate and the contents being encoded. The original channel count of the encoder input is not preserved by the lossy compression.

Regardless of the internal channel count, any Opus stream may be decoded as mono (single channel) or stereo (two channels) by appropriate initialization of the decoder. The "two-channel stream count" field indicates that the first M Opus decoders should be initialized in stereo mode, and the remaining N-M decoders should be initialized in mono mode. The total number of decoded channels (M+N) must be no larger than 255, as there is no way to index more channels than that in the channel mapping.

* '''Channel mapping'''
Contains one index per output channel indicating which decoded channel should be used. If the index is less than 2*M, the output MUST be taken from decoding stream (index/2) as stereo and selecting the left channel if index is even, and the right channel if index is odd. If the index is 2*M or larger, the output MUST be taken from decoding stream (index-M) as mono. As a special case, an index of 255 means that the corresponding output channel MUST contain pure silence.

For channel mapping family 0, the first index defaults to 0, and if c==2, the second index defaults to 1. Neither index is coded.

The number of output channels (c) is not constrained to match the number of decoded channels (M+N). A single index MAY appear multiple times, i.e., the same decoded channel may be mapped to multiple output channels. Some decoded channels might not be assigned to any output channel, as well.

==== Comment header ====

- 8 byte 'OpusTags' magic signature (64 bits)
- rest follows the vorbis-comment header design used in OggVorbis (without the "framing-bit"), OggTheora, and Speex.
** Vendor string (always present)
** tag=value metadata strings (zero or more)

One new comment field is introduced for Ogg Opus:
R128_TRACK_GAIN=-573
representing the volume shift needed to normalize the track's volume. The gain is a Q7.8 fixed point number in dB, as in the OpusHead "output gain" field. This field acts similarly to the [[VorbisComment#Replay_Gain|REPLAYGAIN_TRACK_GAIN field in Vorbis]], although the normal volume reference is the [http://tech.ebu.ch/loudness EBU-R128] standard.

An Ogg Opus file MUST NOT have more than one such field, and if present its value MUST be an integer from -32768 to +32767 inclusive, represented in ASCII with no whitespace. If present it MUST correctly represent the R128 normalization gain (relative to the OpusHead output gain). If a player chooses to make use of the TRACK_GAIN, it MUST be applied ''in addition'' to the OpusHead output gain. If an encoder populates the TRACK_GAIN field, and the output gain is not otherwise constrained or specified, the encoder SHOULD write the R128 gain into the OpusHead output gain and write "R128_TRACK_GAIN=0". If a tool modifies the OpusHead "output gain" field, it MUST also update or remove the R128_TRACK_GAIN comment field.

There is no comment field corresponding to Replaygain's ALBUM_GAIN; that information should instead be stored in the OpusHead "output gain" field.

To avoid confusion with multiple normalization schemes, an OpusTags packet SHOULD NOT contain any of the REPLAYGAIN_TRACK_GAIN, REPLAYGAIN_TRACK_PEAK, REPLAYGAIN_ALBUM_GAIN, or REPLAYGAIN_ALBUM_PEAK fields.

== Other implementation notes ==
As [http://www.xiph.org/vorbis/doc/Vorbis_I_spec.html#x1-130000A.2 in Ogg Vorbis], a granule position on the final page in a stream that indicates less audio data than the final packet would normally return is used to end the stream on other than even frame boundaries. The difference between the actual available data returned and the declared amount indicates how many trailing samples to discard from the decoding process.

When seeking within an Ogg Opus stream, the decoder should start decoding (and discarding the output) at least '''FIXME''' samples prior to the seek point in order to ensure that the output audio is correct at the seek point.

== Test vectors ==

* [[OggOpus/testvectors|Planned test vectors for OggOpus]]
* Opus test vectors

OggOpus

2011-11-16T23:29:23Z

Bemasc: Note about seeking upstream of target

2011-08-17T19:51:13Z

Bemasc: Minor gain-related clarifications

== Ogg mapping for Opus ==

The IETF Opus codec is a low-latency audio codec optimized for both voice and general-purpose audio. See [tools.ietf.org/html/draft-ietf-codec-opus the spec] for technical details.

Almost everything about this codec is either fixed or dynamically switchable, so the usual id and setup header parameters in the header packets of an Ogg encapsulation aren't useful. In particular, bitrate, frame size, mono/stereo, and coding modes are all dynamically switchable from packet to packet. A one-byte header on each data packet defines the parameters for that particular packet.

Remaining parameters we need to signal are:

* magic number for stream identification
* comment/metadata tags

Additionally there's been a desire to support some kind of channel bonding for surround, and some kind of option signalling for "Opus Custom", in particular the granulerate.

=== Draft spec ===

Granulepos is the count of decodeable samples at a fixed rate of 48 kHz.

Two headers: id, comment

Id header:

- "OpusHead" (64 bits)
- version number (8 bits) zero for this spec
- Number of channels 'c' (8 bits) (must be > 0)
- Pre-skip (16 bits)
- Input sample rate (32 bits) (informational only)
- Output gain (16 bits, signed Q7.8 in dB) to apply when decoding
- stream mapping (8 bits)
-- 0 = one stream, RTP order, 1 = channels in vorbis spec order, 2..254 reserved (treat as 255), 255 = no defined channel meaning
if stream mapping > 0
- Number of streams 'N'(8 bits) (must be > 0)
- Number of two-output streams 'M' (8 bits) (M+N strictly smaller than 255)
- for each output channel [0..c]
-- read stream index (8 bits) (255 means silent through the file)

All two-output streams come first, so if the stream index is < 2*M, the channel
decode the (index/2)th opus stream as stereo, selecting the (index%2)th output
(left for even, right for odd). If index >= 2*M, decode the (index - M)th stream
as mono and use that as the output. As a special case, a stream index of 255
means to write silence to that output channel.

Comment header:

- 8 byte 'OpusTags' magic signature (64 bits)
- rest follows the vorbis-comment header design used in OggVorbis, OggTheora, and Speex.
** Vendor string (always present)
** tag=value metadata strings (zero or more)

One new comment field is introduced for Ogg Opus:
R128_TRACK_GAIN=-7.03 dB
representing the volume shift needed to normalize the track's volume. This field is similar to the [[VorbisComment#Replay_Gain|REPLAYGAIN_TRACK_GAIN field in Vorbis]], although the normal volume reference is the [http://tech.ebu.ch/loudness EBU-R128] standard. An Ogg Opus file MUST NOT have more than one such field, and if present its value MUST be of the form "[number] dB" in 7-bit ASCII. When a player makes use of the TRACK_GAIN, it MUST be applied ''in addition'' to the OpusHead output gain. If an encoder populates the TRACK_GAIN field, and the output gain is not otherwise constrained or specified, the encoder SHOULD write the track gain into the OpusHead output gain and set "R128_TRACK_GAIN=0 dB". If a tool modifies the OpusHead "output gain" field, it MUST also update or remove the R128_TRACK_GAIN comment field.

There is no comment field corresponding to Replaygain's ALBUM_GAIN; that information should instead be stored in the OpusHead "output gain" field.

To avoid confusion with multiple normalization schemes, an OpusTags packet SHOULD NOT contain any of the REPLAYGAIN_TRACK_GAIN, REPLAYGAIN_TRACK_PEAK, REPLAYGAIN_ALBUM_GAIN, or REPLAYGAIN_ALBUM_PEAK fields.

Some discussion is in order.

* '''magic signature'''
The signature magic values allow codec identification and are being human readable. Starting with 'Op' helps distinguish them from data packets.

* '''version'''
Version number. Must always be zero for this version of the encapsulation spec. In general revising the spec later isn't a good idea, but this also acts as a null terminator for the signature bytes and helps align the rest of the fields.

* '''stream mapping'''
We want to support multichannel. This defines the order and semantic meaning of the various channels encoded in each Opus packet.

For example, we can't just code 5.1 as three stereo Opus streams, because then LFE ends up sharing a stereo pair with another channel (RR in the Vorbis channel order) which isn't a good idea, while 6 mono channels wastes bandwidth. Or, when routing multitrack audio between mixing boards, it helps to be able to flag which instruments should be treated as mono and which are stereo.

We don't need 8 bits of separate channel meanings, so if we want to make it easier to parse the number of channels, we can make that part of some of the stream mappings: 0 = mono, 1 = stereo, 2 = 5.1 in vorbis order, 3 = 6.0 in some order, 4 = 7.1, etc.

* '''pre-skip'''
This is the number of samples (at 48 kHz) to discard from the decoder output before starting playback. The idea is to mitigate transients, and to allow sample-accurate editing through Ogg chaining.

* '''output gain'''
This is a gain to be applied by the decoder. Virtually all players and media frameworks should apply it by default. Any volume adjustment or gain modification, such as the R128_TRACK_GAIN or a user-facing volume knob, MUST be applied ''in addition'' to this output gain in order to achieve playback at the desired volume.

An encoder SHOULD set the output gain to zero, and apply any gain prior to encoding, when this is possible and does not conflict with the user's wishes. The output gain should only be nonzero when the gain is adjusted after encoding, or when the user wishes to adjust the gain for playback while preserving the ability to recover the original signal amplitude.

* '''input rate'''
This is ''not'' the sample rate for playback of the encoded data.

Opus has a handful of coding modes, supporting 8, 12, 16, 24, and 48 kHz signals. Which mode is chosen can be switched dynamically from packet to packet in the stream, but the reference decoder can generate output at any of those sample rates from the compressed data. Fidelity to the original sample rate of the encode input is not preserved by the lossy compression. Therefore, if the playback system supports one of those modes natively, ''the best option is to not resample'' but to play back directly at 48 kHz for best quality regardless of the value of this field.

However, the Ogg mapping allows the encoder to pass the sample rate of the original input stream as metadata. We felt this could be useful downstream, and as something intended for machine consumption, didn't belong in the tag header. For example, a decoder writing PCM format to disk might choose to resample the output audio back to the original input rate to reduce surprise.

* '''stream count'''
It is necessary to describe the number of streams so the decoder can correctly parse the packed frames inside the packet. We store the count-minus-one here, to remove invalid configuration of zero Opus streams in the this Ogg stream.

* '''stream description'''
For each Opus stream framed into the ogg packets of this logical bitstream, we define whether to decode it as mono or stereo, and give a channel index for how it should be mapped to playback. The semantic meaning of each channel index is defined by the ''stream mapping'' byte. E.g. it might be LEFT_REAR or CENTER.

For example, we can't just code 5.1 and three stereo Opus streams, because then LFE ends up sharing a stereo pair with another channel (RR in the Vorbis channel order) which isn't a good idea, while 6 mono channels wastes bandwidth. So we want to be able to say:

(stream mapping: vorbis channel order)
stream 0: stereo: LEFT_FRONT, RIGHT_FRONT
stream 1: mono: CENTER
stream 2: stereo: LEFT_REAR, RIGHT_REAR
stream 3: mono: LFE

== Test vectors ==

* [[OggOpus/testvectors|Planned test vectors for OggOpus]]
* Opus test vectors

OggOpus

2011-08-17T19:23:10Z

Bemasc: More details regarding output gain

OggOpus

2011-08-17T19:06:36Z

Bemasc: Approaching consensus on gain tags

OggOpus

2011-08-11T20:12:52Z

Bemasc: /* Built-in R128 */ Clarification

== Ogg mapping for Opus ==

The IETF Opus codec is a low-latency audio codec optimized for both voice and general-purpose audio. See [tools.ietf.org/html/draft-ietf-codec-opus the spec] for technical details.

Almost everything about this codec is either fixed or dynamically switchable, so the usual id and setup header parameters in the header packets of an Ogg encapsulation aren't useful. In particular, bitrate, frame size, mono/stereo, and coding modes are all dynamically switchable from packet to packet. A one-byte header on each data packet defines the parameters for that particular packet.

Remaining parameters we need to signal are:

* magic number for stream identification
* comment/metadata tags

Additionally there's been a desire to support some kind of channel bonding for surround, and some kind of option signalling for "Opus Custom", in particular the granulerate.

=== Draft spec ===

Granulepos is the count of decodeable samples at a fixed rate of 48 kHz.

Two headers: id, comment

Id header:

- "OpusHead" (64 bits)
- version number (8 bits) zero for this spec
- Number of channels 'c' (8 bits) (must be > 0)
- Pre-skip (16 bits)
- Input sample rate (32 bits) (informational only)
- Output gain (16 bits, signed Q7.8 in dB) to apply when decoding (under discussion)
- stream mapping (8 bits)
-- 0 = one stream, RTP order, 1 = channels in vorbis spec order, 2..254 reserved (treat as 255), 255 = no defined channel meaning
if stream mapping > 0
- Number of streams 'N'(8 bits) (must be > 0)
- Number of two-output streams 'M' (8 bits) (M+N strictly smaller than 255)
- for each output channel [0..c]
-- read stream index (8 bits) (255 means silent through the file)

All two-output streams come first, so if the stream index is < 2*M, the channel
decode the (index/2)th opus stream as stereo, selecting the (index%2)th output
(left for even, right for odd). If index >= 2*M, decode the (index - M)th stream
as mono and use that as the output. As a special case, a stream index of 255
means to write silence to that output channel.

Comment header:

- 8 byte 'OpusTags' magic signature (64 bits)
- rest follows the vorbis-comment header design used in OggVorbis, OggTheora, and Speex.
** Vendor string (always present)
** tag=value metadata strings (zero or more)

Some discussion is in order.

* '''magic signature'''
The signature magic values allow codec identification and are being human readable. Starting with 'Op' helps distinguish them from data packets.

* '''version'''
Version number. Must always be zero for this version of the encapsulation spec. In general revising the spec later isn't a good idea, but this also acts as a null terminator for the signature bytes and helps align the rest of the fields.

* '''stream mapping'''
We want to support multichannel. This defines the order and semantic meaning of the various channels encoded in each Opus packet.

For example, we can't just code 5.1 as three stereo Opus streams, because then LFE ends up sharing a stereo pair with another channel (RR in the Vorbis channel order) which isn't a good idea, while 6 mono channels wastes bandwidth. Or, when routing multitrack audio between mixing boards, it helps to be able to flag which instruments should be treated as mono and which are stereo.

We don't need 8 bits of separate channel meanings, so if we want to make it easier to parse the number of channels, we can make that part of some of the stream mappings: 0 = mono, 1 = stereo, 2 = 5.1 in vorbis order, 3 = 6.0 in some order, 4 = 7.1, etc.

* '''pre-skip'''
This is the number of samples (at 48 kHz) to discard from the decoder output before starting playback. The idea is to mitigate transients, and to allow sample-accurate editing through Ogg chaining.

* '''input rate'''
This is ''not'' the sample rate for playback of the encoded data.

Opus has a handful of coding modes, supporting 8, 12, 16, 24, and 48 kHz signals. Which mode is chosen can be switched dynamically from packet to packet in the stream, but the reference decoder can generate output at any of those sample rates from the compressed data. Fidelity to the original sample rate of the encode input is not preserved by the lossy compression. Therefore, if the playback system supports one of those modes natively, ''the best option is to not resample'' but to play back directly at 48 kHz for best quality regardless of the value of this field.

However, the Ogg mapping allows the encoder to pass the sample rate of the original input stream as metadata. We felt this could be useful downstream, and as something intended for machine consumption, didn't belong in the tag header. For example, a decoder writing PCM format to disk might choose to resample the output audio back to the original input rate to reduce surprise.

* '''stream count'''
It is necessary to describe the number of streams so the decoder can correctly parse the packed frames inside the packet. We store the count-minus-one here, to remove invalid configuration of zero Opus streams in the this Ogg stream.

* '''stream description'''
For each Opus stream framed into the ogg packets of this logical bitstream, we define whether to decode it as mono or stereo, and give a channel index for how it should be mapped to playback. The semantic meaning of each channel index is defined by the ''stream mapping'' byte. E.g. it might be LEFT_REAR or CENTER.

For example, we can't just code 5.1 and three stereo Opus streams, because then LFE ends up sharing a stereo pair with another channel (RR in the Vorbis channel order) which isn't a good idea, while 6 mono channels wastes bandwidth. So we want to be able to say:

(stream mapping: vorbis channel order)
stream 0: stereo: LEFT_FRONT, RIGHT_FRONT
stream 1: mono: CENTER
stream 2: stereo: LEFT_REAR, RIGHT_REAR
stream 3: mono: LFE

== Test vectors ==

* [[OggOpus/testvectors|Planned test vectors for OggOpus]]
* Opus test vectors

== Proposals for handling Opus gain and/or ReplayGain ==
=== Built-in R128 ===
Three Q7.9 dB fields consisting of
# The "file normalization gain", i.e. the gain needed to make this file R128 normal. (a la REPLAYGAIN_TRACK)
# The "program normalization gain", i.e. the gain that will make some set of files normal. (a la REPLAYGAIN_ALBUM). Most players should use this by default.
# The "raw gain", i.e. a gain to apply if the goal is to reproduce the input faithfully. The true raw gain needed to reproduce the input in a sane encoder is always zero, so the real purpose of this gain is to provide adjustment capability even when decoding through a non-player system that will lose these tags. For example, software like ffmpeg might desire to provide faithful input reproduction for transcoding, and might also fail to expose any parameter for users to activate normalization instead. If the newly encoded file also loses the normalization information, then it will play back at the wrong (i.e. unmodified) volume. If ffmpeg supports raw gain, then at least the user may manipulate the input's raw gain so that the tagless transcoded file has appropriate raw volume. (Note, however, that ffmpeg also provides command-line volume adjustment knobs, so this utility is only relevant if it is more convenient to modify the input header than to modify the transcoding invocation.) Similar arguments may apply to web browsers and other frameworks that (a) may be used for both input reproduction and playback, and (b) do not provide convenient parameters to the user to activate normalization ... although it would typically be sufficient for these frameworks to operate with program normalization, which would also be the library default, and possibly the opusdec default behavior.

The exact normalization scheme might be noted in a comment in OpusTags, for the sake of keeping up with trends in volume measurement algorithms.

For the sake of sanity, in this proposal it would probably be best to outlaw ReplayGain tags altogether, although we have not reproduced the clipping-level functionality.

=== Built-in ReplayGain ===
Include in the Opus header a section for the 4 standard ReplayGain values, stored as binary numbers with a value reserved for "unset". Generalize the "Album gain" to "expected listening context gain" for items that do not come from albums. Include a complete ReplayGain implementation in the Ogg-Opus library so that players get support that is on by default (but can be turned off for WAV output). Mandate that decoders must ignore any replaygain comment fields, and encoders must not add them.
=== Compatible mandatory ReplayGain ===
Preserve the VorbisComment REPLAYGAIN tags, but make support mandatory and provide an implementation, so that new decoders may simply hand over all header packets to the library and get replaygain support for free.
=== Inseparable OpusGain ===
Include a header field that encodes a gain. This gain is regarded as part of the bitstream, and the data is never decoded without applying this gain.

OggOpus

2011-08-11T20:04:49Z

Bemasc: Add description of latest proposal

== Ogg mapping for Opus ==

The IETF Opus codec is a low-latency audio codec optimized for both voice and general-purpose audio. See [tools.ietf.org/html/draft-ietf-codec-opus the spec] for technical details.

Almost everything about this codec is either fixed or dynamically switchable, so the usual id and setup header parameters in the header packets of an Ogg encapsulation aren't useful. In particular, bitrate, frame size, mono/stereo, and coding modes are all dynamically switchable from packet to packet. A one-byte header on each data packet defines the parameters for that particular packet.

Remaining parameters we need to signal are:

* magic number for stream identification
* comment/metadata tags

Additionally there's been a desire to support some kind of channel bonding for surround, and some kind of option signalling for "Opus Custom", in particular the granulerate.

=== Draft spec ===

Granulepos is the count of decodeable samples at a fixed rate of 48 kHz.

Two headers: id, comment

Id header:

- "OpusHead" (64 bits)
- version number (8 bits) zero for this spec
- Number of channels 'c' (8 bits) (must be > 0)
- Pre-skip (16 bits)
- Input sample rate (32 bits) (informational only)
- Output gain (16 bits, signed Q7.8 in dB) to apply when decoding (under discussion)
- stream mapping (8 bits)
-- 0 = one stream, RTP order, 1 = channels in vorbis spec order, 2..254 reserved (treat as 255), 255 = no defined channel meaning
if stream mapping > 0
- Number of streams 'N'(8 bits) (must be > 0)
- Number of two-output streams 'M' (8 bits) (M+N strictly smaller than 255)
- for each output channel [0..c]
-- read stream index (8 bits) (255 means silent through the file)

All two-output streams come first, so if the stream index is < 2*M, the channel
decode the (index/2)th opus stream as stereo, selecting the (index%2)th output
(left for even, right for odd). If index >= 2*M, decode the (index - M)th stream
as mono and use that as the output. As a special case, a stream index of 255
means to write silence to that output channel.

Comment header:

- 8 byte 'OpusTags' magic signature (64 bits)
- rest follows the vorbis-comment header design used in OggVorbis, OggTheora, and Speex.
** Vendor string (always present)
** tag=value metadata strings (zero or more)

Some discussion is in order.

* '''magic signature'''
The signature magic values allow codec identification and are being human readable. Starting with 'Op' helps distinguish them from data packets.

* '''version'''
Version number. Must always be zero for this version of the encapsulation spec. In general revising the spec later isn't a good idea, but this also acts as a null terminator for the signature bytes and helps align the rest of the fields.

* '''stream mapping'''
We want to support multichannel. This defines the order and semantic meaning of the various channels encoded in each Opus packet.

For example, we can't just code 5.1 as three stereo Opus streams, because then LFE ends up sharing a stereo pair with another channel (RR in the Vorbis channel order) which isn't a good idea, while 6 mono channels wastes bandwidth. Or, when routing multitrack audio between mixing boards, it helps to be able to flag which instruments should be treated as mono and which are stereo.

We don't need 8 bits of separate channel meanings, so if we want to make it easier to parse the number of channels, we can make that part of some of the stream mappings: 0 = mono, 1 = stereo, 2 = 5.1 in vorbis order, 3 = 6.0 in some order, 4 = 7.1, etc.

* '''pre-skip'''
This is the number of samples (at 48 kHz) to discard from the decoder output before starting playback. The idea is to mitigate transients, and to allow sample-accurate editing through Ogg chaining.

* '''input rate'''
This is ''not'' the sample rate for playback of the encoded data.

Opus has a handful of coding modes, supporting 8, 12, 16, 24, and 48 kHz signals. Which mode is chosen can be switched dynamically from packet to packet in the stream, but the reference decoder can generate output at any of those sample rates from the compressed data. Fidelity to the original sample rate of the encode input is not preserved by the lossy compression. Therefore, if the playback system supports one of those modes natively, ''the best option is to not resample'' but to play back directly at 48 kHz for best quality regardless of the value of this field.

However, the Ogg mapping allows the encoder to pass the sample rate of the original input stream as metadata. We felt this could be useful downstream, and as something intended for machine consumption, didn't belong in the tag header. For example, a decoder writing PCM format to disk might choose to resample the output audio back to the original input rate to reduce surprise.

* '''stream count'''
It is necessary to describe the number of streams so the decoder can correctly parse the packed frames inside the packet. We store the count-minus-one here, to remove invalid configuration of zero Opus streams in the this Ogg stream.

* '''stream description'''
For each Opus stream framed into the ogg packets of this logical bitstream, we define whether to decode it as mono or stereo, and give a channel index for how it should be mapped to playback. The semantic meaning of each channel index is defined by the ''stream mapping'' byte. E.g. it might be LEFT_REAR or CENTER.

For example, we can't just code 5.1 and three stereo Opus streams, because then LFE ends up sharing a stereo pair with another channel (RR in the Vorbis channel order) which isn't a good idea, while 6 mono channels wastes bandwidth. So we want to be able to say:

(stream mapping: vorbis channel order)
stream 0: stereo: LEFT_FRONT, RIGHT_FRONT
stream 1: mono: CENTER
stream 2: stereo: LEFT_REAR, RIGHT_REAR
stream 3: mono: LFE

== Test vectors ==

* [[OggOpus/testvectors|Planned test vectors for OggOpus]]
* Opus test vectors

== Proposals for handling Opus gain and/or ReplayGain ==
=== Built-in R128 ===
Three Q7.9 dB fields consisting of
# The "file normalization gain", i.e. the gain needed to make this file R128 normal. (a la REPLAYGAIN_TRACK)
# The "program normalization gain", i.e. the gain that will make some set of files normal. (a la REPLAYGAIN_ALBUM). Most players should use this by default.
# The "raw gain", i.e. a gain to apply if the goal is to reproduce the input faithfully. The true raw gain needed to reproduce the input in a sane encoder is always zero, so the real purpose of this gain is to provide adjustment capability even when decoding through a non-player system that will lose these tags. For example, software like ffmpeg might desire to provide faithful input reproduction for transcoding, and might also fail to expose any parameter for users to choose normalization instead. If the newly encoded file also loses the normalization information, then it will play back at the wrong volume. If ffmpeg supports raw gain, then at least the user may manipulate the input's raw gain so that they tagless transcoded file has appropriate raw volume. (Note, however, that ffmpeg also provides command-line volume adjustment knobs, so this utility is more convenient to modify the input than to modify the transcoding invocation.) Similar arguments may apply to web browsers and other frameworks that (a) may be used for both input reproduction and playback, and (b) do not provide convenient parameters to the user to control normalization state ... although it would typically be sufficient for these frameworks to operate with program normalization, which would also be the library default.

The exact normalization scheme might be noted in a comment in OpusTags, for the sake of keeping up with trends in volume measurement algorithms.

For the sake of sanity, in this proposal it would probably be best to outlaw ReplayGain tags altogether, although we have not reproduced the clipping-level functionality.

=== Built-in ReplayGain ===
Include in the Opus header a section for the 4 standard ReplayGain values, stored as binary numbers with a value reserved for "unset". Generalize the "Album gain" to "expected listening context gain" for items that do not come from albums. Include a complete ReplayGain implementation in the Ogg-Opus library so that players get support that is on by default (but can be turned off for WAV output). Mandate that decoders must ignore any replaygain comment fields, and encoders must not add them.
=== Compatible mandatory ReplayGain ===
Preserve the VorbisComment REPLAYGAIN tags, but make support mandatory and provide an implementation, so that new decoders may simply hand over all header packets to the library and get replaygain support for free.
=== Inseparable OpusGain ===
Include a header field that encodes a gain. This gain is regarded as part of the bitstream, and the data is never decoded without applying this gain.