Difference between revisions of "OggOpus"

From XiphWiki
Jump to navigation Jump to search
(Approaching consensus on gain tags)
Line 25: Line 25:
 
  - Pre-skip (16 bits)
 
  - Pre-skip (16 bits)
 
  - Input sample rate (32 bits) (informational only)
 
  - Input sample rate (32 bits) (informational only)
  - Output gain (16 bits, signed Q7.8 in dB) to apply when decoding (under discussion)
+
  - Output gain (16 bits, signed Q7.8 in dB) to apply when decoding
 
  - stream mapping (8 bits)
 
  - stream mapping (8 bits)
 
   --  0 = one stream, RTP order, 1 = channels in vorbis spec order, 2..254 reserved (treat as 255), 255 = no defined channel meaning
 
   --  0 = one stream, RTP order, 1 = channels in vorbis spec order, 2..254 reserved (treat as 255), 255 = no defined channel meaning
Line 46: Line 46:
 
   ** Vendor string (always present)
 
   ** Vendor string (always present)
 
   ** tag=value metadata strings (zero or more)
 
   ** tag=value metadata strings (zero or more)
 +
 +
One new comment field is introduced for Ogg Opus:
 +
R128_TRACK_GAIN=-7.03 dB
 +
similar to the [[VorbisComment#Replay_Gain|REPLAYGAIN_TRACK_GAIN field in Vorbis]].  (There is no field corresponding to Replaygain's ALBUM_GAIN; that information should be stored in the header's "output gain" field.)  An Ogg Opus file MUST NOT have more than one such field, and if present its value MUST be of the form "[number] dB" in 7-bit ASCII.
  
 
Some discussion is in order.
 
Some discussion is in order.
Line 64: Line 68:
 
* '''pre-skip'''
 
* '''pre-skip'''
 
This is the number of samples (at 48 kHz) to discard from the decoder output before starting playback. The idea is to mitigate transients, and to allow sample-accurate editing through Ogg chaining.
 
This is the number of samples (at 48 kHz) to discard from the decoder output before starting playback. The idea is to mitigate transients, and to allow sample-accurate editing through Ogg chaining.
 +
 +
* '''output gain'''
 +
This is a gain to be applied by the decoder.  Virtually all players and media frameworks should apply it by default.  When using the R128_TRACK_GAIN, or any other gain modification, players must also apply the output gain in order to achieve playback at the desired volume.
  
 
* '''input rate'''
 
* '''input rate'''
Line 91: Line 98:
 
* [[OggOpus/testvectors|Planned test vectors for OggOpus]]
 
* [[OggOpus/testvectors|Planned test vectors for OggOpus]]
 
* Opus test vectors
 
* Opus test vectors
 
== Proposals for handling Opus gain and/or ReplayGain ==
 
=== Built-in R128 ===
 
Three Q7.9 dB fields consisting of
 
# The "file normalization gain", i.e. the gain needed to make this file R128 normal. (a la REPLAYGAIN_TRACK)
 
# The "program normalization gain", i.e. the gain that will make some set of files normal. (a la REPLAYGAIN_ALBUM).  Most players should use this by default.
 
# The "raw gain", i.e. a gain to apply if the goal is to reproduce the input faithfully.  The true raw gain needed to reproduce the input in a sane encoder is always zero, so the real purpose of this gain is to provide adjustment capability even when decoding through a non-player system that will lose these tags.  For example, software like ffmpeg might desire to provide faithful input reproduction for transcoding, and might also fail to expose any parameter for users to activate normalization instead.  If the newly encoded file also loses the normalization information, then it will play back at the wrong (i.e. unmodified) volume.  If ffmpeg supports raw gain, then at least the user may manipulate the input's raw gain so that the tagless transcoded file has appropriate raw volume.  (Note, however, that ffmpeg also provides command-line volume adjustment knobs, so this utility is only relevant if it is more convenient to modify the input header than to modify the transcoding invocation.)  Similar arguments may apply to web browsers and other frameworks that (a) may be used for both input reproduction and playback, and (b) do not provide convenient parameters to the user to activate normalization ... although it would typically be sufficient for these frameworks to operate with program normalization, which would also be the library default, and possibly the opusdec default behavior.
 
 
The exact normalization scheme might be noted in a comment in OpusTags, for the sake of keeping up with trends in volume measurement algorithms.
 
 
For the sake of sanity, in this proposal it would probably be best to outlaw ReplayGain tags altogether, although we have not reproduced the clipping-level functionality.
 
 
=== Built-in ReplayGain ===
 
Include in the Opus header a section for the 4 standard ReplayGain values, stored as binary numbers with a value reserved for "unset".  Generalize the "Album gain" to "expected listening context gain" for items that do not come from albums.  Include a complete ReplayGain implementation in the Ogg-Opus library so that players get support that is on by default (but can be turned off for WAV output).  Mandate that decoders must ignore any replaygain comment fields, and encoders must not add them.
 
=== Compatible mandatory ReplayGain ===
 
Preserve the VorbisComment REPLAYGAIN tags, but make support mandatory and provide an implementation, so that new decoders may simply hand over all header packets to the library and get replaygain support for free.
 
=== Inseparable OpusGain ===
 
Include a header field that encodes a gain.  This gain is regarded as part of the bitstream, and the data is never decoded without applying this gain.  Additionally normal replaygain style tags may exist in the comments which are applied (by supporting playback chains) on top of the OpusGain.
 

Revision as of 12:06, 17 August 2011

Ogg mapping for Opus

The IETF Opus codec is a low-latency audio codec optimized for both voice and general-purpose audio. See [tools.ietf.org/html/draft-ietf-codec-opus the spec] for technical details.

Almost everything about this codec is either fixed or dynamically switchable, so the usual id and setup header parameters in the header packets of an Ogg encapsulation aren't useful. In particular, bitrate, frame size, mono/stereo, and coding modes are all dynamically switchable from packet to packet. A one-byte header on each data packet defines the parameters for that particular packet.

Remaining parameters we need to signal are:

  • magic number for stream identification
  • comment/metadata tags

Additionally there's been a desire to support some kind of channel bonding for surround, and some kind of option signalling for "Opus Custom", in particular the granulerate.

Draft spec

Granulepos is the count of decodeable samples at a fixed rate of 48 kHz.

Two headers: id, comment

Id header:

- "OpusHead" (64 bits)
- version number (8 bits) zero for this spec
- Number of channels 'c' (8 bits) (must be > 0)
- Pre-skip (16 bits)
- Input sample rate (32 bits) (informational only)
- Output gain (16 bits, signed Q7.8 in dB) to apply when decoding
- stream mapping (8 bits)
 --  0 = one stream, RTP order, 1 = channels in vorbis spec order, 2..254 reserved (treat as 255), 255 = no defined channel meaning
if stream mapping > 0
- Number of streams 'N'(8 bits) (must be > 0)
- Number of two-output streams 'M' (8 bits) (M+N strictly smaller than 255)
- for each output channel [0..c]
  -- read stream index (8 bits) (255 means silent through the file)

All two-output streams come first, so if the stream index is < 2*M, the channel decode the (index/2)th opus stream as stereo, selecting the (index%2)th output (left for even, right for odd). If index >= 2*M, decode the (index - M)th stream as mono and use that as the output. As a special case, a stream index of 255 means to write silence to that output channel.

Comment header:

- 8 byte 'OpusTags' magic signature (64 bits)
- rest follows the vorbis-comment header design used in OggVorbis, OggTheora, and Speex.
 ** Vendor string (always present)
 ** tag=value metadata strings (zero or more)

One new comment field is introduced for Ogg Opus:

R128_TRACK_GAIN=-7.03 dB

similar to the REPLAYGAIN_TRACK_GAIN field in Vorbis. (There is no field corresponding to Replaygain's ALBUM_GAIN; that information should be stored in the header's "output gain" field.) An Ogg Opus file MUST NOT have more than one such field, and if present its value MUST be of the form "[number] dB" in 7-bit ASCII.

Some discussion is in order.

  • magic signature

The signature magic values allow codec identification and are being human readable. Starting with 'Op' helps distinguish them from data packets.

  • version

Version number. Must always be zero for this version of the encapsulation spec. In general revising the spec later isn't a good idea, but this also acts as a null terminator for the signature bytes and helps align the rest of the fields.

  • stream mapping

We want to support multichannel. This defines the order and semantic meaning of the various channels encoded in each Opus packet.

For example, we can't just code 5.1 as three stereo Opus streams, because then LFE ends up sharing a stereo pair with another channel (RR in the Vorbis channel order) which isn't a good idea, while 6 mono channels wastes bandwidth. Or, when routing multitrack audio between mixing boards, it helps to be able to flag which instruments should be treated as mono and which are stereo.

We don't need 8 bits of separate channel meanings, so if we want to make it easier to parse the number of channels, we can make that part of some of the stream mappings: 0 = mono, 1 = stereo, 2 = 5.1 in vorbis order, 3 = 6.0 in some order, 4 = 7.1, etc.

  • pre-skip

This is the number of samples (at 48 kHz) to discard from the decoder output before starting playback. The idea is to mitigate transients, and to allow sample-accurate editing through Ogg chaining.

  • output gain

This is a gain to be applied by the decoder. Virtually all players and media frameworks should apply it by default. When using the R128_TRACK_GAIN, or any other gain modification, players must also apply the output gain in order to achieve playback at the desired volume.

  • input rate

This is not the sample rate for playback of the encoded data.

Opus has a handful of coding modes, supporting 8, 12, 16, 24, and 48 kHz signals. Which mode is chosen can be switched dynamically from packet to packet in the stream, but the reference decoder can generate output at any of those sample rates from the compressed data. Fidelity to the original sample rate of the encode input is not preserved by the lossy compression. Therefore, if the playback system supports one of those modes natively, the best option is to not resample but to play back directly at 48 kHz for best quality regardless of the value of this field.

However, the Ogg mapping allows the encoder to pass the sample rate of the original input stream as metadata. We felt this could be useful downstream, and as something intended for machine consumption, didn't belong in the tag header. For example, a decoder writing PCM format to disk might choose to resample the output audio back to the original input rate to reduce surprise.


  • stream count

It is necessary to describe the number of streams so the decoder can correctly parse the packed frames inside the packet. We store the count-minus-one here, to remove invalid configuration of zero Opus streams in the this Ogg stream.

  • stream description

For each Opus stream framed into the ogg packets of this logical bitstream, we define whether to decode it as mono or stereo, and give a channel index for how it should be mapped to playback. The semantic meaning of each channel index is defined by the stream mapping byte. E.g. it might be LEFT_REAR or CENTER.

For example, we can't just code 5.1 and three stereo Opus streams, because then LFE ends up sharing a stereo pair with another channel (RR in the Vorbis channel order) which isn't a good idea, while 6 mono channels wastes bandwidth. So we want to be able to say:

(stream mapping: vorbis channel order)
stream 0: stereo: LEFT_FRONT, RIGHT_FRONT
stream 1: mono: CENTER
stream 2: stereo: LEFT_REAR, RIGHT_REAR
stream 3: mono: LFE

Test vectors