Difference between revisions of "OpusFAQ"

From XiphWiki
Jump to: navigation, search
(How do I use Opus?)
 
(90 intermediate revisions by 10 users not shown)
Line 1: Line 1:
 +
If you are looking for info not covered in this FAQ, try the [https://opus-codec.org main Opus website] or the pages included in the [[:Category:Opus|Opus category]] of this wiki.
 +
 
[[Image:Opus logo trans.png|right]]
 
[[Image:Opus logo trans.png|right]]
  
 
== General Questions ==
 
== General Questions ==
  
=== What is Opus? ===
+
=== What is Opus? Who created it? ===
  
Opus is a format for compressing audio to efficiently transmit it across networks. Opus allows for music-grade high quality audio at low data rates while not delaying the signal much. Opus is distinguished from most formats for high quality audio (AAC, Vorbis, MP3) by having low delay and it is distinguished from most low delay formats (G.711, GSM, Speex) by supporting high audio quality. The Opus format itself and the reference implementation are available under liberal royalty-free licenses, making it easy to adopt, compatible with free software, and suitable for for usage as part of the basic infrastructure of the Internet.
+
Opus is a totally open, royalty-free, highly versatile audio codec.
  
=== Who created Opus? ===
+
It is primarily designed for interactive speech and music transmission over the Internet, but is also applicable to storage and streaming applications. It incorporates technology from Skype's '''[https://en.wikipedia.org/wiki/SILK SILK]''' codec and Xiph.Org's '''[http://celt-codec.org/ CELT]''' codec. It has been standardized by the '''[https://www.ietf.org/ Internet Engineering Task Force]''' (IETF) as '''[https://tools.ietf.org/html/rfc6716 RFC 6716]'''.
  
Opus was created by combining Xiph.Org's CELT development codec and Skype's SILK codec as part of a cooperative effort in the IETF codec working group. Opus has been in development since early 2007, and <!-- as of ???? is an IETF proposed standard: '''RFC TBD''' --> has been recommended by the working group for promotion as a proposed standard.
+
Opus has been in development since early 2007. Programmers associated with '''[https://xiph.org/ Xiph.Org]''', '''[https://www.skype.com/ Skype]''' and several other organizations have contributed to its development and to the standardization process as part of the '''[https://datatracker.ietf.org/wg/codec/charter/ IETF's Codec Working Group]'''.
  
=== Is the SILK part of Opus compatible with the SILK implementation shipped in Skype? ===
+
=== How does Opus compare to other codecs? ===
 +
 
 +
Opus is distinguished from most high quality formats (eg: AAC, [[Vorbis]], MP3) by having '''low delay''' (5 ~ 66.5 ms) and distinguished from most low delay formats (eg: G.711, GSM, Speex) by supporting '''high audio quality''' ([https://tools.ietf.org/html/rfc6716#section-2.1.1 details here]).
 +
 
 +
It meets or exceeds existing codecs' quality across a wide range of bitrates, and it operates at lower delay than virtually any existing compressed format.
 +
 
 +
Most importantly, the Opus format and its reference implementation are both available under '''[https://opus-codec.org/license/ liberal, royalty-free licenses]'''.<br />
 +
This makes it:
 +
* easy to adopt
 +
* compatible with free software
 +
* suitable for use as part of the basic infrastructure of the Internet
 +
 
 +
See the Opus '''[https://opus-codec.org/comparison comparison page]''' for more details.
 +
 
 +
=== Does Opus make all those other lossy codecs obsolete? ===
 +
 
 +
Theoretically, yes.
 +
 
 +
From a technical point of view (loss, delay, bitrates, ...) it should replace both [[Vorbis]] and [[Speex]], and the common proprietary codecs too.
 +
 
 +
=== Will Opus replace Vorbis in video files? ===
 +
 
 +
For Ogg [[Theora]] video files, it can, just the overall size reduction will be minimal and it will break compatibility with existing players.
 +
 
 +
For WebM video files, the convention is to use the VP9 video codec when using Opus as an audio codec.
 +
 
 +
=== How do I use Opus? ===
 +
 
 +
For now, the best way to '''encode''' audio into Opus files is to use the '''opusenc''' command-line tool from the '''[https://opus-codec.org/downloads/ opus-tools package]'''.
 +
 
 +
If you want to encode many files at once (e.g. your music library), try the applications listed in the '''[[OpusSupport|Opus Support]]''' page.
 +
 
 +
For rough guidelines on encoding settings, see the '''[[Opus Recommended Settings]]''' page.
 +
 
 +
=== What programs support Opus? ===
 +
 
 +
Opus decoding support is now included in '''[http://caniuse.com/opus some Internet browsers]''' and '''[[OpusSupport|many applications]]''', including '''[https://www.mozilla.org/firefox Firefox]''', '''[https://www.foobar2000.org/ foobar2000]''' and '''[https://www.videolan.org/vlc/ VLC]''', as well as in frameworks such as '''[https://gstreamer.freedesktop.org/ GStreamer]''' and '''[https://ffmpeg.org/ FFmpeg]'''.
 +
 
 +
For real-time applications, Opus support is available in '''[https://www.webrtc.org/ Google's WebRTC codebase]'''.
 +
 
 +
Opus is a relatively new codec: '''[[OpusSupport|many more applications]]''' will support it in the near future.
 +
 
 +
=== Does Opus support higher sampling rates, such as 96 kHz or 192 kHz? ===
 +
 
 +
Yes and no.
 +
 
 +
Opus encoding tools like opusenc will happily encode input files that are sampled at 96 or 192 kHz.
 +
 
 +
However, files at these rates are internally '''converted to 48 kHz''' and then only frequencies '''up to 20 kHz''' are encoded.
 +
 
 +
The reason is simple: lossy codecs are designed to preserve audible details while discarding irrelevant information. Since the human ear can only hear up to 20 kHz at best (usually lower than that), frequency content above 20 kHz is the first thing to go.
  
No. The SILK codec, as submitted by Skype to the IETF, was heavily modified as part of its integration within Opus. The modifications are significant enough that it is not possible to just write a "translator" and even sharing code between Opus and the "old SILK" would be highly non-trivial.
+
See Monty's '''[https://people.xiph.org/~xiphmont/demo/neil-young.html article]''' for more details.
  
=== How do I use Opus / What programs use Opus? ===
+
If you want a codec to handle higher sampling rates losslessly, use '''[[FLAC]]'''!
  
 
=== What are the licensing requirements? ===
 
=== What are the licensing requirements? ===
Line 21: Line 73:
 
The reference Opus source code is released under a three-clause BSD license, which is a very permissive Open Source license. Commercial use and distribution (including in proprietary software) is permitted, provided that some basic conditions specified in the license are met.  
 
The reference Opus source code is released under a three-clause BSD license, which is a very permissive Open Source license. Commercial use and distribution (including in proprietary software) is permitted, provided that some basic conditions specified in the license are met.  
  
Opus is also covered by some patents, for which royalty-free usage rights are granted, under conditions that the authors believe are compatible with most (all?) open source licenses, including the GPL (v2 and v3).
+
Opus is also covered by some patents, for which royalty-free usage rights are granted, under conditions that the authors believe are compatible with (hopefully) all open source licenses, including the GPL (v2 and v3).
  
=== How does the quality of Opus compare to other codecs? ===
+
See the '''[https://www.opus-codec.org/license/ Opus Licensing]''' page for details.
  
See the Opus [http://opus-codec.org/comparison comparison page] for more on this.
+
=== Why make Opus free? ===
 +
 
 +
On the Internet, protocol and codec standards are part of the common infrastructure everyone builds upon.
 +
 
 +
Most of the value of a high-quality standard is the innovation and inter-operation provided by the systems built on top of it. When a few parties have monopoly rights to monetize a standard, that infrastructure stops being so common and everyone else has more reason to use their own solution instead, increasing cost and reducing efficiency.
 +
 
 +
Imagine a road system where each type of car could only drive on its own manufacturer's pavement. We all benefit from living in a world where all the roads are connected.
 +
 
 +
This is why Opus, unlike many codecs, is free.
 +
 
 +
=== Is the SILK part of Opus compatible with the SILK implementation shipped in Skype? ===
 +
 
 +
No.
 +
 
 +
The SILK codec, as submitted by Skype to the IETF, was heavily modified as part of its integration within Opus. The modifications are significant enough that it is not possible to just write a "translator". Even sharing code between Opus and the "old SILK" would be highly complex.
 +
 
 +
=== Why not keep the SILK and CELT codecs separate? ===
 +
Opus is more than just two independent codecs with a switch.
 +
 
 +
In addition to a [https://en.wikipedia.org/wiki/Linear_predictive_coding Linear Prediction] '''SILK mode''' and an [https://en.wikipedia.org/wiki/Modified_discrete_cosine_transform MDCT] '''CELT mode''' it has a '''hybrid mode''', where speech frequencies up to 8 kHz are encoded with LP while those between 8 and 20 kHz are encoded with MDCT. This is what allows Opus to have such high speech quality around 32 kbps.
 +
 
 +
Another advantage of the integration is the ability to switch between these 3 modes seamlessly, without any audible "glitches" and without any out-of-band signalling.
 +
 
 +
=== Now that Opus is standardized, will its development stop or can it be further improved? ===
 +
Yes, Opus '''can''' and '''should''' be improved, because unlike most '''[https://en.wikipedia.org/wiki/ITU-T#Key_standards_published_by_ITU ITU-T codecs]''', Opus is only defined in terms of its decoder.
 +
 
 +
The encoder can keep evolving as long as the bitstream it produces can be decoded by the reference decoder. This is what made it possible for modern MP3 encoders (e.g. '''[https://en.wikipedia.org/wiki/LAME LAME]''') to improve far beyond the original '''[https://en.wikipedia.org/wiki/L3enc L3enc]''' and '''dist10''' reference implementations.
 +
 
 +
Although it is unlikely that Opus encoders will see such a spectacular evolution, we certainly hope that future encoders will become much better than the reference encoder.
 +
 
 +
In fact, the 1.1 libopus release significantly improves on the reference encoder's quality. See '''[https://people.xiph.org/~xiphmont/demo/opus/demo3.shtml Monty's demo]''' for more details.
 +
 
 +
=== Will all future Opus releases comply with the [https://tools.ietf.org/html/rfc6716 Opus specification]? ===
 +
 
 +
Yes.
 +
 
 +
=== In what ways is Opus optimized for the Internet? ===
 +
 
 +
Opus has good packet loss robustness and concealment, but its optimisations go further.
 +
 
 +
One of the first things we've been asked when designing Opus was to make the rate '''really''' adaptable because we never know what kind of rates will be available. This not only meant having a wide range of bitrates, but also being able to vary in small increments.
 +
 
 +
This is why Opus scales from about '''6 ''' to '''512 kb/s''', in increments of '''0.4 kb/s''' (one byte with 20 ms frames). Opus can have '''more than 1200 possible bitrates''' while spending only '''11 bits''' signalling the bitrate because UDP already encodes the packet size.
 +
 
 +
One last aspect is that Opus is simple to transport over RTP, as can be seen from the [https://tools.ietf.org/html/rfc7587 Opus RTP payload format]. For example, it's possible to decode RTP packets without having even seen the SDP or any out-of-band signalling.
 +
 
 +
=== What applications for Android can play Opus? ===
 +
 
 +
Right now, there are just a few but that list is fast growing. Please reference [https://android.stackexchange.com/q/37970/7425 this question on android.stackexchange.com]. Feel free to suggest other applications.
 +
 
 +
=== When will the next version be released? ===
 +
 
 +
When it's done. Seriously, we do not know.
 +
 
 +
Opus is not a large project with a fixed release schedule.
 +
 
 +
That being said, our [https://www.opus-codec.org/downloads/ pre-releases] and even the [https://git.xiph.org/?p=opus.git git repository] are generally pretty stable and given proper testing (which you should always do anyway), are safe to distribute.
 +
 
 +
Just be aware that the API of new features (that have never been included in a stable release) could potentially still change.
 +
 
 +
== Software Developers' Questions ==
  
 
=== On what platforms does Opus run? ===
 
=== On what platforms does Opus run? ===
  
The Opus code base is written in C89 and should run on the vast majority of recent (and not so recent) CPUs. Opus has been tested and is known to run on the following (non-exhaustive list) platforms: x86, x86-64, ARM, Itanium, Blackfin, SPARC.
+
The Opus code base is written in C89 and should run on the vast majority of recent (and not so recent) CPUs.
 +
 
 +
Some of the platforms '''[https://mf4.xiph.org/jenkins/view/opus/ on which Opus has been tested]''' include x86, x86-64, ARM, Itanium, Blackfin, and SPARC.
  
 
=== Is there a fixed-point implementation? ===
 
=== Is there a fixed-point implementation? ===
  
Yes, the fixed-point and floating-point decoder and encoder implementations are part of the same code base. The code defaults to float, so you need to configure with --enable-fixed-point (or defining FIXED_POINT if not using the configure script) to build the code for fixed-point.
+
Yes.
  
=== Why not keep the SILK and CELT codecs separate? ===
+
The fixed-point and floating-point decoder and encoder implementations are part of the same code base.
  
Opus is more than just two independent codecs with a switch. It can not only run in "SILK mode" and in "CELT mode", but also in "hybrid mode" where speech frequencies up to 8 kHz are encoded with SILK while those above 8 kHz are encoded with CELT. This is what allows Opus to have such high speech quality around 32 kb/s. Another advantage of the integration is the ability to switch mode seamlessly. It is possible to change between all the modes without any "glitch" and without any out-of-band signalling.
+
The code defaults to float, so you need to configure with '''--enable-fixed-point''' (or define '''FIXED_POINT''' if not using the configure script) to build the code for fixed-point.
  
== Opus for Software developers ==
+
=== Which implementation should I use? ===
  
=== What is difference between supporting Opus and supporting Speex/G.711/MP3? ===
+
While the implementation in RFC 6716 is what ''defines'' the standard, it is likely not the best and most up-to-date implementation.
  
Opus has variable frame durations which can change on the fly, so an Opus decoder needs to be ready to accept packets with durations that are any multiple of 2.5ms up to a maximum of 120ms.
+
The [https://opus-codec.org/ Opus] website was set up for the purpose of continually improving the implementation — in terms of speed, encoding quality, device compatibility, etc — while still conforming to the standard.
  
The opus encoder and decoder do not need to have matched sampling rates, bandwidths, or channel counts.  Its recommended to always just decode at the highest rate the hardware supports (e.g. 48kHz stereo) so the user gets the full quality of whatever the far end is sending.
+
All Opus implementations are compatible by definition.
  
=== Forward Error correction (FEC) doesn't appear to do anything! HELP! ===
+
=== How is supporting Opus different from supporting Speex/G.711/MP3? ===
  
 +
Opus has variable frame durations which can change on the fly, so an Opus decoder needs to be ready to accept packets with durations that are '''any multiple of 2.5ms''' up to a '''maximum of 120ms'''. 
  
The inband FEC feature of Opus helps reduce the harm of packet loss by encoding some information about the prior packet.
+
The opus encoder and decoder do not need to have matched sampling rates or channel counts.  It is recommended to always just decode at the highest rate the hardware supports (e.g. 48kHz stereo) so the user gets the full quality of whatever the far end is sending.
  
In order to make use of inband FEC the decoder must delay its output by at least one frame so that it can call the decoder with the decode_fec argument on the ''next'' frame in order to reconstruct the missed frame. This works best if it's integrated with a jitter buffer.
+
=== My application doesn't work. Can anyone help me? ===
  
FEC is only used by the encoder under certain conditions: the feature must be enabled via the OPUS_SET_INBAND_FEC CTL, the encoder must be told to expect loss via the OPUS_SET_PACKET_LOSS_PERC CTL, and the codec must be operated in any of the linear prediction or Hybrid modes. Frame durations of <10ms and very high bitrates will use the MDCT modes, where FEC is not available.
+
It's possible to get help, but before doing so, there are a few basic things to try:
  
Even when FEC is not used, telling the encoder about the level of loss will help it make more intelligent decisions. By default the implementation assumes there is no loss.
+
* Implement your application with uncompressed audio instead of Opus. If it still doesn't work, then the problem isn't related to Opus.
 +
* Read the [https://www.opus-codec.org/docs/ Opus documentation].
 +
* Read the [https://git.xiph.org/?p=opus.git;a=blob;f=src/opus_demo.c opus_demo.c] source code to see how to use the encoder and decoder.
 +
 
 +
If you still can't solve the problem, the best option is to ask for help on the [http://lists.xiph.org/mailman/listinfo/opus mailing list] or on the '''#opus''' IRC channel on '''irc.freenode.net'''.
 +
 
 +
=== How do I report a bug? ===
 +
 
 +
If you think you have found a bug in Opus (and not in your application), please [https://trac.xiph.org/newticket?component=Opus file a bug report].
 +
 
 +
Please include a way for us to reproduce the problem. The best way to do this is to provide an input file, along with the opusenc/opusdec/opus_demo command line that causes the bug to occur.
 +
 
 +
If the bug cannot be triggered by the command line tools, please provide a simple patch or C file that can help reproduce it. Please also provide any other relevant information, such as OS, CPU, build options, etc.
 +
 
 +
Don't hesitate to also contact us on the [http://lists.xiph.org/mailman/listinfo/opus mailing list] or on [irc://irc.freenode.net/opus IRC].
  
 
=== What is Opus Custom? ===
 
=== What is Opus Custom? ===
  
Opus Custom is an '''optional''' part of the Opus standard that allows for sampling rates other than 8, 12, 16, 24, or 48 kHz and frame sizes other than multiples of 2.5 ms. Opus Custom requires additional out-of-band signalling that Opus does not normally require and disables many of Opus' coding modes. Also, because it is an optional part of the specification, using Opus Custom may lead to compatibility problems. For these reasons, its use is discouraged outside of very specific applications, e.g.:
+
Opus Custom is an '''optional''' part of the Opus standard that allows for sampling rates other than 8, 12, 16, 24, or 48 kHz and frame sizes other than multiples of 2.5 ms.
 +
 
 +
Opus Custom requires additional out-of-band signalling that Opus does not normally require and disables many of Opus' coding modes. Also, because it is an optional part of the specification, using Opus Custom may lead to compatibility problems.
 +
 
 +
For these reasons, its use is discouraged outside of very specific applications, for example:
 
* ultra low delay applications where synchronization with the soundcard buffer is important.  
 
* ultra low delay applications where synchronization with the soundcard buffer is important.  
 
* low-power embedded applications where compatibility with others is not important.
 
* low-power embedded applications where compatibility with others is not important.
Line 68: Line 201:
 
=== How do I use 44.1 kHz or some other sampling rate not directly supported by Opus? ===
 
=== How do I use 44.1 kHz or some other sampling rate not directly supported by Opus? ===
  
Tools which read or write Opus should inter-work with other sampling rate by transparently performing sample rate conversion behind the scenes. It's generally preferable to run the output at 48kHz even when you know the original input was 44.1kHz because many inexpensive audio interfaces have poor quality output for 44.1k.
+
Tools which read or write Opus should inter-operate with other sampling rates by transparently performing sample rate conversion behind the scenes whenever necessary. In particular, software developers should not use Opus Custom for 44.1 kHz support, except in the very specific circumstances outlined above.
  
In particular, software developers should not use Opus Custom for 44.1 kHz support, except in the very specific circumstances outlined above.
+
Note that it's generally preferable for a decoder to output at 48kHz, even when you know the original input was 44.1kHz. This is not only because you can skip resampling, but also because many cheaper audio interfaces have poor quality output for 44.1kHz.
  
The opus-tools package source code contains a small, high quality, high performance, BSD licensed resampler which can be used where resampling is required.
+
The '''[https://opus-codec.org/downloads/ opus-tools]''' package source code contains a small, high quality, high performance, BSD licensed resampler which can be used where resampling is required.
  
=== I can't use malloc or much stack on my embedded platform how do I make Opus work? ===
+
=== How is the bitrate setting used in VBR mode? ===
  
A normal build of libopus only uses malloc/free in the _create() and _destroy() calls, so Opus is safe for realtime use so long as the codec state is pre-created.
+
Variable bitrate (VBR) mode allows the bitrate to automatically vary over time based on the audio being encoded, in order to achieve a consistent quality.
  
In order to build Opus without any reference to malloc/free at all use init() calls rather than the create() calls in your application and compile with <tt>CFLAGS="-DOVERRIDE_OPUS_ALLOC -DOVERRIDE_OPUS_FREE -D'opus_alloc(x)=NULL' -D'opus_free(x)=NULL' "</tt> you will get a build which does not use malloc/free.
+
The bitrate setting controls the desired quality, on a scale that is calibrated to closely approximate the average bitrate that would be obtained over a large and diverse collection of audio. The actual bitrate of any particular audio stream may be higher or lower than this average.
  
If libopus is built with -DNONTHREADSAFE_PSEUDOSTACK (instead of VAR_ARRAYS, or USE_ALLOCA) it will use a user provided block of heap instead of stack for many things resulting in much lower the stack usage. However this makes the resulting library non-threadsafe and is not recommend on anything except limited embedded platforms.
+
=== What frame size should I use? ===
  
=== Which implementation should I use? ===
+
A '''20ms''' frame size works well for most applications.
 +
 
 +
Smaller frame sizes may be used to achieve lower latency, but have lower quality at a given bitrate.
 +
 
 +
Sizes greater than 20 ms increase latency and are generally beneficial only at fairly low bitrates, or when used to reduce external overhead (e.g. by reducing the number of packets that are sent).
 +
 
 +
=== Forward Error correction (FEC) doesn't appear to do anything! HELP! ===
 +
 
 +
The in-band FEC feature of Opus helps reduce the harm of packet loss by encoding some information about the prior packet.
 +
 
 +
In order to make use of in-band FEC the decoder must delay its output by at least one frame so that it can call the decoder with the decode_fec argument on the ''next'' frame in order to reconstruct the missed frame. This works best if it's integrated with a jitter buffer.
 +
 
 +
FEC is only used by the encoder under certain conditions:
 +
* the feature must be enabled via the OPUS_SET_INBAND_FEC CTL
 +
* the encoder must be told to expect loss via the OPUS_SET_PACKET_LOSS_PERC CTL
 +
* the codec must be operated in any of the linear prediction or Hybrid modes
 +
 
 +
Frame durations shorter than 10ms and very high bitrates will use the MDCT modes, where FEC is not available.
 +
 
 +
Even when FEC is not used, telling the encoder about the expected level of loss will help it make more intelligent decisions. By default the implementation assumes there is no loss.
 +
 
 +
=== I can't use malloc or much stack on my embedded platform. How do I make Opus work? ===
 +
 
 +
A normal build of libopus only uses <tt>malloc/free</tt> in the <tt>_create()</tt> and <tt>_destroy()</tt> calls, making it safe for realtime use as long as the codec state is pre-created.
 +
 
 +
To build Opus without the references to <tt>malloc/free</tt>, you must:
 +
 
 +
* use <tt>init()</tt> calls rather than <tt>create()</tt> calls in your application
 +
* compile with <tt>CFLAGS="-DOVERRIDE_OPUS_ALLOC -DOVERRIDE_OPUS_FREE  -D'opus_alloc(x)=NULL' -D'opus_free(x)=NULL' "</tt>.
 +
 
 +
If libopus is built with <tt>-DNONTHREADSAFE_PSEUDOSTACK</tt> (instead of <tt>VAR_ARRAYS</tt>, or <tt>USE_ALLOCA</tt>), it will use a user-provided block of heap instead of stack for many things, resulting in much lower stack usage.<br>
 +
This makes the resulting library '''non-threadsafe''' and is '''not recommended''' on anything except limited embedded platforms.
 +
 
 +
=== How can I ensure that my software interoperates with other software implementing Opus? ===
 +
 
 +
For applications using Ogg files, there are some [https://people.xiph.org/~greg/opus_testvectors/ Ogg Opus testvectors] to test decoders and you can test encoders with opusdec. For RTP applications, the opusrtp tool can be useful.
 +
 
 +
In general, here's a list of specific issues to check:
 +
* Can your application handle all frame sizes, including changing the frame size from frame to frame?
 +
* Does your application properly react to lost packet by calling the decoder with a NULL packet?
 +
 
 +
=== What is the complexity of Opus? ===
 +
 
 +
The complexity of Opus varies by a large amount based on the settings used.
 +
 
 +
It depends on the mode, audio bandwidth, number of channels, and even a "complexity knob" that can trade complexity for quality. It will run easily on any recent PC or smartphone.
 +
 
 +
For slower embedded CPUs/DSPs, the amount of CPU required will vary depending on the configuration and the exact CPU, so you will need to experiment. Do not expect Opus to run quickly on really slow devices like 8-bit micro-controllers.
 +
 
 +
=== Opus is using too much CPU for my application. What can I do? ===
 +
 
 +
First don't panic and don't start writing assembly just yet.
 +
 
 +
It's possible that you're just not using the right set of options.
 +
 
 +
If you're targeting an embedded/mobile platform, chances are the fixed-point build will be faster, so make sure you're using '''--enable-fixed-point''' or defining '''FIXED_POINT''' in the build system.
 +
 
 +
Opus also has a complexity option that can trade quality for complexity. The default is highest quality and highest complexity. You can control this using '''OPUS_SET_COMPLEXITY()''' (see the '''[https://www.opus-codec.org/docs/ Documentation]''' for details).
 +
 
 +
If all else fails and you need to optimize the Opus code, see the next question.
 +
 
 +
=== I would like to optimize/improve/help with Opus. Where should I start? ===
 +
 
 +
Please '''[https://www.opus-codec.org/contact/ contact us]''' before you start, or at least before you get too far.
 +
 
 +
This will help coordinate the efforts made on Opus and reduce the probability of wasting your time on duplicated effort or going down the wrong path.
 +
 
 +
=== Does Opus have an echo canceller like Speex does? ===
 +
 
 +
Echo cancellation is completely independent from codecs.
 +
 
 +
You can use any echo canceller (including the one from libspeexdsp) along with Opus.
 +
 
 +
That being said, among the free acoustic echo cancelers (AEC) we're aware of, the best is probably the Google AEC from the [https://code.google.com/p/webrtc/ WebRTC codebase].
  
While the implementation in the latest IETF draft (and eventually RFC) of Opus is what ''defines'' the standard, it is likely not the best and most up-to-date implementation. The [http://opus-codec.org/ Opus] website was set up for the purpose of continually improving the implementation— in terms of speed, encoding quality, device compatibility, etc— while still conforming to the standard. All Opus implementations are compatible by definition.
+
[[Category:Opus]]

Latest revision as of 16:10, 12 May 2016

If you are looking for info not covered in this FAQ, try the main Opus website or the pages included in the Opus category of this wiki.

Opus logo trans.png

Contents

General Questions

What is Opus? Who created it?

Opus is a totally open, royalty-free, highly versatile audio codec.

It is primarily designed for interactive speech and music transmission over the Internet, but is also applicable to storage and streaming applications. It incorporates technology from Skype's SILK codec and Xiph.Org's CELT codec. It has been standardized by the Internet Engineering Task Force (IETF) as RFC 6716.

Opus has been in development since early 2007. Programmers associated with Xiph.Org, Skype and several other organizations have contributed to its development and to the standardization process as part of the IETF's Codec Working Group.

How does Opus compare to other codecs?

Opus is distinguished from most high quality formats (eg: AAC, Vorbis, MP3) by having low delay (5 ~ 66.5 ms) and distinguished from most low delay formats (eg: G.711, GSM, Speex) by supporting high audio quality (details here).

It meets or exceeds existing codecs' quality across a wide range of bitrates, and it operates at lower delay than virtually any existing compressed format.

Most importantly, the Opus format and its reference implementation are both available under liberal, royalty-free licenses.
This makes it:

  • easy to adopt
  • compatible with free software
  • suitable for use as part of the basic infrastructure of the Internet

See the Opus comparison page for more details.

Does Opus make all those other lossy codecs obsolete?

Theoretically, yes.

From a technical point of view (loss, delay, bitrates, ...) it should replace both Vorbis and Speex, and the common proprietary codecs too.

Will Opus replace Vorbis in video files?

For Ogg Theora video files, it can, just the overall size reduction will be minimal and it will break compatibility with existing players.

For WebM video files, the convention is to use the VP9 video codec when using Opus as an audio codec.

How do I use Opus?

For now, the best way to encode audio into Opus files is to use the opusenc command-line tool from the opus-tools package.

If you want to encode many files at once (e.g. your music library), try the applications listed in the Opus Support page.

For rough guidelines on encoding settings, see the Opus Recommended Settings page.

What programs support Opus?

Opus decoding support is now included in some Internet browsers and many applications, including Firefox, foobar2000 and VLC, as well as in frameworks such as GStreamer and FFmpeg.

For real-time applications, Opus support is available in Google's WebRTC codebase.

Opus is a relatively new codec: many more applications will support it in the near future.

Does Opus support higher sampling rates, such as 96 kHz or 192 kHz?

Yes and no.

Opus encoding tools like opusenc will happily encode input files that are sampled at 96 or 192 kHz.

However, files at these rates are internally converted to 48 kHz and then only frequencies up to 20 kHz are encoded.

The reason is simple: lossy codecs are designed to preserve audible details while discarding irrelevant information. Since the human ear can only hear up to 20 kHz at best (usually lower than that), frequency content above 20 kHz is the first thing to go.

See Monty's article for more details.

If you want a codec to handle higher sampling rates losslessly, use FLAC!

What are the licensing requirements?

The reference Opus source code is released under a three-clause BSD license, which is a very permissive Open Source license. Commercial use and distribution (including in proprietary software) is permitted, provided that some basic conditions specified in the license are met.

Opus is also covered by some patents, for which royalty-free usage rights are granted, under conditions that the authors believe are compatible with (hopefully) all open source licenses, including the GPL (v2 and v3).

See the Opus Licensing page for details.

Why make Opus free?

On the Internet, protocol and codec standards are part of the common infrastructure everyone builds upon.

Most of the value of a high-quality standard is the innovation and inter-operation provided by the systems built on top of it. When a few parties have monopoly rights to monetize a standard, that infrastructure stops being so common and everyone else has more reason to use their own solution instead, increasing cost and reducing efficiency.

Imagine a road system where each type of car could only drive on its own manufacturer's pavement. We all benefit from living in a world where all the roads are connected.

This is why Opus, unlike many codecs, is free.

Is the SILK part of Opus compatible with the SILK implementation shipped in Skype?

No.

The SILK codec, as submitted by Skype to the IETF, was heavily modified as part of its integration within Opus. The modifications are significant enough that it is not possible to just write a "translator". Even sharing code between Opus and the "old SILK" would be highly complex.

Why not keep the SILK and CELT codecs separate?

Opus is more than just two independent codecs with a switch.

In addition to a Linear Prediction SILK mode and an MDCT CELT mode it has a hybrid mode, where speech frequencies up to 8 kHz are encoded with LP while those between 8 and 20 kHz are encoded with MDCT. This is what allows Opus to have such high speech quality around 32 kbps.

Another advantage of the integration is the ability to switch between these 3 modes seamlessly, without any audible "glitches" and without any out-of-band signalling.

Now that Opus is standardized, will its development stop or can it be further improved?

Yes, Opus can and should be improved, because unlike most ITU-T codecs, Opus is only defined in terms of its decoder.

The encoder can keep evolving as long as the bitstream it produces can be decoded by the reference decoder. This is what made it possible for modern MP3 encoders (e.g. LAME) to improve far beyond the original L3enc and dist10 reference implementations.

Although it is unlikely that Opus encoders will see such a spectacular evolution, we certainly hope that future encoders will become much better than the reference encoder.

In fact, the 1.1 libopus release significantly improves on the reference encoder's quality. See Monty's demo for more details.

Will all future Opus releases comply with the Opus specification?

Yes.

In what ways is Opus optimized for the Internet?

Opus has good packet loss robustness and concealment, but its optimisations go further.

One of the first things we've been asked when designing Opus was to make the rate really adaptable because we never know what kind of rates will be available. This not only meant having a wide range of bitrates, but also being able to vary in small increments.

This is why Opus scales from about 6 to 512 kb/s, in increments of 0.4 kb/s (one byte with 20 ms frames). Opus can have more than 1200 possible bitrates while spending only 11 bits signalling the bitrate because UDP already encodes the packet size.

One last aspect is that Opus is simple to transport over RTP, as can be seen from the Opus RTP payload format. For example, it's possible to decode RTP packets without having even seen the SDP or any out-of-band signalling.

What applications for Android can play Opus?

Right now, there are just a few but that list is fast growing. Please reference this question on android.stackexchange.com. Feel free to suggest other applications.

When will the next version be released?

When it's done. Seriously, we do not know.

Opus is not a large project with a fixed release schedule.

That being said, our pre-releases and even the git repository are generally pretty stable and given proper testing (which you should always do anyway), are safe to distribute.

Just be aware that the API of new features (that have never been included in a stable release) could potentially still change.

Software Developers' Questions

On what platforms does Opus run?

The Opus code base is written in C89 and should run on the vast majority of recent (and not so recent) CPUs.

Some of the platforms on which Opus has been tested include x86, x86-64, ARM, Itanium, Blackfin, and SPARC.

Is there a fixed-point implementation?

Yes.

The fixed-point and floating-point decoder and encoder implementations are part of the same code base.

The code defaults to float, so you need to configure with --enable-fixed-point (or define FIXED_POINT if not using the configure script) to build the code for fixed-point.

Which implementation should I use?

While the implementation in RFC 6716 is what defines the standard, it is likely not the best and most up-to-date implementation.

The Opus website was set up for the purpose of continually improving the implementation — in terms of speed, encoding quality, device compatibility, etc — while still conforming to the standard.

All Opus implementations are compatible by definition.

How is supporting Opus different from supporting Speex/G.711/MP3?

Opus has variable frame durations which can change on the fly, so an Opus decoder needs to be ready to accept packets with durations that are any multiple of 2.5ms up to a maximum of 120ms.

The opus encoder and decoder do not need to have matched sampling rates or channel counts. It is recommended to always just decode at the highest rate the hardware supports (e.g. 48kHz stereo) so the user gets the full quality of whatever the far end is sending.

My application doesn't work. Can anyone help me?

It's possible to get help, but before doing so, there are a few basic things to try:

  • Implement your application with uncompressed audio instead of Opus. If it still doesn't work, then the problem isn't related to Opus.
  • Read the Opus documentation.
  • Read the opus_demo.c source code to see how to use the encoder and decoder.

If you still can't solve the problem, the best option is to ask for help on the mailing list or on the #opus IRC channel on irc.freenode.net.

How do I report a bug?

If you think you have found a bug in Opus (and not in your application), please file a bug report.

Please include a way for us to reproduce the problem. The best way to do this is to provide an input file, along with the opusenc/opusdec/opus_demo command line that causes the bug to occur.

If the bug cannot be triggered by the command line tools, please provide a simple patch or C file that can help reproduce it. Please also provide any other relevant information, such as OS, CPU, build options, etc.

Don't hesitate to also contact us on the mailing list or on IRC.

What is Opus Custom?

Opus Custom is an optional part of the Opus standard that allows for sampling rates other than 8, 12, 16, 24, or 48 kHz and frame sizes other than multiples of 2.5 ms.

Opus Custom requires additional out-of-band signalling that Opus does not normally require and disables many of Opus' coding modes. Also, because it is an optional part of the specification, using Opus Custom may lead to compatibility problems.

For these reasons, its use is discouraged outside of very specific applications, for example:

  • ultra low delay applications where synchronization with the soundcard buffer is important.
  • low-power embedded applications where compatibility with others is not important.

For almost all other types of applications, Opus Custom should not be used.

How do I use 44.1 kHz or some other sampling rate not directly supported by Opus?

Tools which read or write Opus should inter-operate with other sampling rates by transparently performing sample rate conversion behind the scenes whenever necessary. In particular, software developers should not use Opus Custom for 44.1 kHz support, except in the very specific circumstances outlined above.

Note that it's generally preferable for a decoder to output at 48kHz, even when you know the original input was 44.1kHz. This is not only because you can skip resampling, but also because many cheaper audio interfaces have poor quality output for 44.1kHz.

The opus-tools package source code contains a small, high quality, high performance, BSD licensed resampler which can be used where resampling is required.

How is the bitrate setting used in VBR mode?

Variable bitrate (VBR) mode allows the bitrate to automatically vary over time based on the audio being encoded, in order to achieve a consistent quality.

The bitrate setting controls the desired quality, on a scale that is calibrated to closely approximate the average bitrate that would be obtained over a large and diverse collection of audio. The actual bitrate of any particular audio stream may be higher or lower than this average.

What frame size should I use?

A 20ms frame size works well for most applications.

Smaller frame sizes may be used to achieve lower latency, but have lower quality at a given bitrate.

Sizes greater than 20 ms increase latency and are generally beneficial only at fairly low bitrates, or when used to reduce external overhead (e.g. by reducing the number of packets that are sent).

Forward Error correction (FEC) doesn't appear to do anything! HELP!

The in-band FEC feature of Opus helps reduce the harm of packet loss by encoding some information about the prior packet.

In order to make use of in-band FEC the decoder must delay its output by at least one frame so that it can call the decoder with the decode_fec argument on the next frame in order to reconstruct the missed frame. This works best if it's integrated with a jitter buffer.

FEC is only used by the encoder under certain conditions:

  • the feature must be enabled via the OPUS_SET_INBAND_FEC CTL
  • the encoder must be told to expect loss via the OPUS_SET_PACKET_LOSS_PERC CTL
  • the codec must be operated in any of the linear prediction or Hybrid modes

Frame durations shorter than 10ms and very high bitrates will use the MDCT modes, where FEC is not available.

Even when FEC is not used, telling the encoder about the expected level of loss will help it make more intelligent decisions. By default the implementation assumes there is no loss.

I can't use malloc or much stack on my embedded platform. How do I make Opus work?

A normal build of libopus only uses malloc/free in the _create() and _destroy() calls, making it safe for realtime use as long as the codec state is pre-created.

To build Opus without the references to malloc/free, you must:

  • use init() calls rather than create() calls in your application
  • compile with CFLAGS="-DOVERRIDE_OPUS_ALLOC -DOVERRIDE_OPUS_FREE -D'opus_alloc(x)=NULL' -D'opus_free(x)=NULL' ".

If libopus is built with -DNONTHREADSAFE_PSEUDOSTACK (instead of VAR_ARRAYS, or USE_ALLOCA), it will use a user-provided block of heap instead of stack for many things, resulting in much lower stack usage.
This makes the resulting library non-threadsafe and is not recommended on anything except limited embedded platforms.

How can I ensure that my software interoperates with other software implementing Opus?

For applications using Ogg files, there are some Ogg Opus testvectors to test decoders and you can test encoders with opusdec. For RTP applications, the opusrtp tool can be useful.

In general, here's a list of specific issues to check:

  • Can your application handle all frame sizes, including changing the frame size from frame to frame?
  • Does your application properly react to lost packet by calling the decoder with a NULL packet?

What is the complexity of Opus?

The complexity of Opus varies by a large amount based on the settings used.

It depends on the mode, audio bandwidth, number of channels, and even a "complexity knob" that can trade complexity for quality. It will run easily on any recent PC or smartphone.

For slower embedded CPUs/DSPs, the amount of CPU required will vary depending on the configuration and the exact CPU, so you will need to experiment. Do not expect Opus to run quickly on really slow devices like 8-bit micro-controllers.

Opus is using too much CPU for my application. What can I do?

First don't panic and don't start writing assembly just yet.

It's possible that you're just not using the right set of options.

If you're targeting an embedded/mobile platform, chances are the fixed-point build will be faster, so make sure you're using --enable-fixed-point or defining FIXED_POINT in the build system.

Opus also has a complexity option that can trade quality for complexity. The default is highest quality and highest complexity. You can control this using OPUS_SET_COMPLEXITY() (see the Documentation for details).

If all else fails and you need to optimize the Opus code, see the next question.

I would like to optimize/improve/help with Opus. Where should I start?

Please contact us before you start, or at least before you get too far.

This will help coordinate the efforts made on Opus and reduce the probability of wasting your time on duplicated effort or going down the wrong path.

Does Opus have an echo canceller like Speex does?

Echo cancellation is completely independent from codecs.

You can use any echo canceller (including the one from libspeexdsp) along with Opus.

That being said, among the free acoustic echo cancelers (AEC) we're aware of, the best is probably the Google AEC from the WebRTC codebase.