XiphWiki - User contributions [en]

OggCELT

2009-08-29T23:21:05Z

DavidSchleef: /* Ogg mapping (experimental) */

__NOTOC__
{{draft}}

CELT is an experimental audio codec for use in low-delay communication.

The name stands for "Code-Excited Lapped Transform". It applies some of the [[Wikipedia:CELP|CELP]] principles, but does everything in the frequency domain, which removes some of the limitations of CELP.

The CELT codec is meant to close the gap between [[Vorbis]] and [[Speex]] for applications where both high quality audio and low delay are desired. It is a

Current features include:

* '''Ultra-low latency (typically from 3 to 9 ms)'''
* Full audio bandwidth (44.1 kHz and 48 kHz)
* Stereo support
* Packet loss concealment
* Constant bit-rates from 32 kbps to 128 kbps and above
* A fixed-point version of the encoder and decoder

The [http://www.celt-codec.org/ CELT homepage] has all additional info as well as samples.

==CELT Tuning==

CELT currently provides three classes of tuning knobs.

===Bit allocation for the spectrum details===
The frequency-dependent allocation of bits used to represent the 'details' of each frame's spectrum is by far the most important set of tuneables in CELT.

There is a matrix in modes.c called "band_allocation". It has BITALLOC_SIZE lines and BARK_BANDS columns. Each line determines how many bits will be allocated to each critical band.

Even in CBR mode (the only mode currently supported) the available number of bits will change from frame to frame because the space consumed by the compressed spectral envelope is not constant. However, at a single constant bitrate the available number of bits does not change tremendously much from frame to frame, as a result most of the values in the matrix will have little to no effect at any particular bit-rate setting.

The particular bit-rate that a line is used for is not explicitly specified but is instead inferred by the actual bit consumption of that line.

When the amount of bits available is somewhere between the amounts provided by explicitly configured lines the encoder will interpolate between the two nearest matching lines to achieve the needed bit-rate.

===Minimum width of bands===
At low frequencies critical bands are too narrow to be useful so CELT imposes a minimum width for the bands.

The minimum width is defined in MDCT frequency bins and is controlled by MIN_BINS in modes.c.

If MIN_BINS is set to a small value there will be more bands and as a result more energy data to encode which hurts the encoder's efficiency. If MIN_BINS is set to a large value there may not be enough frequency-resolution to encode the spectral shape in a manner which conforms well to human perception.

===Band energy resolution===

The last set of parameters is the resolution of the quantizer used to encode the 'shape' of the spectrum of every frame. Ideally the shape of the spectrum would be encoded as as accurately as possible, but every bit used to accurately represent the shape of the spectrum is unavailable to represent the spectrum details. The sensitivity of the human perceptual system to the exact amount of energy in a critical band is limited, especially at higher frequencies.

The resolution is defined in the "frac" array in quant_bands.c. The values themselves are the number of subdivisions to use in 6 dB, i.e. for band i, the resolution used is 6/frac[i] dB. So the higher the value, the more accurate the encoding and the more bits it takes.

== Design decisions ==

* How do we allocate the fine energy bits?
* Do we have a fixed static bit allocation or do we transmit it in a header?
* Dynamic bit allocation?

== Ogg mapping (experimental) ==

Default field type: LITTLE ENDIAN unsigned integer

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1| Byte
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| codec_id: Identifier char[8]: 'CELT ' | 0-3
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | 4-7
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| codec_version: char[20] | 8-11
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | 12-15
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | 16-19
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | 20-23
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | 24-27
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| version_id | 28-31
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| header_size | 32-35
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| sample_rate | 36-39
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| nb_channels | 40-43
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| frame_size | 44-47
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| overlap | 48-51
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| bytes_per_packet | 52-55
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| extra_headers | 56-59
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

[[Category:Ogg Mappings]]

Summer of Code 2008

2006-04-27T01:54:35Z

DavidSchleef: /* Ogg Dirac mapping and integration in players */

== DRAFT ==

This is a draft proposal for Google Summer of Code projects with Xiph.org.

We need a primary and backup mentor volunteer for any project that is to become an official proposal.

=== Optimize Theora encoding/decoding speed, SSE/SSE2 ===
Work on MMX, SSE/SSE2 implementations of the crucial encoding and
decoding elements in libtheora. This could continue in theora-exp
where with ruik's work or consolidate theora-mmx, theora-oil work into
trunk. The results must still build cleanly on other archs and do
run-time capability detection.

''Mentor: Ralph Giles, backup: Jan Gerber, Mike Smith''

=== Theora reference encoder quality optimization ===
The libtheora encoder could make more use of some features present in the spec
but not currently implemented in the encoder. This is a little open ended, but
suggestions are: quant matrix tuning, per-block qi choice, 4:2:2 and 4:4:4 chroma
support.

''Mentor: Ralph Giles, backup: Timothy Terriberry''

=== Encode support in theora-exp ===
Implement a rate-distortion optimized encoding mode for theora-exp,
including R-D optimzed mode decision and quantization (e.g., constant
lambda). Then, use the above routines to implement a medium-latency ABR
encoding mode (e.g., varying lambda), with a default target buffer size
of approximately 2 seconds.

''Mentor: Timothy "Derf" Terriberry, backup: Mike Smith, Ralph Giles''

=== Development assistant for the "Ghost" audio codec ===
Designing a cutting edge perceptual codec is a very daunting task. Xiph
is in the research stage on a new low-latency, general purpose audio codec,
code-named "Ghost". This is basically a "code assistant" position, where you
will be asked to implement, test, and give feedback on ideas from Christopher
Montgomery, designer of the Ogg Vorbis format. Be prepared to learn a lot about
audio coding, or apply what you already know. While there's less "ownership"
potential in this project proposal, it will be a great opportunity to learn
about compression algorithm design, practice your programming chops, and learn
to work in team.

''Mentor: Christopher "Monty" Montgomery, backup: Jean-Marc Valin''

=== OggMNG implementation ===
Implement the OggMNG decode support in gstreamer and/or illi's dshow filters.
Implement encoding support in based on byzanz or istambul. Bonus points for
overlay support.

''Mentors: Mike Smith, Ralph Giles''

=== Subtitle Editor ===
this project would also consolidate the various proposals for
subtitles. from what i saw on #annodex,
using CMML might be the way to go. Coordinate with Annodex.net.

another option is to pick an existing format (e.g. srt) and get a
gui mockup done. It should: give you playback with scrubbing, let
you set in and out points, and write out the results in an Ogg File.

=== OggSkeleton tool support ===
Implement OggSkeleton production in various xiph tool packages. Also
implement some playback support at least one popular playback framework:
illi's dshow filters, gstreamer/totem, VLC, Quicktime, etc.

''Mentor: Conrad Parker, backup: Silvia Pfeiffer''

=== MXF support in gstreamer ===
Implement an MXF mux/demux for gstreamer, with mappings for Vorbis and Theora.
This should probably be a gstreamer project instead.

''Mentors: Christian Schaller, Mike Smith''

=== Hardware implementation of Theora decoding ===
Working on a hardware theora decoder, that can be used in embedded
devices, dvd players and video pods. Presumedly GPL verilog source
to run on an FPGA.

=== Ogg Dirac mapping and integration in players ===
this could be a) writing down the spec for ogg dirac that is floating
around on the dirac and carid mailing list and
implementing it in i.e. vlc or gstreamer.

j^ note: in whats state is ogg dirac right now anyway?

rillian note: OggDirac is pretty much waiting on schoedinger and a spec revision from Tim at the BBC. I'm not sure this would make a good project, but maybe ds would like help with something.

ds note: probably not a good project for working on the OggDirac definition itself, but implementing it in all the players/encoders out there could be.

=== Intel to AT&T x86 assembly translation ===
There is a general need for cross platform projects to be able to compile the same asm accelleration code on both GCC and MSVC. Unfortunately, at least of x86, they have incompatible assembly formats. Currently people either convert one to the other by hand (a maintenance nightmare) or require an external compile/assemble step on one or the other platform.

Start with the (unmaintained?) intel2gas script. Spruce it up to support all of recent MMX, SSE, SSE2, SSE3 instructions. Then implement the reverse translation. Once both are working, write some glue code so it can be easily used as part of a GNU autotools build to derive one set of source from the other at build or package time.