XiphWiki - User contributions [en]

GranulePosAndSeeking

2015-10-31T19:35:30Z

Daala-ts: /* But how do I "seek to the desired time"? */ Fix a minor grammatical error.

== Granulepos encoding and How seeking really works ==

This describes how to seek on a multiplexed stream which contains logical bitstreams with granuleshift, such as
Theora, Kate, CMML or OggText.
The purpose is to locate the earliest page that is required for rendering a given time offset.
Due to the fact that two time-seeking operations are required, this procedure is commonly referred to as a "double seek".

=== Definitions ===

Overload '''time''' to mean the time represented by a granulepos. Hence the time of a page is the time represented by the page granulepos.

Define '''seek''' to mean: for each logical bitstream, locate the bytewise-latest page in the bitstream with a time < the
target time, then choose the bytewise-earliest among these pages. Thus if two pages have the same time, seeking will locate the bytewise-earlier page.

==== Granules and Granuleshift ====

We use the term '''granule''' to refer to time measured in the units of the codec. For audio codecs this is usually samples, and for video codecs it is frames or fields.

In some formats, pages have a dependency on the data of an earlier page; for example in Theora, interframes have a dependency on an earlier keyframe -- the keyframe data is required to decode the interframe. We encode both the time of the page and the time of the page it depends on into the granulepos. In order to do this we treat the granulepos as a bitfield as follows:

+---------------------+-------------+
| prev_granule | offset |
+---------------------+-------------+

Then if a page has time in units of codec granules <tt>curr_granule</tt>, and the page it depends on has time
<tt>prev_granule</tt>, we define <tt>offset</tt> as the difference between these:

offset = curr_granule - prev_granule

We refer to the number of bits used to encode the offset as the "granuleshift". This is fixed for all pages in
that track (logical bitstream). So we encode the later page's granulepos as:

granulepos = (prev_granule << granuleshift) | offset

When decoding, we can extract the current_granule from a granulepos by simply adding these fields:

curr_granule = prev_granule + offset

Which expands to this expression of the page granulepos:

curr_granule = (granulepos >> granuleshift) + (granulepos & ((1 << granuleshift) - 1)))

Keyframes, and other data with no dependency on earlier packets, are encoded with:

prev_granule = curr_granule, offset = 0

=== Procedure for seeking on a single track ===

In order to locate the earliest page in a track (a logical bitstream)
that is required for rendering a given time offset:

# seek to the desired time
# read the prev_granule out of the granulepos
# seek to the time represented by the prev_granule

=== Procedure for seeking on a multitrack file ===

In order to locate the earliest page in a multitrack file (a physical
bitstream) that required for rendering all tracks from a given time
offset:

# seek to the desired time
# scan forward until a page has been seen from all of the tracks that use granuleshift; while doing so, record the prev_granule of the bytewise-earliest page encountered from each track
# seek to the minimum of the prev_granules of those pages

It is useful to put a bound on the forward scan; the distance scanned
only depends on the way the stream is constructed, so it can be large
if pages in a particular logical bistream is sparse.

=== But how do I "seek to the desired time"?===
The above assumes that you already know how to seek to a particular granpos within the stream efficiently.

This isn't as simple as it sounds because the Ogg format does not include an index. The lack of an index is a feature rather than a deficiency and it is one of the primary reasons to use Ogg over some other formats. Because Ogg doesn't have in index infinite streams and partial streams are automatically supported by correctly written applications. There is no risk of truncation or minor corruption making a stream unseekable. No memory is required to store an index, no bandwidth is wasted to transmit it, and seeking granularity is not limited to the precision of the index. On the other hand non-indexed formats can require a little more intelligence in the application and many applications have gotten it wrong (although this intelligence is also needed in a well written application fo indexed formats so that it can seek with a corrupted index or below the index granularity).

If you are thinking about seeking in an Ogg file by first building your own complete index: STOP. This is not the correct procedure. It seems simple but it requires a costly read of the entire stream (which may be gigabytes in size, or even infinite). There is a better way.

The correct way to seek to a particular granule value in Ogg is by using a [http://en.wikipedia.org/wiki/Bisection_method bisection search]: Seek to the middle of the stream, obtain sync, and compare your target granule position with the current position. If the target is less than the current position repeat these steps on the left side, if it's greater repeat it on the right side. By applying this recursive algorithm you are guaranteed to find your target location quickly.

To correctly support chaining you should first use this kind of search to locate the stream endpoints. Then the above approach can be applied within the streams to seek to any location within a chained file.

Doing this correctly is somewhat more complicated than it seems due to the existence of continued pages and the risk of a small valid page being contained within a packet. Both of these challenges can be addressed but the solution is left as an exercise for the reader. 

The bisection search is very good compared to the obvious alternatives (a linear scan of the whole file), often taking just a couple of reads to locate the correct location in a file gigabytes in size but the truly obsessive can out-perform the bisection on average by using the local bitrate to pick a better target than the half way point used in a bisection search ([http://en.wikipedia.org/wiki/Secant_method secant method] but be careful about the worst case becoming linear; see [http://en.wikipedia.org/wiki/Brent%27s_method brents method]). The improvement possible from better-than-bisection approaches is probably only relevant for seeking across a high latency network. In typical applications the added complexity may not be worth the cost.

== References ==

From an Email by Monty, [http://web.archive.org/web/20031201054855/http://www.xiph.org/archives/theora-dev/200209/0040.html 13th Sept 2002]

'''Note that this document is obsolete, and incorrect with respect to seeking in multiplexed streams.''' It does accurately describe the rationale behind the two-part granulepos scheme (option 3 below) now use in Theora, Dirac, CMML and other codecs in Ogg.

Folks have noticed that the documentation is semi-silent about how to properly encode the granule position and interleave synchronization of keyframe-based video. The primary reasons for this:

* we at Xiph hadn't had to do it yet

* there are several easy possibilities, and the longer we had to think about it before mandating One True Spec, the better that spec would likely be.

The lack of a painfully explicit spec has led to the theory that it's not possible; that's not true, there are a few ways to do it. Several require no extension to Ogg stream v 0. A last way requires an extra field (a point against it), but does not actually break any stream that currently exists.

The time has come to lay down the spec as we're currently building the real abstraction layers in a concrete Ogg framework now where the Ogg engine, the codecs, and the overarching Ogg control layers are neatly put into boxes connected in formalized ways. Below I go into detail about each scheme in a 'thinking aloud' sort of way. This is not because I haven't already given the matter sufficient thought, it is because I wish to give the reader sufficient background information to understand why one way is better than the others. This is not a call for input so much as an educational effort (and a public sanity check of my thinking; please do pipe up if it appears I missed a salient point).

==== Starting Assumptions: ====

1) Ogg is not a non-linear format. It is not a replacement for the scripting system of a DVD player. It is a media transport format
designed to do nothing more than deliver content, in a stream, and have all the pieces arrive on time and in sync. It is not designed to *prevent* more complex use of content, it merely does not implement anything beyond a linear representation of the data contained within. If you want to build a real non-linear format, build it *from* Ogg, not *into* Ogg. This has been the intent from day 1.

2) The Ogg layer does not know specifics of the codec data it's multiplexing into a stream. It knows nothing beyond 'Oooo, packets!', that the packets belong to different buckets, that the packets go in order, and that packets have position markers. Ogg does not even have a concept of 'time'; it only knows about the sequentially increasing, unitless position markers. It is up to higher layers which have access to the codec APIs to assign and convert units of framing or time.

3) Given pre-cached decode headers, a player may seek into a stream at any point and begin decode. It may be the case that audio may start after video by a fraction of a second, or video might be blank until the stream hits the next keyframe, but this simplest case must just work, and there will be sufficient information to maintain perfect cross-media sync.

4) (This departs from current reality, but it will be the reality very soon; vorbisfile currently blurs the careful abstraction I'm about to describe) Seeking at an arbitrary level of precision is a distributed abstraction in the larger Ogg picture. At the lowest-level Ogg stream abstraction, seeking is one operation: "find me the page from logical stream 'n' with granule position 'x'". All more complex seeking operations are a function of a higher-level layer (with knowledge of the media types and codec in use) making intelligent use of this lowest Ogg abstraction. The Ogg stream abstraction need deal with nothing more complex than 'find this page'.

The various granulepos strategies for keyframes concern this last point.

The basic issue with video from which complexity arises is that frames often depend on previous and possibly future frames. This happens in a larger, general category of codecs whose streams may not begin decode from just any packet as well as packets that may not represent an entire frame, or even a fixed-time sampling algorithm. It is a mistake to design a seeking system tied to an exact set of very specific cases. While one could implement an explicit keyframe mechanism at the Ogg level, this mechanism would not cover any of the other interesting seeking cases while, as I'll show below, the mechanism would not actually be necessary.

There will be a few complaints that Ogg is being unnecessarily subtle and shifts a great deal of complexity into software which a few extra page header fields could eliminate. Consider the following:

1) Ogg was designed to impose a roughly .5-1% over the raw packet data over a wide range of packet usage patterns. 'A few extra fields' begins inflating that figure for specific special cases that only apply to a few stream types. Right now there is no header field that is not general to every stream. There is no fat in the page headers.

2) The Ogg-level seeking algorithm is exceptionally simple and can be described in a single sentence: "Find the earliest page with a granulepos less than but closest to 'x'". This shifts the onus of assembling more complex seeking operation requiring knowledge of a specific media type into a higher layer that has knowledge of that media type. The higher layer becomes responsible for determining for what 'x' Ogg should search. The division of labor is clear and
sensible.

3) Complex, precise seeking operations are still contained entirely within the framework, just at a higher layer than Ogg-stream. At no time is an application developer required to deal with seeking mechanisms within an Ogg stream or to manually maintain stream
synchronization.

==== High level handwaving- How seeking really works ====

The granulepos is intended to mean, roughly, 'If I stop decode at the end of this page, I will get data from my decoder up to position 'granulepos'. The granulepos simultaneously provides seeking information and a 'length-of-stream' indicator. Depending on the codec, it can also usually be used to indicate a timebase, but that isn't our problem right now.

By inference, the granulepos is also used to construct a value 'y' such that 'if I begin decode *from* point 'y', I will get data
beginning at position 'granulepos'. Although in some codecs, y == granulepos, that is not necessarily the case when decode can't begin at any arbitrary packet. The granulepos encoding method candidates I will now describe affect exactly the 'granulepos' to 'y' conversion process. Note also that none of these affect Ogg, only the higher decision-making layers... Different circumstanced necessitated by different codecs can lead to different valid choices, all of which work as far as Ogg is concerned. However, for our I-/P-/B-frame video case, there is a pretty clear winner.

===== Strategy 1: Straight Granulepos, Keyframes Are Not Our Problem. =====

In this scheme, the granulepos is a simple frame counter. The seeking decision-maker in the codec's framework plugin is responsible for determining if a frame is a keyframe or not, and if it can't begin decode from a given frame, it must request another earlier frame until it finds a keyframe. If the codec so desires, it can store 'what is my keyframe?' information in the stream packets.

This case means that each seek to a *specific* frame in a video stream will generally result in two Ogg seeks; a first seek to the the requested frame, then a second seek backwards to find that frame's keyframe.

A larger concern is the semantic accuracy of the granulepos; it's intended to reflect position accurately when decoding forward. In this scheme, it's fine for a P-frame to update the counter (as it can be decoded going strictly forward), but B frames will also advance the counter; they can't be decoded without subsequent P or I frames. Thus, the semantic value of granulepos no longer strictly represents 'we can decode up to 'granulepos' at the end of this frame'.

===== Strategy 2: Granulepos Represents Keyframes Only =====

In this scheme, only keyframes update the granulepos (monotonically or non-monotonically). It simplifies the seeking process to a keyframe as an Ogg-level seek to page 'x' will always yield a page with a keyframe. In addition, granulepos will also always mean 'we can decode up to *at least* this point in the stream. If the stream is truncated at P or B frames past granulepos, the extra frames can be discarded. (A special case would need to be defined to terminate a stream that doesn't end on an I frame).

The difficulty with this scheme is that it presents slightly more for the software level decoder to track; a proper frame number could not be determined internally without tracking from an I frame. Also, the granulepos an Ogg page would not necessarily map to the last packet on the page, or even any packet on that page; multiple sequential pages could have the same granulepos. It is conceptually slightly messy, although the 'messiness' does not make it at all impractical.

===== Strategy 3: Granulepos Encodes Some State =====

In some ways, this strategy is the most semantically 'over clever', but also the easiest to implement and the one that gives the most correct, up to date sync information. Pending comments, it is the I/P/B video strategy I currently favor.

The granulepos is 64 bits, a size that is absolutely necessary if, for example, it represents the PCM sample count in an audio codec. When being used to encode video frame number, however, it is comparatively absurdly large*.

* note that although granulepos is not permitted to wrap around, we can simply begin a new logical stream segment with a new serial number should a 30fps video stream ever hit the ten-billion year mark.

Thus we clearly have room to skim a few bits off the bottom of granulepos to represent I, P or B frame. These bits are not used as flags, but rather, frame representation becomes a counting problem; We do this such that the count is still always strictly increasing.

For example, we know that I frames will never be more than 256 frames apart and P frames no more than 31 B frames apart, the granulepos of an I frame can be defined to always be granulepos | 0xff == 0. If we can have up to seven intervening P frames, they could be numbered in granulepos-of-iframe + 0x20, 0x40, 0x60... 0xe0. B frames between the I and P frames would use the remaining five bits and be numbers as sub-I and sub-P frames 1 through 31. Thus, starting from zero, the frames/packets in the pattern IPBBPBBI would be numbered 0x000, 0x020, 0x021, 0x022, 0x040, 0x041, 0x042, 0x100.

If we wish to preserve the ability to represent a timebase, the granulepos number for I frames need not be increased monotonically and shifted; it can be used to represent the frame number. The above example becomes 0x000, 0x020, 0x021, 0x022, 0x040, 0x041, 0x042, 0x700. To get real frame number (from an I frame), we just shift granulepos >> 8. This scheme can be taken further or modified to get frame number from any video frame.

In this way, we can always seek, first time, to a desired key frame page (by seeking to Ogg page 'x' where x | 0xff == 0). In addition, each frame still has a unique frame number and also a clear 'group' number, potentially useful information to the decoder. Lastly, granulepos is still semantically correct, although it is now, in a sense, representing a whole.fractional frame number for buffering purposes.

===== Scheme Four: Extra 'Seekpos' Field / Straw Man =====

Another possibility requires extension of the current Ogg page format. Although older players would reject any such extended pages as invalid, we do have versioning and typing fields, so there's not actually any compatibility problems with current Ogg pages... in the future.

The idea in this scheme is to keep the current granulepos as a frame number field (ala scheme 1), but also add a new field 'seekpos' that is used, rather than granulepos, in seeking. The seekpos would represent the number of the last keyframe that passed by.

advantages:

1) The net effect of this strategy is to modify scheme 1 to only require one bisection seek rather than two. Some amount of code simplification (over scheme 1) at the decision-making level.

disadvantages:

1) The Ogg format will need to be revved. No current (ala 1.0) Ogg code will understand the new pages.

2) The header becomes larger, from a minimum size of 27 bytes to a minimum size of 35.

3) This strategy only enhances keyframes; it is of no use in other odd seeking cases.

4) Gives no more information than scheme 3, but is still more complicated, both in code and API (Ogg would have to understand keyframes).

Thus, there's no substantial reason to prefer extending the format over a scheme that's possible within the existing framework. Note that schemes 1-3 can all be implemented within the Ogg stream today.

Monty

[[Category:Ogg]]

Daala

2015-10-16T13:38:55Z

Daala-ts: /* Communication */ Update wiki to account for the loss of Etherpad.

Daala is the codename for a new video compression technology.

The effort is a collaboration between the [https://www.mozilla.org/en-US/research/ Mozilla Foundation], the [https://www.xiph.org/ Xiph.Org Foundation] and any other contributors that wish to help.

The goal of the project is to provide a video format that's free to implement, use and distribute, and a reference implementation with technical performance superior to [https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding H.265].

Please see the links below or the [https://www.xiph.org/daala/ main page] for more information.

== Wiki Pages ==
* [[Daala Quickstart|Daala Quickstart (Linux/MacOS)]]
* [[Daala Quickstart Windows|Daala Quickstart (Windows)]]
* [[Daala MinGW64 Environment]]

* [[Daala Weekly Meetings|Daala Weekly Meetings]]

* [[AreWeCompressedYet]]
* [[RD Curve Data Format]]

* [[DaalaTodo|Daala To-do List]]
* [[DaalaRoadmap|Daala Roadmap]]

* [[Intra|Intra-prediction within Daala]]

* [[Videos|Digital Primers]] - educational videos about audio/video technology

== Communication ==
You are '''encouraged''' to join the
* [irc://irc.freenode.net/daala '''#daala''' IRC channel at freenode.net] - if you don't have an IRC client, you can use Freenode's '''[https://webchat.freenode.net/?channels=%23daala webchat]''' instead.
* [http://lists.xiph.org/mailman/listinfo/daala Daala Email List]

=== Weekly Meetings ===
You are also welcome to attend the public [[Daala Weekly Meetings|weekly progress meetings]] by installing and using [http://wiki.mumble.info Mumble]. 
The address is '''mf4.xiph.org''' and the port is '''64738''' (you can run '''mumble://mf4.xiph.org:64738''' within your browser as a shortcut). 
The meetings occur on '''Tuesdays''' at '''[http://www.timeanddate.com/worldclock/fixedtime.html?msg=Daala+Weekly+Meeting&iso=20150428T09&p1=1241 9AM Pacific Time]''' (5PM UTC/GMT).
The meeting agenda used to be available at '''[https://daala.etherpad.mozilla.org/weekly-meeting this Etherpad]''', but Mozilla took their Etherpad down and is transitioning to Etherpad lite. In the meantime, the October 13, 2015 meeting is available on [https://docs.google.com/document/d/1JP_Ko3wPuyDWhooZcp_m9kndyfZ75xN5YOi5yIMCW0s/edit?pli=1 Google Docs].

=== Other ===
* [http://forum.doom9.org/showthread.php?t=168004 Doom9 Forum discussion] - generic forum thread regarding Daala
* [https://daala.etherpad.mozilla.org/ep/padlist/all-pads Daala Etherpads] - you can [https://daala.etherpad.mozilla.org/ep/account/request-account request a free account] to view these. You should receive access within a few days.
* [http://benjamin.smedbergs.us/weekly-updates.fcgi/project/daala Daala Project Status Board] - what Daala bits the Mozilla people are working on

== Coding ==
You can get a copy of the latest Daala Source Code from [https://git.xiph.org/?p=daala.git;a=summary '''git.xiph.org'''] or [https://github.com/xiph/daala '''GitHub''']. Please stick to the '''[https://git.xiph.org/?p=daala.git;a=blob_plain;f=doc/coding_style.html Coding Style Guide]'''.

* [https://review.xiph.org/all?limit=100 Xiph Code Reviews] - there is a proposal on the review process '''[[DaalaReview|here]]'''
* [https://github.com/xiph/daala/issues Daala's issues] - Issue/bug tracker on Github
* [https://mf4.xiph.org/jenkins/view/daala/ Continuous Integration Tests] - these run every time a new commit is made to the Daala git master, to make sure the new code hasn't broken existing functionality.

== Demos ==
* [https://people.xiph.org/~xiphmont/demo/daala/player-demo.shtml Daala Video Player] - an example implementation of a Daala decoder and player, ported to Javascript using [https://github.com/kripken/emscripten Emscripten].

=== Codec Techniques ===
* [https://people.xiph.org/~xiphmont/demo/ Demo Articles] - explanations on certain techniques used in Daala (and other Xiph.Org projects)
* [http://exp.martres.me/edi/ Edge-Directed Interpolation] ([https://github.com/smarter/edi source code])
* [https://people.xiph.org/~ds/edi/info.html More Edge-Directed Interpolation]
* [https://people.xiph.org/~unlord/demo/intra.html Intra-prediction]
* [https://people.xiph.org/~unlord/zigzags.html Macroblock Coefficient Zigzag Graph] - HTML page generated using [https://github.com/xiph/daala/blob/master/tools/draw_zigzags.c tools/draw_zigzags.c] from the Daala source code.

== Documents ==
* [https://people.xiph.org/~unlord/spie_cfl.pdf Chroma from Luma (CfL)]
* [http://jmvalin.ca/papers/spie_pvq.pdf Perceptual Vector Quantisation (PVQ)]
* [https://people.xiph.org/~tterribe/daala/vbsobmc.pdf Overlapped Block Motion Compensation (OBMC)]
* [https://mf4.xiph.org/jenkins/job/daala-autotools/ws/doc/html/index.html C API Documentation]
* [https://people.xiph.org/~yushin/tmp__/yushin_phd_thesis.pdf Image Coding Thesis] by Yushin Cho
* [http://arxiv.org/pdf/1411.4290v1.pdf Maximising Coding Efficiency Through Block Rotation] and why it [http://lists.xiph.org/pipermail/daala/2015-January/000054.html won't work well within Daala]
* [http://jmvalin.ca/video/theoretical_results.pdf JMSpeex' Journal of Dubious Theoretical Results] - "take with an entire shaker-full of salt"
* [https://people.xiph.org/~unlord/pcs_daala.pdf Using Daala Intra Frames for Still Picture Coding]

=== IETF Drafts ===
* [https://tools.ietf.org/html/draft-egge-videocodec-tdlt Time-Domain Lapped Transforms (TDLT)] - documents the Lapped Transform pre- and post-filters used for block-edge decorrelation
* [https://tools.ietf.org/html/draft-valin-videocodec-pvq Perceptual Vector Quantisation (PVQ)] -
* [https://tools.ietf.org/html/draft-terriberry-codingtools Coding Tools] - documents Entropy Coding, Integer Transforms and other techniques
* [https://tools.ietf.org/html/draft-moffitt-netvc-requirements Internet Video Codec (NetVC) Requirements] - explains what requirements and use cases Daala is trying to cater for
* [https://tools.ietf.org/html/draft-daede-netvc-testing Internet Video Codec (NetVC) Testing and Quality Measurement]
* [https://tools.ietf.org/html/draft-terriberry-ipr-license Example IPR Licence Terms]

Additional drafts can be found at the [https://datatracker.ietf.org/wg/netvc/documents/ IETF DataTracker].

== Presentations ==
* 2015-09-19 VideoLAN Dev Days - [https://www.youtube.com/playlist?list=PLQLpBN3oI7E44HIdTOovThc1MNHLchgHE YouTube Playlist]
* 2015-07-22 IETF 93 - [http://recordings.conf.meetecho.com/Playout/watch.jsp?recording=IETF93_NETVC_II&chapter=chapter_1 NetVC Session 2/2] - [https://datatracker.ietf.org/meeting/93/agenda/netvc/ Agenda] - [http://www.meetecho.com/ietf93/netvc_II Video and Chat] - Slides - [https://www.ietf.org/jabber/logs/netvc/2015-07-22.html Jabber Log]
* 2015-07-20 IETF 93 - [http://recordings.conf.meetecho.com/Playout/watch.jsp?recording=IETF93_NETVC&chapter=chapter_1 NetVC Session 1/2] - [https://datatracker.ietf.org/meeting/93/agenda/netvc/ Agenda] - [http://www.meetecho.com/ietf93/netvc Video and Chat] - [https://www.ietf.org/proceedings/93/slides/slides-93-netvc-3.pdf Slides] - [https://www.ietf.org/jabber/logs/netvc/2015-07-20.html Jabber Log]
* 2015-03-24 IETF 92 - [http://recordings.conf.meetecho.com/Playout/watch.jsp?recording=IETF92_NETVC&chapter=chapter_0 NetVC Session] - Audio as [https://people.xiph.org/~tdaede/audio/ietf92-venetian-20150324-0900-am1.opus Opus] (29MB) or [https://www.ietf.org/audio/ietf92/ietf92-venetian-20150324-0900-am1.mp3 MP3] (119MB, action starts at 14:50) - [https://www.ietf.org/proceedings/92/slides/slides-92-netvc-0.pdf Slides] - [https://www.ietf.org/mail-archive/web/video-codec/current/msg00235.html Notes] - [https://www.ietf.org/jabber/logs/netvc/2015-03-24.html Jabber Log]
* 2015-02-11 SPIE talks:


** [http://people.xiph.org/~tdaede/video/SPIE_Nathan.webm Chroma from Luma (CfL)] - [https://people.xiph.org/~unlord/SPIE-2015-CfL.pdf Slides] - [https://people.xiph.org/~unlord/spie_cfl.pdf Paper]
** [http://people.xiph.org/~tdaede/video/SPIE_PVQ.webm Perceptual Vector Quantisation (PVQ)] - [http://people.xiph.org/~tterribe/daala/spie_pvq_slides.pdf Slides] - [http://jmvalin.ca/papers/spie_pvq.pdf Paper]
** [http://people.xiph.org/~tdaede/video/SPIE_Tim.webm Overlapped Block Motion Compensation (OBMC)] - [http://people.xiph.org/~tterribe/daala/spie_pvq_slides.pdf Slides] - [https://people.xiph.org/~tterribe/daala/vbsobmc.pdf Paper]
* 2015-01-31 [http://ftp.osuosl.org/pub/fosdem/2015/devroom-open_media/daala.mp4 Daala Project Update at FOSDEM 2015] - [https://fosdem.org/2015/schedule/event/daala/ summary] - [https://fosdem.org/2015/schedule/event/daala/attachments/slides/569/export/events/attachments/daala/slides/569/Daala_FOSDEM_2015.pdf Slides]
* 2015-01-14 [https://www.youtube.co.uk/watch?v=Dmho4gcRvQ4 Linux Conf 2015] - [http://lca2015.linux.org.au/schedule/30187/view_talk presentation summary] - [https://people.xiph.org/~tterribe/pubs/lca2015/daala.pdf Slides]
-------
* 2014-09-16 [https://air.mozilla.org/daala-are-we-compressed-yet/ Daala: Are We Compressed Yet?]
* 2014-06-25 [https://air.mozilla.org/sparsity-induced-prediction-for-images-and-video/ Sparsity Induced Prediction for Images and Video]
* 2014-06-06 VP9 Summit (no video available) - [https://people.xiph.org/~xiphmont/demo/daala/daala-vp9summit-20140606.pdf Slides]
-------
* 2013-10-23 [https://people.xiph.org/~xiphmont/video/Free_Codecs_Update_Opus_and_Daala.ogv Opus and Daala: State of the Art Royalty-free Codecs] - [https://people.xiph.org/~greg/gstreamer-daala-opus.pdf Slides]
* 2013-09-30 [https://people.xiph.org/~tterribe/daala/coding_party2/?C=M;O=A Daala Coding Party 2] - [https://people.xiph.org/~unlord/Daala-Intra.pdf Slides]
* 2013-05-02 [https://people.xiph.org/~xiphmont/tim-terriberry-presents-daala/ Tim Terriberry Presents Daala]
-------
* 2012-01-24 [https://media.basilgohar.com/derf-talks/?C=M;O=A Introduction to Video Coding] - [https://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf Slides] (no video for slides 1-50)

== Other Websites ==
* [https://www.zazzle.com/daala_tee_shirt-235139149596175944 Daala T-shirts] - if you'd like a free one, help out with the project and ask the Mozilla guys nicely for one :-)
* [https://www.xiph.org/donate/ Donate to Xiph.Org]
* [[Daala_on_Wheels|Historical Daala wiki page]]

[[Category:Daala]]

Daala and Cygwin

2015-04-15T00:24:31Z

Daala-ts: Add code tags (I thought I did that already. Did I overwrite one of my edits?)

This is a guide on how to build Daala on Windows with [https://cygwin.com/ Cygwin]. Here's a different guide about this, on which this one is based: [http://nwgat.ninja/compiling-daala-on-cygwin/ Compiling Daala on Cygwin x64]

==Installing Cygwin==

Download and run [https://cygwin.com/setup-x86_64.exe https://cygwin.com/setup-x86_64.exe].

===Install apt-cyg===

wget rawgit.com/XadillaX/apt-cyg/master/apt-cyg
install apt-cyg /bin

==Install Packages==

Install packages necessary to build Daala:

apt-cyg install gcc-core make pkg-config autoconf automake libtool libogg-devel libpng-devel libjpeg-devel check git

==Compile Daala==

git clone https://git.xiph.org/daala.git
cd daala
./autogen.sh
./configure --disable-player
make -j22
make tools

If the compile fails, try commenting out these lines in configure.ac:
CFLAGS="-std=c89 -pedantic -Wall -Wextra -Wno-parentheses -Wno-long-long -Wno-overlength-strings $CFLAGS"

CC_CHECK_CFLAGS_APPEND([-U__STRICT_ANSI__])

by putting a # on front of the lines in configure.ac file

Note: make tools will fail if sdl is missing.

Daala and Cygwin

2015-04-15T00:23:16Z

Daala-ts: Minor writing change

This is a guide on how to build Daala on Windows with [https://cygwin.com/ Cygwin]. Here's a different guide about this, on which this one is based: [http://nwgat.ninja/compiling-daala-on-cygwin/ Compiling Daala on Cygwin x64]

==Installing Cygwin==

Download and run [https://cygwin.com/setup-x86_64.exe https://cygwin.com/setup-x86_64.exe].

===Install apt-cyg===

wget rawgit.com/XadillaX/apt-cyg/master/apt-cyg

install apt-cyg /bin

==Install Packages==

Install packages necessary to build Daala:

apt-cyg install gcc-core make pkg-config autoconf automake libtool libogg-devel libpng-devel libjpeg-devel check git

==Compile Daala==

git clone https://git.xiph.org/daala.git

cd daala

./autogen.sh

./configure --disable-player

make -j22

make tools

If the compile fails, try commenting out these lines in configure.ac:
CFLAGS="-std=c89 -pedantic -Wall -Wextra -Wno-parentheses -Wno-long-long -Wno-overlength-strings $CFLAGS"

CC_CHECK_CFLAGS_APPEND([-U__STRICT_ANSI__])

by putting a # on front of the lines in configure.ac file

Note: make tools will fail if sdl is missing.

Daala and Cygwin

2015-04-15T00:02:15Z

Daala-ts: Explain reference better

This is a guide on how to build Daala on Windows with [https://cygwin.com/ Cygwin]. Here's a different guide about this, on which this guide is based: [http://nwgat.ninja/compiling-daala-on-cygwin/ Compiling Daala on Cygwin x64]

==Installing Cygwin==

Download and run [https://cygwin.com/setup-x86_64.exe https://cygwin.com/setup-x86_64.exe].

===Install apt-cyg===

wget rawgit.com/XadillaX/apt-cyg/master/apt-cyg

install apt-cyg /bin

==Install Packages==

Install packages necessary to build Daala:

apt-cyg install gcc-core make pkg-config autoconf automake libtool libogg-devel libpng-devel libjpeg-devel check git

==Compile Daala==

git clone https://git.xiph.org/daala.git

cd daala

./autogen.sh

./configure --disable-player

make -j22

make tools

If the compile fails, try commenting out these lines in configure.ac:
CFLAGS="-std=c89 -pedantic -Wall -Wextra -Wno-parentheses -Wno-long-long -Wno-overlength-strings $CFLAGS"

CC_CHECK_CFLAGS_APPEND([-U__STRICT_ANSI__])

by putting a # on front of the lines in configure.ac file

Note: make tools will fail if sdl is missing.

Daala and Cygwin

2015-04-14T23:54:49Z

Daala-ts: Annotate link.

This is a guide on how to build Daala on Windows with [https://cygwin.com/ Cygwin]. Here's a different guide about this: [http://nwgat.ninja/compiling-daala-on-cygwin/ Compiling Daala on Cygwin x64]

==Installing Cygwin==

Download and run [https://cygwin.com/setup-x86_64.exe https://cygwin.com/setup-x86_64.exe].

===Install apt-cyg===

wget rawgit.com/XadillaX/apt-cyg/master/apt-cyg

install apt-cyg /bin

==Install Packages==

Install packages necessary to build Daala:

apt-cyg install gcc-core make pkg-config autoconf automake libtool libogg-devel libpng-devel libjpeg-devel check git

==Compile Daala==

git clone https://git.xiph.org/daala.git

cd daala

./autogen.sh

./configure --disable-player

make -j22

make tools

If the compile fails, try commenting out these lines in configure.ac:
CFLAGS="-std=c89 -pedantic -Wall -Wextra -Wno-parentheses -Wno-long-long -Wno-overlength-strings $CFLAGS"

CC_CHECK_CFLAGS_APPEND([-U__STRICT_ANSI__])

by putting a # on front of the lines in configure.ac file

Note: make tools will fail if sdl is missing.

Daala and Cygwin

2015-04-14T23:53:32Z

Daala-ts: Clean up attribution to nwgat.

This is a guide on how to build Daala on Windows with [https://cygwin.com/ Cygwin]. Here's a different guide about this: [http://nwgat.ninja/compiling-daala-on-cygwin/ Compiling Daala on Cygwin x64]

==Installing Cygwin==

Download and run [https://cygwin.com/setup-x86_64.exe].

===Install apt-cyg===

wget rawgit.com/XadillaX/apt-cyg/master/apt-cyg

install apt-cyg /bin

==Install Packages==

Install packages necessary to build Daala:

apt-cyg install gcc-core make pkg-config autoconf automake libtool libogg-devel libpng-devel libjpeg-devel check git

==Compile Daala==

git clone https://git.xiph.org/daala.git

cd daala

./autogen.sh

./configure --disable-player

make -j22

make tools

If the compile fails, try commenting out these lines in configure.ac:
CFLAGS="-std=c89 -pedantic -Wall -Wextra -Wno-parentheses -Wno-long-long -Wno-overlength-strings $CFLAGS"

CC_CHECK_CFLAGS_APPEND([-U__STRICT_ANSI__])

by putting a # on front of the lines in configure.ac file

Note: make tools will fail if sdl is missing.

Daala and Cygwin

2015-04-14T23:49:52Z

Daala-ts: General cleanup, wikifying

This is a guide on how to build Daala on Windows with [https://cygwin.com/ Cygwin].
CC-BY [http://nwgat.ninja/compiling-daala-on-cygwin/ nwgat]

==Installing Cygwin==

Download and run [https://cygwin.com/setup-x86_64.exe].

===Install apt-cyg===

wget rawgit.com/XadillaX/apt-cyg/master/apt-cyg

install apt-cyg /bin

==Install Packages==

Install packages necessary to build Daala:

apt-cyg install gcc-core make pkg-config autoconf automake libtool libogg-devel libpng-devel libjpeg-devel check git

==Compile Daala==

git clone https://git.xiph.org/daala.git

cd daala

./autogen.sh

./configure --disable-player

make -j22

make tools

If the compile fails, try commenting out these lines in configure.ac:
CFLAGS="-std=c89 -pedantic -Wall -Wextra -Wno-parentheses -Wno-long-long -Wno-overlength-strings $CFLAGS"

CC_CHECK_CFLAGS_APPEND([-U__STRICT_ANSI__])

by putting a # on front of the lines in configure.ac file

Note: make tools will fail if sdl is missing.

Daala

2015-03-26T17:16:29Z

Daala-ts: /* Video Presentations */ Better name for IETF 92

Daala

2015-03-26T17:14:32Z

Daala-ts: /* Video Presentations */ Add link to IETF 92 audio

Daala

2015-03-24T22:58:39Z

Daala-ts: /* Video Presentations */ Add link to presentation slides

Daala

2015-03-24T20:56:17Z

Daala-ts: /* Video Presentations */ These links work in HTTPS too.

Daala

2015-03-24T20:55:31Z

Daala-ts: /* Video Presentations */ Adding link to IETF 92. No idea if there's video.

Daala

2015-03-24T09:15:15Z

Daala-ts: Add link to issue tracking page in Github.

Daala

2015-03-24T08:24:49Z

Daala-ts: Revise lead section

Daala Quickstart

2015-03-23T08:55:26Z

Daala-ts: /* Decoding/Playing a Video */ Wrong heading type

This is a guide to getting a copy of the latest code and encoding a video.

There is also a '''[[Daala Quickstart Windows]]''' page.

== Installation ==

=== Obtaining dependencies ===
==== Linux ====
You'll need:

* Standard build tools (autoconf, automake v1.11 or later, libtool, pkg-config, and a C compiler)
* git
* libogg (v1.3 or later)
* libpng
* libjpeg
* libcheck (v0.9.8 or later, can be skipped if you pass --disable-unit-tests to ./configure)
* libsdl (1.2, not 2!) (can by skipped if you pass --disable-player to ./configure)

Do not use linuxbrew.

Instructions for installing these packages are OS-specific (feel free to contribute some here, especially if you tried installing these somewhere and ran into difficulties; you will likely save other people some pain). If you have a package manager that has separate -dev versions with the public headers, make sure you install those in addition to the actual libraries.

==== Mac OS X ====
Install Apple's command line developer tools. E.g. install [https://developer.apple.com/xcode/ Xcode] from the App Store and select 'Command Line Tools' from the Preferences::Downloads panel, or download and install the pkg directly from [https://developer.apple.com/downloads/ developer.apple.com].

Install [http://brew.sh/ Homebrew]

Run the following command to install dependencies:
brew install autoconf automake libtool libogg libpng libjpeg check sdl

=== Installation Procedure ===

Just run these commands:

git clone https://git.xiph.org/daala.git
cd daala
./autogen.sh
./configure
make

Note that the git clone can take several minutes to complete.

And optionally

make tools

Make sure you run the git clone operation on the same machine where you intend to use the code. Checking out a copy on Windows and then trying to use it on Linux will not work, as executable permissions and line-endings will not be set properly.

== Encoding a Video ==

=== Test Media ===
If you do not have one, get a sample video or two in .y4m format from [https://media.xiph.org/video/derf/ media.xiph.org]. These videos are relatively large and will take a long time to encode. There are also subsets of 1 second long videos for faster encoding:
* [https://people.xiph.org/~tdaede/video-1-short/ video-1-short]

We also maintain a set of still-image collections in .y4m format:
* [https://people.xiph.org/~tterribe/daala/subset1-y4m.tar.gz Subset 1] (50 images, small training set)
* [https://people.xiph.org/~tterribe/daala/subset2-y4m.tar.gz Subset 2] (50 images, small testing set)
* [https://people.xiph.org/~tterribe/daala/subset3-y4m.tar.gz Subset 3] (1000 images, large training set)
* [https://people.xiph.org/~tterribe/daala/subset4-y4m.tar.gz Subset 4] (1000 images, large testing set)

=== Using the encoder ===
Encode a video:

./examples/encoder_example -v 30 video.y4m -o video.ogv

where
* video.y4m is the input video you want to encode,
* video.ogv is the name of the encoded video file to output,
* -v specifies the quality (currently from 0 to 511, where 0 is lossless)

== Decoding/Playing a Video ==

Play the video in a window:

./examples/player_example video.ogv

For information on the controls available while playing, run

./examples/player_example --help

If you want to use a different player, you can decode the video back to .y4m with

./examples/dump_video video.ogv -o decoded_video.y4m

Many other players can play back these .y4m files, and other tools can convert them to various other formats.

== The Stream Analyzer ==

There is also a bitstream analyzer. To run it:

./examples/analyzer video.ogv

where video.ogv is the file you want to open. If you omit it, you can open it via the analyzer's GUI.

== Using PNG Images ==

To encode a series of images:

make tools
./tools/png2y4m video%05d.png -o video.y4m

where %05d means your input images are named video00000.png, video00001.png, etc. You can leave out the %05d tag if you only want to convert a single image (which does not need to be numbered).

To convert a y4m back to PNGs:

./tools/y4m2png video.y4m -o video%05d.png

If you are converting a .y4m file that only contains a single frame (e.g., from one of the still-image subsets linked above), you can leave out the %05d tag. Conversion from PNG to Y4M uses the Rec 709 matrix with video levels, a box filter for chroma subsampling, and a triangular dither. Conversion back from Y4M to PNG uses the same matrix, levels, and box filter, but does not dither.

== Creating y4m from other formats ==

You can use the ffmpeg tool to generate y4m from any of it supported video formats:

ffmpeg -i video.webm -pix_fmt yuv420p video.y4m

Note that ffmpeg is optimized for speed. You may not get repeatable results across machines.

[[Category:Daala]]

Daala Quickstart

2015-03-23T08:54:58Z

Daala-ts: Minor copyediting and add info about the stream analyzer

This is a guide to getting a copy of the latest code and encoding a video.

There is also a '''[[Daala Quickstart Windows]]''' page.

== Installation ==

=== Obtaining dependencies ===
==== Linux ====
You'll need:

* Standard build tools (autoconf, automake v1.11 or later, libtool, pkg-config, and a C compiler)
* git
* libogg (v1.3 or later)
* libpng
* libjpeg
* libcheck (v0.9.8 or later, can be skipped if you pass --disable-unit-tests to ./configure)
* libsdl (1.2, not 2!) (can by skipped if you pass --disable-player to ./configure)

Do not use linuxbrew.

Instructions for installing these packages are OS-specific (feel free to contribute some here, especially if you tried installing these somewhere and ran into difficulties; you will likely save other people some pain). If you have a package manager that has separate -dev versions with the public headers, make sure you install those in addition to the actual libraries.

==== Mac OS X ====
Install Apple's command line developer tools. E.g. install [https://developer.apple.com/xcode/ Xcode] from the App Store and select 'Command Line Tools' from the Preferences::Downloads panel, or download and install the pkg directly from [https://developer.apple.com/downloads/ developer.apple.com].

Install [http://brew.sh/ Homebrew]

Run the following command to install dependencies:
brew install autoconf automake libtool libogg libpng libjpeg check sdl

=== Installation Procedure ===

Just run these commands:

git clone https://git.xiph.org/daala.git
cd daala
./autogen.sh
./configure
make

Note that the git clone can take several minutes to complete.

And optionally

make tools

Make sure you run the git clone operation on the same machine where you intend to use the code. Checking out a copy on Windows and then trying to use it on Linux will not work, as executable permissions and line-endings will not be set properly.

== Encoding a Video ==

=== Test Media ===
If you do not have one, get a sample video or two in .y4m format from [https://media.xiph.org/video/derf/ media.xiph.org]. These videos are relatively large and will take a long time to encode. There are also subsets of 1 second long videos for faster encoding:
* [https://people.xiph.org/~tdaede/video-1-short/ video-1-short]

We also maintain a set of still-image collections in .y4m format:
* [https://people.xiph.org/~tterribe/daala/subset1-y4m.tar.gz Subset 1] (50 images, small training set)
* [https://people.xiph.org/~tterribe/daala/subset2-y4m.tar.gz Subset 2] (50 images, small testing set)
* [https://people.xiph.org/~tterribe/daala/subset3-y4m.tar.gz Subset 3] (1000 images, large training set)
* [https://people.xiph.org/~tterribe/daala/subset4-y4m.tar.gz Subset 4] (1000 images, large testing set)

=== Using the encoder ===
Encode a video:

./examples/encoder_example -v 30 video.y4m -o video.ogv

where
* video.y4m is the input video you want to encode,
* video.ogv is the name of the encoded video file to output,
* -v specifies the quality (currently from 0 to 511, where 0 is lossless)

== Decoding/Playing a Video ==

Play the video in a window:

./examples/player_example video.ogv

For information on the controls available while playing, run

./examples/player_example --help

If you want to use a different player, you can decode the video back to .y4m with

./examples/dump_video video.ogv -o decoded_video.y4m

Many other players can play back these .y4m files, and other tools can convert them to various other formats.

=== The Stream Analyzer ===

There is also a bitstream analyzer. To run it:

./examples/analyzer video.ogv

where video.ogv is the file you want to open. If you omit it, you can open it via the analyzer's GUI.

== Using PNG Images ==

To encode a series of images:

make tools
./tools/png2y4m video%05d.png -o video.y4m

where %05d means your input images are named video00000.png, video00001.png, etc. You can leave out the %05d tag if you only want to convert a single image (which does not need to be numbered).

To convert a y4m back to PNGs:

./tools/y4m2png video.y4m -o video%05d.png

If you are converting a .y4m file that only contains a single frame (e.g., from one of the still-image subsets linked above), you can leave out the %05d tag. Conversion from PNG to Y4M uses the Rec 709 matrix with video levels, a box filter for chroma subsampling, and a triangular dither. Conversion back from Y4M to PNG uses the same matrix, levels, and box filter, but does not dither.

== Creating y4m from other formats ==

You can use the ffmpeg tool to generate y4m from any of it supported video formats:

ffmpeg -i video.webm -pix_fmt yuv420p video.y4m

Note that ffmpeg is optimized for speed. You may not get repeatable results across machines.

[[Category:Daala]]

Daala Quickstart

2015-03-23T08:42:11Z

Daala-ts: fixing my own typo

This is a guide to getting a copy of the latest code and encoding a video.

There is also a '''[[Daala Quickstart Windows]]''' page.

== Installation ==

=== Obtaining dependencies ===
==== Linux ====
You'll need:

* Standard build tools (autoconf, automake v1.11 or later, libtool, pkg-config, and a C compiler)
* git
* libogg (v1.3 or later)
* libpng
* libjpeg
* libcheck (v0.9.8 or later, can be skipped if you pass --disable-unit-tests to ./configure)
* libsdl (1.2, not 2!) (can by skipped if you pass --disable-player to ./configure)

Do not use linuxbrew.

Instructions for installing these packages are OS-specific (feel free to contribute some here, especially if you tried installing these somewhere and ran into difficulties; you will likely save other people some pain). If you have a package manager that has separate -dev versions with the public headers, make sure you install those in addition to the actual libraries.

==== Mac OS X ====
Install Apple's command line developer tools. E.g. install [https://developer.apple.com/xcode/ Xcode] from the App Store and select 'Command Line Tools' from the Preferences::Downloads panel, or download and install the pkg directly from [https://developer.apple.com/downloads/ developer.apple.com].

Install [http://brew.sh/ Homebrew]

Run the following command to install dependencies:
brew install autoconf automake libtool libogg libpng libjpeg check sdl

=== Installation Procedure ===

Just run these commands:

git clone https://git.xiph.org/daala.git
cd daala
./autogen.sh
./configure
make

Note that the git clone can take several minutes to complete.

And optionally

make tools

Make sure you run the git clone operation on the same machine where you intend to use the code. Checking out a copy on Windows and then trying to use it on Linux will not work, as executable permissions and line-endings will not be set properly.

== Encoding a Video ==

=== Test Media ===
If you do not have one, get a sample video or two in .y4m format from [https://media.xiph.org/video/derf/ media.xiph.org]. These videos are relatively large and will take a long time to encode. There are also subsets of 1 second long videos for faster encoding:
* [https://people.xiph.org/~tdaede/video-1-short/ video-1-short]

We also maintain a set of still-image collections in .y4m format:
* [https://people.xiph.org/~tterribe/daala/subset1-y4m.tar.gz Subset 1] (50 images, small training set)
* [https://people.xiph.org/~tterribe/daala/subset2-y4m.tar.gz Subset 2] (50 images, small testing set)
* [https://people.xiph.org/~tterribe/daala/subset3-y4m.tar.gz Subset 3] (1000 images, large training set)
* [https://people.xiph.org/~tterribe/daala/subset4-y4m.tar.gz Subset 4] (1000 images, large testing set)

=== Using the encoder ===
Encode the video:

./examples/encoder_example -v 30 video.y4m -o video.ogv

where
* video.y4m is the input video you want to encode,
* video.ogv is the name of the encoded video file to output,
* -v specifies the quality (currently from 0 to 511, where 0 is lossless)

== Decoding/Playing a Video ==

Play the video in a window:

./examples/player_example video.ogv

For information on the controls available while playing, run

./examples/player_example --help

If you want to use a different player, you can decode the video back to .y4m with

./examples/dump_video video.ogv -o decoded_video.y4m

Many other players can play back these .y4m files, and other tools can convert them to various other formats.

== Using PNG Images ==

To encode a series of images:

make tools
./tools/png2y4m video%05d.png -o video.y4m

where %05d means your input images are named video00000.png, video00001.png, etc. You can leave out the %05d tag if you only want to convert a single image (which does not need to be numbered).

To convert a y4m back to PNGs:

./tools/y4m2png video.y4m -o video%05d.png

If you are converting a .y4m file that only contains a single frame (e.g., from one of the still-image subsets linked above), you can leave out the %05d tag. Conversion from PNG to Y4M uses the Rec 709 matrix with video levels, a box filter for chroma subsampling, and a triangular dither. Conversion back from Y4M to PNG uses the same matrix, levels, and box filter, but does not dither.

== Creating y4m from other formats ==

You can use the ffmpeg tool to generate y4m from any of it supported video formats:

ffmpeg -i video.webm -pix_fmt yuv420p video.y4m

Note that ffmpeg is optimized for speed. You may not get repeatable results across machines.

[[Category:Daala]]

Daala Quickstart

2015-03-23T08:41:32Z

Daala-ts: Changing headings

This is a guide to getting a copy of the latest code and encoding a video.

There is also a '''[[Daala Quickstart Windows]]''' page.

== Installation ==

=== Obtaining dependencies ===
==== Linux ====
You'll need:

* Standard build tools (autoconf, automake v1.11 or later, libtool, pkg-config, and a C compiler)
* git
* libogg (v1.3 or later)
* libpng
* libjpeg
* libcheck (v0.9.8 or later, can be skipped if you pass --disable-unit-tests to ./configure)
* libsdl (1.2, not 2!) (can by skipped if you pass --disable-player to ./configure)

Do not use linuxbrew.

Instructions for installing these packages are OS-specific (feel free to contribute some here, especially if you tried installing these somewhere and ran into difficulties; you will likely save other people some pain). If you have a package manager that has separate -dev versions with the public headers, make sure you install those in addition to the actual libraries.

==== Mac OS X ====
Install Apple's command line developer tools. E.g. install [https://developer.apple.com/xcode/ Xcode] from the App Store and select 'Command Line Tools' from the Preferences::Downloads panel, or download and install the pkg directly from [https://developer.apple.com/downloads/ developer.apple.com].

Install [http://brew.sh/ Homebrew]

Run the following command to install dependencies:
brew install autoconf automake libtool libogg libpng libjpeg check sdl

=== Installation Procedure ===

Just run these commands:

git clone https://git.xiph.org/daala.git
cd daala
./autogen.sh
./configure
make

Note that the git clone can take several minutes to complete.

And optionally

make tools

Make sure you run the git clone operation on the same machine where you intend to use the code. Checking out a copy on Windows and then trying to use it on Linux will not work, as executable permissions and line-endings will not be set properly.

== Encoding a Video ==

=== Test Media ===
If you do not have one, get a sample video or two in .y4m format from [https://media.xiph.org/video/derf/ media.xiph.org]. These videos are relatively large and will take a long time to encode. There are also subsets of 1 second long videos for faster encoding:
* [https://people.xiph.org/~tdaede/video-1-short/ video-1-short]

We also maintain a set of still-image collections in .y4m format:
* [https://people.xiph.org/~tterribe/daala/subset1-y4m.tar.gz Subset 1] (50 images, small training set)
* [https://people.xiph.org/~tterribe/daala/subset2-y4m.tar.gz Subset 2] (50 images, small testing set)
* [https://people.xiph.org/~tterribe/daala/subset3-y4m.tar.gz Subset 3] (1000 images, large training set)
* [https://people.xiph.org/~tterribe/daala/subset4-y4m.tar.gz Subset 4] (1000 images, large testing set)

=== Using the encoer ===
Encode the video:

./examples/encoder_example -v 30 video.y4m -o video.ogv

where
* video.y4m is the input video you want to encode,
* video.ogv is the name of the encoded video file to output,
* -v specifies the quality (currently from 0 to 511, where 0 is lossless)

== Decoding/Playing a Video ==

Play the video in a window:

./examples/player_example video.ogv

For information on the controls available while playing, run

./examples/player_example --help

If you want to use a different player, you can decode the video back to .y4m with

./examples/dump_video video.ogv -o decoded_video.y4m

Many other players can play back these .y4m files, and other tools can convert them to various other formats.

== Using PNG Images ==

To encode a series of images:

make tools
./tools/png2y4m video%05d.png -o video.y4m

where %05d means your input images are named video00000.png, video00001.png, etc. You can leave out the %05d tag if you only want to convert a single image (which does not need to be numbered).

To convert a y4m back to PNGs:

./tools/y4m2png video.y4m -o video%05d.png

If you are converting a .y4m file that only contains a single frame (e.g., from one of the still-image subsets linked above), you can leave out the %05d tag. Conversion from PNG to Y4M uses the Rec 709 matrix with video levels, a box filter for chroma subsampling, and a triangular dither. Conversion back from Y4M to PNG uses the same matrix, levels, and box filter, but does not dither.

== Creating y4m from other formats ==

You can use the ffmpeg tool to generate y4m from any of it supported video formats:

ffmpeg -i video.webm -pix_fmt yuv420p video.y4m

Note that ffmpeg is optimized for speed. You may not get repeatable results across machines.

[[Category:Daala]]

Daala

2015-03-04T07:37:45Z

Daala-ts: /* Video Presentations */ Remove extra slash in link address

Daala is the code-name for a new video compression technology.

The effort is a collaboration between [https://www.mozilla.org/en-US/research/ Mozilla Foundation], the [https://www.xiph.org/ Xiph.Org Foundation] and any other contributors that wish to help.

The goal of the project is to provide a free-to-implement, -use and -distribute digital media format 
and reference implementation with technical performance superior to [https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding H.265].

Please see the links below or the [https://www.xiph.org/daala/ main page] for more information.

== Resources ==
=== Wiki Pages ===
* [[Daala Quickstart|Daala Quickstart (Linux/MacOS)]]
* [[Daala Quickstart Windows|Daala Quickstart (Windows)]]
* [[Daala Weekly Meetings|Daala Weekly Meetings]]
* [[AreWeCompressedYet]]
* [[DaalaTodo|Daala To-do List]]
* [[DaalaRoadmap|Daala Roadmap]]
* [[Intra|Intra-prediction within Daala]]

* [[Daala_on_Wheels|Historical Daala wiki page]]

=== Project Communication ===

You are '''encouraged''' to join the
* [irc://irc.freenode.net/daala '''#daala''' IRC channel at freenode.net]
* [http://lists.xiph.org/mailman/listinfo/daala Daala Email List]

You are also welcome to attend the [[Daala Weekly Meetings|weekly progress meetings]] using [http://wiki.mumble.info Mumble]. The address is '''mf4.xiph.org''' and the port is '''64738'''. 
They occur on '''Tuesdays''' at '''[http://www.timeanddate.com/worldclock/fixedtime.html?msg=Daala+Weekly+Meeting&iso=20150210T09&p1=1241 9AM Pacific Time]''' (5PM UTC/GMT).

=== Websites ===
You can get a copy of the latest Daala Source Code from [https://git.xiph.org/?p=daala.git;a=summary '''git.xiph.org'''] or [https://github.com/xiph/daala '''GitHub'''].

* [https://review.xiph.org/ Xiph Code Reviews] - there is a proposal on the review process '''[[DaalaReview|here]]'''
* [https://people.xiph.org/~xiphmont/demo/ Monty's Demo Pages] - explanations on certain techniques used in Daala
* [http://benjamin.smedbergs.us/weekly-updates.fcgi/project/daala Daala Project Status Board] - what Daala bits the Mozilla people are working on
* [https://daala.etherpad.mozilla.org/ep/padlist/all-pads Daala Etherpads] - you can [https://daala.etherpad.mozilla.org/ep/account/request-account request a free account] to view these. You should receive access within a few days.
* [https://www.zazzle.com/daala_tee_shirt-235139149596175944 Daala T-shirts] - if you'd like a free one, help out with the project and ask the Mozilla guys nicely for one :-)

=== JavaScript Demos ===

* [https://people.xiph.org/~xiphmont/demo/daala/player-demo.shtml Daala Video Player]
* [http://exp.martres.me/edi/ Edge-Directed Interpolation]
* [https://people.xiph.org/~unlord/demo/intra.html Intra-prediction]

=== Documents ===
* [https://tools.ietf.org/html/draft-terriberry-codingtools-01 Coding Tools for Daala IETF Draft]
* [https://tools.ietf.org/html/draft-egge-videocodec-tdlt-00 Time-Domain Lapped Transforms (TDLT) IETF Draft]
* [https://tools.ietf.org/html/draft-valin-videocodec-pvq-01 Perceptual Vector Quantisation (PVQ) IETF Draft] - there is also a [http://jmvalin.ca/papers/spie_pvq.pdf PDF file]
* [https://people.xiph.org/~unlord/spie_cfl.pdf Chroma from Luma (CfL)]
* [https://mf4.xiph.org/jenkins/job/daala-autotools/ws/doc/html/index.html C API Documentation]
* [http://arxiv.org/pdf/1411.4290v1.pdf Maximising Coding Efficiency Through Block Rotation] - [http://lists.xiph.org/pipermail/daala/2015-January/000054.html why it won't work well within Daala]

=== Video Presentations ===
* 2015-02-11 SPIE talk about Chroma from Luma (video not available yet) - [https://people.xiph.org/~unlord/SPIE-2015-CfL.pdf slides]
* 2015-01-31 [http://ftp.osuosl.org/pub/fosdem/2015/devroom-open_media/daala.mp4 Daala Project Update at FOSDEM 2015] - [https://fosdem.org/2015/schedule/event/daala/ summary] - [https://fosdem.org/2015/schedule/event/daala/attachments/slides/569/export/events/attachments/daala/slides/569/Daala_FOSDEM_2015.pdf slides]
* 2015-01-14 [https://www.youtube.co.uk/watch?v=Dmho4gcRvQ4 Tim Terriberry's presentation at Linux Conf 2015] - [http://lca2015.linux.org.au/schedule/30187/view_talk presentation summary] - [https://people.xiph.org/~tterribe/pubs/lca2015/daala.pdf slides]
-------
* 2014-09-16 [https://air.mozilla.org/daala-are-we-compressed-yet/ Daala: Are We Compressed Yet?]
* 2014-06-25 [https://air.mozilla.org/sparsity-induced-prediction-for-images-and-video/ Sparsity Induced Prediction for Images and Video]
-------
* 2013-10-23 [https://people.xiph.org/~xiphmont/video/Free_Codecs_Update_Opus_and_Daala.ogv Opus and Daala: State of the Art Royalty-free Codecs] - [https://people.xiph.org/~greg/gstreamer-daala-opus.pdf accompanying slides]
* 2013-09-30 [https://people.xiph.org/~tterribe/daala/coding_party2/?C=M;O=A Daala Coding Party 2] - [https://people.xiph.org/~unlord/Daala-Intra.pdf unlord's slides]
* 2013-05-02 [https://people.xiph.org/~xiphmont/tim-terriberry-presents-daala/ Tim Terriberry Presents Daala]
-------
* 2012-01-24 [https://media.basilgohar.com/derf-talks/?C=M;O=A Tim Terriberry Talks] (in 3 different formats) - [https://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf accompanying slides] (no video for slides 1-50 unfortunately)

[[Category:Daala]]

Daala

2015-02-20T01:13:10Z

Daala-ts: /* Project Communication */ HTTPS doesn't work on this link.

Daala is the code-name for a new video compression technology.

The effort is a collaboration between [https://www.mozilla.org/en-US/research/ Mozilla Foundation], the [https://www.xiph.org/ Xiph.Org Foundation] and any other contributors that wish to help.

The goal of the project is to provide a free-to-implement, -use and -distribute digital media format 
and reference implementation with technical performance superior to [https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding H.265].

Please see the links below or the [https://www.xiph.org/daala/ main page] for more information.

== Resources ==
=== Wiki Pages ===
* [[Daala Quickstart|Daala Quickstart (Linux/MacOS)]]
* [[Daala Quickstart Windows|Daala Quickstart (Windows)]]
* [[Daala Weekly Meetings|Daala Weekly Meetings]]
* [[AreWeCompressedYet]]
* [[DaalaTodo|Daala To-do List]]
* [[DaalaRoadmap|Daala Roadmap]]
* [[Intra|Intra-prediction within Daala]]

* [[Daala_on_Wheels|Historical Daala wiki page]]

=== Project Communication ===

You are '''encouraged''' to join the
* [irc://irc.freenode.net/daala '''#daala''' IRC channel at freenode.net]
* [http://lists.xiph.org/mailman/listinfo/daala Daala Email List]

You are also welcome to attend the [[Daala Weekly Meetings|weekly progress meetings]] using [http://wiki.mumble.info Mumble]. The address is '''mf4.xiph.org''' and the port is '''64738'''. 
They occur on '''Tuesdays''' at '''[http://www.timeanddate.com/worldclock/fixedtime.html?msg=Daala+Weekly+Meeting&iso=20150210T09&p1=1241 9AM Pacific Time]''' (5PM UTC/GMT).

=== Websites ===
You can get a copy of the latest Daala Source Code from [https://git.xiph.org/?p=daala.git;a=summary '''git.xiph.org'''] or [https://github.com/xiph/daala '''GitHub'''].

* [https://review.xiph.org/ Xiph Code Reviews] - there is a proposal on the review process '''[[DaalaReview|here]]'''
* [https://people.xiph.org/~xiphmont/demo/ Monty's Demo Pages] - explanations on certain techniques used in Daala
* [http://benjamin.smedbergs.us/weekly-updates.fcgi/project/daala Daala Project Status Board] - what Daala bits the Mozilla people are working on
* [https://daala.etherpad.mozilla.org/ep/padlist/all-pads Daala Etherpads] - you can [https://daala.etherpad.mozilla.org/ep/account/request-account request a free account] to view these. You should receive access within a few days.
* [https://www.zazzle.com/daala_tee_shirt-235139149596175944 Daala T-shirts] - if you'd like a free one, help out with the project and ask the Mozilla guys nicely for one :-)

=== JavaScript Demos ===

* [https://people.xiph.org/~xiphmont/demo/daala/player-demo.shtml Daala Video Player]
* [http://exp.martres.me/edi/ Edge-Directed Interpolation]
* [https://people.xiph.org/~unlord/demo/intra.html Intra-prediction]

=== Documents ===
* [https://tools.ietf.org/html/draft-terriberry-codingtools-01 Coding Tools for Daala IETF Draft]
* [https://tools.ietf.org/html/draft-egge-videocodec-tdlt-00 Time-Domain Lapped Transforms (TDLT) IETF Draft]
* [https://tools.ietf.org/html/draft-valin-videocodec-pvq-01 Perceptual Vector Quantisation (PVQ) IETF Draft] - there is also a [http://jmvalin.ca/papers/spie_pvq.pdf PDF file]
* [https://people.xiph.org/~unlord/spie_cfl.pdf Chroma from Luma (CfL)]
* [https://mf4.xiph.org/jenkins/job/daala-autotools/ws/doc/html/index.html C API Documentation]

=== Video Presentations ===
* 2015-02-11 SPIE talk about Chroma from Luma (video not available yet) - [https://people.xiph.org/~unlord/SPIE-2015-CfL.pdf slides]
* 2015-01-31 [https://fosdem.org/2015/schedule/event/daala/ FOSDEM 2015] (video not available yet) - [https://fosdem.org/2015/schedule/event/daala/attachments/slides/569/export/events/attachments/daala/slides/569/Daala_FOSDEM_2015.pdf slides]
* 2015-01-14 [https://www.youtube.co.uk/watch?v=Dmho4gcRvQ4 Tim Terriberry's presentation at Linux Conf 2015] - [http://lca2015.linux.org.au/schedule/30187/view_talk presentation summary] - [https://people.xiph.org/~tterribe/pubs/lca2015/daala.pdf slides]
-------
* 2014-09-16 [https://air.mozilla.org/daala-are-we-compressed-yet/ Daala: Are We Compressed Yet?]
* 2014-06-25 [https://air.mozilla.org/sparsity-induced-prediction-for-images-and-video/ Sparsity Induced Prediction for Images and Video]
-------
* 2013-10-23 [https://people.xiph.org/~xiphmont/video/Free_Codecs_Update_Opus_and_Daala.ogv Opus and Daala: State of the Art Royalty-free Codecs] - [https://people.xiph.org/~greg/gstreamer-daala-opus.pdf accompanying slides]
* 2013-09-30 [https://people.xiph.org/~tterribe/daala/coding_party2/?C=M;O=A Daala Coding Party 2] - [https://people.xiph.org/~unlord/Daala-Intra.pdf unlord's slides]
* 2013-05-02 [https://people.xiph.org/~xiphmont/tim-terriberry-presents-daala/ Tim Terriberry Presents Daala]
-------
* 2012-01-24 [https://media.basilgohar.com/derf-talks/?C=M;O=A Tim Terriberry Talks] (in 3 different formats) - [https://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf accompanying slides] (no video for slides 1-50 unfortunately)

[[Category:Daala]]

Daala

2015-02-14T05:10:57Z

Daala-ts: /* Websites */ I feel like git.xiph.org shouldn't be capitalized. Feel free to revert me on that.

Daala is the code-name for a new video compression technology.

The effort is a collaboration between [https://www.mozilla.org/en-US/research/ Mozilla Foundation], the [https://www.xiph.org/ Xiph.Org Foundation] and any other contributors that wish to help.

The goal of the project is to provide a free-to-implement, -use and -distribute digital media format 
and reference implementation with technical performance superior to [https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding H.265].

Please see the links below or the [https://www.xiph.org/daala/ main page] for more information.

== Resources ==
=== Wiki Pages ===
* [[Daala Quickstart|Daala Quickstart (Linux/MacOS)]]
* [[Daala Quickstart Windows|Daala Quickstart (Windows)]]
* [[Daala Weekly Meetings|Daala Weekly Meetings]]
* [[AreWeCompressedYet]]
* [[DaalaTodo|Daala To-do List]]
* [[DaalaRoadmap|Daala Roadmap]]
* [[Intra|Intra-prediction within Daala]]

* [[Daala_on_Wheels|Historical Daala wiki page]]

=== Project Communication ===

You are '''encouraged''' to join the
* [irc://irc.freenode.net/daala '''#daala''' IRC channel at freenode.net]
* [https://lists.xiph.org/mailman/listinfo/daala Daala Email List]

You are also welcome to attend the [[Daala Weekly Meetings|weekly progress meetings]] using [http://wiki.mumble.info Mumble]. The address is '''mf4.xiph.org''' and the port is '''64738'''. 
They occur on '''Tuesdays''' at '''[http://www.timeanddate.com/worldclock/fixedtime.html?msg=Daala+Weekly+Meeting&iso=20150210T09&p1=1241 9AM Pacific Time]''' (5PM UTC/GMT).

=== Websites ===
You can get a copy of the latest Daala Source Code from [https://git.xiph.org/?p=daala.git;a=summary '''git.xiph.org'''] or [https://github.com/xiph/daala '''GitHub'''].

* [https://review.xiph.org/ Xiph Code Reviews] - there is a proposal on the review process '''[[DaalaReview|here]]'''
* [https://people.xiph.org/~xiphmont/demo/ Monty's Demo Pages] - explanations on certain techniques used in Daala
* [http://benjamin.smedbergs.us/weekly-updates.fcgi/project/daala Daala Project Status Board] - what Daala bits the Mozilla people are working on
* [https://daala.etherpad.mozilla.org/ep/padlist/all-pads Daala Etherpads] - you can [https://daala.etherpad.mozilla.org/ep/account/request-account request a free account] to view these. You should receive access within a few days.
* [https://www.zazzle.com/daala_tee_shirt-235139149596175944 Daala T-shirts] - if you'd like a free one, help out with the project and ask the Mozilla guys nicely for one :-)

=== JavaScript Demos ===

* [https://people.xiph.org/~xiphmont/demo/daala/player-demo.shtml Daala Video Player]
* [http://exp.martres.me/edi/ Edge-Directed Interpolation]
* [https://people.xiph.org/~unlord/demo/intra.html Intra-prediction]

=== Documents ===
* [https://tools.ietf.org/html/draft-terriberry-codingtools-01 Coding Tools for Daala IETF Draft]
* [https://tools.ietf.org/html/draft-egge-videocodec-tdlt-00 Time-Domain Lapped Transforms (TDLT) IETF Draft]
* [https://tools.ietf.org/html/draft-valin-videocodec-pvq-01 Perceptual Vector Quantisation (PVQ) IETF Draft] - there is also a [http://jmvalin.ca/papers/spie_pvq.pdf PDF file]
* [https://people.xiph.org/~unlord/spie_cfl.pdf Chroma from Luma (CfL)]
* [https://mf4.xiph.org/jenkins/job/daala-autotools/ws/doc/html/index.html C API Documentation]

=== Video Presentations ===
* 2015-02-11 SPIE talk about Chroma from Luma (video not available yet) - [https://people.xiph.org/~unlord/SPIE-2015-CfL.pdf slides]
* 2015-01-31 [https://fosdem.org/2015/schedule/event/daala/ FOSDEM 2015] (video not available yet) - [https://fosdem.org/2015/schedule/event/daala/attachments/slides/569/export/events/attachments/daala/slides/569/Daala_FOSDEM_2015.pdf slides]
* 2015-01-14 [https://www.youtube.co.uk/watch?v=Dmho4gcRvQ4 Tim Terriberry's presentation at Linux Conf 2015] - [http://lca2015.linux.org.au/schedule/30187/view_talk presentation summary] - [https://people.xiph.org/~tterribe/pubs/lca2015/daala.pdf slides]
-------
* 2014-09-16 [https://air.mozilla.org/daala-are-we-compressed-yet/ Daala: Are We Compressed Yet?]
* 2014-06-25 [https://air.mozilla.org/sparsity-induced-prediction-for-images-and-video/ Sparsity Induced Prediction for Images and Video]
-------
* 2013-10-23 [https://people.xiph.org/~xiphmont/video/Free_Codecs_Update_Opus_and_Daala.ogv Opus and Daala: State of the Art Royalty-free Codecs] - [https://people.xiph.org/~greg/gstreamer-daala-opus.pdf accompanying slides]
* 2013-09-30 [https://people.xiph.org/~tterribe/daala/coding_party2/?C=M;O=A Daala Coding Party 2] - [https://people.xiph.org/~unlord/Daala-Intra.pdf unlord's slides]
* 2013-05-02 [https://people.xiph.org/~xiphmont/tim-terriberry-presents-daala/ Tim Terriberry Presents Daala]
-------
* 2012-01-24 [https://media.basilgohar.com/derf-talks/?C=M;O=A Tim Terriberry Talks] (in 3 different formats) - [https://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf accompanying slides] (no video for slides 1-50 unfortunately)

[[Category:Daala]]

Daala

2015-02-13T00:23:44Z

Daala-ts: /* Video Presentations */ Add date

Daala is the code-name for a new video compression technology.

The effort is a collaboration between [https://www.mozilla.org/en-US/research/ Mozilla Foundation], the [https://www.xiph.org/ Xiph.Org Foundation] and any other contributors that wish to help.

The goal of the project is to provide a free-to-implement, -use and -distribute digital media format 
and reference implementation with technical performance superior to [https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding H.265].

Please see the links below or the [https://www.xiph.org/daala/ main page] for more information.

== Resources ==
=== Wiki Pages ===
* [[Daala Quickstart|Daala Quickstart (Linux/MacOS)]]
* [[Daala Quickstart Windows|Daala Quickstart (Windows)]]
* [[Daala Weekly Meetings|Daala Weekly Meetings]]
* [[AreWeCompressedYet]]
* [[DaalaTodo|Daala To-do List]]
* [[DaalaRoadmap|Daala Roadmap]]
* [[Intra|Intra-prediction within Daala]]

* [[Daala_on_Wheels|Historical Daala wiki page]]

=== Project Communication ===

You are '''encouraged''' to join the
* [irc://irc.freenode.net/daala '''#daala''' IRC channel at freenode.net]
* [https://lists.xiph.org/mailman/listinfo/daala Daala Email List]

You are also welcome to attend the [[Daala Weekly Meetings|weekly progress meetings]] using [http://wiki.mumble.info Mumble]. The address is '''mf4.xiph.org''' and the port is '''64738'''. 
They occur on '''Tuesdays''' at '''[http://www.timeanddate.com/worldclock/fixedtime.html?msg=Daala+Weekly+Meeting&iso=20150210T09&p1=1241 9AM Pacific Time]''' (5PM UTC/GMT).

=== Websites ===
You can get a copy of the latest Daala Source Code from [https://git.xiph.org/?p=daala.git;a=summary '''Git.Xiph.Org'''] or [https://github.com/xiph/daala '''GitHub'''].

* [https://review.xiph.org/ Xiph Code Reviews] - there is a proposal on the review process '''[[DaalaReview|here]]'''
* [https://people.xiph.org/~xiphmont/demo/ Monty's Demo Pages] - explanations on certain techniques used in Daala
* [http://benjamin.smedbergs.us/weekly-updates.fcgi/project/daala Daala Project Status Board] - what Daala bits the Mozilla people are working on
* [https://daala.etherpad.mozilla.org/ep/padlist/all-pads Daala Etherpads] - you can [https://daala.etherpad.mozilla.org/ep/account/request-account request a free account] to view these. You should receive access within a few days.
* [https://www.zazzle.com/daala_tee_shirt-235139149596175944 Daala T-shirts] - if you'd like a free one, help out with the project and ask the Mozilla guys nicely for one :-)

=== JavaScript Demos ===

* [https://people.xiph.org/~xiphmont/demo/daala/player-demo.shtml Daala Video Player]
* [http://exp.martres.me/edi/ Edge-Directed Interpolation]
* [https://people.xiph.org/~unlord/demo/intra.html Intra-prediction]

=== Documents ===
* [https://tools.ietf.org/html/draft-terriberry-codingtools-01 Coding Tools for Daala IETF Draft]
* [https://tools.ietf.org/html/draft-egge-videocodec-tdlt-00 Time-Domain Lapped Transforms (TDLT) IETF Draft]
* [https://tools.ietf.org/html/draft-valin-videocodec-pvq-01 Perceptual Vector Quantisation (PVQ) IETF Draft] - there is also a [http://jmvalin.ca/papers/spie_pvq.pdf PDF file]
* [https://people.xiph.org/~unlord/spie_cfl.pdf Chroma from Luma (CfL)]
* [https://mf4.xiph.org/jenkins/job/daala-autotools/ws/doc/html/index.html C API Documentation]

=== Video Presentations ===
* 2015-02-11 Next up: SPIE talk about Chroma from Luma - [https://people.xiph.org/~unlord/SPIE-2015-CfL.pdf slides]
* 2015-01-31 [https://fosdem.org/2015/schedule/event/daala/ FOSDEM 2015] - [https://fosdem.org/2015/schedule/event/daala/attachments/slides/569/export/events/attachments/daala/slides/569/Daala_FOSDEM_2015.pdf slides] (Video not available yet.)
* 2015-01-14 [https://www.youtube.co.uk/watch?v=Dmho4gcRvQ4 Tim Terriberry's presentation at Linux Conf 2015] - [http://lca2015.linux.org.au/schedule/30187/view_talk presentation summary] - [https://people.xiph.org/~tterribe/pubs/lca2015/daala.pdf slides]
-------
* 2014-09-16 [https://air.mozilla.org/daala-are-we-compressed-yet/ Daala: Are We Compressed Yet?]
* 2014-06-25 [https://air.mozilla.org/sparsity-induced-prediction-for-images-and-video/ Sparsity Induced Prediction for Images and Video]
-------
* 2013-10-23 [https://people.xiph.org/~xiphmont/video/Free_Codecs_Update_Opus_and_Daala.ogv Opus and Daala: State of the Art Royalty-free Codecs] - [https://people.xiph.org/~greg/gstreamer-daala-opus.pdf accompanying slides]
* 2013-09-30 [https://people.xiph.org/~tterribe/daala/coding_party2/?C=M;O=A Daala Coding Party 2] - [https://people.xiph.org/~unlord/Daala-Intra.pdf unlord's slides]
* 2013-05-02 [https://people.xiph.org/~xiphmont/tim-terriberry-presents-daala/ Tim Terriberry Presents Daala]
-------
* 2012-01-24 [https://media.basilgohar.com/derf-talks/?C=M;O=A Tim Terriberry Talks] (in 3 different formats) - [https://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf accompanying slides] (no video for slides 1-50 unfortunately)

[[Category:Daala]]

Daala

2015-02-12T22:54:37Z

Daala-ts: /* Video Presentations */ Removing Next up and adding note that no video is available.

Daala is the code-name for a new video compression technology.

The effort is a collaboration between [https://www.mozilla.org/en-US/research/ Mozilla Foundation], the [https://www.xiph.org/ Xiph.Org Foundation] and any other contributors that wish to help.

The goal of the project is to provide a free-to-implement, -use and -distribute digital media format 
and reference implementation with technical performance superior to [https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding H.265].

Please see the links below or the [https://www.xiph.org/daala/ main page] for more information.

== Resources ==
=== Wiki Pages ===
* [[Daala Quickstart|Daala Quickstart (Linux/MacOS)]]
* [[Daala Quickstart Windows|Daala Quickstart (Windows)]]
* [[Daala Weekly Meetings|Daala Weekly Meetings]]
* [[AreWeCompressedYet]]
* [[DaalaTodo|Daala To-do List]]
* [[DaalaRoadmap|Daala Roadmap]]
* [[Intra|Intra-prediction within Daala]]

* [[Daala_on_Wheels|Historical Daala wiki page]]

=== Project Communication ===

You are '''encouraged''' to join the
* [irc://irc.freenode.net/daala '''#daala''' IRC channel at freenode.net]
* [https://lists.xiph.org/mailman/listinfo/daala Daala Email List]

You are also welcome to attend the [[Daala Weekly Meetings|weekly progress meetings]] using [http://wiki.mumble.info Mumble]. The address is '''mf4.xiph.org''' and the port is '''64738'''. 
They occur on '''Tuesdays''' at '''[http://www.timeanddate.com/worldclock/fixedtime.html?msg=Daala+Weekly+Meeting&iso=20150210T09&p1=1241 9AM Pacific Time]''' (5PM UTC/GMT).

=== Websites ===
You can get a copy of the latest Daala Source Code from [https://git.xiph.org/?p=daala.git;a=summary '''Git.Xiph.Org'''] or [https://github.com/xiph/daala '''GitHub'''].

* [https://review.xiph.org/ Xiph Code Reviews] - there is a proposal on the review process '''[[DaalaReview|here]]'''
* [https://people.xiph.org/~xiphmont/demo/ Monty's Demo Pages] - explanations on certain techniques used in Daala
* [http://benjamin.smedbergs.us/weekly-updates.fcgi/project/daala Daala Project Status Board] - what Daala bits the Mozilla people are working on
* [https://daala.etherpad.mozilla.org/ep/padlist/all-pads Daala Etherpads] - you can [https://daala.etherpad.mozilla.org/ep/account/request-account request a free account] to view these. You should receive access within a few days.
* [https://www.zazzle.com/daala_tee_shirt-235139149596175944 Daala T-shirts] - if you'd like a free one, help out with the project and ask the Mozilla guys nicely for one :-)

=== JavaScript Demos ===

* [https://people.xiph.org/~xiphmont/demo/daala/player-demo.shtml Daala Video Player]
* [http://exp.martres.me/edi/ Edge-Directed Interpolation]
* [https://people.xiph.org/~unlord/demo/intra.html Intra-prediction]

=== Documents ===
* [https://tools.ietf.org/html/draft-terriberry-codingtools-01 Coding Tools for Daala IETF Draft]
* [https://tools.ietf.org/html/draft-egge-videocodec-tdlt-00 Time-Domain Lapped Transforms (TDLT) IETF Draft]
* [https://tools.ietf.org/html/draft-valin-videocodec-pvq-01 Perceptual Vector Quantisation (PVQ) IETF Draft] - there is also a [http://jmvalin.ca/papers/spie_pvq.pdf PDF file]
* [https://people.xiph.org/~unlord/spie_cfl.pdf Chroma from Luma (CfL)]
* [https://mf4.xiph.org/jenkins/job/daala-autotools/ws/doc/html/index.html C API Documentation]

=== Video Presentations ===
* 2015-02-?? Next up: SPIE talk about Chroma from Luma - [https://people.xiph.org/~unlord/SPIE-2015-CfL.pdf slides]
* 2015-01-31 [https://fosdem.org/2015/schedule/event/daala/ FOSDEM 2015] - [https://fosdem.org/2015/schedule/event/daala/attachments/slides/569/export/events/attachments/daala/slides/569/Daala_FOSDEM_2015.pdf slides] (Video not available yet.)
* 2015-01-14 [https://www.youtube.co.uk/watch?v=Dmho4gcRvQ4 Tim Terriberry's presentation at Linux Conf 2015] - [http://lca2015.linux.org.au/schedule/30187/view_talk presentation summary] - [https://people.xiph.org/~tterribe/pubs/lca2015/daala.pdf slides]
-------
* 2014-09-16 [https://air.mozilla.org/daala-are-we-compressed-yet/ Daala: Are We Compressed Yet?]
* 2014-06-25 [https://air.mozilla.org/sparsity-induced-prediction-for-images-and-video/ Sparsity Induced Prediction for Images and Video]
-------
* 2013-10-23 [https://people.xiph.org/~xiphmont/video/Free_Codecs_Update_Opus_and_Daala.ogv Opus and Daala: State of the Art Royalty-free Codecs] - [https://people.xiph.org/~greg/gstreamer-daala-opus.pdf accompanying slides]
* 2013-09-30 [https://people.xiph.org/~tterribe/daala/coding_party2/?C=M;O=A Daala Coding Party 2] - [https://people.xiph.org/~unlord/Daala-Intra.pdf unlord's slides]
* 2013-05-02 [https://people.xiph.org/~xiphmont/tim-terriberry-presents-daala/ Tim Terriberry Presents Daala]
-------
* 2012-01-24 [https://media.basilgohar.com/derf-talks/?C=M;O=A Tim Terriberry Talks] (in 3 different formats) - [https://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf accompanying slides] (no video for slides 1-50 unfortunately)

[[Category:Daala]]

Daala

2015-02-09T17:15:55Z

Daala-ts: /* Video Presentations */ Add link to FOSDEM 2015 presentation slides

Daala is the code-name for a new video compression technology.

The effort is a collaboration between [https://www.mozilla.org/en-US/research/ Mozilla Foundation], the [https://www.xiph.org/ Xiph.Org Foundation] and any other contributors that wish to help.

The goal of the project is to provide a free-to-implement, -use and -distribute digital media format 
and reference implementation with technical performance superior to [https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding H.265].

Please see the links below or the [https://www.xiph.org/daala/ main page] for more information.

== Resources ==
=== Wiki Pages ===
* [[Daala Quickstart|Daala Quickstart (Linux/MacOS)]]
* [[Daala Quickstart Windows|Daala Quickstart (Windows)]]
* [[Daala Weekly Meetings|Daala Weekly Meetings]]
* [[AreWeCompressedYet]]
* [[DaalaTodo|Daala To-do List]]
* [[DaalaRoadmap|Daala Roadmap]]
* [[Intra|Intra-prediction within Daala]]

* [[Daala_on_Wheels|Historical Daala wiki page]]

=== Project Communication ===

You are '''encouraged''' to join the
* [irc://irc.freenode.net/daala '''#daala''' IRC channel at freenode.net]
* [https://lists.xiph.org/mailman/listinfo/daala Daala Email List]

You are also welcome to attend the [[Daala Weekly Meetings|weekly progress meetings]] using [http://wiki.mumble.info Mumble]. The address is '''mf4.xiph.org''' and the port is '''64738'''. 
They occur on '''Tuesdays''' at '''[http://www.timeanddate.com/worldclock/fixedtime.html?msg=Daala+Weekly+Meeting&iso=20150210T09&p1=1241 9AM Pacific Time]''' (5PM UTC/GMT).

=== Websites ===
You can get a copy of the latest Daala Source Code from [https://git.xiph.org/?p=daala.git;a=summary '''Xiph.Org'''] or [https://github.com/xiph/daala '''GitHub'''].

* [https://review.xiph.org/ Xiph Code Reviews] - there is a proposal on the review process '''[[DaalaReview|here]]'''
* [https://people.xiph.org/~xiphmont/demo/ Monty's Demo Pages] - explanations on certain techniques used in Daala
* [http://benjamin.smedbergs.us/weekly-updates.fcgi/project/daala Daala Project Status Board] - what Daala bits the Mozilla people are working on
* [https://daala.etherpad.mozilla.org/ep/padlist/all-pads Daala Etherpads] - you can [https://daala.etherpad.mozilla.org/ep/account/request-account request a free account] to view these. You should receive access within a few days.
* [https://www.zazzle.com/daala_tee_shirt-235139149596175944 Daala T-shirts] - if you'd like a free one, help out with the project and ask the Mozilla guys nicely for one :-)

=== JavaScript Demos ===

* [https://people.xiph.org/~xiphmont/demo/daala/player-demo.shtml Daala Video Player]
* [http://exp.martres.me/edi/ Edge-Directed Interpolation]
* [https://people.xiph.org/~unlord/demo/intra.html Intra-prediction]

=== Documents ===
* [https://tools.ietf.org/html/draft-terriberry-codingtools-01 Coding Tools for Daala IETF Draft]
* [https://tools.ietf.org/html/draft-egge-videocodec-tdlt-00 Time-Domain Lapped Transforms (TDLT) IETF Draft]
* [https://tools.ietf.org/html/draft-valin-videocodec-pvq-01 Perceptual Vector Quantisation (PVQ) IETF Draft] - there is also a [http://jmvalin.ca/papers/spie_pvq.pdf PDF file]
* [https://people.xiph.org/~unlord/spie_cfl.pdf Chroma from Luma (CfL)]
* [https://mf4.xiph.org/jenkins/job/daala-autotools/ws/doc/html/index.html C API Documentation]

=== Video Presentations ===
* 2015-01-31 Next up: [https://fosdem.org/2015/schedule/event/daala/ FOSDEM 2015] - [https://fosdem.org/2015/schedule/event/daala/attachments/slides/569/export/events/attachments/daala/slides/569/Daala_FOSDEM_2015.pdf slides]
* 2015-01-14 [https://www.youtube.co.uk/watch?v=Dmho4gcRvQ4 Tim Terriberry's presentation at Linux Conf 2015] - [http://lca2015.linux.org.au/schedule/30187/view_talk presentation summary] - [https://people.xiph.org/~tterribe/pubs/lca2015/daala.pdf slides]
-------
* 2014-09-16 [https://air.mozilla.org/daala-are-we-compressed-yet/ Daala: Are We Compressed Yet?]
* 2014-06-25 [https://air.mozilla.org/sparsity-induced-prediction-for-images-and-video/ Sparsity Induced Prediction for Images and Video]
-------
* 2013-10-23 [https://people.xiph.org/~xiphmont/video/Free_Codecs_Update_Opus_and_Daala.ogv Opus and Daala: State of the Art Royalty-free Codecs] - [https://people.xiph.org/~greg/gstreamer-daala-opus.pdf accompanying slides]
* 2013-09-30 [https://people.xiph.org/~tterribe/daala/coding_party2/?C=M;O=A Daala Coding Party 2] - [https://people.xiph.org/~unlord/Daala-Intra.pdf unlord's slides]
* 2013-05-02 [https://people.xiph.org/~xiphmont/tim-terriberry-presents-daala/ Tim Terriberry Presents Daala]
-------
* 2012-01-24 [https://media.basilgohar.com/derf-talks/?C=M;O=A Tim Terriberry Talks] (in 3 different formats) - [https://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf accompanying slides] (no video for slides 1-50 unfortunately)

[[Category:Daala]]

Speex

2015-02-07T01:10:42Z

Daala-ts: /* Speex is deprecated; Xiph recommends you use Opus instead */ Add a period

{{historical}}

= Speex is deprecated; Xiph recommends you use [[Opus]] instead. =

== Website ==

The [http://www.speex.org/ Speex homepage] has all the project info.

There is also a '''[[Speex FAQ]]'''.

== Hardware ==

See [[Speex hardware]] for a partial list of supported hardware

== Tasks ==

These are some improvements that could be made to Speex.

[mailto:speex-dev@xiph.org Let us know] if you'd like to work on one of them.

* Speech/signal processing (DSP design)
** Improve noise suppression (get rid of musical noise) and residual echo suppression
** Improve packet-loss concealment (PLC)
** Re-write the built-in voice activity detector (VAD)
** Improve the 2.15 kbps vocoder mode (there are even 4 unused bits left to use)
** Algorithmic optimizations (see if some searches can be simplified/approximated)

* Complete fixed-point (DSP development)
** Wideband
** VBR
** Rest of the narrowband modes
** Preprocessor (noise suppression, AGC)
** Jitter buffer
** Arch-specific optimization
** More...

* Tune (playing with parameters)
** Noise weighting filter
** Perceptual enhancement

* Features (plain C programming)
** Implement maximum VBR bit-rate
** Implement peeling (write functions to strip some of the bits)
*** Peel high-band (wideband -> narrowband)
*** Transform 24.6 kbps mode to 15 kbps mode

* Documentation
** Use questions from the mailing list to create a better FAQ on the wiki
** Update the Speex manual based on recent papers
** Improve libspeex documentation
** Write good example code

== External links ==
* [[Applications that use Speex]]
* [[Games that use Speex]]
* [[Wikipedia: Speex]]

[[Category:Speex]]

Ogg

2015-02-07T01:10:04Z

Daala-ts: Minor copyediting

The '''Ogg''' transport bitstream is designed to provide framing, error protection and seeking structure for higher-level codec streams that consist of raw, unencapsulated data packets, such as the [[Opus]], [[Vorbis]] and [[FLAC]] audio codecs or the [[Theora]] and [[Dirac]] video codecs.

== Name ==

Ogg derives from "ogging", jargon from the computer game Netrek. Ogg is not an acronym and should not be mentioned as "OGG".

== Design constraints for Ogg bitstreams ==

* True streaming; we must not need to seek to build a 100% complete bitstream.
* Use no more than approximately 1-2% of bitstream bandwidth for packet boundary marking, high-level framing, sync and seeking.
* Specification of absolute position within the original sample stream.
* Simple mechanism to ease limited editing, such as a simplified concatenation mechanism.
* Detection of corruption, recapture after error and direct, random access to data at arbitrary positions in the bitstream.

== Specification / standard==

The Ogg transport bitstream and file format is defined in RFC 3533 approved 2003-May. As RFC documents are invariable once approved, there will never be newer versions of RFC 3533, but an [[RFC_3533_Errata]] exists instead. Existing flaws are discussed at [[OggIssues]], ideas for the future at [[TransOgg]].

== Detecting Ogg files and extracting information ==

Ogg files begin with a signature "OggS". This signature also repeats many times inside the file, at the beginning of every page. There are several tools to get information about Ogg files:
* Ogginfo - part of Vorbis-Tools, supports Vorbis codec only (historical Ogg-vs-Vorbis issue), other codecs cause it to report garbage
* Opusinfo - part of Opus-Tools, supports only Opus codec well, only minimal Vorbis support
* Oggz ???
* MediaInfo [http://sourceforge.net/projects/mediainfo/ sf.net/projects/mediainfo] - provides information about media (and some other) files, supports many types, also Ogg with various codecs, generic audio and video information only, no Ogg-specific details

== Projects using Ogg ==

=== Codecs ===

* [[Opus]]
* [[CMML]]
* [[FLAC]] ([http://flac.sourceforge.net/ogg_mapping.html Ogg mapping])
* [[OggKate|Kate]]
* [http://opus-codec.org/ Opus] ([[OggOpus|Ogg mapping]])
* [[OggPCM|PCM]]
* [[Ogg Skeleton|Skeleton]]
* [[Speex]] ([[OggSpeex|Ogg mapping]])
* [[Theora]] ([[OggTheora|Ogg mapping]])
* [[Vorbis]] ([[OggVorbis|Ogg mapping]])
* [[OggWrit|Writ]]

=== Servers ===

* [http://www.annodex.net/ Annodex]
* [[Icecast]]
* [http://www.metavid.org/ Metavid]

== Developer info ==

* [[GranulePosAndSeeking]] -- a discussion of the interpretation of granulepos, and the algorithm for seeking on Ogg files

=== Ogg page format ===

The LSb (least significant bit) comes first in the Bytes. Fields
with more than one byte length are encoded LSB (least significant
byte) first.

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1| Byte
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| capture_pattern: Magic number for page start "OggS" | 0-3
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| version | header_type | granule_position | 4-7
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | 8-11
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | bitstream_serial_number | 12-15
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | page_sequence_number | 16-19
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | CRC_checksum | 20-23
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |page_segments | segment_table | 24-27
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | 28-
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

== Implementations ==

The Ogg encapsulation format can be handled with the following libraries:

* libogg: [http://svn.xiph.org/trunk/ogg/ libogg svn] (C, cross-platform) Low-level Ogg parsing and writing.
* liboggz: [http://git.xiph.org/?p=liboggz.git liboggz git] or [http://annodex.net/software/liboggz/index.html DEAD:liboggz] (C, cross-platform) liboggz wraps libogg and provides features such as seeking.
* the Ogg Directshow filters: see [http://www.illiminable.com/ogg/ illiminable] (C++, Win32)
* [http://www.kfish.org/software/hogg HOgg] (pure Haskell)
* [http://www.jcraft.com/jorbis/ JOrbis] (pure Java) contains com.jcraft.jogg
* [http://www.sacredchao.net/quodlibet/wiki/Development/Mutagen Mutagen] (pure Python)

== See also ==

* [[Oggless]]
* [[MIME Types and File Extensions]]
* [[RFC_3533_Errata]] - errors and flaws in the specification
* [[Nut_Container]]

== External links ==

* [http://www.xiph.org/ogg/doc/ Ogg documentation]
* [http://www.ietf.org/rfc/rfc3533.txt Ogg RFC]
* [http://en.wikipedia.org/wiki/Ogg Ogg at Wikipedia]
* [http://wiki.multimedia.cx/index.php?title=Ogg Ogg at Multimedia Wiki]

[[Category:Ogg]]

Daala Weekly Meetings

2015-02-07T00:27:03Z

Daala-ts: /* 2012 */ Add dashes, remove an extraneous zero

You are encouraged to join the weekly progress meetings using [http://wiki.mumble.info Mumble] at '''mf4.xiph.org''' on '''Tuesdays''' at '''[http://www.time.gov/ 9AM PST]''' (5PM GMT).

== Progress Meetings ==

To enable text-wrapping within meeting logs, try using:
<pre>
<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
</pre>

=== 2012 ===
* 2012 June 4 - [https://people.xiph.org/~giles/2012/daala_20120604.txt minutes] (actually a work week)
* 2012 June 22 - [https://people.xiph.org/~giles/2012/daala_20120622.txt minutes]
* 2012 June 29 - [https://people.xiph.org/~giles/2012/daala_20120629.txt minutes] [https://people.xiph.org/~giles/2012/daala_20120629.opus recording]
* 2012 July 6 - [https://people.xiph.org/~giles/2012/daala_20120706.txt minutes]
* 2012 July 13 - [https://people.xiph.org/~giles/2012/daala_20120713.txt minutes]
* 2012 July 20 - [https://people.xiph.org/~giles/2012/daala_20120720.txt minutes]
* 2012 July 27 - [https://people.xiph.org/~giles/2012/daala_20120727.txt minutes]
* 2012 August 3 - [https://people.xiph.org/~giles/2012/daala_20120803.txt minutes] [https://people.xiph.org/~giles/2012/daala_20120803.opus recording]
* 2012 August 10 - [https://people.xiph.org/~giles/2012/daala_20120810.txt minutes]
* 2012 August 17 - no meeting
* 2012 August 24 - [https://people.xiph.org/~giles/2012/daala_20120824.txt minutes] [https://people.xiph.org/~giles/2012/daala_20120824.opus recording]
* 2012 August 31 - no meeting
* 2012 September 7 - no meeting
* 2012 September 14 - no meeting
* 2012 September 21 - [https://people.xiph.org/~giles/2012/daala_20120921.txt minutes]
* 2012 September 28 - [https://people.xiph.org/~giles/2012/daala_20120928.txt minutes] [https://people.xiph.org/~giles/2012/daala_20120928.opus recording]
* 2012 October 5 - [https://people.xiph.org/~giles/2012/daala_20121005.opus recording]
* 2012 October 26 -
* 2012 Novemeber 2 - no meeting
* 2012 December 7 - [https://people.xiph.org/~giles/2012/daala_20121207.txt minutes]

=== 2013 ===
* [[DaalaMeeting20130911|2013 September 11]] - logistics, status updates, coding party agenda
* [[DaalaMeeting20130917|2013 September 17]] - summit plans, research talks, coding party agenda, monty's TF ideas
* [[DaalaMeeting20130924|2013 September 24]] - training results, 32x32 code, and 2nd stage TF
* [[DaalaMeeting20131008|2013 October 8]] - pvq status, training, chroma from luma
* [[DaalaMeeting20131022|2013 October 22]] - even/odd quantizer, determinant 1, IETF, chroma from luma
* [[DaalaMeeting20131029|2013 October 29]] - gstreamer conf, ietf, chroma from luma
* [[DaalaMeeting20131112|2013 November 12]] - chroma from luma, pvq
* [[DaalaMeeting20131203|2013 December 3]] - chroma from luma, pvq, misc
* [[DaalaMeeting20131217|2013 December 17]] - luma intraprediction, pvq decoder, interns

=== 2014 ===
* [[DaalaMeeting20140107|2014 January 07]] - reviews, pvq projects, and det1
* [[DaalaMeeting20140114|2014 January 14]] - pvq, det1, new projects
* [[DaalaMeeting20140121|2014 January 21]] - project planning
* [[DaalaMeeting20140128|2014 January 28]] - pvq, work week
* [[DaalaMeeting20140211|2014 February 11]] - 32x32, activity masking, current issues
* [[DaalaMeeting20140218|2014 February 18]] - input scaling, prediction error
* [[DaalaMeeting20140225|2014 February 25]] - det1, pvq, fastssim
* [[DaalaMeeting20140304|2014 March 4]] - det1, quant matrix, metrics
* [[DaalaMeeting20140318|2014 March 18]] - det1, fastssim, activity masking
* [[DaalaMeeting20140401|2014 April 1]] - crushing jpeg, pvq
* [[DaalaMeeting20140408|2014 April 8]] - reviews, pvq presentation
* [[DaalaMeeting20140415|2014 April 15]] - pvq experiments, activity masking
* [[DaalaMeeting20140422|2014 April 22]] - block size switching, gpu jpeg
* [[DaalaMeeting20140506|2014 May 6]] - workweek debrief, trained vq
* [[DaalaMeeting20140513|2014 May 13]] - trained vq, intraprediction
* [[DaalaMeeting20140520|2014 May 20]] - dc encoding
* [[DaalaMeeting20140527|2014 May 27]] - standard metrics
* [[DaalaMeeting20140603|2014 June 3]] - deringing filter
* [[DaalaMeeting20140624|2014 June 24]] - deringing filter
* [[DaalaMeeting20140701|2014 July 1]] - multi-frame motion compensation, jean-marc crazy ideas
* [[DaalaMeeting20140708|2014 July 8]] - intrapred ideas, journaling, rd_collect issues
* [[DaalaMeeting20140715|2014 July 15]] - VIPC, intrapred
* [[DaalaMeeting20140722|2014 July 22]] - demo progress, video results, edi, deringing, new intra
* [[DaalaMeeting20140729|2014 July 29]] - edi, intra paint
* [[DaalaMeeting20140805|2014 August 5]] - edi, intra paint
* [[DaalaMeeting20140812|2014 August 12]] - multi-frame MC, intra paint, edi
* [[DaalaMeeting20140826|2014 August 26]] - intra paint, halfpel BMA
* [[DaalaMeeting20140902|2014 September 2]] - demos, multi-frame motion compensation, deringing
* [[DaalaMeeting20140909|2014 September 9]] - reviews, deringing, intraprediction, non-photographic video
* [[DaalaMeeting20140916|2014 September 16]] - reviews, unlapping
* [[DaalaMeeting20140923|2014 September 23]] - reviews, VDD debrief, deringing demo, intraprediction tests
* [[DaalaMeeting20140930|2014 September 30]] - steinar's tests, intra update
* [[DaalaMeeting20141007|2014 October 7]] - pvq<->entropy coding dependencies on intra, fate of deringing, scalar gone, intern projects
* [[DaalaMeeting20141014|2014 October 14]] - monty's directional transforms, HV intra, gain/theta/skip coding, deringing
* [[DaalaMeeting20141104|2014 November 4]] - reviews, misc
* [[DaalaMeeting20141118|2014 November 18]] - avx intrinsics, directional transforms
* [[DaalaMeeting20141125|2014 November 25]] - non-intra work, jean-marc's crazy ideas
* [[DaalaMeeting20141209|2014 December 9]] - 32x32, misc
* [[DaalaMeeting20141216|2014 December 16]] - update1, goals, yushin question time
* [[DaalaMeeting20141223|2014 December 23]] - update1, bpyramid

=== 2015 ===

* [[DaalaMeeting20150106|2015 January 6]] - SPIE

[[Category:Daala]]

DaalaMeeting20131217

2015-02-07T00:15:54Z

Daala-ts: Add word wrapping

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2013-12-17

Mumble: mf4.xiph.org:64738

# Agenda

- Interns!
- Research grant
- Luma intrapredictor TF to 4x4
- JM's PVQ decoder

# Attending

- greg, monty, jmspeex, jack, derf

# electrical engineering quiz

- do *you* know how to wind toroids?
- I know how to wind toroids

# Interns!

- derf: we met a promising candidate at PCS. she's worked on the same training problem nathan has worked on but on predictors in time domain.
- jack: i will talk to her about applying
- derf: the other interesting bit was that she worked on open loop stuff. like if we TFed down to 4x4 predicted each block and then TFed back up to compute the residual at the end.
- jm: you mena avoiding being unstable?
- derf: exactly. she's not doing anything explicitly, but she runs the open loop over the whole block and tries to minimize least squared error.
- jm: i wouldn't be surprised if it was a little unstable but it wouldn't explode.
- jack: smarter's going to work on edge directed interpolation from mid-feb to august. doing paperwork for that now.
- jack: smarter won't count against our 2 interns, so we need to find more.
- derf: i talked to folks at perdue and at a german university (hanover). we can ask the northwestern folks.
- jack: need to follow up on this now, because interns are already getting offers from other companies.

# research grant

- derf: northwestern team has two ideas. one is adaptive pre/post filtering. ie, use different filters for different block edges, which would be signaled. i warned them that there are potential IPR issues with that, but it's not impossible to do.
- jm: the idea is to balance the sharp/smooth filters with edges? or is there more to it?
- derf: he wasn't super clear about the effects he desired, but i think it's something like that.
- derf: the second part is decoder-as-estimator. by itself doesn't seem that interesting, but when you optimize the filters in that framework he comes up with a different optimization goal, and we know ours isn't hte best, so having more things to look at would be useful there.
- jack: for decoder-as-estimator, isn't decode pixel perfect? is this only useful for packet loss?
- derf: no, you can perhaps make up quantization loss with knowledge about the type of image
- greg: edge directed interpolation is an example of this class of technique— using prior information about the structure of the image to do a better reconstruction in the decoder.
- derf: but they're planning on doing this in the pre/post processing.

# Luma intrapredictor

- greg: i reviewed nathan's patch (outstanding), and the patch works. i have a general concern that i came to while testing this thing. i'm seeing a lot of bheavior with the new predictors which don't just use DC for the 16x16 case. it moves some low frequency energy into HF with intraprediction and then it doesn't have enough bits to move the bits back out. it's realy obvious in the background in fruits.
- jm: hopefully this is something that pvq can trivially fix. for scalar quantization this can be an issue.
- greg: good point. otherwise i was considering low passing the predictions. or try to address this in training.
- jm: is this really a problem even without pvq? it's minimizing the energy so it's getting it right to compensate for the times when it's wrong. if it's doing that it means that it's helping something and so i'd like to understand what.
- derf: we don't use adaptive quantization so we should be spending more bits than we actually are.
- greg: this pattern of copying LF to HF i expect to continue to be a problem.
- derf: you may be right
- greg: maybe pvq solves it or it can be improved in the training. i'm not saying it can't be solved.
- jm: if the large block sizes are trained on stuff that has edges, then that could cause the HF effect as opposed to training on stuff that should be 16x16.
- greg: i think we were escaping the problem previously because they only predict DC due to previous training problems. it's real easy to churn off ideas to do something better here. i don't think this should block landing this patch.
- jm: it's worth retraining with excluding the edges or weighting the edges very low.
- greg: training on all the offsets might help too. tim was getting some more images.
- derf: working on it. it would be nice to use larger intrapredictors for edges.
- greg: i don't want to exclude edges, but perhaps some regularization to weight more for modes where we get improvement is something we should be doing. i don't think we want to train it on edges that it doesn't even help on. nice long straight edges, sure.
- jm: we can even specialize based on that. it would be tricky because you're using different data for different modes, but you can have intra modes that are trying to do edges but also have modes that explicitly trained to not have edges and only do vaguely directional gradients and that kind of thing.
- derf: we certainly could.

# PVQ decoder

- jm: it decodes. it works on two different images!
- derf: twice as much testing as i expected.
- jm: that's 2 more tests than when i committed it! it's better than the previous pvq decoder. the good news is i did not have to fix anything in the encoder to make it work.
- derf: the encoder was actually working?
- jm: yes. encoded all the information
- derf: twice?
- jm: can't say, but i don't think so :) I only have code for 8x8. if you go to encode.c and set run_pvq=1 it still works. you're going to get scalar quant for 4x4 and 16x16 but pvq for 8x8. i wouldn't guarantee you'd be able to export the files because it's using double precision but it might work. double precision is probably safer than float for this kind of thing. you have to be pretty unlucky to end up at the rounding threshold. i have verified that hte bitstream is actually correct. what should the next thing be?
- jack: 4x4 and 16x16?
- jm: that or removing double precision or context adaptation.
- greg: how can we increase confidence that it's correct?
- derf: or useful
- greg: i meant that we're implementing the stuff we think we're implementing and that it works better than the prior pvq stuff.
- jm: it's been in review for 2 months, that would have been a first step.
- derf: i started going through it and have a bunch of comments but i haven't finished it yet.
- jm: ideally everyone would look at it an understand it and see what i may have been doing wrong.
- derf: when are you doing that monty?
- xiphmont: after i skim the PCS papers.
</pre>

DaalaMeeting20131203

2015-02-07T00:15:29Z

Daala-ts: Add word wrapping, remove blank line at the top

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2013-12-03

## Agenda

- CfL experiments (unlord)
- PVQ writeup (demo5, xiphmont)

## Attending

xiphmont, jmspeex, unlord, jack, gmaxwell, derf

## CfL experiments

- u: i did what jack suggest and did correlation between luma and chroma planes and between the chroma planes to see why we saw a smaller prediction gain in stats tool. what i'm looking at so far shows that there is some correlation between the correlation and the prediction gain. when i get it done completely i'll post the images. i haven't finished integrating the optimal linear regression yet since i got sidetracked landing lots of patches. it will show the optimal RD curve for the optimal linear predictor. will show us how much better we could be.
- d: optimal in regards to what?
- u: this is finding the very best linear fit in the encoder given that we know what hte chroma should be.
- g: this gives you the optimal alpha and beta
- u: i can print an RD curve so that if we had a magic oracle, this is the curve we could get. it should be showing up soon.
- g: in your performance results did you characterize it based on blocksize to see if it was blocksize related?
- u: the cfl stats tool?
- g: yeah
- u: i don't handle the 4x4 size correction. the tool doesn't work with luma 4x4. i tihink i have it pegged to 8x8 but it should work for 16x16.
- g: so you're not handling the hard case yet
- u: not really. that's interesting also. i haven't landed the patch for the tool yet although i did look at half the comments that monty provided. i'll finish addressing those with him and get it landed.
- jm: should we be trying to do chroma from luma and then the next chroma with chroma and the old chroma?
- g: we just the neighboring chroma
- jm: i mean using the other chroma plane
- d: it might be too expensive
- u: computationally?
- d: yeah
- g: no but using it some cheap way...
- d: that still has a parallelism cost.
- g: fair enough. what are you thinking with respect to parallelism now?
- d: well, parallelism is not the right word.. pipelining. all this stuff gets deeply pipelined.
- g: we could potentially do some things to interleave the code more tightly. the problem is that we serialize in the bitstream.
- d: i think those processes wind up being independent in hardware.
- g: ??
- x: i wanted to ask you more about them. let's talk about that offline.
- g: they are just simple regenerates of what you did before with the bugs fixed in the code.
- x: if we're going to have demos that are live documents, i might want to go back in and change the wording to make that clear. i've been posting these as archival documents. the only things that i'd like to change in the stuff i put up is typos; i haven't wanted to change the content. jm wanted to do this for the opus 1.1 demo as well. i think this will end up confusing and annoying people.
- d: the two cases are different. you've done updates before.
- x: that was the case where something was broken and the content didn't reflect the text. i want to put a little bit of thought into it.
- g: this case was a similar gaffe as the rotated blocks. the cfl stuff as i implemented it got broken during code review.
- x: i should replace it if it's broken data. sorry i misunderstood. i will go back and fix it. irc me the links again and i'll get it replaced right away.

## PVQ writeup

- x: reviews had been lagging so I have been doing that (by jack's request).
- d: thank you for that.
- x: i'll get back to pvq after i do the opus 1.1 addendum

## Travel

- d: greg and i are at the picture coding symposium next week. it's in san jose. we're not giving a talk. we're looking for potential interns and potential research grants.

## Resources

- g: i'm getting bottlenecked on cpu. i need a big machine.
- j: let's talk about that offline and try to figure out what's needed. maybe the whole research team could share some big 64-way machine.

## PVQ decoder

- jm: i started on this in vancouver, but haven't finished yet. it's next on my list.

</pre>

DaalaMeeting20131112

2015-02-07T00:15:00Z

Daala-ts: Add word wrapping

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2013-11-12

## Agenda

* CFL fixes

## Attending

gmaxwell, jmspeex, nathan

Tim, Xiphmont not in today

(From Nathan's status report:) Fixed bug in od_resample_luma_coeffs, -6% on rate +0.3dB in PSNR: https://pastebin.mozilla.org/3586591
http://people.xiph.org/~unlord/head_vs_cfl-psnr.png
http://people.xiph.org/~unlord/head_vs_cfl-psnrhvs.png
http://people.xiph.org/~unlord/head_vs_cfl-ssim.png
http://people.xiph.org/~unlord/head_vs_cfl-fastssim.png
http://people.xiph.org/~unlord/head_vs_cfl-luma4x4-psnr.png
http://people.xiph.org/~unlord/head_vs_cfl-luma4x4-psnrhvs.png
http://people.xiph.org/~unlord/head_vs_cfl-luma4x4-ssim.png
http://people.xiph.org/~unlord/head_vs_cfl-luma4x4-fastssim.png
http://people.xiph.org/~unlord/head_vs_cfl-luma8x8-psnr.png
http://people.xiph.org/~unlord/head_vs_cfl-luma8x8-psnrhvs.png
http://people.xiph.org/~unlord/head_vs_cfl-luma8x8-ssim.png
http://people.xiph.org/~unlord/head_vs_cfl-luma8x8-fastssim.png
http://people.xiph.org/~unlord/head_vs_cfl-luma16x16-psnr.png
http://people.xiph.org/~unlord/head_vs_cfl-luma16x16-psnrhvs.png
http://people.xiph.org/~unlord/head_vs_cfl-luma16x16-ssim.png
http://people.xiph.org/~unlord/head_vs_cfl-luma16x16-fastssim.png

Ran cfl_stats on subset1 and subset3, Cb predicts well, Cr not so much: https://pastebin.mozilla.org/3586503

In CFL fix the 4x4 case doesn't improve very much compared to the 8x8 and 16x16 case, but this isn't entirely surprising because the fix's effect was not as substantial in this case.

We thought the Cr/Cb difference sounded weird, might be explained by content, suggested swapping the two planes on input.

Jm replaced the PVQ encoder code in the encoder, but did not replace the code in the decoder as there is no decoder written yet.

So PVQ no longer works in the codebase right now. The current encoder code is 8x8 only as well, so it will us scalar for other blocksizes.

[We lapsed into Opus discussion at this point, targeting 1.1-rc release perhaps next monday]

</pre>

DaalaMeeting20131029

2015-02-07T00:14:30Z

Daala-ts: Add word wrapping, remove blank line at the top

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2013-10-29

## Agenda

- gstreamer post mortem
- IETF plans
- research team staff meeting
- roadmap

## Attending

derf, jack, gmaxwell, jmspeex, unlord

## gstreamer conference

- g: The conference went pretty well. I made a couple of interesting contacts. I presented on Daala and Opus and I crammed a bit too much into the presentation. There were a lot of people itnerested and grabbing me after. It took me a half hour to leave the room.
- j: How many people were there?
- g: Roughly 200 fit in the room, but there was standing room only for the talk. We got a couple of people who hang out in IRC who came to the talk. I believe there will be coverage about it in the next LWN.
- d: I was hoping either him or Corbett would show up.
- g: They emailed me after and asked for the slides.
- d: You were quote of the week in LWN: "Audio's boring since bandwidth is so cheap these days; just send everything uncompressed. Let's talk about video."
- j: Seems like it was time wells pent.
- g: I got some interesting feedback from people who wanted multi-channel in RTP.
- jm: Why did they want that?
- g: One person had a bid on multi-entertainment system in aircraft and they are going to provide 5.1 surround to the seats in aircraft.
- d: Give everyone quadraphone earphones for people with four ears.

## IETF plans

(some discussion of IETF plans)

## research team meeting

- j & d: plan is to talk for 15 mins, then take questions as they come.

## roadmap

- j: keep this stuff in mind and we'll sit down and write it all down in vancouver.

## PVQ review

- jm: when can I expect a review?
- j: so today?
- d: not today. I can try for tomorrow.
- jm: I'd like to have all three reviewers review it.
- g: I need to do Nathan's reviews first. And the SAD patches. One thing that goes along with that is that we have some problem in the MC code. The two code paths disagree and I haven't tracked that down yet.
- u: I'll get my review done this week.

## Chrome from Luma

- u: I've got some tools for this. By comparing it to what happens if we use the mode from teh luma block and does freq domain intraprediction. It showed 1.5dB of prediction gain which I thought was good. My other test I want to do is to do something similar to an RD curve and look at how the prediction gain changes as we scale up the quantizer. I'm currently doing lossless but it might be interesting to see if the accuracy of chrom from luma falls off as the Q goes up.
- jm: i've been thinking that there are going to be itneresting interactions between CfL and PVQ once we get PVQ hooked up. Because if you are looking not at DC but at AC then if we get the gain of the CfL wrong it won't affect PVQ all that much because PVQ would be able to encode a state of 0 and code a change in the gain that's required to get the actual chroma. To a point where if we're doing that we'd just want to predict the gain that we give to the PVQ; ie, not compute an actual prediction...
- u: this is with partitioning?
- jm: with or without partitioning. In both cases the only thign that changes is how to compute that particular gain. When you are using PVQ the actual normalized prediction is the same regardless of whatever the gain is. There's probably going to be cases where CfL fails for scalar quantizaiton but works pretty well for PVQ quantization.
- u: I can definitely see that. The other question I had is the stuff Greg did creating the modes for the three neighbors based on the DC. Which filters was it using?
- g: The filters that were in at the time.
- u: The same weights for the modes at all block sizes?
- g: Yeah. Just 4x8 filters. It was done before we had multiple block sizes. I would expect that the mode mapings might not be the same. It's easy enough to train values for those. i can post the littel tool I used to do it.
- u: If you post that I can integrate it into the tools I have that use the actual filters so it will spit them out. The other question: the thouht that CfL has worse PSNR. Is that because it includes the cost to signal the mode?
- d: Yes. It had somewhat lower rate and a good chunk lower PSNR. I didn't have good graphing tools then. We could do three things: CfL, intraprediction using Luma's mode, and we could also signal a mode.
- g: We had some speculation that when CfL was making mistakes it might follow constrasts in the luma plain and this might mask things. What we really should have is some SSIM metric that combines all three planes and measures each of the planes against that. Everyone in the literature does all monochrome stuff. We could use the TID2008 database to check this out. They took a bunch of images and sent them through various distortions and then got lots of people to rate the quality of images. You can take your perceptual metric and have it rate the images and compare to see how much agreement there is with the TID database. This is one of the things that people used ot argue that metrics like SSIM are useful. The TID database shows they agree with humans. We could modify an SSIM tool and see if that agrees better with the TID database than monochrome SSIM does. It doesn't have to be a lot of work, but I haven't considered it terribly important.
</pre>

DaalaMeeting20131022

2015-02-07T00:14:05Z

Daala-ts: Add word wrapping

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2013-10-22

## Agenda

- even/odd quantizer behavior
- determinent 1 search results (gmaxwell)
- IETF
- CFL using luma sse

## Attending

derf, jmspeex, unlord, gmaxwell, xiphmont

# even/odd quant zigzags

>>- g: Anyone have any theories on the difference of the even/odd behavior with >>scalar quantizers.
>>- no
>>- jm: we may not be doing the rounding right
>>- m: I can look at that.
>>implying

- m: Still intend to look at this next I think

## determinent 1

Greg dropped off mumble, can't seem to get back, ah there he is
has filters for all the sizes, not run videos through them
not trained intra stuff for 16x32; the training stuff is not working
nathan has patch that does it in the codec, waiting for greg to review

CG for raw transform, no prediction: 'I don't know yet'

Does it quilt/block/do 'funny' things?: Checked 4x8, 8x16 'nothing too funny' but not quite a smooth-- not quilt/block just not quite as smooth.

## IETF

IS it good to continue having Xiph/Moz there in force when we're not there to stack votes? :-D
We don't vote in the IETF!
In-duh-viduals
And we don't stack either (yeah, all four of us)
...this never happened?
TO THE TIME SLIDER

- m: OK, established I'm going, need to get onboarding more in hand to book travel

# CFL using luma sse

http://people.xiph.org/~unlord/crfl_v_luma_sse_4x4.png
"Chroma red from luma vs Luma"

http://people.xiph.org/~unlord/cbfl_v_luma_sse_4x4.png
"Chroma blue from luma vs Luma"

X axis is prediction error (residual) in the chroma planes
Y axis is prediction error (residual) in the luma plane
All 4x4, no basis scaling
scatter plots of summed squared errors for Chroma from Luma prediction vs simple luma prediction (fruits only)

dude, mismatched axes

dude, axes not labelled

No obvious correlation between errors

JM suggests looking at the goodness of the least squares fit as a hurestic in the bitstream for how good the CFL is working

GM points out that at least in some cases there appears to be a dynamic range issue in the coeffs that aren't used in the training, so looking at the goodness of the training fit may not help there
Goal is to come up with a CfL that, when it screws up, is not worse than nothing, or is seldom worse than nothing
Lots of discussion of looking at optimal results vs the corner cases we're seeing....

Nathan is on it

JM at AES saw a FHG booth directly going after Opus:

http://jmvalin.ca/misc_stuff/AES_show_FhG.jpg

They had a couple samples all at very low rate comparing Opus to USAC and prerelease versions of the 3GPP EVS.
</pre>

DaalaMeeting20131008

2015-02-07T00:13:44Z

Daala-ts: Add Word Wrapping

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2013-10-08
## Agenda
## Attending
jmspeex, unlord, derf
jmspeex wants as many people as possible to review his PVQ code.
derf can't promise to do it for several days, possibly a week, but jmspeex thinks that's fine.
unlord promises to take a look.
derf suggests explaining it to Sean Silva, who seems to be interested and not an idiot.
unlord almost done modifying the codec to TF down everything to 4x4 for intra prediction, will have code to review shortly. TF'd 8x16 appears to be no worse, despite predicting from half the data.
Should we have a flag to switch this on and off?
derf: Will we ever run non-TF'd 16x32 predictors again?
This idea is mostly for testing. Still useful to have rd-curves of full predictors (non-sparse, maybe non-TF) to compare to sparse alternatives.
jmspeex: Perhaps to speed up training things we should use a couple iterations of Gauss-Seidel instead of re-solving the system every time (cutting out a factor of O(N)).
derf: More important than that would be to use a robust least-squares solver to handle outliers. We could probably modify things to use Gauss-Seidel as part of that, since these things are usually iterative anyway.
unlord: We're currently running all our training unweighted, because it wasn't clear that the weights were the right way to get rid of outliers.
jmspeex: The other thing we should look at is L1.
derf: We'd need a sparse linear programming solver.
unlord promises to look into getting one.
derf will dig up the computer vision paper that describes how to convert L1 minimizations into LP.
unlord: Talked about Chroma from Luma, what exactly is involved there?
derf: Deciding when to use the AC prediction or not (have a symbol to clamp alpha to zero, figure out probabilities from sudden luma changes, etc.)
jmspeex: PVQ already has a flag that decides whether or not to use the predictor for AC.
jmspeex: Decision between "use prediction" and "no prediction or gain is zero" had an entropy around 1.2 bits for all 4 bands of Lena image with 8x8 blocks.
Consider adding a few lines of code in Chroma from Luma to turn off the alpha's (non DC component) based on a luma threshold
Gather statistics to know how to handle cases where CfL predicts poorly. Need to measure:
1) How well does CfL predict the current block from current Luma
2) How well does current luma "match" neighboring luma
3) How well does intra prediction work on current chroma (from neighboring chroma).
-In the case where there are 4x luma blocks for each chroma block, use the luma UL mode for prediction.
</pre>

DaalaMeeting20130924

2015-02-07T00:12:57Z

Daala-ts: Add word wrapping

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2013-09-24

## Agenda

- grant
- unlord's training results
- getting some other eyes on gmaxwell's 32x32 code ( https://review.xiph.org/65 )
- monty's latest 'I'm so confused', but at least it probably explains unlord's TF2 training results

## Attending

derf, gmaxwell, jack, jmspeex, unlord, xiphmont

## grant

(discussion of research grant)

## unlord's training results

- unlord: looking at the graphs, it was dramatically better than subset1. when it got to the end it went right to zero.
- derf: we didn't save any of the intermediate steps did we?
- unlord: i don't believe so. that would be a good approach though is to start from the intermediate steps and work backwards.
- gmaxwell: my copy of hte training tool saves intermediate steps.
- derf: why isn't that in git?
- gmaxwell: i thought only i thought this was useful.
- jack: is it bad that everythign went to zero, and do we have to start over?
- jm: you probably need most of subset3 for 16x16.
- derf: the whole point of this was we needed lots of data to prevent overfitting. as for going to zero, we knew that would happen at some point, but weren't sure where that point was. now we know where the cliff is.
- jm: how far is it from a reasonable amount of coefficients?
- derf: look at the graph
http://people.xiph.org/~unlord/subset3_16x32_modes.png
- derf: it had dropped most of the coefficients before it fell off the cliff.
- jm: is it dropping them linearly?
- unlord: yes
- derf: that's much worse then.
- unlord: I considered dropping them weighted by the variance. if we run it again we may see something better.
- derf: we talked about switching at the end to something that is computationally tractable by doing something smarter.
- jm: maybe we can save a lot of cpu time by assigning modes upfront. doing a bit of training without decimating, and then freezing that. it will be a lot faster.
- gmaxwell: it does do that. it doesn't retrain after every drop; it batches.
- jm: you drop one coeff at a time?
- unlord: no, 256 at a time i think.
- jm: but you drop one at a time in terms of least squares fit?
- unlord: yes
- jm: i think you can safely drop a whole lot up front and it's only near the end that you need to be really careful.
- g: i don't think that's true but ti's certainly something we can test.
- jm: how many coeffs do you have? 4*256^2?
- unlord: no. 16x16x4. it's 1024 multiplies at the end
- jm: you're trying to rpedict 256 coeffs from 1024 inputs each. so that's 4x256^2=262k.
- derf: that's where we start yes
- jm: i think i fyou pick the first 10k that had the least impact, then it's quite unlikely that you should have not dropped that and used it to the end.
- g: the problem with that is that the coefficients are co-linear so they start with small magnitudes but if you drop them they have large magnitudes.
- jm: i think the first half would have very little impact.
- g: We should be cautious about taking this as fact because it's clearly not true when taken to an extreme
- unlord: one that is to compare with 8x16 (tf from 4x8) and see if this is any better.
- jm: if you do all sorts of approximations like dropping more coeffs at a time. these shortcuts will make training worse but i expect it won't interact with the other stuff you'll try. if it takes 3 weeks you can't run enough experiments.
- x: are you seraching on quantized coeffs.
- u: these are doubles
- x: the final target is 6 bits? 7 bits?
- derf: yeah. the question is how much do we need to not sacrifice too much performance
- g: if there are fewer mults it's easier to have higher precision.
- x: i had some thoughts on how not to get hung up on corners for low precision training. i'll try it on my stuff first. i have some neat ideas about machine search.
- derf: can you write them down
- x; i can't code things unless i write them down carefuly first. the only reaosn i haven't done so yet is because it's not what i'm supposed to be working on. i'm supposed to be working on demo4.
- jm: what is demo4?
- x: chroma from luma. we had some demo stuff and i wasn't going to a hell of a lot of detail. we've got a pretty good conceptual handle on the other demos, and this one we aren't quite as specific. my search thoughts are about how to constrain exhaustive search. it's all straightforward i just wanted to see if it worked first.

## greg's 32x32 code

- g: i have a patch for 32x32 in the review tool and it's working fine except that it hurts performance. there's no obvious bugs, but i can't figure out why it doesn't help.
- derf: what is it now?
- g: maybe 10% larger files for same quality. the images look fine and it runs fine in lossless.
- derf: but 10% is a lot.
- jm: compared to what?
- g: compared to code in git on a few images.
- x: what kind of measurements?
- g: psnr and bi-i?
- jm: how do you adjust the quant step size of 32x32? you're using a mixed block size or just everything 16 or 32?
- g: mixed block size
- jm: could it be an issue of 32x32 not having the right scale factors?
- g: i tried changing around the scaling and wasn't able to make it work better
- derf: could you make it work worse?
- g: yes. this wasn't an instance of running git stash before compiles :)
- g: i'd appreciate if someone could look over teh patch.
- x: how much intermediate stuff does it output?
- g: i cut the patch back so it's implementing only the simplest form, no prediction or anything
- jm: one test you could try is to tak ea synthetic gradient, something relaly smooth, and encode it with 16x16 for 16x16 and 32x32 forced.
- g: that's a good idea, i can try that. i've been comparing with air force which has really smooth backgrounds.
- jm: for example making sure it's not the block switching making the wrong decisions
- derf: the point of using a smooth gradient you can just compute the right answer
- g: i don't expect complete flat to help because it's already predicted???
- jm: if you end up coding all zeros, you'd have 4x fewer zeros so it still should be smaller
- g: i hadn't tried it because i expected 16x to be smaller or tied in any case.
- jm: you could even take an actual image and blur it.
- derf: why were you setting runpvq to zero in your patch?
- g: to remove pvq as a source of problems
- derf: so you're testing both ways?
- g: when i'm speaking here i'm talking about all scalar comparison.
- j: who's going to review the patch?
- derf: i'm looking at it.

## monty's confusion

- x: last week i was supposed to look at the coding gain of dct + tf + 2nd stage tf to see how it compared to dct of native block size. i wanted to see how much more efficient 2nd stage tf was. and whether we could ditch bigger dcts entirely and just tf 4x4 and 8x8. i quantified just what the efficiency difference was. one of the unexpected things if i took a 16x16 block and transformed with 4x4 dcts and tfed up to 16x16 that was reasonable efficiency. however if 4x4 lapped and tfed up to 16x16 the 2nd stage tf decreased coding gain. one thing i noticed, if i did 4x4s that were lapped as 16 and then tfed up, it was nearly identical. i have not chased this to its logical conclusion. i wonder if my 2nd stage filter is doing in freq domain what lapping does in the spatial domain. it would be useful to know that because it's an additional tool in the toolbox, but it's one of those things that i didn't expect it to be a liability. i didn't expect it to be similar to spatial lapping.
- def: i'm confused on what you said was identical
- x: if i take 4x4 blocks that have not been lapped 4x4 and tf them up to 16x16 my seodn stage TF filter produces coding gain X. if i take the 4x4 blocks and lap them as 4x4 and TF them up to 16x16 using 1 stage TF the coding gain and output turns out to be identical.
- d: they can't be because of blocking artifacts
- g: do you have artifacts?
- d: yes, he's not lapping outside, so at least 16x16 blocking artifacts.
- jm: but if it works inside then you can apply the same reasoning with blocks outside lapping
- x: it appears the 2nd stage TF does something similiar to 4x4 internal lapping. if we leave aside external lapping, only looking at 16x16 block internally, and whether lapping is done within the block, it appears that 2nd TF stage is doing something similar to 4x4 lapping.
- d: at this point i would spend some time with malvar's original papers on ???
- g: i recall there was a diagram in those papers that looks a lot like you're improved TF thing. i'll see if i can find the paper i'm thinking of.
- x: i was going to chase that a little farther because it would be an interesting connection between those two things
- jm: one that that could be interesting is couldn't you compute the matrix that's perfectly equivaelnt to lapping in spatial domain?
- x: yes. as well as the matrix that would compute the DCT.
- jm: maybe the most interesting thing to look at is the basis functions for the lapping and the TF 2nd stage. what they look like in the spatial domain and how close they are.
- x: i generated graphs to look at that but didn't look at it from this angle. it's also the case that if i want to look fora 2nd stage TF filter that improves the lap transform i'm going to have to do that directly. we know now the second stage TF is not going to improve the lap transform as we currently have it. it's going to improve the transform when we aren't lapped. i'm not sure which is better, lapping or 2nd stage TF.
- jm: efficient computationally?
- x: yes. coding gain appears identical.
- jack: when do we not lap
- x: we always lap. i've been exploring ???. if we could dispense with DCT and use TF we'd gain some computational margin or prediction could work form the exact transform instead of approximation.
- un: i saw that at high rates it was a slight improvement, but low rates it was a liability.
- x: when you were Tfing to retrain the predictors. were you going 4x4 to 16x16? were they fully lapped?
- u: what i did was go between 4x4 and 8x8 and when i did the training I trained on having 8x8 lapped with 4x4.
- x: the second stage TF would have been a liability in this case. i did not realize that until the end of last week.
- u: we're not going to be lapping 4x4s with 8x8.
- x: unless we decided that variable block size was using variable lapping. that was one of the things i was exploring. it might not be a good idea.
- d: i don't see that being worth it. that starts to matter when we start to get beyond 16x16. that's why greg's doing 32x32 the way he's doing it. but i dont' see it being useful at smaller sizes.
- x: how does the complexity of hte lapping filter scale?
- d: linearly
- x: what about register pressure?
- d: sub-linearly
- g: way sub-linearly.
- x: there was the other observation that if it really is equivalent then does that enable any interesting hack where we can reuse traditional architecture?
- d: the thing that makes everything complicated is the lapping, not the block transform.
- x: does the support only matter to low frequencies. i had a theory that it mattered very little to high freqs because even in textures there not so regular that you were going ot get extra compaction in HF. that was false. the HF benefits from larger supports as well.
- d: jm has me wondering if that's the right measure?
- jm: which?
- d: coding gain.
- jm: it's not.
- x: going by coding gain, it was the correct measure in this case because it's a theoretical higher bound. it's the most optimistic assessment of what we're getting out of it. the additonal compaction from greater lapping is not primarily a benefit to DC and near-DC. assuming no bugs in my code.
- d: at least your result confirms what i suspected to be true.
- x: it's not the transform it's the support. i was looking at can we reuse traditional architecutre if ???. we can't reuse traditional dct infrastrcuture that way.
- x: it's also nice to know why JPEG-XR didn't work.
- d: begs the question of why they did that
- x: maybe they didn't test with anything meaningful. the visual output seems to confirm what you think is going to happen. DC has much larger support. it elminates a lot of the blocking. but at the same time, it's not actually more efficient coding wise. there's still the thing that the flat gradients in the background are a lot smoother. you haven't improved coding gain but have improved visual stuff perhaps. if you've decided thisi is better and build a test to confirm, then you confirm it.
- jm: another thought i had. did you try instead of designing 2nd stage TF matrix actually using PCA?
- x: pCA?
- jm: KLT
- x: i didn't try KLT
- jm: i'd be curious if the KLT looks like the 2nd stage you've designed. i suspect it might.
- x: how would i compare those?
- jM: you look at resulting basis functions.
- x: the KLT basis functions will be different for every input
- jm: you train a KLT
- d: you can't train a KLT on one input you need a collection
- x: if you train on a bunch of inputs, then that bunch is your input
- d: but you're assuming it's representative of some population
- jm: if you compute teh KLT for spatial domain coeeff you find it's close to DCT.
- x: i have not done that but i have no shortage of things to try next.

</pre>

DaalaMeeting20130917

2015-02-07T00:12:34Z

Daala-ts: Add word wrapping

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2013-09-17

## Agenda

- daala-private list?
- summit innovation fair
- research talks (strategy, pvq, neon)
- coding party agenda
- TF ideas (monty)

## Attending

greg, jm, derf, unlord, jack, xiphmont

## daala-private

- jack: should we create this for private discussions?
- derf: no. this won't be a problem once monty leaves RH.
- ???: To be fair, it's not a problem so much as RH policy is still to keep any inventions secret until preliminary filing

## innovation fair

- jack: should we do this?
- gmaxwell: yes, we should do this.
- jack: i'll sign us up. probably just for toronto (me) and santa clara (greg and others)

## research talks

- jack: tim will give a talk to the research team on halloween. we also thought it might be good to give a talk on pvq as well.
- jm: it probably makes more sense in something at the coding party. i wanted to have these talks to introduce a topic.
- jack: let's give them at the coding party and then decide whether they warrant wider distribution.
- jm: my goal is to get everyone on the same page and a wider talk would be more for presenting something that already works. for this talk i want to be interrupted with specific questions and that would bore people from outside.

## coding party agenda

- jack: invited alon to the meeting so we should give talks on monday or tuesday. probably need a talk on the training infrastructure by nathan since he has a ML background too. pvq talk as well.
- greg: training infra is a good way to onboard new people.
- nathan: we need metrics for them to see if what they are getting is ok.
- jack: we have hte metrics already right, so we just need to document what they are and what hte numbers mean.
- nathan: i meant guidance for what is a valid test and what we're looking for. for example, is it sufficient to test on still images or do we need to include video.
- derf: that depends on whether or not we're testing motion compensation.
- jack: let's plan for tuesday pre-lunch for pvq and training talk.
- derf: we should probably come up with a bunch of projects that people can work on that people can complete in a week.
- greg: there's additional interest around the office. a few people plan to show up.
- jack: we have lots of infrastructural things like merging from reviews, etc.
- jm: I don't like automatic merges.
- greg: separate from the merge stuff i'm fine with it being manual. a buildbot that automatically did reviews by attempting to test and collect metrics.
- nathan: this can feed into arewecompressedyet.
- greg: it would be the same infrastructre but running on uncommitted things.
- jm: it would be nice to upload stuff to try and see what the metrics are. having some kidn of aggregated metric when you make a decision about whether patch is good. ideally you want a single metric to see whether th enet effect is positive or negative.
- greg: nathan's stuff already gives you two numbers like this. because it's across multiple rates it's sometimes opaque.
- jm: when you say multiple rates, you mean it doesn't get the same SNR on one point but at multiple quality levels
- derf: it's looking at two RD curves and computing the gap between them.
- jack: seems like we could farm all this stuff to amazon.
- jm: i guess we want both the numbers and the curves as output. if i get those then i know if the patch is any good.

## TF ideas

- greg: jm, do you understand what monty has been doing?
- jm: i think i do. if it works it makes a lot of sense and it seems to work.
- greg: it sort of works. as tim was pointing out on irc it has problems in the textured areas.
- jm: what kind of problems? you're talking about the version with TF on AC?
- greg: or not
- jm: what's hte problem?
- greg: the area on chapel building doesn't look as good as the lapped transform case.
- jm: maybe 2 months ago i proposed something similar. basically i proposed doing lapping on 4x4 and another level of transform, and all i got as a comment from Tim was JPEG-XR and i haven't figured out why it's bad.
- greg, derf: we said the same thing to monty.
- greg: the same thing that wavelets coders are doing where you don't have ringing but you don't geting any coding gain.
- jm: i'm talking about the case where you do TF the AC.
- greg: that's not what JPEG-XR does.
- jm: my original suggestions was do 4x4 lapping and 4x4 dcts and then do another level of lapping for each coefficient.
- greg: that's not what monty is doing
- jm: i'd like to know what's bad about this. i thought the quality looked generally good? what if we were to apply the lapping on the AC too?
- greg: the only other problem is that it leaves you to switch between 4x support and 16x support. there's no way to do the lapping with 2x that's not just really trivial. there's no room for the rotation.
- jm: 4x4 lapping 8x8 may not be that bad. if you do it as a 4x4 and go up to 8x8, one issue is that you can't do a lapping of 2. i'm saying don't do lapping there. wich is 8x8 transform with 4x4 lapping, and i don't see why this would be bad.
- greg: it has a lot less coding gain than 8x8 lapping.
- jm: it's going to have worse coding gain but better ringing characteristics. and if you're 8x8 you are close to somewhere that coudl ring. it's worth testing but i suspect overall you won't be worse.
- greg: when i switch 8x8 to 4x4 lap everywhere it costs 0.3dB. it's rather a lot. that's including on places where it would be on the border of a 8x8 and 16x16 block.
nathan: your test included 16x16 blocks?
- greg: correct
- nathan: the version i ran only has 4x and 8x. that's why we're different.
- greg: i ran it both ways and got similar results. that's interesting.
- jack: this seems very rube goldberg machine
- greg: this is actually simpler than what we currently have in the codec. things like h.264 and vp8 have second level transforms. vp8 and h.264 both do a degree of second level processing but are limited. monty has been talking about doing more complicated things at the second level.
- jm: do you have a link to the stuff it does on the front of the chapel?
- jack: why do we TF at small sizes. i thougtht it was for 16x to 32x
- jm: because of intra-prediction we don't have a chocie. unless you want intra predictors for every combination of modes you have to use TF. if you have a 16x block and above you use a 4x4 block then you have to TF to 16x16 to do a intraprediction.
- nathan: this is because we trained where all your neighbors are 16x16
- greg: monty is exploring replacing smaller transforms with TF versions.
- jack: does this buy us speed?
- greg: potentially.
- nathan: this is also an alternative to banding because you can lump together the DCs.
- greg: i wouldn't say that. it let's us have signalling choices.
- jm: looking at the front of hte chapel, i'm not sure it's actually worse.
- greg: you have flat areas and textured areas which is the classic wavelet problem.
- jm: with or without AC coeefs
- nathan: we use TF on chroma from luma to predict 4x4 luma blocks from 2x2 chroma blocks.
- jm: this hapens if the HF have too little support. this is definitely not something we should do. i always considered the version where you do TF on AC as well.
- jack: why don't we train on all combinations?
- jm: if you considered all the possibilies you would do millions of things.
- nathan: there is also overhead on signalling. you might have more info than you need.
- jm: if you didn't mind carrying hundreds of MB of tables, then we should do brute force. even if we could train this at scale, there's no way we could include all the coefficients.
- jack: how much data can we ship with encoder and decoder?
- greg: we want this data to fit into L1 cache. couple hundred kilobytes. once we've converted to fixed point and dealt with precision we don't have a ton of free space.
- jack: why do we do so much work on intra prediction in general?
- greg: it's a lot of bits and it's where hte interesting challenges are.
- jm: if we do the predictors up to 16x16 considering all the possible combinations of sizes, this would multiple the number of predictors by 83k. the three predictors we have take 150k of space. if we included everything the decoder would be 10GB.
- greg: if we plot out the FF download size over time, we'll fit this in a couple of years.
- derf: target cpu complexity should not be a modern desktop processor.
- jm: every 16x16 block can be split 17 ways so there are 17^4 combinations.

## Opus 1.1

- jack: what will it take to ship this?
- jm: i'm temporarily off the critical path i think. what's left is merging the arm neon stuff and then a lot of testing.
- jack: who's doing the testing
- greg: i am
- jm: i did what i wanted to do for opus. there's one minor feature that coudl be done by someone else. the only opus stuff on my todo list is writing an article for the IETF journal. the only other stuff i have is daala.
- jm: i keep getitng interrupted with issues like google dropping opus for complexity reasons.
- jm: i've changed focus on pvq writing stuff with double precision. it will take longer to get to a decoder but the research will be more reliable.

# JM updates on PVQ work

- jm: Checked in a double precision rewrite. Checked on still view hardly codes anything on the still blocks
- jm: Been wondering what to do with the gain, quantize the gain or the gain difference. The difference is that if it quantizes the gain directly then the reference value may not be a possible quantized value.
- tim: Yes, thats a problem.
- jm: One option is to code the gain as an integer, the other option is to code it as an integer difference. tim: I think it should be the second one.
- jm: [what happens when symetric deltas would result in negative absolute gains]
</pre>

DaalaMeeting20130911

2015-02-07T00:12:10Z

Daala-ts: Add word wrapping

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2013-09-11

## Agenda

- logistical
- status updates
- brainstorm coding party agenda

## Attending

xiphmont, jack, jmspeex, gmaxwell, derf

## logistics

### meeting change

- jack: i suggest 9am pacific on tuesday

### #daala channel

- jack: I suggest we do daala talk in #daala.
- (not much argument)

### status updates

#### travel

- Monty: Daala coding party: Sept27-Oct1
Disneyworld with family: Oct7-Oct14
- derf
12-17 Sep. IBC (Amsterdam)
1 Oct.: SF MusicTech XIV (probably not going, but someone maybe should?)
4-6 Oct.: Summit (MV)
9-11 Oct.: IEEE Broadcast Symposium (San Diego)
3-8 Nov.: IETF 88
11-15 Nov.: TPAC (not going to Shenzhen, defecting and going to Seattle)
17-18 Nov.: FOMS (San Francisco)
19-21 Nov.: WebRTC World West (Santa Clara... maybe not going)
8-11 Dec.: PCS 2013 (San Jose)
12-13 Dec.: Packet Video (San Jose)
- gmaxwell:
4-6 Oct.: Summit (MV)
3-8 Nov.: IETF 88
17-18 Nov.: FOMS (San Francisco)
8-11 Dec.: PCS 2013 (San Jose)
12-13 Dec.: Packet Video (San Jose)
- jmspeex:
CCBE Sep 26-28
Coding Party Sep 28-Oct 3
Summit Oct 4-7
AES Convention Oct 16-20
IETF 88
- unlord:
Daala coding party 9/26-10/7
IETF 88
- jack:
Coding Party
IETF 88
Platform rendering work week (paris) oct 21-25
Samsung (Seoul) Nov 14th-??
Moz Summit
Strange Loop Sep 16-19
LCA Jan ??

- xiphmont: I am shaving yaks. None of the mastroska tools can generate [completely] valid WebM. mkvmerge doesn't obey the profile. mkclean doesn't reliably get timestamps in ascending order. Apparently this isn't Matroska but is in WebM and Chrome hangs if they timestamps aren't in order. And it's not deterministic. What you get out of mk clean dpeends on what you put into it.
This week I have RH legal stuff (re 2-stage TF).
I'm still working on getting machine search of lapped transforms working. If we're going tform 4x4 to 8x8 we have to account for the lapping between the 4x4s. If we're splitting, then the second stage TF filter will need to simulate adding overlap between the two blocks that we split. I'm not optimizistic that we're goign to find anything, but I'm still looking.
This week: continue machine searching for lapping filters, lawyerness, and get started on next demo (ChromaFromLuma)
- jmspeex: "What lapping are you actually assuming we're going to use?"
- xiphmont: i'm assuming that our transforms are lapped with the matching lapping and that's what i'm trying to simulate. i think that's the right place to start. if we go to building 16x16 out of 4x4s then we lose some of the benefit of having the greater support.
-jmspeex: that is mostly useful for the larger blocksizes.
- xiphmont: this is all an approximation and i'm not sure it's good enough yet that that actually matters. I have bugs and have to find them. I got really lucky finding the first filter that I did.
- jm: lucky in what sense?
- xiphmont: I made a wild guess and the second stage TF popped up, but so far I've found nothing else.

### jmspeex

- jm: i've been working on getting Opus 1.1 out the door. Nothing really new on Daala.

### gmaxwell

- greg: I'm still hung up with 32x32 support. I went off on a tangent trying to search for lapping filters with the TF in place. So I have a modified coding gain tool that measure it. We prefer different prefilters when teh TF is in place.
- derf: what are you trying to optimize for with TF?
- greg: Basically I'm doing empirical coding gain using the variance on the image data.
- derf: what does that mean?
- greg: I'm TFing 16x16 up to 32x32 and then changing the lap params for the 16x32 prefilter.
- derf: ok
- unlord: this is just the part that's on the inside of hte block?
- greg: no, inside and outside. the reason i started messing around with this was to make sure that i was applying the TF correctly. So I broke that out and started measuring.
- unlord: this is sort of like the questions i've been having. should you do something different than the normal 16x16 if you're going to TF it together. this is sort of what finding a better filter on the inside looks like.

### unlord

- unlord: http://people.xiph.org/~unlord/subset3_16x32_modes.png
that's been running along. it doesn't look like it's ....
I can post the individual data too so you can see the coding and prediction gain output.
- greg: good. it's not totally busted anymore.
- unlord: before when i was doing the rollup that looked terrible. it was totally because of the overfitting moving everything to mode 0. this may be a reason to rerun all the 8x8 on subset3. i'll know more when i plot it bits / pg.
- jm: that figure is the prediction gain after full sparsification?
- unlord: no, we're at step 150 or so. this is after sparsifying 60% of the way. it has 40% of the coeffs.
- jm: you're trying to reduce it how far?
- unlord: 1/262. quite a bit. this doesn't mean anything except that before when looking at subset1 this graph was terrible. http://people.xiph.org/~unlord/subset1_16x32_modes.png
the other thing i've looked at is in the traiing tool. i modified the codec to only use 4x4 and 8x8 blocks, and then i ran the RD curve generation on subset1 with the second stage TF enabled and disabled. basically our current codec and the application of hte filters TF up and down. I thgouth maybe we are seeing an improvement on the smaller blocks. For low rates TF was slightly better and high rates was slightly worse. Greg brought up that doing it both ways, with TF and with TF + filter. So I'd have to run the experiment four times. My theory is that pairing these correctly should work.
- jm: there's two reaosns you can see it doens't imrpove. one is that it doesn't work well enough and the other is without it it works well enough.
- unlord: maybe this is being overwhelmed by compression loss somewhere else. is there a different suggestion for how to test that?
- jm: how to test whether the current one is good enough?
- unlord: yes
- jm: the only way i can think is to do this with 4x4 and 8x8 but lapping of 4 all the time and you can doa losless TF for that. with that config you can do lossless TF between 4x4 and 8x8.
- xiph: that's worth playing with but i'm concerned about deploying it.
- jm: no, i was talking just measuring performance with that, and if it's not better compared to the stupid TF then it means you've squeezed the area and you aren't goign to get any better with the second stage, but if there's a gap then you can see where the second stage gets you within the gap.
- unlord: the predictors are horrible if you do that.
- jm: no you've trained the 4x4 on 4x4 and 8x8 on 8x8. if you do it lossless it means your predictors are optimal.
- unlord: the training was on 8x8 lapping
- jm: you'd have to retrain with 8x8 blocks and 4x4 lapping. doing lossless TF would give you the best case. running without monty's second stage gives you the wrost case and then you see if there's any meaningful difference that's worth trying to get back.
- unlord: ok. i can run that one.

### derf

- derf: i have been mostly focusing on opus 1.1. most of my daala work is dumping stuff into jack's brain.

## travel schedule

- everyone but monty headed to IETF in vancouver week of nov 4th.
- jm: i'm only going to one day at CCB in toronto before the coding party. and going to NYC for AES in october
- derf: What are people working on next week?
- xiphmont: TF, and then the next demo (chroma from luma)
- jmspeex: Continuing on PVQ (play around with theta quantizer and see if we can use prediction properly)
- gmaxwell: actually finish 32x32 support, put images up
- unlord: finish experiment with TF and intra prediciton
- derf: Coding party planning
</pre>

DaalaMeeting20140715

2015-02-07T00:10:19Z

Daala-ts: Remove extraneous blank line

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2014-07-15

Mumble: mf4.xiph.org:64738

# Agenda

- reviews
- code party
- https://daala.etherpad.mozilla.org/coding-party-201408
- IETF 90
- https://daala.etherpad.mozilla.org/video-codec-ietf90
- Visual Information Processing and Communication submissions
- tech discussion: new intra pred idea (jmspeex)

# Attending

jmspeex, td-linux, derf, bkoc, smarter

# Reviews

nothing to report. assigned a few to nathan since he wasn't here

# Coding party

Need to ping people you haven't heard from next week.

# IETF 90

tim needs to add agenda items

# VIPC

- d: sent email about this to the internal alias. extended abstracts are due next week.
- jm: i don't like copyright assignment. that's my only issue.
- d: i will raise that with the organizers.
- j: who's going to submit stuff to this? we should submit at least two things, since they want to do a special session on daala.
- jm: i think it makes sense to have a pvq paper
- j: should we have an overview paper?
- jm & d: probably not, can just cover the needed intro stuff in the more meaty presentations.
- d: we could do something on chroma from luma.
- smarter: what about the entropy coder?
- d: what's new there?
- jm: if we did, it would be use of multi-symbol in video coding.
- d: we could talk about the intra coding probabilities and the laplace encoder.
- jm: we could mention it, but i have a feeling that it will be completely rewritten.
- j: what about your motion compensation paper?
- d: it's less interesting if you dont' do CGI, but i can try to come up with some results to make that interesting.
- jm: for what pieces of daala do we actually have results that we can present?
- j: nathan might be a good person for CfL
- d: nathan might want to write up stuff on training the transforms.
- jm: if extended abstract is 3-4 pages, how long is the actual paper?
- d: fantastic question!
- u: oct 6 is notification.
- d: there's no guidelines on length. make it as long as you want.

# jean-marc's intrapred ideas

- jm: i mentioned it to tim last week but wanted to see if anyone had ideas for how to implement it. my latest idea is to instead of doing prediction at the same time as coding, do a first pass trying to get all the edges approximately right and trying to code the difference. for instance, chop the image into differnt block sizes. you code a 1d dct of all the edges, and then you apply an intrapredictor to that (standard type of intrapred based on the border). you get a coarse image and you use this as a standard reference just as if we had motion compensation.
- d: i don't see any reaosn it wouldn't work, but i don't know how you plan on handling the sudden changes at block boundaries.
- jm: you mean dealing with blocking? i'm still trying to figure that out. the other one i'm having a harder time with is how not to do it completely dumbly. the easiest way is to take all the border pixels and code them all with a 1d dct. once you have them all you find the best predictor and then fill all the contents. with the dct you are applying you're only looking at the border pixels not inside the block, and you should probably use those. when you're quantizing the 1d dct, you're not even looking at what's inside the block. you're only looking at what's in the border. so if there is noise in the block, you're not taking that into account.
- d: what you want to do is given that this is the prediction you're going to be using, what is the optimum set of pixels to code there.
- jm: i would like to have it structured such that having a greedy search is not completely dumb.
- d: i don't htink this breaks that. i think the pixels you want to code are going to be different based on the mode you chose.
- jm: anytime you're encoding a block you'll already have the top and left edge of the block with the dct already. then you need to figure out the intra within that block and a vague idea i had is that hte top and left edges are already coded. you decide on the mode you'll be using for every block of the image, and then you code the 1d dct, and every time you code one you look at both sides of that edge. and then you do this kind of blending. on every block you would extend from 2 edges and you do both extensions and then do a blending within the block. this was my idea for trying to not have discontinuities.
- d: my point is that if i'm looking at the right edge of a block and i want to code some dct there. i'm going to use it to predict to the left and right. if the pred mode in the block on my left is horizontal, then that blocks contribution should be the average across each row. but if i look to the right and it says to predict up-right, then i shoudl be looking at the diagonal contribution and predict that. the pixels are going to be different for every choice of intramodes.
- jm: how would you do the blending?
- d: no idea. maybe you would just do the obmc thing. if the mode is horizontal, you could do bilinear weight of left edge to right edge.

(discussion too fast for me to capture about this)

- d: the other thing i dont' quite understand is switching between inter and intra.
- jm: you don't.
- d: i'd really l ike to.
- jm: this scheme is for keyframes. i have no solution for mixed frames. i see one way to do it, but it is another variant and i don't know how i would implement it with reasonable complexity. the idea would be to do it in the lapped domain. backtracking.... instead of encoding left edge and top edge and applying the filter. another way to do this would be to code the 1d dct along the diagonal that is orthogonal to the prediction direction. so this is another way you can code the information there.
- d: you would code the avg. along each of those prediction lines right?
- jm: exactly. you can also get lapping to work with that. i am coding the average as if there was no lapping and it would be like you generate this kind of pattern for an infinite block and then apply the lapping to that. it would tell you what's inside this particular block. that brings out the problem with the scheme you suggested. with my modified scheme, you'd be able to say i predict this pattern in this direction, but the actual 1d dct could not be predicted from anything else. you wouldnt' be able to predict it from the neighboring block.
- d: you'd be able to do DC prediction right?
- jm: yes, but it wouldn't add anything. it would just be an overcomplete codebook. back to the other scheme for keyframes. in the case where you're actually using the left and top edge and predicting, your prediction gives you the bottom and right edge which you can code an error for. if you have a perfect pattern for the entire image your 1d dcts will be zeros. imagine if your entire image is a 45 degree pattern. once you've coded the top and left edge of your entire image, you propagate your pattern with the predictor within a block. once you propagate you get a prediction for the bottom and right edge and then you only need to encode the difference.
- d: i can see why that's harder to do if you're lapping things, but i'm not sure it's impossible.
- jm: if it was possible it would also work for freq domain prediction. i think i want to play with it a little a bit. is anyone else interested in this?
- td: i am i guess.
- jm: i'm not sure how to get started but i'd like to see what results we can get. i don't think we need lapping working as a first step.
- d: i think you can start with the keyframe approach and just not worry about it.
- jm: i'd like to see how good we can get fruits with just this and nothing extra.

# other intra ideas

- u: we have unlapping that we're working on. i have some bad results but no good results yet. the other one is the stuff that greg an i have been talking about, which is to manufacture a 4d covariance matrix and do sparsification on that. no training on actual images just the synthesized stuff.
- jm: what you mean is figuring out the exact modes that vp8 does?
- u: any directional mode. some kind of 2d AR95
- d: you want a limit.
- u: if we have 32 of these at a 16x16 level we coudl sparsify them.
- jm: on unlapping we still have the block size mixing problem right?
- u: yes. what you end up doing is that you try to separate the edges. as you look at certain edges in certain configurations, some of them you can undo all the way to space domain and some are half unlapped. it seems like you'd train for all possible combinations, but that seems an intractable amount.
- jm: what are your bad results? still debugging or what's the issue?
- u: i've got a model that's learned on say the left edge. and i ahve a model for the horizontal mode but i'm trying to learn something for a diagonal mode. i haven't got something that makes sense because it's basically taking trained data for that. i'll post some images of what's predicted and maybe it will make more sense.
- u: the next steps are that horizontal mdoe it works ok. the same issues we had before are still there. if the block sizes are different, how does lapping interact. for a given angle you should be able to do a much larger least squares regression. you should be able to predict a much larger region around that border. i haven't had a lot of luck with that one yet. the idea is to do something that was separable, because then you could do them independently, but i don't think you can really separate them.
- jm: on a different note, what is monty up to?
- j: monty has been doing his journal this week! http://people.xiph.org/~xiphmont/journal.txt
</pre>

DaalaMeeting20140107

2015-02-07T00:08:24Z

Daala-ts: Add word wrapping

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2014-01-07

Mumble: mf4.xiph.org:64738

# Agenda

- review policy changes
- review tool emails
- tim day
- pvq projects
- 16x16 support
- activity masking
- convert to fixed point
- Interns!
- Det1 training ( http://people.xiph.org/~greg/daala.det1.training.20140107/ )
- Work week

# Attending

gmaxwell, xiphmont, jmspeex, unlord, jack, td-linux,

# review policy

- jack: reviews due in a week. in this meeting we'll go through quickly and assign and those should get done by end of the week.
- jmspeex: i would also auggest we review only important things.
assigning reviews
# pvq projects
- jack: jean-marc wants these three projects done by someone other than him. volunteers?
- xiphmont: me me me
- jm: here are the projects:
1) Implementing PVQ for 16x16
2) Activity masking
3) Fixed-point conversion
- jm: these are sequential and easiest to hardest
- greg: why is activity masking not two lines of code?
- jm: it's not a lot of code, but it's trying to figure out what's going on. there's a bunch of constants that would influence the results that are implicitly 1.
- xiphmont: there's some amount of wanking with constants I imagine.
- jack: can greg do activity masking in parallel with monty doing 16x16?
- jm: yes
- greg: sounds good
- jm: for 16x16 we don't predict anything so pvq is not that useful.
- unlord: i've landed the TF predictors
- jm: are we predicting something useful?
- unlord: perhaps. we're not feeding the 16x16 blocks directional data. when we do block size splitting we take all the directional blocks and making them 8x8 or 4x4.
- jm: not sure i'm following
- unlord: the training is done on all blocks in an image.
- jm: but you are at least training something and you predict in HF?
- unlord: yes.
- jm: the other thing is det1 stuff at least when doing pvq with interframes.

# interns

- jack: email people. recruit!

# det1

- g: on http://people.xiph.org/~greg/daala.det1.training.20140107/fastssim.png the blue line is git. the green line is the results from training. this was the best settings for prediction gain. i freaked out because this wasn't the results i got back when we first tried this. so i redid the training for the det1 coeffs and then having nothing else left to check i changed the training tool settings back to the defaults. then i got the red line. i think i have it sorted out now. it is odd that the changes in the training settings make such a big different. if you look at http://people.xiph.org/~greg/daala.det1.training.20140107/training.png you can see
- d, u: what is the difference between the settings on the fastssim graph?
- g: the green and reds are the same on both those images. the green line the training is set to use weighting and the bits metric. when i was working on this back in december, i noticed setting weight and bits in the training tool are setting higher prediction gain numbers.
- jm: is this on still images or videos?
- g: still images. the det1 stuff didn't hurt the performance at the last coding party when we first tested this. in any case, i'm happy with the results i have now. at least it's not worse anymore.
- u: using no weights and no bits is giving you best performance?
- g: on the images, yes.
- x: what are the axis labels on the last graph?
- g: that's iteration steps vs. actual coding gain from intraprediction.
- u: are you testing against subset 2 in your rd curves?
- g: the rd curves are testing against subset1 and i'm training against subset3.
- jm: it would be interesting to see these curves for pvq as well.
- g: sure.
- jm: it's worth investigating. at some poitn we need to commit to pvq or scalar or whatever, and assuming we use pvq, i realized it will have impacts on everything. similar to how the lapped transform changed the rest of the codec. at least it would change the metrics all over the place.
- g: ok. i have it running right now. i just started that data collection. i have the det1 patch ready to review. there's little to review.
- d: do you know what settings the stuff in git was trained with?
- g: i was going to ask if we should change the training tools to write out the settings in the header files.
- u: it's no weights and no bits.
- g: do you know what subsets it was trained on?
- u: do you know hte sizes?
- g: 4x4
- u: i think it's just subset3. it might have been subset1. i didn't modify those.
- g: that may be a reason i'm struggling against subset1.
- u: this is why i've moved to only testing on subset2. i remember that no weights and no bits got us the best results. i think i was graphing the prediction gain not the overall gain.
- g: that shouldn't make any difference.
- jm: what's weights vs. bits again?
- u: actuallly it might make a difference because we classify them based on prediction gain right?
- g: yes. jm, the weights vs. bits is where we weight them as the best over next best???
- g: there didn't appear to be any signs of overfitting. none of the weights were really high or zero, and i trained on subset3 which is a lot of images.
- u: i'd wnat to see this graph on prediction gain instead of overall gain.
- jm: it seems like it doesn't have to be all or nothing. reduce weights or bits, etc.
- g: not just that. part of the problem is that i don't know what to look at. here we have prediction gain telling us it's a lot better, but it's not so much better in the codec.
- jm: if you have something that seems to be working it's time to check it in. pvq will change the cost function anyway.
- d: based on these graphs i think we can check it in.
- u: we want to change the prediction metric for pvq?
- jm: we definitely want to change the selection metric. among the changes you'd want to consider individual bands and looking at the selection metric in the encoder, if any particular band is completely wrong or slightly wrong it's the same. if the prediction is 0 or way wrong, it doesn't change anything for the pvq code.
- g: as far as more changes to training tools. beyond changing for pvq, the thing we should be looking at is doing some L1 optimization, but this is probably a good intern projects.

</pre>

DaalaMeeting20140114

2015-02-07T00:08:07Z

Daala-ts: Add word wrapping

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2014-01-14

Mumble: mf4.xiph.org:64738

# Agenda

- work week update ( https://daala.etherpad.mozilla.org/workweek-201402 )
- interns
- Anyone waiting on anyone else for reviews now?
unlord: I would still like a review of https://review.xiph.org/132/
jm: have two more to review, rest is still up

# Attending

jack, derf, unlord, jmspeex, gmaxwell, td-linux

xiphmont: will be asleep for tomorrow's meeting :-( I'll see you in my early morning and early evening.

# pvq

- jm: what's the plan for pvq now?
- derf: greg is doing activity masking and monty is going the 16x16

# det 1

- derf: how did that turn out?
- g: behavior is the same as i showed last week. i couldn't get anything better than that.

# next things

- nathan: we came up with two things for me. the first is an idea for better coding of the motion vectors.
- derf: i want him to look into the square polar stuff you were doing. the other one was my idea for having multiple predictors and partitioning the coding space that way.
- g: in terms of the MVs, there are a bunch of ways we can encode x and y. we can code them independently or we could code sums and differences between coordinates. there are two obvious projections that we could potentially use to get them into angle and magnitude form. one is sum an difference whicih gets you a diamond shape, and the other is coding one magnitude and a radius around that magnitude which gets you a square shape instead of a diamond shape. i did some stats on real MVs and you could make some guesses about which structure was better.
- derf: the other idea is multiple predictors. you code the decision but it's easy to code. most motion fields are piecewise continuous. the theory is that if you make one predictor the median of neighbors and the other the 0,0 vector. inside one piecewise continuous region you'd predict the median and at the boundary you'd use 0,0 instead. so the flags are only not what you'd expect along the boundary. even in the case where there's no advantage to having two predictors you aren't wasting any bits. the idea is that it will always give you a gain.
- jack: which one are you going to start on first?
- nathan: probably the polar one.
- derf: yes, the question that will come out of that work is what does closest mean?
- g: the easy definition of closest that makes it easy to prune depends on how you code the MVs.
- jm: i was thinking that i'd look at pvq from another angle. what should we change in the rest of the code to account for PVQ? and i've got some IETF stuff to do.
- derf: the stuff i tried was .... removing restriction that neighboring motion partitions only differ by one level ... ripping out CGI and changing out dynamic programming algorithm.
- g: how do you do OBMC windows in that case?
- d: for the largest block along an edge, you can only have two motion vectors on that edge. so i can split the MVs down on one side of that edge, but i can't add any MVs on the edge.
- g: you'll get weirdly shaped OBMC boundaries right?
- d: no. it works fine. since you can fold all those into one extra interpolation it requires no extra work per pixel.
</pre>

DaalaMeeting20140121

2015-02-07T00:07:56Z

Daala-ts: Add word wrapping

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2014-01-21

Mumble: mf4.xiph.org:64738

# Agenda

- work weeks
- 2014 project plan https://daala.etherpad.mozilla.org/daala-plan-2014

# Attending

gmaxwell, unlord, jmspeex, xiphmont, derf

# work week stuff

# project plan 2014

- jmspeex: how do you measure quality of MC?
- derf: you measure the distortion of the MC residual
- gmaxwell: you can do it high bitrate
- derf: you could do it lossless. not sure if you can do that with VP9 but i think you can.
- xiphmont: i thougth last time i tried to run VP9 lossless it only ran in iframe only mode.
</pre>

DaalaMeeting20140128

2015-02-07T00:07:42Z

Daala-ts: Add word wrapping

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2014-01-28

Mumble: mf4.xiph.org:64738

# Agenda

- review review (27 open issues)
- 144
- 132
- pvq: activity masking http://people.xiph.org/~greg/daala/pvq_activity/psnr.png
- pvq: 16x16
- pvq: energy preservation
- pvq: demo 5
- work week agenda bashing - https://daala.etherpad.mozilla.org/workweek-201402

# Attending

xiphmont, jmspeex, jack, unlord, gmaxwell, derf

# activity masking

g: i achieved an impossible result. i mananged to improve PSNR. what i'm doing here is changing the computation in teh pvq code that selects the resolution and also changing how it quantizes the gain. in changing it to achieve implicit activity masking so that we give it more rate in smoother areas where noise is more visible. this is achieve straightforwardly in the current code by companding and uncompanding the gain. at least in the case where there is no theta, is there more to it?
jm: even when there's no theta, tehre's factors of 1+2alpha that come up in a few places.
g: i'm aware of that.
jm: computing k is a function of the gain.
x: missing teh big picture here
jm: monty, did you read the video pvq doc that i wrote? the whole diea here is that flat areas should have more resolution. highly textured areas should have less resolution. the first thing you do is when you quantize the gain you want more resolution in flat areas. instead of using uniform scalar quantization on gain itself you use it on a warped version of the gain. on the gain raised to a power less than one.
x: yes, i played with that.
jm: once you do that you also you need to update how you determine k as a function of that gain.
x: so setting activity has to be taken care of elsewhere in the code. got it.
g: i didn't expect to achieve an improvement in PSNR. that blew my mind. the PSNR graph was surprising. this change should hurt PSNR and help SSIM, but i got an across the board improvement. maybe jm has ideas about where we were suboptimal before.
jm: i notice you're increasing the rate across the board which also influences other things. i wonder if while applying activity masking you could just scale the quality so that you achieve similar rate and see what happens?
g: i'm confused.
jm: basically adding this activity masking increases resolution across the board because all gain steps get closer. so i was thinking in the function itself you could multiple also the quality (-V option) by a constant factor that just gives us equivalent rate.
g: i can do that. i'm still at a loss as to where our behavior is non-optimimal such that i get a PSNR improvement.
x: i found that we're currently in the case where the vast majority of the pvq partitions are quantizing to 0. maybe you got out of that. this is not a product of the RDO search. most gains are coming in below 0.5 and they quantize to 0. at moderate and low rates, when i look at pvq compared to scalar, ???? the current behavior of pvq completely defeats the whole purpose of using it. that is energy preservation is not happening. high end detail is getting thrown away.
g: i think what we were tryign to achieve is that we expect that pvq would actually result in a straight up MSE improvement as well. the old stuff did not result in a MSE improvement so the point of the reimplementation exercises would fix that.
jm: you are summarizing that perfectly.
x: i agree. however in my mind, i was hoping to get primarily psycho-visual results.
- jm: i thought about doing that as a second step. first get an MSE improvement, then imrpove visuals.
- x: quantizing all the low energy stuff to 0 appears to baked in to the way it's being done.
- jm: it's absolutely not. baked into the encoder or the format?
- x: simply into the way we handle it right now.
- jm: i think a lot of this is the cost function
- x: i don't. the cost function is applied after things are quantized to zero.
- jm: right now the cost function does not value gain preservation.
- x: it is always quantized to zero before that point.
- jm: oh, you want to preserve 0.1 gains? that is for activity masking. the only way to do this is activity masking or some variant of it.
- x: i disagree with that but i do think that activity masking is a fine way to experiment.
- g: activity masking compands the gain and makes it less likely that small values will quantize to zero.
- x: i hope that's the case.
- g: as an aside, this code is hellaciously slwo right now.
- jm: the loop for the non-predicted case tries all the gains instead of a small range. we're doing many calls to the pvq encoder instead of a single one. and the pvq encoder itself doesn't have the fast pass.
- g: we may need to improve perofrmnace to finish the work on this stuff because it's really slow.
- jm: one thing i realized from greg's results and it's going up always. and not touching anything it should go down. did you only change activity?
- g: no i put in the constants, but i'll check that i didn't do it backwards. the reason i'm perplexed about this is that my expectation is that nothing i'm doing should improve PSNR.
- jm: i've also got the activity variable backwards compared to my document. what did you set it to?
- g: i set it to 4/3rds.
- jm: it should be set to 3/4ths. the document says that 1 + 2alpha should be 4/3rds.
- g: i'll check that i have it right. that still doesn't explain PSNR improvement by 0.5db.
- jm: i can see why it does. right now the pvq code is preserving energy than is MSE optimal.
- g: how?
- jm: at high rate it is optimal. at low rate, i did the derivations of MSE optimal at low rate, and what is actually MSE optimal is to have a black hole in the middle. if you have 1000 coeffs, anything that has a gain below 5 or 10 you better have it just at zero. essentially if you have a really large N and the gain of your vector is 2, then you look at what would the error be if i set k according to gain = 2and it gives you a distortion. you find that setting gain at 0 gives you a lower distortion. i suspect by doing activity masking backwards that would cause the PSNR improvement.
- g: i'll go figure that out. i can see how that could happen now.
- jm: just look at where activity is applied. maybe it's worth not changing the value of activity but reversing where we use the inverse activity verses the activity itself.
- g: i'll go fix it.

# 16x16

- x: there's not a lot to say about it other than it was reasonably easy. except for the constants sprinkled all over the code. the fact there were 4 PVQ partitions was hardcoded in 4 differnet places. it's straightforward. it's implemented and working. the only caveat is that i don't know what CDF stands for. the CDF for 16x16 is a placeholder and we need to code more than 16 choices. so it's coded as 4 bits using 8x8 and then 3 bits with flat probability. if it turns out not to be a premature optimization. i've been traipsing through this code, and perhaps i should document it. it's not so much that the code doesn't make sense, but there are anumber of things that are named strangely and in weird places. i am preparing a patch that moves a few things to more logical places and fixes up the names. pvq_encoding is now laplace_encode_vector. the details aren't important; we can bikeshed on the patch.
- g: i agree. it's come up as a source of confusion a few times now.
- x: all these chagnes are minor though there will be some bikeshedding of naming.
- jm: you didn't see anything the code was doing that was stupid?
- x: there was some dead code and a few places where we were dealing with coding conventions. we can go into more details offline.
- greg: testing shows dead code:
https://mf4.xiph.org/jenkins/view/daala/job/daala-coverage/ws/unix/coverage/src/pvq_encoder.c.gcov.html#165
- x: as for removing old pvq thing, it's already gone.
- jm: what's the status of your patches?
- x: they are in my local git. i was going to do the mild refactoring and commenting before submitting all of it.
- jm: if you have some that are ready to apply i have no problem looking and merging right away.
- x: ok
- jm: do you have them in public git? if you push them somewhere i can review from git.

# energy preservation

- x: we were looking to pvq for two things. it's elegant and perhaps more efficient. the third thing is, is energy preservation actually useful? i think we should experiment with that. i'm certain it is going to behave in surprising ways and that it will be useful. i'm not sure that the small amount of efficiency we're seeing is worth the additional complexity. it's not particularly impressive in my testing. i'm seeing visual effects that are not as good as scalar. there are large sections of the iamge that just wash out.
- jack: can you bring some sample images to the work week so we can all look at them together?
- x: yes. even if you discard the images that are surprising, the other images look better with scalar.
- jm: these are regions wher eyou have gain < 1?
- x: yes. these are regions where energy is disappearing. happens in both HF and LF. a large area of the image grays out slightly, but compared to scalar it's obvious.
- jm: activity masking is one part of hte solution. another thing to experiment with (in parallel) is to have something explicitly handling really low gains. like adding 1 pulse at 0.2. some kind of spreading for just this issue.
- x: what i was going to propose experimenting with is to do away with the partitioning and apply a spectral tilt. try to keep the rough energy across the spectrum more or less intact, but do it in a reversible way like with CELT. (spectral tilt = whitening filter). what i'm loking for is nenergy preservation so that energy doesn't move as we quantize but we smoothly decorrelate. i want to see what it looks like when our quantization process moves to noise and grain that is true to the original spectral distribution rather than moving towards everything becoming a flat block.
- jm: if you apply this whitening filter and then inject noise...
- x: i dont' want to inject noise. i want it to pick up noise instead of drop energy
- jm: if you apply the spectral tile then code then apply it backwards, you're going to give better resolution to the stuff that was weaker.
- x: ype. we don't want to do a straight version of that. we're allocating our resolution based on energy, and when the LF is low energy you lose contrast. when the HF is low energy it turns into a flat area of perfect color.
- jm: that part i get. if the gain doesn't quantize to zero you'll be using a bunch of pulses, and maybe what we need is a first step is a single pulse for small gain so that you're not completely washed out.
- x: that's a good test as well. one reason i want to drop partitioning is that diiving a big number doesn't give you two little numbers, but two more big numbers that are just slightly smaller.
- jm: i was hoping the gains would be easy to code because we could code them jointly and predict them.
- x: that may be the case. what i'm seeing right now, is that until we get to that point it's not worth discussing because they are all zero.
- g: are you comparing these things at equal rates?
- x: yes. i realize that the rate is sort of a guess, but i'm running with RDO and matching rate.
- g: i think it's well accepted wisdom that at low bitrates you should be reducing resolution of your images. and that remains true even if you have good partioning schemes in the codec. it's positive to me that our quantizer is making similar decisions. i wanted to make sure that what you were complaining about wasn't what was expected to be a good behavior.
- x: if i was taking this in a vacuum, i'd agree.
- jm: what is reported as the quantization factor ?
- x: i have been testing at between 32 and 48. low enough rate that it's causing obvious differences but well above where it would totaly fall apart.
- jm: the function that has k as a function of hte gain is MSE optimal, and maybe we should deviate from that. it has the same error in the direction of the gain as it does in other directions. maybe it shouldn't do that.
- x: at the moment things are disappointing and we have reason to be concerned. perhaps activity masking will shape it right up.
- jm: the one i'm mentioning right now is what i consider to be an easy one that is complemtary to activity masking. i always thought it was reasonable to check that early on.
- g: i will play with that. i wonder if there is a better way to test these visual tradeoffs.
- jm: that's one of theprolbme with pvq. there are thousands of interacting things.
- x: i think we might want to turn prediction off and try it that way. the one thing that pvq is doing right that scalar is botching is that scalar is waffling like hell and pvq is not.
- g: this may be the zigzags.
- all: ZIGZAGS!
- g: can you check to see if there is a difference in waffling between even and odd quantizer values?
- x: pvq shows no hint of this.
- g: pvq gets rid of hte zigzag. i could see a rounding difference causingi that.
</pre>

DaalaMeeting20140211

2015-02-07T00:07:30Z

Daala-ts: Add word wrapping

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2014-02-11

Mumble: mf4.xiph.org:64738

# Agenda

- review review (11 open issues)
- current issues
- activity masking
- project plan updates

# Attending

jmspeex, jack, unlord, gmaxwell, derf

# review and current codec results

- d: i'm rebasing 32x32.
- g: it didn't work even with pre-filters turned off.
- d: ok. i can look at that.
- d: nathan are you reviewing greg's det1 training patch?
- u: yes
- jm: what's the status with the scaling patch?
- u: i sent an updated one but you found some rounding thing that i'm looking at now.
- jm: there were two things. one was removing the scaling for lossless and the other one was shifting wrong.
- u: we also talked about using FP on the commandline to specify quantization.
- d: we want to map integers on the ocmmandline. we ahve to code these in the bitstream.
- jm: i don't think it matters in the short term
- d: sure, but it's not hard, let's just do it correctly. the thing you want is to store them in log space. this is the way we should be testing too.
- u: i proposed some mapping where 1 would go to 1 + smallest increment
- d: we don't want the spacing to be linear. there's no reason it needs to be.
- u: what would we like it to be?
- d: logarithmic.
- g: because that will be linear in rate
- u: how do you pick what that space looks like?
- d: you say it doubles every N number of steps
- jm: as soon as we introduce activity masking it will be a bit more interesting
- g: teh status of activity masking is that it's producing much worse performance on SSIM than not having it. that's the opposite of what we want.
- jm: can you send me whatever intermediate patch you have so I can look?
- g: sure.

# current problems

- jm: i've been doing experiments with everything off and 8x8 only and 16x16 only and it's worse with 16x16. i'm looking at psnr vs. rate, no BSS, no nothing. pure coding gain.
- u: shouldn't we expect 16x16 to be worse than 8x8 in certain places?
- jm: yes, but overall it should do better.
- u: because we're measuring it on AR95 and some images aren't AR95
- jm: have we measured the 2d coding gain?
- d: yes. that's how the current filters were trained. 2d coding gain on actual images.
- jm: i need to do more experiments to make sure it's not hte entropy coding screwing up. that looked pretty fishy. adding intra meant that 8x8 was clearly better. there's no way to get better with 16x16 because it's much worse. i think as soon as you turn on lapping filters, 8x8 is already better than 16x16. and once everything is on, the gap is too big to make up.
- u: you were just running transform + prefilters and scalar?
- jm: only scalar. no intra. even tried without prefilter.
- d: in my status report, i said i woudl be working on putting a quantization matrix in
- jm: if you do that, can you make sure to split basis function magnitude and the quant matrix? we'll want this to work for pvq
- d: i planned to do exactly that.

# activity masking

- jm: i've been investigating what to do close to zero. whether we should code a single pulse or adding noise, etc.
- d: i'm hoping you could work with monty to see if you both could come up with something.
- jm: i think the first step would be to get 16x16 working.
- d: i agree
- jm: the 8x8 seems to not be too bad. before the scaling patch the intra on 8x8 made things worse, but now it's helping a bit.
- d: how much is it helping?
- jm: at high rate it's not helping at all. at low rate it seems to be improving distortion a bit, by maybe 0.2dB. it doesn't change rate.
- d: it should be helping more than that.
- jm: prefilter definitely helps for 8x8, mostly on distortion. it gives us about 2dB at low rate. assuming i didn't screw up these experiments. hold on, this is fastssim, which is crap. intra makes a tiny difference at high rate and at low rate it's 0.1dB.
- d: that doesn't sound like what we were expecting.
- jm: PSNR is on the order of a dB for the filter.
- d: i think intra should be bigger than the lapping filter. one thing we never looked at is the mode decision stuff. that isn't adapted to rate at all is i?
- jm: no
- g: we've known that the mode deicsion stuff was broken. a year ago i inverted it and got better results.
- u: that might account for these things not showing improvmeent.
- d: maybe i will take a look at that also.
- jm: disabling RDO in the mode decision and not use curves that don't account for the rate of the mode decisions?
- d: that would certianly give you a best case.
- jm: the weighting we've known was wrong. i believe i came up with a proof of why it was wrong, but i don't remember the details. the weighting of the distortion metric for intra mode selection.
- u: is that like RDO?
- d: except that it doesn't consider rate
- u: other than the transform scaling, what would you weight SATD by?
- jm: we're trying to improve the entropy not just hte final distortion.
- u: where are these weights coming from?
- jm: we're going to code a residual after that. if you're reducing the error on DC by a small amount it wont' change the rate too miuch because coding one extra pulse in DC is cheap. if you're reducing HF then improving that can elp the rate more. on the other hand if you weren't going to code it it doesn't help you much.
- u: is this pvq only?
- jm: no it affects scalar as well. i think this is what was wrong because there's a point at which you're not going to code anything and at that point the weighting is completely useless.
- u: we're looking at the residual after prediction... you're going to bias SATD by position of coeff?
- jm: essentially yes
- u: aren't we coding these with same prob in sclaar?
- jm: absolutely not. the prob for close to DC is a lot higher. the first few AC coeff. the last one will have a small variance and encoding a 1 there is going to be really expensive. for small values, we're actually interesting in distortion because we're not going to be coding anything. it should grow as the square of the value. for large values it should grow as a weight times the absolute value. we ended up in the square region more often than not, which is why teh reverse scaling worked better. at least in terms of fixing the metric, it's probably something is hould have a look at. i at least have some delusion that i understand what's going on.
- d: normal codecs pick a lambda and optimize for d + lambda*r. i'll probalby try that on thursday.
</pre>

DaalaMeeting20140218

2015-02-07T00:07:17Z

Daala-ts: Add word wrapping

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2014-02-18

Mumble: mf4.xiph.org:64738

# Agenda

- scaling issues
- det1
- 32x32
- Prediction error numbers: https://pastebin.mozilla.org/4329150

# Attending

unlord, jmspeex, derf, jack, xiphmont, gmaxwell

# input scaling

- d: have a patch that works, but nathan is having trouble testing it. i'll take a look today.
- Currently set to 4 bit (seems good for now, just a #define)
- Need to check for overflows in CfL
- Mostly disables RDO in Intral prediction at the moment

# det1

- g: I posted graphs in my status report. I have it trained and ready to go now. It didn't do anything terrible tot eh RD plots.
- g: https://people.xiph.org/~greg/det8/
- d: I will take not terrible.
- j: is the training tool getting checked in?
- u: that already landed
- g: i've been on a bit of a training lark becuase I made the mistake of running the training again after updating the tools. i made some improvements to make it faster and then found better prefilters on friday, saturday and sunday. i haven't found a better one since sunday so I left it there.
- jm: this is only 4x4?
- g: all sizes but only changing 8x8.
- jm: what about 4x4?
- g: that one i searched exhaustively. in the 8x16 case, the non-negative case makes it somewhat spare, so there still may be better ones.
- d: the important thign is to get something in.
- j: do we still need a 16x16 one?
- g: that's training right now with the new training code.
- j: how long does that take?
- g: I'll put in whatever it comes up with by tomorrow.
- j: is it possible to search 8x8 and 16x16 exhaustively?
- g: not at all.
- j: do we need to retrain the intra stuff now?
- g: that's already baked into this task. that's part of what i'm doing.
- u: these are still independent right?
- g: correct
- j: do we need to retrain with scaling patch?
- d: no, they were already scaled up during training.
- g: i thought you used integer transforms in training?
- u: i thought we used the double version. i'll find out.
- u: it's using integer filters and transforms. but we dont' have a problem because we're in 32 bits.
- d: there are multiples in the transform that multiply by 15 bits and produce a 32bit result. so if od_coeff is more than 17 bits you have a problem.
- u: our input is in y4m space right?
- d: yes.
- u: we're scaling by 6 bits in the training.
- d: that's already 14 and prefilter adds 2 bits in each direction for 16x16.
- g: we'll definitely be overflowing somewhere there.
- d: that could be one reason that 16x16 is coming out crap.
- x: the appearance of stairs at the high end, is that innocuous?
- d: that patch hasn't landed, so that is innocuous.
- j: why train at 6 bits and run at 4 bits?
- jm: it makes since to use more than necessary.
- j: how do we solve this problem?
- d: we have to fix the transform to work with 4 its if that's what we're using.
- u: you were looking at trans tools with 16x16 and seeing overflow also?
- d: that wasn't overflow, that was precision error.
- g: i' will check for overflow. we should change the scaling down because it's risky in training.

# 32x32

- j: made progress on this?
- d: not yet. waiting on det1.
- jm: i've been running experiements and 16x16 is performing noticeably worse than 8x8. maybe that's the cause for 32x32.
- g: if that's true, why would we expect 32x32 to be better?
- d: that's concerning.
- jm: i have no idea what could cause that.
- d: is it an entropy coding issue?
- jm: i meant to check but if i did I don't remember

# prediction error

- d: this leads me into the next thing i wanted to alk about. i was collecting data from teh quant matrix stuff, and one thing i was looking at is the square root of the average squared prediction error for each coeff for all 16x16, 8x8, and 4x4 blocks. i ran with k=3 so it should be 1/3 intra and 2/3 inter, and https://pastebin.mozilla.org/4329150 shows what it looks like.
- g: i noticed that in the 8x16 training, none of the predictors predict the highest frequency coeff.
- d: greg is referring to something else i found on friday, but that effect is gone after running more video through it. the fact that the vertical prediction error is more than twice as large as horizontal is not gone.
- jm: i would tend to blame that on having more energy in that direction.
- d: yeah, this may be that video is wide. this also showed up in 4x4 and 8x8 but not as bad as 16x16. i'll go look at it and see whether that is true.
- jm: can we get the coding gain values for our transforms?
- u: the trans2d tool does it.
- g: on what input?
- jm: the coding gain we think we are achieving.
- g: without intra?
- jm: yes
- g: you want transform + prefilter?
- jm: yes.
- g: ok, will have in a few minutes.
</pre>

DaalaMeeting20140225

2015-02-07T00:06:57Z

Daala-ts: Add word wrapping

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2014-02-25

Mumble: mf4.xiph.org:64738

# Agenda

- patch status
- fastssim metric?
- ietf89 planning

# Attending

jmspeex, unlord, derf, xiphmont, gmaxwell, jack

# patch status

- derf: 8x8 is in, but 16x16 is not in
- derf: i have not made a lot of progress on 32x32
- greg: i screwed up the training on 16x16 and that's why it's not done yet. i left it running for 2 days with the old code, and i didn't notice it until yesterday.
- xiphmont: i am running my own RDO tests but was curious what the status was.
- unlord: you're testing RDO?
- xiphmont: i'm just keeping track of what works and doesn't. i'm mostly focused on PVQ and trying to inform my intuition about where we are getting gains. i want to be able to command the knowledge without thinking about it a lot.
- unlord: RDO or RD curves?
- xiphmont: RD curves, sorry.
- jack: did the scaling patch land?
- derf: yes. we still need to do all the followup things: transform multiples might overflow now. CfL needs to be revisited since we were at the precision limit before, and i'm not sure what the third one was.
- greg: i ran the training tools in clang with integer overflow checker and it wasn't overflowing on me at 16x32. i only ran it on subset1.
- jm: when did the transforms land for 8x8?
- derf: yes. same patch as the tools. wait. there were two patches. one did non-neg constraint for search (was 4x4 only last I checked) and the other was changes to 8x8 including updating the tools to use the new transforms.
- jm: i'm looking at git, and trying to find 8x8 coeffs.
- derf: did you check that in greg?
- greg: maybe i forgot to push it. will fix.

# fastssim

- jack: fastssim was/is broken. should we delete it?
- derf: we can certainly stop running it, but it's probably a good idea to understand why, but it's a low priority. the nice advantage is of fastssim is that it's faster and the implementation is multi-scale which has interesting properties. one of those properties is that it's one of the few metrics that shows changes in quant matrices beyond flat to jpeg-style. multiscale SSIM will move where the other metrics don't.
- unlord: i can remove it from rd_plot.
- jack: perhaps also bd_rate.
- unlord: sounds good.
- xiphmont: what about people who notice we collect data but don't use it?
- jack: file a bug and put the bug # in the comment that disables it so people know why
- derf: traceability!

# pvq

- jm: i ran some experiments with joint coding of gains, and in theory we may want to do that, but i've been disappointed with the results.
- derf: this is jointly coding the gains within a single block?
- jm: the octave experiment was dumping the values of the 3 gains in the top octave and looking at the sum of independent entropy for each of the three gains and then looking at the joint entropy. the two differ by 3-4%. not nearly as much as i would have hoped. this is without prediction because i wasn't sure how to mix gain prediction into that. gain prediction gets us less than 1% currently.
- derf: less than 1% of the rate on the gains?
- jm: less than 1% of total rate. the 5% was % of gain itself.
- derf: 1% of total rate is a lot.
- jm: it was less than 1%. the difference in total rate is 0.5%. the gain itself is 10%. the gain prediction is about 5% relative to the total gain thing.
- derf: we're spending 10% of the bits on gains. huh. that seems like an awful lot.
- greg: that's crazy
- jm: we're dividing into bands now
- x: we turn one large DC and a bunch of small AC into one large DC and 5-6 large AC coeefs. it defeats a large amount of the purpose.
- jm: it's about 80% of the bits for pvq.
- derf: how many dimensions in bands on avg?
- jm: this is with block switching. the band sizes are 15, 8, 32, 64, and maybe 16s.
- derf: maybe not so terrible but it's a little concerning.
- jm: i didn't see that it matched. i dumped all the gains and measured the entropy, but i iddn't check that it matched the rate that it spent in the encoder. i'm not that concerned about that. it seemed to mostly match. there seems to be something like 10% on DC and all kinds of signaling. 70% on pulses. 10% on gains, and the rest on no-ref flags and thetas. i'm not sure i like the no-ref thing.
- derf: i'm sure i don't like it.
- jm: why?
- derf: seems like an odd thing to have. in particular, what this is saying is that our predictors are harming in a bunch of cases and we're going to spedn bits to correct that.
- jm: hwich is what you do for scalar, except that we don't need to spend pulses on ignoring the predictor. for scalar if a block is flat and you predict all kinds of crap... ???.. with pvq if you have flat and predict grab you say ignore band 0. two choices i had: no-ref flag. the other was flip the angle. have the angle be from 180 to 90 degrees. but that basically meant the equivalent of subtracting the prediction which i didn't really like.
- d: the point was if you're going to do this in scalar, you could code a flag to ignore the prediction and you wouldn't have anything to cancel out. if we're going to compare pvq to scalar we need to compare apples to apples.
- jm: i'm trying to think if there is a better way. the only advantage of having hte sign flip with theta is that there's no redundancy whereas ignore prediction has some redundancy. the same point can be coded with the theta or without.
- g: it is also potentially redundant with our mode bits. in theory we should have a mode that is not predicting a whole bunch. so if really the prediction is terrible, that's the mode we should be using.
- jm: it's easy to say if we predict mode 0 then these flags are hardcoded and you don't do that.
- greg: ok. i retract that objection.
- derf: it's easy to do but we're not doing it.
- jm: i had to disable prediction for my experiments with gains. jointly coding thetas with gains when they don't have a theta becomes really messy and i didn't really like it.
- d: i'm not sure how you get around that.
- g: does it mess up the context modelling? i imagine the stats are pretty different.
- jm: right now there is not much context modelling. in terms of context for coding gains and thetas there are so many potential things to use and they are actually hard to use because we are switching and some have prediction and some don't. i'm not sure what the gains would be. they don't seem to be that large. for the gains and thetas you can use the gains and thetas of neighboring bands and blocks, the prediction itself.... i'm finding it hard to actually use this.
- d: this is the sort of thing that we need to spend an awful lot of time on at some point. this is a little more of an art than a science. i am hoping we can makek reasonable progress without sinking a lot of time on it up front.
- jm: among the things that need figuring out is the band layout. maybe monty wants to look at that.
- x: i've been thinking about making it part of demo5 with some js so you could play with band partitioning in real time.
- jm: looking at curves?
- x: no. actual image output.
- jm: we'll need to figure out band layout. i'm not sure how to make it easy to change the layout and second how to actually decide that one is better than another one.
- d: first one soundsn like an engineering problem. i think we can figure that one out. for the second one, maybe looking at images will tell us something.
- jm: you need decent enough encoding that you can reliably use the image you're looking at.
- d: i'm not following. you mean the issue is being able to change the layout and have the stats updated?
- jm: yep.
- d: you're right, that is a little more challenging.
- jm: i did a few experiments with the k value for the g=1. i think we can do a little bit better, but not huge gains. what i'm finding is that we need fewer pulses for small gains. this is with curves so needs to be checked on real images. there's all the activity masking that goes into that as well.
- derf: activity masking would suggest we want more pulses?
- jm: no it says we want more gain resolution at small gains. k is a function of what we're quantizing. this is orthogonal to activity masking.
- d: is it?
- jm: mostly, although you can combinei them.
- d: i guess you're right.
- jm: the idea is that MSE optimal means you want to skip gain=1 and not have pulses there. and of course we're not doing that, but at the same time having fewer pulses for k is good since you can use a smaller stepsize for the gain.
- d: this is what i've been telling people is great about pvq all the time
- jm: i'm still telling you where the energy is, just with not very much resolution.
- d: maybe we'll get to a point where we can experiment with folding or something
- jm: i did play with injecting noise. it still means you need to signal the gain
- x: quad trees!
- d: we can make that decision independently but scalar cannot. you can think of several examples where you'd want to do that.
- jm: what it also told me is that we also want to figure out activity masking before figuring out gain = 1.
- d: presumably greg will have spare time while 16x32 filters are training. maybe he can figure out while that patch is hurting instead of helps.
- jm: on metrics on actual images?
- g: both. obviously something was busted.
- jm: hurting metrics is not that surprising.
- d: hurting SSIM is surprising.
- g: i expect it to help SSIM.
- d: when we added activity masking to theora we got a 3dB improvement to SSIM. that was using the old SSIM which was doubled, so 1.5dB with the current tool. but it was still quite a lot.
- jm: i think we want activity masking to be adaptive and not apply it to 4x4.
- g: that may be part of what i was messing up.
- jm: there's a patch that goes a long way for that (bss ???).

# ietf89

- jack: anything we should be doing?
- jm: get cullen and magnus to agree on ptime.
</pre>

DaalaMeeting20140304

2015-02-07T00:06:40Z

Daala-ts: Add word wrapping

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2014-03-4

Mumble: mf4.xiph.org:64738

# Agenda

Status of current patches waiting review and commit...
- roll back det1?
- JM's quant matrix stuff etc
- metrics (and CI)
- reviews
LibrePlanet

# Attending

xiphmont, gmaxwell

gmaxwell: working on the det1 DC basis problem; trying the easiest most obvious thing first (optimizing based solely on DC leakage) didn't work out so well.
gmaxwell: next tried coding cain + leakage, results not impressive
http://jmvalin.ca/video/signal_dc_nondet1.png
http://jmvalin.ca/video/signal_dc_det1.png vs https://people.xiph.org/~greg/railway.png
These are all DC only images with the last being the det optiimzation with it limited to low leakage results and using leakage as a weak weight in the search.
xiphmont: right now, given that scaling has 'taken the pressure' off of the need for det1, if we have to choose det1 vs the smoother filter, smoother wins.
gmaxwell: Right. Though there is a difference in trade-off for low vs high rate. E.g. at lossess the det1 is a big win— we can't do the scaling there. But I've also wondered if we're not wasting or time dorking again with the filter stuff at this point.
xiphmont: agreed
gmaxwell: Also, putting down and picking up the toolsets/states repeatedly can waste some time and effort swapping. It makes some sense to keep chasing this for now, because we ideally want det1 that's smooth.
xiphmont: I've been working on visual comparisons of the recent changes, as we seem to have not been making them of all changes.
gmaxwell: I have been comparing visually, though mostly at high rate. There det1 appeared to be a wash and the metrics mostly agreed.
xiphmont: I compare at lower rates so things are easier to see/notice-- but that may not be giving fully representative results either
gmaxwell: I agree that the lower rates are way more useful. I have a somewhat bad habbit of looking at the higher rates because they're more useful for outright broken code. (The image should look perfect or the code is broken) but thats not a good approach when evaluating trade-offs. I'm making a mental note to avoid continuing to make the same mistake. Probably as practice we should make a habbit of doing both a low and a high rate comparison.
xiphmont: re, jm's 209 set of patches (quant matrix for PVQ) they're an obvious win on smooth backgrounds that would otherwise block out badly (eg, subset2 Abandoned Railway signal) and are even not bad for detail there. OTOH, it's harder to tell if it's a good think on images with more sharp forground edges like Streptopelia. Probably a win overall, need to look at this more without det1.
gmaxwell: perceptual theory says that quantization matrixes should be flat when all distortion is visible, so there is a bunch of rate sensitive tradeoffs with the quantization matrix stuff.
xiphmont: ah, right. I should be looking at a range of rates. And activity masking.
gmaxwell: which got us big SSIM improvements in Theora. In daala, activity masking looks better but does not improve SSIM. This is a mystery we should discover, because it may indicate a bug.
xiphmont: BTW, shall we back out det1?
gmaxwell: jm wants det1 backed out, gmaxwell agrees, xiphmont agrees. No vote yet from derf, unlord, jack.
gmaxwell: other subjects… what do we want to prep for libreplanet?
xiphmont: I was hoping to have one more demo by then, and was considering putting off demo5 and doing a demo update covering some of the recent activity including the det1 stuff. I would be interesting to talk about some of the recent failures.
gmaxwell: might be fun to have a realtime motion vector analysis display running for demonstration purposes. Currently fast enough at low res (probably).
xiphmont: just do it on the GPU!
gmaxwell: die.
xiphmont: I tried to upload an updated patchset to issue 209 and rietveld nacked me for not being the owner. Nathan said that was undesireable and should get fixed.... did he go off and fix it?
gmaxwell: dunno.

</pre>

DaalaMeeting20140318

2015-02-07T00:06:19Z

Daala-ts: Add word wrapping

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2014-03-18

Mumble: mf4.xiph.org:64738

# Agenda

- reviews
- future of det1
- fastssim
- Activity masking

# Attending

unlord, jmspeex, derf, gmaxwell, jack

# future of det1

- gmaxwell: jean-marc doesn't think it's necessary. becaue we're scaling up we lose the bi-directional round-tripping feature.
- jm: the only case where det1 makes a difference now is lossless, not even high rate.
- g: that's assuming we don't turn down the scaling at some high rate
- d: i think it'll be a 0/1 switch. we won't turn it off.
- jm: there's really no point in not scaling up if we're not lossless. so det1 would only ever be useful at lossless.
- g: would we consider having separate prefilters at lossless vs. non-lossless?
- jm: i don't think we should make the normal non-lossless case worse for lossless.
- d: i don't think it's about making it worse. in theory it made it better except for 8x8.
- g: the dc basis function is not very smooth so it causes blocking artifacts.
- d: we had outlined 3 or 4 ways to smooth the basis functions. you tried yet another one and i don't know which one you tried.
- g: it was one of the ones we tried before. we measured the leakage (freq response to dc funciton). i'm not unconvinced that that could do something useful. the question is what does it buy us?
- d: it buys us stuff at lossless?
- g: at the cost of more multiplies. the rotations are comparable to the number of ?? in the steps. and the scales are related to the number of inputs.
- jm: if there's more multiples it's because there are more degrees of freedom.
- d: there are fewer degrees of freedom.
- g: we definitely have more.
- d: then something is wrong and we need to sit down and figure out what.
- jm: speaking of DoF i came up with what we should normally have for a certain size. i pasted that in irc at the time.
- d: what you're trying to do is come up with DoF for an arbitrary det1 matrix, but that's not what we're doing.
- jm: what i got was consistent with some of the papers i saw. for the case of 16x8, there's 128 possible different transforms.
- d: you already lost me.
- jm: 16x8 transform. there's 16x8 coeefs. if the only constraint you have is invertibility then you can put whatever coeff you want in any element and it will work. then if you impose symmetricity, you go to 64 DoF. if you don't want DC leakage, there's another 7 DoF that you lose. that means you're left with 57 DoF. if you consider that it doesn't matter if one basis function is scaled by 2 then you remove another 8 DoF (49). if you impose det1, then you get 48 DoF for a 16x8 transform. so i went the other way, if i impose that the core of hte transform is orthonormal, how many degrees do i have for 8x8 orthonormal? 28. my reasoning here is that i have 48 DoF for the bi-orthogonal transform. 28 are taken by the DCT. so there are 20 DoF remaining.
- d: that's too many. that may be correct for a completely arbitrary lapped transform. but a good number of those DoF are useless because the guy who did this work did exhaustive searches and determined they were useless.
- jm: i looked at the paper and it said 8x8 had 21 DoF.
- d: what paper?
- jm: monty's demo. in monty's demo, see the v matrix near the bottom. s0 to s_n-1. you have the PNU type 3 or type 4. both of these would have 21 DoF for an 8x8 transform.
- d: what you're looking at has exactly 10.
- jm: what is n? 4 or 8?
- d: n is 4jj
- jm: this still has 12
- g: you can see it has the banded structure, so this has less.
- jm: would an arbitrary V matrix still have all the properties like symmetry and leakage?
- d: yes. the butterflies on the input is what guarantees symmetry. the prefilter and postfilter U and U^-1 matrices in them, the U matrix is arbitrary but it is set to identity, so lots of DoF goes away. by setting it to the identity it gets rid of DC leakage.
- jm: why are some DoF completely useless?
- d: i recommended reading trans original research on the subject. nothing he did to U helped. for the V matrix, the stuff he did comes out as nearly band diagonal. that's what led to the type 3 and type 4 factorizations, which allow you to specify the coeffs on the diagonal very well and leave the off-diagonal stuff very small which is what you want. you can do a more general version of hte type 3 where you have pairwise rotations among all the pairs. n^2 multiplies instead of 2n.
- jm: the full n^2 would be redundant. in the case of 16x8 some of them may be useless but if you have more than 20 DoF it's not going to work.
- d: my point is that if you do enough of the pairs so that you don't have any redundancy then it works out correctly. doing extra pairs doesn't buy you anything. i only have his old paper that is split into part 1 and part 2. in part 2 section 3 he goes through the derivation and explains whey these things are useless. i'll have to go over it with greg later and figure out how he got more DoF.
- jm: we can try det1 and we can keep it if it's good, but i wouldn't restrict myself to that.
- d: it has nice properties and when it made everything better it seemed like a good idea.
- jm: ??
- g: it did reduce the support of the transform. i think that on the 4x8 case, the ones we have checked in right now aren't problematic. now that i did something wrong with the lifters i should look into that. looking at the factorization i should lose 1 DoF. i'm not sure what i did, but i ended up with more.
- d: i know we did an early test to add in the extra pairs to make sure it didn't make a difference.
- g: i might have picked up the old code that had that. that's why i've been complaining about the search being a problem.
- jm: you mentioned you did some training with the leakage
- g: so far the results i'm getting out of that are crap. once i fix the DoF here that may change. taking 9 DoF out of your search is kind of magical.
- d: any point that was good in the old space doesn't mean you'd ever find it.
- g: i know what i need to do immediately. let's table this until i've done that.

# reviews

stuff is looking good

# fastssim

- jm: i want this back on
- j: it's broken
- g: monty also had positive opinions on fastssim.
- d: he's not on the call :(. my suspicion is that it's likely there are bugs in that code. someone should look at it and figure that out. monty had signed up to do that.
- g: i run it against the TID database and it was so bad as to be broken on that. it didn't do the same thing as the SSIM in their paper and they didn't have numbers for hte fast formulation, but it worked reasonable well on that database.
- d: we should coordinate with monty.
- g: i think jm is saying that it agrees with his visual inspection particularly on blocking artifacts.
- d: becaues of multiscale.
- jm: can we reenable it for now?
- d: i think that's fine. if you make graphs for public consumption then turn it off until we know more stuff.
- u: should i turn it back on in bd rate too?
- d: yes.
- u: i'll submit a patch.

# activity masking

- jm: i think we need to understand how AM works, but i don't. how do we figure it out? it sounds like the kind of thing we could spend two years doing fundamental research.
- g: what precise question should we ask?
- jm: what i want to do is understand the equivalent of what are the video critical bands when it comes to masking.
- d: right. the research i've seen suggests that they are directional and i've even seen there are 6 directions, which doesn't particularly work well for our square grids.
- g: the cells in the eyes are in a hexagonal pattern.
- d: whether that corresponds to how hte brain interprets things god only knows. after that things are octave band structrured. where those bands lie depends on how far away you sit from the screen. an octave can cover a pretty large range of the freq space. it's not like audio where pitch is fixed at an absolute scale.
- jm: audio is not logarithmic in terms of freq.
- d: my point is the boundaries are not at fixed locations. it's like audio as if you were on a speeding train.
- jm: you could offset everything by half a critical band and it would still work. this is not about it's betwen this and this frequency. it's around this point the critical band is this wide.
- g: i can give you a hand wavy argument: the vision system works when objects are at different distances.
- d: i don't have a good handle on what kind of threshold differences will ???
- jm: if we assume this then it's really different from the AM in theora.
- d: yes. i don't know if theora was brilliant. i tried something and it worked.
- g: x264 was very similar to theora's analysis.
- d: it does things at the macroblock level and stuff too. there's a lot of details in the stuff that jm cares about.
- g: all x264 is doing is trying to classify regions that are textured. a whole wide general class of techniques may be effective but doesn't help you decide what to do per band or on directionality. 6 directoins also doesn't fall nicely into our 2d separable transforms.
- jm: i would do an approximation there. horizontal, vertical, and diagonal. my main question is there masking between horiz and vert or between two difference octaves.
- g: i can propose a theory about how it might work. you've got difference detectors that run in each direction and there's effectively a high pass filter that comes out of hte feedback detectors, so if there's a lot .... in areas with lots of contrast, the sensitivity to changes goes down. i fyou assume they run independently and there are multiple scales then you end up with a conclusion that you expect them to be indepdendent in horiz and vert at finest scales and independence between bands.
- d: then you read hte papers and they say somethin else. short story is that no one has any idea how this stuff fucking works.
- g: keep in mind that these studies are done with really simple stimulus images.
# code party
- j: SFO office, week of 6/9 work for everyone?
- jm: i'll need to check that.
</pre>

DaalaMeeting20140401

2015-02-07T00:05:35Z

Daala-ts: Add word wrapping

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2014-04-01

Mumble: mf4.xiph.org:64738

# Agenda

- reviews
- Do we keep mumble?
-- But it finally works!
- the path to crushing jpeg
- path with PVQ?
- Are we finally happy the transforms are working right?
- Basis magnitude scaling

# Attending

jack, derf, unlord, gmaxwell, jmspeex

# reviews

- g: i'll take 226, 228, 229

# mumble

- jack: do we keep mumble? what alternative?
- nathan: tested vline with josh aas and it worked fine.
- jm: i would consider vidyo.
- j: we can invite external people, but they'll need the client.
- jm: can they dial in?
- j: yes.
- jm: we can keep mumble for now.
- j: ok. stay on mumble for now, switch the next time we have problems.

# path to crushing jpeg

- j: we're beating jpeg but not enough to impress in honolulu.
- jm: metrics or actual images?
- j: whatever helps
- g: both is best.
- d: we need to make things actually look better. that is the ultimate goal. we can talk around curves that don't look so great if we can show that shutting off things that make image better make the curves better. it's a good part of our process that we don't have to care about the metrics.
- u: i feel like we should be able to quantify this in a meaningful fashion.
- d: that's a research project in itself
- u: josh aas is doing exactly this
- d: we added the sensible ones and only one metric is showing improvements based on jm's visual improvements.
- j: all the metrics are some flawed approx of human vision.
- u: PSNR-HVS-M has human in the name!
- g: for the honolulu meeting, we need to figure out a way to convert better visual performance and convert it to something that looks good on paper. we can't present images in the IETF, we must present numbers.
(discussion of honolulu bof strategy)
- g: as far as where we need to be, do we need to beat webm?
- d: that would be nice.
- g; it would certainly help. where are we on the graphs?
- j: not even half way to vp8.
- j: do we need to beat webp?
(some discussion)
- j: summary: we don't need to beat webp for honolulu but do need to be better than where we are. what about what to do to fix performnace?
- d: i don't want to spend more time on intrapred. but i have some ideas. instead of trying to train a predictor for specific modes, start with just horiz and vert modes. first, undo dct part of transform and then can i explicitly undo the lapping? given that i have my AR95 model of my image, what is the optimal predictor for the pixels that i don't have for hte pixels i do have for the output fo the transform.
- jm: AR95 in one direction and trying to predict in the other direction?
- d: i'm looking at this as the 1d problem.
- g: what is the extension of the 1d line assuming you just go horiz and vert.
- d: right. this is where it starts to get very fuzzy. once you do that can you do it for diagonals?
- g: if that actually works it may make the rest of what we're doing already work better. you drop that in as a fixed mode and then see what happens to the rest.
- d: all kinds of stuff yfou can do. that's why i don't want to spend time to make it better.
- g: i think with all the mode stuff it may be the one of those cases that as long as we nail a few things very well, teh perofrmance of the others is not as important.
- d: clearly it doesn't seem true since h265 people added tons of additional diagonal modes.
- jm: so you're thinking of finding a closed form solution
- d: specifically there's a different one for each kind of lapping you're using.
- jm: i think that's part of the problem we have right now. we're using the same predictors regardless of which transform size we're doing. we're using TF and mixing stuff that has different basis magnitudes.
- d: that's certainly a problem mixing det1 and not.
- jm: even without it you won't get the same magnitudes. if your intrapredictor is trying to be fancy you're going to be way off.
- u: our predictors are coupled t the size of the transform. for the other codecs, that's nto the case.
- j: open loop can solve that right? do everything at 4x4?
- d: i still want to try that.
- jhM; still has basis problems
- d: yes, but that's what we slated for interns to look at.
- u: also more modes at larger sizes may help.
- jm: right now 45 degrees is not working.
- d: i want to spend time on this other idea, and figure out how to extend it to diagonal modes.
- jm: until we fix the ones we have it's no use adding more.
- j: what do we want to work on besides intrapred?
- d: block size switching
- jm: i don't think the problem is with switching
- d: right. we just want larger transofrms to work.
- g: i'm convinced that the transforms are not the problem. i wanted to know if other people are convinced of this?
- jm: do you mean implementation or use of dct?
- g: our implementation.
- jm: i've always been convicned of this.
- d: you were the only one. i wasn't convinced because i wrote it.
- j: why else might they not be working?
- jm: in the other codecs, 16x16 works, but not for the reason people pretend it works. right now it's not helping us to understand that. i don't think it's about coding gain.
- d: what's it about?
- jm: i don't know. basically i've spent some time measuring coding again on nonlapped 16x16. it's a tiny bit higher than 8x8 and if i actually compute distributions and their entropy then the entropy is worse or similar to 8x8. nothign there to show that it will perform better than 8x8. other codecs are getting better rd curves with those block sizes, but that's why i think it must be something else.
- d: you've never tried forcing everything to 16x16 or 8x8 in the other codecs?
- jm: i have not. i have tried measuring our forced 16x16 dfistributions. for example in one case, what i did was tune the BSS code to tend towards smaller block sizes so only flat blocks get 16x16. so then i used that classification and did it 8x8 and 16x16 so it was apples to apples. and even in that case it wasn't great. one thought i had was that for low bitrate you're not actually coding the high frequencies much, and that appears to be why 16x is doing worse than 8x.
- d: you're assumption of order 0 entropy may not be accurate?
- jm: it's possible but not the first thing i think of. it's also possible that there are many problems with the larger transforms each getting us a tiny bit worse. the problem there is you can't isolate anything.
- j: do you compute CG from math or from implementation?
- jm: i am measuring stddev from actual coeffs.
- g: do these agree with covariance matrices?
- jm: i didn't spend too much time, but there was nothing suspicious there. i've also been interested in looking at distribution gain which is entropy of distribution - log2 of stddev. basically how hard the actual distribution is to code. for this i was seeing 4x4 easier than 8x8 easier than 16x16 at equal stddev. it seems like ti would help to have someone who knows what they are doing to look at this.
- d: which things should i not look at? :)
- g: most things we can turn off an get anomalous results so it must be entropy coding.
- d: run some images in another codec and see what it does for block sizes.
- g: if we code images of sky with no landscape it should be all large blocks.
- d: go run that in another codec an dlook at the decisions.
- g: the fact that BS decisions are based on perceptual model and not CG.
- u: what about brute force BSS that minimizes rate, but we don't just care about rate. but ???
- d: we should figure out how to set it, everyone else does
- g: i should go try to implemetn this then. i think i need to implement some stuff to snapshot entropy coder state first, but it's all pretty simple.
- u: as part of these alternatives, instead of using lapped, do no lapping and use time domain prediction and code the rest using dct and pvq. that would be a different test that isn't us trying to analyze other codecs.
- jm: from the tests i ran, before running actual distributions, and 16x16 with no intra and nothing else was also not helping.
- u: i'm wondering if having time domain prediction is what's making the larger transforms work well.
- g: there are image codecs that are state of the art that odn't use intra at all, just BSS. that's why i think it wasn't intra that's causing most of our problems.
- u: do you have a name of one of those?
- g: ADCT. no open source, but windows binary.

# path to pvq

- jm: mostly i want to figure out what is the next steps?
- d: it's getting increasingly harder to compare to scalar as it grows more thigns that scalar doesn't have. i don't know what to do about that. unless you want to implement RDO for scalar?
- jm: AM is something i can do, but RDO i tried and failed before.
- d: i don't think anyone else has any hope of figuring out your k tokenizer.
- jm: the final rate it should be using is not much different than if you were coding coeffs independently. coding one extra pulse still has a cost and coding low frequencies is still cheaper than HF. it shouldnt' be really different than coding coeffs separately.
- j: so work on AM first then RDO?
- d: let me clean up quan matrices first.
- j: what's jm working on until then?
- jm: there's a whole bunch of cases in pvq where the ref vector is 0. i haven't yet checked whether it's because the area is so flat that there's nothing to predict or ... in any case, there's no point in coding flags and such in those cases. so right not we code them uselessly. ref = 0 is 1/3 of the cases i looked at. i still have on my todo list to include actual rate metrics, running the dct inside BSS.
- g: i was planning on doing something that was brute forcey. just code block every different way and measure performance coming out and not using perceptual metrics at all. what you're suggesting is halfway between those and i should try that too. i want to see what the largest improvement we can get from that path is.
- jm: with or without the prefilter?
- g: i'll have to figure out how to handle the prefilter.
- d: you're going to have to make some approximations.
- jm: an obvious one is not to run the prefilter. what's the status on training them?
- d: not spending more time on that.
- g: i don't want to spend any more time on it right now. read my status report from last week. i don't believe there are any prefilters in that design space that dont' have worse DC response but better coding gain.
- jm: my thoguht was let's get rid of det1 for now, but right now we can use the DC basis metric to improve our non-det1 filters. especially the 16x. in some cases it seemed to be doing a bit of blocking.
- g: it's easy to spin up the code to do that. i'll do it.
- jm: and also shoudl we move 4x away from det1?
- d: keeping it det1 means we can use it for lossless. the question is is there a lot to be gained by making it non-det1.
- jm: didn't it make fastssim worse?
- d: i don't remember
- g: we can look. i'll check that. i don't think it made anything worse and i did visual inspection too.
- d: i think it also wasn't much better and that was disappointing. i can't remember if it was before or after scaling.
- jm: in general these metrics are not that reliable.
- jm: all of the basis functions have a scaling less than 1.
- d: yep
- jm: so when I apply the scailing the rd curves all change and nothing matches
- d: because none of your lambdas match
- jm: the RD is really not doing anything now that we're scaling up.

</pre>

DaalaMeeting20140408

2015-02-07T00:04:34Z

Daala-ts: Add word wrapping (though this page has no really long lines)

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2014-04-08

Mumble: mf4.xiph.org:64738

# Agenda

- reviews
- ???

# Attending

jmspeex, jack, derf, unlord, gmaxwell, xiphmont

# reviews

- lots of stuff for derf to review
- gmaxwell reviewed 226, still has two outstanding

# pvq presentation questions

- reviewed tim's pvq presentation

</pre>

DaalaMeeting20140415

2015-02-07T00:04:11Z

Daala-ts: Add word wrapping

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">
# Meeting 2014-04-15

Mumble: mf4.xiph.org:64738

# Agenda

- reviews
- Discuss JM's experiments
- Google meeting

# Attending

gmaxwell, unlord, jack, jmspeex, derf

# reviews

no one waiting on reviews

# JM's experiments

- jm: did a bunch of experiments in random directions. see status update for the list. i tried splitting the lowest band in two. i tried both splitting along horiz and vert and putting the diagonal on either side. and i tried having horiz and vert in one band and diagonal in one band. that one was the worst of the two. splitting horizontal vs. vertical gave similar results--maybe slightly worse.
- derf: i assume the issue is overhead of the additional parameters.
- jm: i don't know
- d: how do we answer that question?
- jm: i don't know
- g: we could measure it while cheating with the additonal parameters
- jm: it was very close. let me get the numbers. i was getting stuff like some metrics being 0.6% better, psnrhvsm being 1% worse. that sort of scale.
# Google meeting
- d: i don't know if you noticed the range of the changes the google team was talking about while we were there.
- u: it felt like a one way conversation. they would like us to prove these techniques would work in libvpx and they'd love to use them.
- d: they phrased it better than they have in the past. the way they put it was that they wanted to build a library of techniques and then pick the things that worked. i don't think they quite fully understand that our techniques require some fundamental changes to the codec.
- d: ?? would they be upset if they took some of our ideas and the answer is absolutely no.
- u: they wanted us to show it would be an improvement in libvpx
- d: that's what debargha wanted :)
- g: jim gave a different perspective on it. after we left the meeting we had some additional conversation about the ietf.
- d: they wanted to know what the ideal ask was. so i sent him an email with that.
- u: that sounds promising.
- jm: we should discuss lower level things with them like our investigations on entropy in block
- j: let's make an etherpad for an agenda and get the next one schedule. put that item on there.
- d: i talked to them about june. will ping them.
- jm: in general we can talk about minor things we tried and things we are trying to figure it out.
- d: i tried to bring up metrics, but they didn't seem to have any great ideas there either.
# JM's experiments part 2
- jm: that splitting thing was worse with all metrics with activity masking.
- d: did you do interband masking?
- jm: i did not. that may be part of the problem. basically it was between 1 and 2% worse. i tried doing RDO tuning and i was able to improve all 4 metrics and the images look worse
- j: patent that and submit it to mpeg!
- jm: if i disable intra on every band except n=15 for 4x4 and 8x8 it's a wash.
- d: it was like 0.3%.
- jm: less than that.
- d: it's not a surprise that the current predictors are not doing much in HF.
- jm: my thought was that especially for 16x16, ?? trying to predict the first 15 properly. and not waste our nonzero coeffs on HF since we're not going to nail that anyway.
- d: isn't our training already doing that/
- jm: maybe for scalar it's not such a bad idea but for pvq having one nonzero coeff in the high band is completely useless. if you predict just one HF coeff you don't gain anything in pvq. in scalar you might have a tiny gain.
- g: i think if we're not predicitng the HF then we lose a lot of what intra is supposed to gain us.
- jm: for 16x16 we're not even able to predict the LFs enough that it becomes useful, so i don't see how we're going to predict HF anytime soon. one thing i was thinking aobut was can we do something along the lines of intra block prediction--predicting HF from LF?
- g: our original training tools were set up to assume a particular scan order and it added the previously scanned coeffs to the prediction
- jm: that won't work for pvq. something linear is not going to work. i don't see how something linear would work because if it did work, then the transform would suck. the dct is supposed to decorrelate all components. i don't see how it is going to help to have something linear.
- d: the nonlinearity comes from a) you're looking at the residue at a particular mode and b) biorthogonality issues, but those are probably minor.
- jm: even if there's a biortho issue, it probably means your thing shouldn't be orthogonal.
- u: whatever happened to the idea of undoing the lapping?
- d: i don't know how to do and it wasn't baked yet. i haven't worked on it yet.
- jm: i couldn't think of the entire system, but the general idea i had was something along the lines of trained VQ. training a codebook of HF (or one for each band), and then your codebook is 256 entries and based on the freq ???? that gives you a set of maybe 4 or 8 code vectors that are allowed given the low frequencies. it makes your search faster and what you need to code is not 8 bits but 2 or 4.
- d: i've been thinking of something along those lines for trying to use when the predictor is 0 or bad. somethign we could use to fill in HF to match whatever energy we had encoded. i hadn't thought of coding it conditionally based on the LF stuff. i'm not sure how that would work.
- jm: i know a way it could work but it would be too complex. take the LF and do a quantization (k-means and figure out which vector is closer among a set of 256).
- d: that's too complex. you could do it over 4.
- jm: my idea is to have just enough to be able to learn what happens if you have an edge in a particular direction.
- d: we could just classify "do i have an edge in a direction" for a small set of directions to start with right?
- jm: what do you mean?
- d: instead of learning 128 of these things, build a classifier that tells you that you have an edge in direction X for some small set of directions.
- jm: we need directions and offsets. this is not a predictor
- d: that makes it much more of a mess. i don't know how to do that in the freq domain.
- jm: the other thing i thought about is instead of using trained VQ to classify the LF, using PVQ. But then the problem is that at most we'd be able to use three pulses and they'd end up in the lowest freqs so there wouldn't be many combinations in practice.
- d: yep. i think you have almost no information about whether or not you have an edge.
- jm: the only thing i tried so far was the case where i didn't code anything, just trying to predict. doing VQ with a codebook of 256 in the low band and using that to predict the high band. that had very little prediction effect. it's quite possible that having just a few bits in the HF would have much more impact. i assume in many cases the HF could go one way or another but if you average them you get nothing.
- d: you're going to run into phase issues
- jm: you resolve phase with ???. i tried to predict ??? with the sine coeffs. there are a few factors that are skewed by 25-75% but if you looked at overall we'd end up saving 0.1%. i haven't looked at what we gain by coding.
- d: how would this work for inter?
- jm: why would you want to do this? MC should be a lot better than what we get out of this.
- d: it's a lot better than what our inter is doing right now. but correlations in the HF is going to be around 75%. it's a lot small than .99 which is what i would like.
- jm: i was tyring to get to a point where it's better than no prediction at all, because that's where we are now. ???
- g: the next thing we could look at is a time domain predictor
- d: have we measured the prediction power in the HF with the VP8 predictors? turn off lapping and use the DCT and we have the VP8 predictors implemented.
- jm: we see from h265.. the fruits image at ridiculous rate... we can see it's possible to predict something.
- d: my question is if using the same metrics you're using how it compares. that should tell us what is possible to do.
- jm: that's totally different. right now we have nothing. doing anything at all is already better.
- d: but i want to know what is achievable.
- jm: i don't know.

# activity masking

- d: i did some playing around but i didn't find a whole lot. didn't do that badly on the worst examples i could come up with. it does make text a little bit worse, but it's more moving noise around and maybe adding some, more than severely degrading anything. it's not that bad. i wanted to compare it to some of the scalar quant stuff i've been working on, but it's not ready yet. you point out that we can reduce luma DC resolution at 4x4.
- jm: i was thinking the larger ones. probably all of them though.
- d: the way scalar quant matrices work is that DC needs more resolution than the first two AC coeffs.
- jm: one thing i experimented with was trying to have DC be much worse for luma and i found it surprising how much error we could tolerate in the DC. i suspected it was because in JPEG the error in DC causes more blocking and we dont' have the problem.
- d: humans are insensitive to DC shifts. we may be able to get away with worse than traditional codecs do there. i assume that we're terrible on metrics on your images?
- jm: i didn't even look. tim, you mentioned not being good on text. AM is only enabled on 8x8 and 16x16 and disabled on 4x4 because you don't want to do it on edges. it would be interesting to see what it does on 4x4 but i haven't really tried because i didn't expect it to improve.
- d: i'll turn it on and run it on the text again and we'll find out.
- jm: http://jmvalin.ca/video/iena0.png http://jmvalin.ca/video/iena1.png http://jmvalin.ca/video/iena2.png http://jmvalin.ca/video/iena3.png
- jm: 0 is normal. changing resolution by 2^n. 8 is definitely extreme.
- d: did you just change the quantizer?
- jm: i only changed the quantizer. the effect is more noticeable on airforce.
- d: that's because that image is only DC.
- g: you can see in #3 that there is a band of darkness in the upper left corner.
- d: what it suggests is that the sky in the other images happens to be close to a quantization interval.
- jm: one thing i considered also was look at having large steps except for the very first step. if you're on a gradient you can code the small differences but for larger changes you don't have that small of resolution. image 0 is 104kB. image 1 is 101.5kB. image 2 is 99kB. image 3 is 97kB. it's about 2.5kB every time i double the error.
- d: a bit over 2%. it seems like the step size you needed to code gradients correctly depends on the slope of the gradient.
- jm: why would it?
- d: i have some huge gap in my dc quant levels. at some point i guess it's slow enough we can take one step.
- jm: yes, i mean just in this region. i'll show you airforce and you'll see what i mean.
- jm: http://jmvalin.ca/video/air_dc0.png http://jmvalin.ca/video/air_dc1.png http://jmvalin.ca/video/air_dc2.png http://jmvalin.ca/video/air_dc3.png
- jm: Maybe most of the rate would be in that first step.
- d: Yes, but you keep pointing out how much the tails matter.
- jm: Because of scaling I don't think we can even predict a flat image correctly when the block size changes
- d: We certantly need to do something about the scaling at least for DC
- g: So we use a table of scaling factors
- jm: There are too many even just for dc
- g: so we scale things to a common reference e.g. 16x16 and then scale back so it's two multiplers per size
- d: there were reports on IRC that 4:4:4 was broken, anyone want to look at that?
- u: I'll look at that, I think I noticed that a while ago and didn't follow up on that

</pre>