XiphWiki - User contributions [en]

Daala

2015-10-26T08:35:10Z

Tomvanbraeckel: Link to new Etherpad Lite

Daala is the codename for a new video compression technology.

The effort is a collaboration between the [https://www.mozilla.org/en-US/research/ Mozilla Foundation], the [https://www.xiph.org/ Xiph.Org Foundation] and any other contributors that wish to help.

The goal of the project is to provide a video format that's free to implement, use and distribute, and a reference implementation with technical performance superior to [https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding H.265].

Please see the links below or the [https://www.xiph.org/daala/ main page] for more information.

== Wiki Pages ==
* [[Daala Quickstart|Daala Quickstart (Linux/MacOS)]]
* [[Daala Quickstart Windows|Daala Quickstart (Windows)]]
* [[Daala MinGW64 Environment]]

* [[Daala Weekly Meetings|Daala Weekly Meetings]]

* [[AreWeCompressedYet]]
* [[RD Curve Data Format]]

* [[DaalaTodo|Daala To-do List]]
* [[DaalaRoadmap|Daala Roadmap]]

* [[Intra|Intra-prediction within Daala]]

* [[Videos|Digital Primers]] - educational videos about audio/video technology

== Communication ==
You are '''encouraged''' to join the
* [irc://irc.freenode.net/daala '''#daala''' IRC channel at freenode.net] - if you don't have an IRC client, you can use Freenode's '''[https://webchat.freenode.net/?channels=%23daala webchat]''' instead.
* [http://lists.xiph.org/mailman/listinfo/daala Daala Email List]

=== Weekly Meetings ===
You are also welcome to attend the public [[Daala Weekly Meetings|weekly progress meetings]] by installing and using [http://wiki.mumble.info Mumble].<br />
The address is '''mf4.xiph.org''' and the port is '''64738''' (you can run '''mumble://mf4.xiph.org:64738''' within your browser as a shortcut).<br />
The meetings occur on '''Tuesdays''' at '''[http://www.timeanddate.com/worldclock/fixedtime.html?msg=Daala+Weekly+Meeting&iso=20150428T09&p1=1241 9AM Pacific Time]''' (5PM UTC/GMT).
The meeting agenda used to be available at '''[https://daala.etherpad.mozilla.org/weekly-meeting this Etherpad]''', the October 13, 2015 meeting is available on [https://docs.google.com/document/d/1JP_Ko3wPuyDWhooZcp_m9kndyfZ75xN5YOi5yIMCW0s/edit?pli=1 Google Docs] and, following the migration to Etherpad Lite, the meeting agenda and minutes are now available at [https://public.etherpad-mozilla.org/p/daala-weekly-meeting this Etherpad].

=== Other ===
* [http://forum.doom9.org/showthread.php?t=168004 Doom9 Forum discussion] - generic forum thread regarding Daala
* <del>[https://daala.etherpad.mozilla.org/ep/padlist/all-pads Daala Etherpads] - you can [https://daala.etherpad.mozilla.org/ep/account/request-account request a free account] to view these. You should receive access within a few days.</del> Mozilla are transitioning to Etherpad Lite.
* [http://benjamin.smedbergs.us/weekly-updates.fcgi/project/daala Daala Project Status Board] - what Daala bits the Mozilla people are working on

== Coding ==
You can get a copy of the latest Daala Source Code from [https://git.xiph.org/?p=daala.git;a=summary '''git.xiph.org'''] or [https://github.com/xiph/daala '''GitHub''']. Please stick to the '''[https://git.xiph.org/?p=daala.git;a=blob_plain;f=doc/coding_style.html Coding Style Guide]'''.

* [https://review.xiph.org/all?limit=100 Xiph Code Reviews] - there is a proposal on the review process '''[[DaalaReview|here]]'''
* [https://github.com/xiph/daala/issues Daala's issues] - Issue/bug tracker on Github
* [https://mf4.xiph.org/jenkins/view/daala/ Continuous Integration Tests] - these run every time a new commit is made to the Daala git master, to make sure the new code hasn't broken existing functionality.

== Demos ==
* [https://people.xiph.org/~xiphmont/demo/daala/player-demo.shtml Daala Video Player] - an example implementation of a Daala decoder and player, ported to Javascript using [https://github.com/kripken/emscripten Emscripten].

=== Codec Techniques ===
* [https://people.xiph.org/~xiphmont/demo/ Demo Articles] - explanations on certain techniques used in Daala (and other Xiph.Org projects)
* [http://exp.martres.me/edi/ Edge-Directed Interpolation] ([https://github.com/smarter/edi source code])
* [https://people.xiph.org/~ds/edi/info.html More Edge-Directed Interpolation]
* [https://people.xiph.org/~unlord/demo/intra.html Intra-prediction]
* [https://people.xiph.org/~unlord/zigzags.html Macroblock Coefficient Zigzag Graph] - HTML page generated using [https://github.com/xiph/daala/blob/master/tools/draw_zigzags.c tools/draw_zigzags.c] from the Daala source code.

== Documents ==
* [https://people.xiph.org/~unlord/spie_cfl.pdf Chroma from Luma (CfL)]
* [http://jmvalin.ca/papers/spie_pvq.pdf Perceptual Vector Quantisation (PVQ)]
* [https://people.xiph.org/~tterribe/daala/vbsobmc.pdf Overlapped Block Motion Compensation (OBMC)]
* [https://mf4.xiph.org/jenkins/job/daala-autotools/ws/doc/html/index.html C API Documentation]
* [https://people.xiph.org/~yushin/tmp__/yushin_phd_thesis.pdf Image Coding Thesis] by Yushin Cho
* [http://arxiv.org/pdf/1411.4290v1.pdf Maximising Coding Efficiency Through Block Rotation] and why it [http://lists.xiph.org/pipermail/daala/2015-January/000054.html won't work well within Daala]
* [http://jmvalin.ca/video/theoretical_results.pdf JMSpeex' Journal of Dubious Theoretical Results] - "take with an entire shaker-full of salt"
* [https://people.xiph.org/~unlord/pcs_daala.pdf Using Daala Intra Frames for Still Picture Coding]

=== IETF Drafts ===
* [https://tools.ietf.org/html/draft-egge-videocodec-tdlt Time-Domain Lapped Transforms (TDLT)] - documents the Lapped Transform pre- and post-filters used for block-edge decorrelation
* [https://tools.ietf.org/html/draft-valin-videocodec-pvq Perceptual Vector Quantisation (PVQ)] -
* [https://tools.ietf.org/html/draft-terriberry-codingtools Coding Tools] - documents Entropy Coding, Integer Transforms and other techniques
* [https://tools.ietf.org/html/draft-moffitt-netvc-requirements Internet Video Codec (NetVC) Requirements] - explains what requirements and use cases Daala is trying to cater for
* [https://tools.ietf.org/html/draft-daede-netvc-testing Internet Video Codec (NetVC) Testing and Quality Measurement]
* [https://tools.ietf.org/html/draft-terriberry-ipr-license Example IPR Licence Terms]

Additional drafts can be found at the [https://datatracker.ietf.org/wg/netvc/documents/ IETF DataTracker].

== Presentations ==
* 2015-09-19 VideoLAN Dev Days - [https://www.youtube.com/playlist?list=PLQLpBN3oI7E44HIdTOovThc1MNHLchgHE YouTube Playlist]
* 2015-07-22 IETF 93 - [http://recordings.conf.meetecho.com/Playout/watch.jsp?recording=IETF93_NETVC_II&chapter=chapter_1 NetVC Session 2/2] - [https://datatracker.ietf.org/meeting/93/agenda/netvc/ Agenda] - [http://www.meetecho.com/ietf93/netvc_II Video and Chat] - Slides - [https://www.ietf.org/jabber/logs/netvc/2015-07-22.html Jabber Log]
* 2015-07-20 IETF 93 - [http://recordings.conf.meetecho.com/Playout/watch.jsp?recording=IETF93_NETVC&chapter=chapter_1 NetVC Session 1/2] - [https://datatracker.ietf.org/meeting/93/agenda/netvc/ Agenda] - [http://www.meetecho.com/ietf93/netvc Video and Chat] - [https://www.ietf.org/proceedings/93/slides/slides-93-netvc-3.pdf Slides] - [https://www.ietf.org/jabber/logs/netvc/2015-07-20.html Jabber Log]
* 2015-03-24 IETF 92 - [http://recordings.conf.meetecho.com/Playout/watch.jsp?recording=IETF92_NETVC&chapter=chapter_0 NetVC Session] - Audio as [https://people.xiph.org/~tdaede/audio/ietf92-venetian-20150324-0900-am1.opus Opus] (29MB) or [https://www.ietf.org/audio/ietf92/ietf92-venetian-20150324-0900-am1.mp3 MP3] (119MB, action starts at 14:50) - [https://www.ietf.org/proceedings/92/slides/slides-92-netvc-0.pdf Slides] - [https://www.ietf.org/mail-archive/web/video-codec/current/msg00235.html Notes] - [https://www.ietf.org/jabber/logs/netvc/2015-03-24.html Jabber Log]
* 2015-02-11 SPIE talks:


** [http://people.xiph.org/~tdaede/video/SPIE_Nathan.webm Chroma from Luma (CfL)] - [https://people.xiph.org/~unlord/SPIE-2015-CfL.pdf Slides] - [https://people.xiph.org/~unlord/spie_cfl.pdf Paper]
** [http://people.xiph.org/~tdaede/video/SPIE_PVQ.webm Perceptual Vector Quantisation (PVQ)] - [http://people.xiph.org/~tterribe/daala/spie_pvq_slides.pdf Slides] - [http://jmvalin.ca/papers/spie_pvq.pdf Paper]
** [http://people.xiph.org/~tdaede/video/SPIE_Tim.webm Overlapped Block Motion Compensation (OBMC)] - [http://people.xiph.org/~tterribe/daala/spie_pvq_slides.pdf Slides] - [https://people.xiph.org/~tterribe/daala/vbsobmc.pdf Paper]
* 2015-01-31 [http://ftp.osuosl.org/pub/fosdem/2015/devroom-open_media/daala.mp4 Daala Project Update at FOSDEM 2015] - [https://fosdem.org/2015/schedule/event/daala/ summary] - [https://fosdem.org/2015/schedule/event/daala/attachments/slides/569/export/events/attachments/daala/slides/569/Daala_FOSDEM_2015.pdf Slides]
* 2015-01-14 [https://www.youtube.co.uk/watch?v=Dmho4gcRvQ4 Linux Conf 2015] - [http://lca2015.linux.org.au/schedule/30187/view_talk presentation summary] - [https://people.xiph.org/~tterribe/pubs/lca2015/daala.pdf Slides]
-------
* 2014-09-16 [https://air.mozilla.org/daala-are-we-compressed-yet/ Daala: Are We Compressed Yet?]
* 2014-06-25 [https://air.mozilla.org/sparsity-induced-prediction-for-images-and-video/ Sparsity Induced Prediction for Images and Video]
* 2014-06-06 VP9 Summit (no video available) - [https://people.xiph.org/~xiphmont/demo/daala/daala-vp9summit-20140606.pdf Slides]
-------
* 2013-10-23 [https://people.xiph.org/~xiphmont/video/Free_Codecs_Update_Opus_and_Daala.ogv Opus and Daala: State of the Art Royalty-free Codecs] - [https://people.xiph.org/~greg/gstreamer-daala-opus.pdf Slides]
* 2013-09-30 [https://people.xiph.org/~tterribe/daala/coding_party2/?C=M;O=A Daala Coding Party 2] - [https://people.xiph.org/~unlord/Daala-Intra.pdf Slides]
* 2013-05-02 [https://people.xiph.org/~xiphmont/tim-terriberry-presents-daala/ Tim Terriberry Presents Daala]
-------
* 2012-01-24 [https://media.basilgohar.com/derf-talks/?C=M;O=A Introduction to Video Coding] - [https://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf Slides] (no video for slides 1-50)

== Other Websites ==
* [https://www.zazzle.com/daala_tee_shirt-235139149596175944 Daala T-shirts] - if you'd like a free one, help out with the project and ask the Mozilla guys nicely for one :-)
* [https://www.xiph.org/donate/ Donate to Xiph.Org]
* [[Daala_on_Wheels|Historical Daala wiki page]]

[[Category:Daala]]

Daala Weekly Meetings

2014-09-24T07:15:11Z

Tomvanbraeckel: Add minutes of September 23th, 2014 from https://daala.etherpad.mozilla.org/weekly-meeting

We've been having weekly progress meetings on mumble.

* 2012 June 4 [https://people.xiph.org/~giles/2012/daala_20120604.txt minutes] (actually a work week)
* 2012 June 22 [https://people.xiph.org/~giles/2012/daala_20120622.txt minutes]
* 2012 June 29 [https://people.xiph.org/~giles/2012/daala_20120629.txt minutes] [https://people.xiph.org/~giles/2012/daala_20120629.opus recording]
* 2012 July 6 [https://people.xiph.org/~giles/2012/daala_20120706.txt minutes]
* 2012 July 13 [https://people.xiph.org/~giles/2012/daala_20120713.txt minutes]
* 2012 July 20 [https://people.xiph.org/~giles/2012/daala_20120720.txt minutes]
* 2012 July 27 [https://people.xiph.org/~giles/2012/daala_20120727.txt minutes]
* 2012 August 3 [https://people.xiph.org/~giles/2012/daala_20120803.txt minutes] [https://people.xiph.org/~giles/2012/daala_20120803.opus recording]
* 2012 August 10 [https://people.xiph.org/~giles/2012/daala_20120810.txt minutes]
* 2012 August 17 - no meeting
* 2012 August 24 - [https://people.xiph.org/~giles/2012/daala_20120824.txt minutes] [https://people.xiph.org/~giles/2012/daala_20120824.opus recording]
* 2012 August 31 - no meeting
* 2012 September 7 - no meeting
* 2012 September 14 - no meeting
* 2012 September 21 - [https://people.xiph.org/~giles/2012/daala_20120921.txt minutes]
* 2012 September 28 - [https://people.xiph.org/~giles/2012/daala_20120928.txt minutes] [https://people.xiph.org/~giles/2012/daala_20120928.opus recording]
* 2012 October 5 - [https://people.xiph.org/~giles/2012/daala_20121005.opus recording]
* 20120 October 26 -
* 2012 Novemeber 2 - no meeting
* 2012 December 7 - [https://people.xiph.org/~giles/2012/daala_20121207.txt minutes]
* [[DaalaMeeting20130911|2013 September 11]] - logistics, status updates, coding party agenda
* [[DaalaMeeting20130917|2013 September 17]] - summit plans, research talks, coding party agenda, monty's TF ideas
* [[DaalaMeeting20130924|2013 September 24]] - training results, 32x32 code, and 2nd stage TF
* [[DaalaMeeting20131008|2013 October 8]] - pvq status, training, chroma from luma
* [[DaalaMeeting20131022|2013 October 22]] - even/odd quantizer, determinant 1, IETF, chroma from luma
* [[DaalaMeeting20131029|2013 October 29]] - gstreamer conf, ietf, chroma from luma
* [[DaalaMeeting20131112|2013 November 12]] - chroma from luma, pvq
* [[DaalaMeeting20131203|2013 December 3]] - chroma from luma, pvq, misc
* [[DaalaMeeting20131217|2013 December 17]] - luma intraprediction, pvq decoder, interns
* [[DaalaMeeting20140107|2014 January 07]] - reviews, pvq projects, and det1
* [[DaalaMeeting20140114|2014 January 14]] - pvq, det1, new projects
* [[DaalaMeeting20140121|2014 January 21]] - project planning
* [[DaalaMeeting20140128|2014 January 28]] - pvq, work week
* [[DaalaMeeting20140211|2014 February 11]] - 32x32, activity masking, current issues
* [[DaalaMeeting20140218|2014 February 18]] - input scaling, prediction error
* [[DaalaMeeting20140225|2014 February 25]] - det1, pvq, fastssim
* [[DaalaMeeting20140304|2014 March 4]] - det1, quant matrix, metrics
* [[DaalaMeeting20140318|2014 March 18]] - det1, fastssim, activity masking
* [[DaalaMeeting20140401|2014 April 1]] - crushing jpeg, pvq
* [[DaalaMeeting20140408|2014 April 8]] - reviews, pvq presentation
* [[DaalaMeeting20140415|2014 April 15]] - pvq experiments, activity masking
* [[DaalaMeeting20140422|2014 April 22]] - block size switching, gpu jpeg
* [[DaalaMeeting20140506|2014 May 6]] - workweek debrief, trained vq
* [[DaalaMeeting20140513|2014 May 13]] - trained vq, intraprediction
* [[DaalaMeeting20140520|2014 May 20]] - dc encoding
* [[DaalaMeeting20140527|2014 May 27]] - standard metrics
* [[DaalaMeeting20140603|2014 June 3]] - deringing filter
* [[DaalaMeeting20140624|2014 June 24]] - deringing filter
* [[DaalaMeeting20140701|2014 July 1]] - multi-frame motion compensation, jean-marc crazy ideas
* [[DaalaMeeting20140708|2014 July 8]] - intrapred ideas, journaling, rd_collect issues
* [[DaalaMeeting20140715|2014 July 15]] - VIPC, intrapred
* [[DaalaMeeting20140722|2014 July 22]] - demo progress, video results, edi, deringing, new intra
* [[DaalaMeeting20140729|2014 July 29]] - edi, intra paint
* [[DaalaMeeting20140805|2014 August 5]] - edi, intra paint
* [[DaalaMeeting20140812|2014 August 12]] - multi-frame MC, intra paint, edi
* [[DaalaMeeting20140826|2014 August 26]] - intra paint, halfpel BMA
* [[DaalaMeeting20140902|2014 September 2]] - demos, multi-frame motion compensation, deringing
* [[DaalaMeeting20140909|2014 September 9]] - reviews, deringing, intraprediction, non-photographic video
* [[DaalaMeeting20140916|2014 September 16]] - reviews, unlapping
* [[DaalaMeeting20140923|2014 September 23]] - reviews, VDD debrief, deringing demo, intraprediction tests

DaalaMeeting20140923

2014-09-24T07:13:12Z

Tomvanbraeckel: Add weekly meeting report of 20140923

<pre>
# Meeting 2014-09-23

Mumble: mf4.xiph.org:64738

# Agenda
- VDD debrief (derf, unlord)
- reviews
- Deringing demo (jm)
- more intra pred tests (unlord) http://people.xiph.org/~unlord/intra/
- travel plans IETF-91
- ?

# Attending

unlord, jack, derf, bkoc, jmspeex, td-linux

# Reviews

- td-linux will review gal's patch
- monty will review td-linux's quantizer scaling

# VDD

- u: Tim gave a great talk.
- d: I thought it was good. We gave away some tshirts and got a potential intern for next year.
- u: We have patches from new contributors in rietveld. Hopefully they will continue!
- d: katarina has been trying to correct the scale factor between the different levels. she managed to make the curves go down a bit, but she hasn't gotten very far.
- jm: all our dcs have similar scaling now.
- d: they were within about 0.9.
- jm: Still awesome that someone is looking at this.
- u: I'm also looking at it. I think I figured about how to handle the Haar DC fi we want to not code the blocks outside the frame. I'll try and send a patch for that in the next day or so.

# deringing

- jm: Who's seen the demo I came up with?
- j: I haven't seen it.
- jm: I haven't had any comments yet.
- d: In general I thought it was great and I used it in my presentation. I can give you more detailed feedback today.
- jm: While working on this I think I came up with a decent way to do the search. Basically in terms of complexity in each direction I would have only one add per pixel and two multiplies per line. Meaning for horizontal and vertical you have N lines and for 45 degree and diagonal you have 2N-1 but still relatively cheap.
- d: What are you actually doing there to get that?
- jm: Nothing fundamentally different. I just did a bit of simpliciation. My criterion is the MSE compared to the painted image. The painted image is just an average along a certain direction. MSE - avg is basically the variance. If I go back to the variance computation, you are doing sum of the squares minus sum squared divided by N and the sum of the squares is constant across all the directions you search for. In the ned all you need to do is for each direction you follow each line summing, and then divide by N and sum all the lines.
- d: You are just recentering your variance calculation. That's fine.
- jm: You're just computing all the sums for each direction and then squaring and divinging.
- d: Do you really get away with squaring things along the line?
- jm: It all cancels out between directions. All the x^2 terms cancel each other.
- d: This is starting to sound more promising.
- jm: It runs pretty fast now.
- d: How fast?
- jm: I still haven't optimized the painting but if I look at the total time it goes from .5s to .7s instead of going to 3s. This is on fruits.
- d: I still think there should be a way to do this without computing the paint image for every direction.
- jm: I suspect most of the complexity is no longer the search. I suspecting painting has become more complex because I'm still doing the square roots and divisions and stuff.
- d: I'd like to avoid doing the paints for all or most of the directions as part of the search.
- jm: I don't see a way to avoid that. Mayber there is but I haven't seen it. I think the way I'm doing it now is viable. In terms of per pixel operations it is 9 now.
- d: Yes, but maybe you can look at the first few AC coefficents and compute a gradient and then your per pixel operations is zero.
- jm: I think we're going to have a really hard time figuring out a direction in the general case that I"m trying to handle. LIke I"ve got a boundary between texture and something smooth or I have a whole bunch of lines in different directions. If we could see AC coefficients could we code it efficiently?
- d: I still think it is worth experimenting with. I don't think it will be as good as what you are doing now, but maybe it will be close and you'll only have to search a few directions.
- jm: It's hard to do much. If you have a few directions to search you haven't reduced the complexity very much. The base complexity is not high.
- d: I don't have to do all the painting for those directions
- jm: I don't do the painting there.
- d: Oh, nevermind.
- jm: I paint only in the one direction, and that hasn't been optimized. Right now I'm painting the whole image even if some blocks are disabled. The decoder will be able to avoid painting disabled blocks.
- d: Maybe not as big of a deal as a thought, but probably still worth taking a look at.
- jm: It seems like anything you would try would be more complex than what I'm currently doing. Right now I'm just searching for 8 directions and considering whether to do a refinement to 16, but maybe the search for 8 could be 4 plus refinement. I'm not sure if it's worth it. http://pastebin.mozilla.org/6594773 This is my directional search.
- d: This looks like pretty good progress.
- jm: For the on/off switch I think I want to do something else. At low rates just coding one flag per 8x8 block would be bad. I need to figure out what I want to do instead. Either going to 16x16 or automatically disabling the postfilter in blocks where we don't think it will be useful.
- d; Either that or you do some hierarchical thing. As in you code at 32x32 whether there are any blocks to dering and split down.
- jm: Right now if I just let it choose which ones seem best it picks about half/half. At -v 80. At lower v it picks fewer blocks. If i just let it choose with no constraint there's no way to reduce the rate. It's going to cost me one bit per block.
(missed some)
- jm: Unlike other stuff I've done this one is basically just a post filter. Is anyone up for helping merge this?
- d: If I get time I'm interested. I'll see what tomorrow looks like.
- jm: Monty did you look at this?
- x: It was spooky because it looked like something I would write.
- jm: http://jmvalin.ca/daala/demo1/

# intrapred tests

- u: I continued the work from last week. There's a URL in the agenda above that has a bunch of images.
- j: This looks amazing. Unlapping closed most of the gap with VP8.
- u: I'm not including mode signaling cost yet, so still need to do that and Haar DC.
- jm: for the cp ones, you should try copying only the coefficients which have horizontal energy. I would like to see this graph as it gets closer to master's features.
- u: I'm working on that.
(missed some)
- jm: You would look at left blocka nd see horitonzal energy. And look up at vertical energy. Let's say it's the one on the left. Youy would take the N horizontal coeffs to predict the current block. If it's bad you just signal noref. In this case, you'd have no signaling for the mode.
- u: The mode selection has to take it into account.
- jm: There is no signaling needed here. The decoder picks automatically.
- u: We could try this right now.
- jm: It wouldn't be entirely free because you're signaling noref, but it's pretty cheap compared to 8 modes. There's no potential for making bad decisions like spending 3 bits to code a mode which is thrown away.
- u: Currently we should take into account noref when we spend those three bits.
- d: This needs research because it sounds vaguely like what mpeg4 part 2 did with copying one row or column of AC.
- u: There's more things I want to experiment with. 1) Take the intrapredictors (TFing neightbors down to 4x4) and add it to this graph 2) Go back and retrain where you are using just AC coeffs. 3) Instead of training as we do now, train which coeffs to copy.
- jm: Are these variable block size?
- d: These are all 8x8 but adding the rest is in my plan.
</pre>

Daala Weekly Meetings

2014-09-16T13:01:05Z

Tomvanbraeckel: Add minutes of September 9th, 2014 from https://daala.etherpad.mozilla.org/weekly-meeting

DaalaMeeting20140909

2014-09-16T12:57:07Z

Tomvanbraeckel: Unscrew the formatting

<pre>
# Meeting 2014-09-09

Mumble: mf4.xiph.org:64738

# Agenda

- reviews
- deringing
- intraprediction
- non-photographic video

# Attending
- unlord, tmatth, smarter, jack

# Reviews

https://review.xiph.org/455/
https://review.xiph.org/456/

# derining

- jm: I've continued on using paint as deringing. I have results that are slightly cheating but getting better. http://jmvalin.ca/video/dering/ What this is doing is computing a paint image and from the coded image figuring out (from variances and stuff) the gain that should apply to the derining. Ie, how much of the paint image we should use vs. how much of the decoded, pre-paint image we should use. It's doing a reasonable job but it's not perfect. On top of this I'm adding a single flag per block which disables the filter entirely for the block (gain of 0). I don't have crossfade, but I didn't notice much of an issue with blocking artifacts. I didn't generate images because I wasn't quite sure what the best thing to generate was.
- d: Howa bout Clovis?
- x: Good torture test.
- jm:
off: http://jmvalin.ca/video/dering/clovis0.png
on: http://jmvalin.ca/video/dering/clovis.png
We can see easily. It's at least helping on the edges of the dunes. The color edges are just chroma contrast so they don't show up. I'm not sure yet how to handle chroma. THere's many ways to handle it. One way is through CfL. Ie, not apply deringing to chroma at all, but just on luma.
- d: That means you have to delay chroma until a whole line of superblocks has beend one. That is specifically what the HW people didn't like about CfL. We have an advantage because of doing it in the frequency domain.
- jm: Actually more than that because of the filter. Two superblock rows.
- jm: Those are -v60. Output is 18k. This is much below what I normally used for testing.
- d: I notice it does a fairly good job on the big balloon in the center on the upper left edge. But on the right edge seems like it does nothing.
- jm: It can't do much for features that are horizontal or vertical because the 2d dct is already a directional basis and its artifacts will be aligned with the direction of the image. If you look just below the bulge where its 45 degrees you see a huge difference.
- d: It seems bad that can't do anything about horizontal or vertical.
- jm: The rining is worst on the diagonals usually.
- d: The big question is "how slow is this?"
- jm: The normal encoder would take 0.4 seconds, and deringing takes a little under 3s. The vast majority is spent in the direction search.
- j: 6x slower!
- d: To be clear, just for this feature he is using 150x the decoder budget!
- d: Does the operation scale linearly with blocksize?
- jm: 16x16 is at most 2x as complex as 8x8 but it depends on what directional resolution we want. So per-pixel it's the same cost. The number of ops per pixel is on the order of 50. Assuming we don't do any downscaling before we find the direction.
- d: That's not completely unreasonable so why is it 150x slower?
- jm: I'm doing optimal interpolation weights around all four edges etc. If you look at intrapaint.c there is a pixel_interp function and that takes most of the cost. You call this for every pixel and it uses square roots etc. Then numbers I gave you are too optimistic. I forgot computing the distortion. It may be closer to 100 or 150.
- d: ekr is trying to hook us up with someone who's done speed optimizations for opensll, etc. Would it make sense for you to talk to him for trying to make this function super fast?
- jm: Yeah. Unlike openssl, I'm pretty flexible on the outputs.
- d: If all you want is a direction, why not just take an edge detection algoirthm and run an arctan?
- jm: I'm not a video guy so I've never run an edge detector. Actually that wouldn't work. I am not necesarily looking at edges, i'm looking at stuff like low variance in one direction and low variance in another. Like the tail of the horse.
- d: I'm talking about running a gradient operator. If one direction is dominant you should be able to find it. At lower rate you need a larger support for your gradient operator. We need to figure out if it is possible to make it sufficiently fast. Are you interested Monty?
- x: I think it is doomed. I'm not convinced it is generally useful. It does give a pleasing effect at low bitrates. I dont' see what it buys Daala; it's not codec specific.
- jm: Right now it's about the only thing we have to deal with ringing since we don't have intra.
- d: Monty to be clear, this is not a post processor. It's in-loop.
- x: Oh, I missed that. This is interesting, but we have other problems to ship a codec and I don't see how this solves our problems.
- d: Right now directional edges and features are not captured well since we haven't figured out intra.
- x: But we've also failed to make deringing from other codecs work. Figuring that out seems like a much more interesting problem.
- jm: I don't think the 265 stuff can get rid of the ringing we get at low bitrates. It just changes individual samples.
- x: Won't do that in 265 either right?
- jm: No, but 265 has intra prediction!
- j: Sounds like you want to spend more time on intra?
- x: Yes.
- d: Nathan is working on this. Sounds like you should move on to the next thing in your pipeline.

# intraprediction

- u: I have the still image test bed and it produces RD curves now. I'll post some of the curves. I don't think that AC coeefs from neighbors is a reasonable approach. It requires too large of support.
- d: Especially for shallow angles.
- u: At 45 you can go one block away, but at others you ahve to go 2 away. I'll post curves later today. I can't point you at the code Monty. It's super simple.

# non-photographic video

- jm: I think this is very useful and we should start planning for it.
- d; I was not going to worry about it until next year.
- jm: This has the potential to be a killer feature.
- x: That all by itself would get RedHat's attention.
- j: I brought htis up as a differentiator but Tim didn't seem super supportive.
- x: It's hard.
- d: I don't htink it's that hard.
- jm: There is a big piece that is missing from what I did for paint. Being able to encode antialiased text.
- d: Anything you can imagine you can do in the spatial domain and then transform it into the frequency domain.
- jm: Why would you want to do that?
- d: Because that way you don't have to worry about mode switching and all this other stuff. IT makes a bunch of stuff simpler.
- jm: If you get that to work, then you get paint to work.
- u: Didn't we talk about this with Greg in Orlando?
- d: I don't remember this discussion at all. I remember the Thai food, but not hte discussion. Go ask Greg if he remembers.
- jm: This would be even worse than paint. How do you disable it? Having no safe fallback... On top of that, if you code something, you're going to ring all over the place. If you ever code a DCT on top of the first step then it rings all over the place. In every piece that is non-photographic you can't code a dct on top of a basis function anyway. Attempting to code the whole thing with DCT and then using beefed up version of deringing to fix things up, ensuring the background for the text is flat, etc, you would already have wasted tons of bits on other things. I don't see a 2 pass version really working.
- d; Try to think of something that does work and in conjunction with motion compensation.
- jm: A separate codec that you switch block by block basis is the kind of thing I'm thinking of. In the desktop case, the motion will be flat. Either you're moving not at all or large areas at the same speed.
- j: With all the animations window managers have, that kidn of motion doesn't seem realistic.
- jm: The text is really the thing to focus on.
- d: The particular tool you use is not important. How it integrates into the codec is the hardest part of this problem.
</pre>

DaalaMeeting20140909

2014-09-16T12:56:35Z

Tomvanbraeckel: Add minutes from https://daala.etherpad.mozilla.org/weekly-meeting

# Meeting 2014-09-09

Mumble: mf4.xiph.org:64738

# Agenda

- reviews
- deringing
- intraprediction
- non-photographic video

# Attending
- unlord, tmatth, smarter, jack

# Reviews

https://review.xiph.org/455/
https://review.xiph.org/456/

# derining

- jm: I've continued on using paint as deringing. I have results that are slightly cheating but getting better. http://jmvalin.ca/video/dering/ What this is doing is computing a paint image and from the coded image figuring out (from variances and stuff) the gain that should apply to the derining. Ie, how much of the paint image we should use vs. how much of the decoded, pre-paint image we should use. It's doing a reasonable job but it's not perfect. On top of this I'm adding a single flag per block which disables the filter entirely for the block (gain of 0). I don't have crossfade, but I didn't notice much of an issue with blocking artifacts. I didn't generate images because I wasn't quite sure what the best thing to generate was.
- d: Howa bout Clovis?
- x: Good torture test.
- jm:
off: http://jmvalin.ca/video/dering/clovis0.png
on: http://jmvalin.ca/video/dering/clovis.png
We can see easily. It's at least helping on the edges of the dunes. The color edges are just chroma contrast so they don't show up. I'm not sure yet how to handle chroma. THere's many ways to handle it. One way is through CfL. Ie, not apply deringing to chroma at all, but just on luma.
- d: That means you have to delay chroma until a whole line of superblocks has beend one. That is specifically what the HW people didn't like about CfL. We have an advantage because of doing it in the frequency domain.
- jm: Actually more than that because of the filter. Two superblock rows.
- jm: Those are -v60. Output is 18k. This is much below what I normally used for testing.
- d: I notice it does a fairly good job on the big balloon in the center on the upper left edge. But on the right edge seems like it does nothing.
- jm: It can't do much for features that are horizontal or vertical because the 2d dct is already a directional basis and its artifacts will be aligned with the direction of the image. If you look just below the bulge where its 45 degrees you see a huge difference.
- d: It seems bad that can't do anything about horizontal or vertical.
- jm: The rining is worst on the diagonals usually.
- d: The big question is "how slow is this?"
- jm: The normal encoder would take 0.4 seconds, and deringing takes a little under 3s. The vast majority is spent in the direction search.
- j: 6x slower!
- d: To be clear, just for this feature he is using 150x the decoder budget!
- d: Does the operation scale linearly with blocksize?
- jm: 16x16 is at most 2x as complex as 8x8 but it depends on what directional resolution we want. So per-pixel it's the same cost. The number of ops per pixel is on the order of 50. Assuming we don't do any downscaling before we find the direction.
- d: That's not completely unreasonable so why is it 150x slower?
- jm: I'm doing optimal interpolation weights around all four edges etc. If you look at intrapaint.c there is a pixel_interp function and that takes most of the cost. You call this for every pixel and it uses square roots etc. Then numbers I gave you are too optimistic. I forgot computing the distortion. It may be closer to 100 or 150.
- d: ekr is trying to hook us up with someone who's done speed optimizations for opensll, etc. Would it make sense for you to talk to him for trying to make this function super fast?
- jm: Yeah. Unlike openssl, I'm pretty flexible on the outputs.
- d: If all you want is a direction, why not just take an edge detection algoirthm and run an arctan?
- jm: I'm not a video guy so I've never run an edge detector. Actually that wouldn't work. I am not necesarily looking at edges, i'm looking at stuff like low variance in one direction and low variance in another. Like the tail of the horse.
- d: I'm talking about running a gradient operator. If one direction is dominant you should be able to find it. At lower rate you need a larger support for your gradient operator. We need to figure out if it is possible to make it sufficiently fast. Are you interested Monty?
- x: I think it is doomed. I'm not convinced it is generally useful. It does give a pleasing effect at low bitrates. I dont' see what it buys Daala; it's not codec specific.
- jm: Right now it's about the only thing we have to deal with ringing since we don't have intra.
- d: Monty to be clear, this is not a post processor. It's in-loop.
- x: Oh, I missed that. This is interesting, but we have other problems to ship a codec and I don't see how this solves our problems.
- d: Right now directional edges and features are not captured well since we haven't figured out intra.
- x: But we've also failed to make deringing from other codecs work. Figuring that out seems like a much more interesting problem.
- jm: I don't think the 265 stuff can get rid of the ringing we get at low bitrates. It just changes individual samples.
- x: Won't do that in 265 either right?
- jm: No, but 265 has intra prediction!
- j: Sounds like you want to spend more time on intra?
- x: Yes.
- d: Nathan is working on this. Sounds like you should move on to the next thing in your pipeline.

# intraprediction

- u: I have the still image test bed and it produces RD curves now. I'll post some of the curves. I don't think that AC coeefs from neighbors is a reasonable approach. It requires too large of support.
- d: Especially for shallow angles.
- u: At 45 you can go one block away, but at others you ahve to go 2 away. I'll post curves later today. I can't point you at the code Monty. It's super simple.

# non-photographic video

- jm: I think this is very useful and we should start planning for it.
- d; I was not going to worry about it until next year.
- jm: This has the potential to be a killer feature.
- x: That all by itself would get RedHat's attention.
- j: I brought htis up as a differentiator but Tim didn't seem super supportive.
- x: It's hard.
- d: I don't htink it's that hard.
- jm: There is a big piece that is missing from what I did for paint. Being able to encode antialiased text.
- d: Anything you can imagine you can do in the spatial domain and then transform it into the frequency domain.
- jm: Why would you want to do that?
- d: Because that way you don't have to worry about mode switching and all this other stuff. IT makes a bunch of stuff simpler.
- jm: If you get that to work, then you get paint to work.
- u: Didn't we talk about this with Greg in Orlando?
- d: I don't remember this discussion at all. I remember the Thai food, but not hte discussion. Go ask Greg if he remembers.
- jm: This would be even worse than paint. How do you disable it? Having no safe fallback... On top of that, if you code something, you're going to ring all over the place. If you ever code a DCT on top of the first step then it rings all over the place. In every piece that is non-photographic you can't code a dct on top of a basis function anyway. Attempting to code the whole thing with DCT and then using beefed up version of deringing to fix things up, ensuring the background for the text is flat, etc, you would already have wasted tons of bits on other things. I don't see a 2 pass version really working.
- d; Try to think of something that does work and in conjunction with motion compensation.
- jm: A separate codec that you switch block by block basis is the kind of thing I'm thinking of. In the desktop case, the motion will be flat. Either you're moving not at all or large areas at the same speed.
- j: With all the animations window managers have, that kidn of motion doesn't seem realistic.
- jm: The text is really the thing to focus on.
- d: The particular tool you use is not important. How it integrates into the codec is the hardest part of this problem.