DaalaMeeting20140527

# Meeting 2014-05-27

Mumble:  mf4.xiph.org:64738

# Agenda

- reviews
- work week & code party
  - https://daala.etherpad.mozilla.org/workweek-201406
  - https://daala.etherpad.mozilla.org/coding-party-201408
- Standard benchmark?

# Attending

derf, gmaxwell, jack, jmspeex, unlord

# reviews

# work week & code party

add agenda if you see anything missing

# standard benchmark

- jm: it seems like we want an inter benchmark that we can test against.
- g: it's harder for inter, because 50 videos to get coverage is a lot of data.
- jm: i'm aware of this. we'll have to experiment.
- d: there are two tasks here. one is to find a test to evaluate the residual performance, and the other is a collection of videos that we all agree to test on
- j: was jm talking about the latter?
- d: i think so
- jm: i don't know exactly what i want or need, but i watn to know whether my code is good or not good. i suspect we'll want to do things like caching MVs just to make it go faster. can we just take two frames from all kinds of different videos and cache the MVs etc?
- g: the search uses information from multiple frames. we can do that if we're not benchmarking the actual search.
- d: what you can do depends on what you are trying to test.
- jm: maybe we need more than one of these benchmarks. but we certainly need at least one?
- j: can we just pick one to start with?
- jm: we've had problems with that in the past
- u: what does 265 do?
- d: no idea. the 265 model was pretty terrible?
- jm: was it 264 that was only tested on QCIF?
- d: yeah, and they sliced the large videos into tiny slices
- u: so this is about our ability to find MVs and code them?
- d: the current code does a massive amount of RDO in its motion compensation
- j: the MV search is RDOed
- jm: where does the R come from?
- d: it makes something up.
- jm: i did a quick test and a lot of the bitrate goes to MVs. is that normal?
- d: usually about 50% should go to MVs.
- jm: I was seeing 20kB per frame on parkjoy 780p.
- d: what numbers are you comparing?
- jm: 50-60kB depending on the `-v` setting.
- d: that's normal
- jm: that seems high considering it's mostly static motion. another thing i was thinking about is won't we have a problem because the points in the motion vecture signaling are not the same as the actual blocks we're using?
- d: yes, that's an issue, but it's not clear how big of one it is. we can always not code the MVs and use prediction to fill them in.
- jm: but we've wasted actually coding the MVs. you code MVs for the entire block and then some MVs you end up barely using at all because you intra-code that block.
- d: my point was that you don't really have to code them. you could use prediction to fill them in. you need something there to produce a motion compensated reference so you can blend it, but you don't have to spend a lot of bits coding it. this winds up being a pretty hard problem.
- jm: because of hte overlap it still slightly affects something else and you wouldn't be able to do stupid things like not coding at all.
- d: you can not code it at all as long as you predict to fill it in. if i know a partition is going to be intra, then i can say i don't code the MV that's here or i can say that the prob of not coding anything different is 99.99% whatever.
- jm: assuming you want to code intra/inter flags before MV search
- d: you don't have to code the intra/inter flags before the search. you may wind up coding them in the bitstream before MV info. the way it is set up now, deleting MVs from the MV grid is a basic operation. this is how the RD search works. you can go back at any point and delete a bunch of stuff. i agree it is a hard problem and it's not clear what the best thing to do is there.
- jm: back to benchmarks, maybe we want a separate benchmark for every aspect. we want some way to have good measurements. otherwise we end up going in teh wrong directions.
- g: the frame pairs idea was very interesting
- j: can we choose frames that are minimally correlated
- d: it's not clear how to measure that.
- j: do we just pick a random frame from a number of videos?
- jm: i dont' think we should pick more than one pair from each
- g: the other problem is that you're coding against a previously decoded inter frame, not a previously decoded intra frame.
- jm: we could have triplets.
- j: pick 4 frames, an intra frame, the next frame, and then the two frames before the next intra frame (last inter frames). that tests both cases
- d: basically impossible to do ???. need a better name for this.
- jm: how about perceptual VQ instead of pspherical VQ. or the other one, geodesic prediction encoding. geodesic update encoding, or GPUs!
- d: the best person to build this test framework is greg.
- g: yep
- d: can you work with jm to figure out what will meet his needs?
- g: i will make something good happen
- jm: my needs are simple. no more than 10 minutes running and get some kind of curve out of it. taking chroma into account for chroma would be nice, perhaps even in the intra case. the problem i'm seeing is that sometimes you make changes that indirectly affect Chroma vs. luma balance, and it biases your results. if you give more bits to luma, then you will see an improvement even if it's not a real improvement, and just starving chroma.
- g: our metrics collect info for chroma, and the other thing is to only output a rate number that only counts luma bits.
- d: you can't really. particularly for motion comp, it predicts all the planes from one MV. which bits count toward luma and not chroma?
- g: you can turn off chroma in motion search too
- d: you think that makes this better?
- g: it prevents the bias.
- jm: not looking for something perfect, but it would b enice to realize when you're gain is from starving chroma
- d: we already have bits metrics that spit out the rate by plane.
- j: i think we can get this right now from that metrics stuff.
- jm: you think this is better than weighting in some chroma metrics?
- d: we don't know how to do that. so this may not be better but it's easier to figure out. the other option is to spend some time trying to figure out a good way to weight these things appropriately.
- jm: are there chroma artifacts in our DB?
- d: you're talking about TID 2008?
- jm: yeah.
- d: i don't remember chroma being involved there. none of those metrics don't look at colors.
- g: most of the images are color, although i should check their paper that they didn't convert to b&w.
- j: we talked about cross checking RGBSSIM against TID before with josh
- u: i convinced him to only use luma now
- d: we should look at the psylab stuff that cbloom did. that's where i would start
- jm: could we add a fifth metric that uses the optimal weight of PSNR, HVSM, SSIM, etc.
- d: we could run them through a neural net. not joking. we don't have a hope of understanding this.
- jm: NN means that you are out of what you trained, you can have really silly things oging on. but if you have linear weights and force them to be not be less than zero, then at least have a good chance that the outputs are better than the inputs.
- jm: do we have code to do motion search on the uncompressed one?\
- d: we don't keep uncompressed images around, but it would be easy to make that change.
- jm: seems to me that this is what we want to do right?
- d: that's what we did for theora. the reason is that because theora doesn't have delta coding for MV, once you code a MV you will use it for subsequent blocks. Daala doesn't have that problem.
- jm: in general, when you have a certain motion you want the artifacts to move at the same speed.
- d: yes. we may wind up with a different effect, a different rationale for wanting to do this. because we injected all this noise into the video, we may need to follow true motion to match the artifacts.
- g: even in theora this was only a win for coarse MV, not subpixel stuff.
- d: subpixel is a mess in the current code and will probably all have to change anyway.
- jm: i was reading a bit on HEVC and they had one feature listed i had no idea about. sample adaptive offsets...
- d: that's the deringing filter.

# asm.js daala

- g: i didn't find anything yet. i noticed that mik was talking about this on IRC and haven't caught up with that.
- d: i convinced him to start integrating it. he ran into the problem that we require libpng and libjpeg as dependencies.
- u: can we take out that stuff?
- d: we have --disable-player, but the tools can't be disabled yet
- j: this is fixable by some easy configure hacking.

# misc

- jm: what would be the normal way to code the skip flags? right now we dont' have any skip.
- d: i don't know what you mean by normal. generally this is baked into macro-block-mode in generations prior to HEVC. but even h.264 has several levels of skip.
- jm: skip this macro skip this block etc?
- d: exactly. code the DC skip all AC.
- jm: is that useful?
- d: yes. 90% of chroma.
- jm: good to know especially as i have DC factored out now. how do you search skip flags?
- d: generally you want to test with exhaustive search and at the end you start adding heuristics to make it not hurt too badly. in theora we got 15% speedup with negligible PSNR loss after some tuning.
- jm: i hate exhaustive search.
- d: it's not that much slower and you need it as a reference to compare against.