TDLT: Difference between revisions
mNo edit summary |
(→4x8) |
||
| Line 10: | Line 10: | ||
== 4x8 == | == 4x8 == | ||
Optimal real-valued coefficients for V: | Optimal real-valued coefficients for V: | ||
| Line 21: | Line 18: | ||
CG = 8.60603 | CG = 8.60603 | ||
Optimal integer-valued coefficients (d=6) for V with no linear ramp: | |||
p0 = -11/64 = | |||
Optimal integer-valued coefficients (d=6) for V: | Optimal integer-valued coefficients (d=6) for V: | ||
| Line 37: | Line 38: | ||
CG = 8.60446 | CG = 8.60446 | ||
{| | |||
! | |||
!p0 | |||
!q0 | |||
!s0 | |||
!s1 | |||
!CG | |||
|- | |||
|R=f | |||
| -11/64<br>-0.171875 | |||
| 36/64<br>0.5625 | |||
| 91/64<br>1.421875 | |||
| 85/64<br>1.328125 | |||
| <br>8.63473 | |||
|- | |||
|R=t,D=f | |||
| -12/64<br>-0.1875 | |||
| 41/64<br>0.640625 | |||
| <br>1.4375 | |||
| <br>1.423177 | |||
| <br>8.60486 | |||
|- | |||
|R=t,D=t | |||
| -16/64<br>-0.25 | |||
| 41/64<br>0.640625 | |||
| 92/64<br>1.4375 | |||
| 93/64<br>1.453125 | |||
| <br>8.59886 | |||
|} | |||
== 8x16 == | == 8x16 == | ||
Revision as of 07:21, 22 May 2012
This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data here).
The coding gain objective used as the objective is taken from slide 13 of Tim's presentation An Introduction to Video Coding
<need figure with block matrix diagrams>
The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].
Given p's and q's and assuming a linear ramp constrains the s's.
4x8
Optimal real-valued coefficients for V:
p0 = -0.18117338915051454
q0 = 0.6331818230771687
CG = 8.60603
Optimal integer-valued coefficients (d=6) for V with no linear ramp:
p0 = -11/64 =
Optimal integer-valued coefficients (d=6) for V:
p0 = -12/64 = -0.1875
q0 = 41/64 = 0.640625
CG = 8.60486
Optimal integer-valued coefficients (d=6) were (1-p0*(1-q0)) is divisible by 3:
p0 = -13/64 = -0.203125
q0 = 41/64 = 0.640625
CG = 8.60446
| p0 | q0 | s0 | s1 | CG | |
|---|---|---|---|---|---|
| R=f | -11/64 -0.171875 |
36/64 0.5625 |
91/64 1.421875 |
85/64 1.328125 |
8.63473 |
| R=t,D=f | -12/64 -0.1875 |
41/64 0.640625 |
1.4375 |
1.423177 |
8.60486 |
| R=t,D=t | -16/64 -0.25 |
41/64 0.640625 |
92/64 1.4375 |
93/64 1.453125 |
8.59886 |
8x16
Optimal real-valued coefficients for V:
p0 = -0.39460731547057293
p1 = -0.33002212811740816
p2 = -0.12391270981321137
q0 = 0.822154737511288
q1 = 0.632488694485779
q2 = 0.40214668677553894
CG = 9.56867
Optimal [maybe] integer-valued coefficients (d=6) for V:
p0 = -26/64 = -0.40625
p1 = -22/64 = -0.34375
p2 = -8/64 = -0.125
q0 = 53/64 = 0.828125
q1 = 41/64 = 0.640625
q2 = 26/64 = 0.40625
9.56627