TDLT: Difference between revisions

From XiphWiki
Jump to navigation Jump to search
Line 18: Line 18:


CG = 8.60603
CG = 8.60603
Optimal integer-valued coefficients (d=6) for V with no linear ramp:
p0 = -11/64 =
Optimal integer-valued coefficients (d=6) for V:
p0 = -12/64 = -0.1875
q0 = 41/64 = 0.640625
CG = 8.60486
Optimal integer-valued coefficients (d=6) were (1-p0*(1-q0)) is divisible by 3:
p0 = -13/64 = -0.203125
q0 = 41/64 = 0.640625
CG = 8.60446


{|
{|
Line 57: Line 37:
| -12/64<br>-0.1875
| -12/64<br>-0.1875
| 41/64<br>0.640625
| 41/64<br>0.640625
| &nbsp;<br>1.4375
| 92/64<br>1.4375
| &nbsp;<br>1.423177
| 1093/768<br>1.423177
| &nbsp;<br>8.60486
| &nbsp;<br>8.60486
|-
|-

Revision as of 08:39, 22 May 2012

This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data here).

The coding gain objective used as the objective is taken from slide 13 of Tim's presentation An Introduction to Video Coding

<need figure with block matrix diagrams>

The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].

Given p's and q's and assuming a linear ramp constrains the s's.

4x8

Optimal real-valued coefficients for V:

p0 = -0.18117338915051454

q0 = 0.6331818230771687

CG = 8.60603

p0 q0 s0 s1 CG
R=f -11/64
-0.171875
36/64
0.5625
91/64
1.421875
85/64
1.328125
 
8.63473
R=t,D=f -12/64
-0.1875
41/64
0.640625
92/64
1.4375
1093/768
1.423177
 
8.60486
R=t,D=t -16/64
-0.25
41/64
0.640625
92/64
1.4375
93/64
1.453125
 
8.59886

8x16

Optimal real-valued coefficients for V:

p0 = -0.39460731547057293

p1 = -0.33002212811740816

p2 = -0.12391270981321137

q0 = 0.822154737511288

q1 = 0.632488694485779

q2 = 0.40214668677553894

CG = 9.56867

Optimal [maybe] integer-valued coefficients (d=6) for V:

p0 = -26/64 = -0.40625

p1 = -22/64 = -0.34375

p2 = -8/64 = -0.125

q0 = 53/64 = 0.828125

q1 = 41/64 = 0.640625

q2 = 26/64 = 0.40625

9.56627