TDLT
This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data here).
The coding gain objective used as the objective is taken from slide 13 of Tim's presentation An Introduction to Video Coding
<need figure with block matrix diagrams>
The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [1,1].
Given p's and q's and assuming a linear ramp constrains the s's.
4x8
Optimal realvalued coefficients for V:
p0 = 0.18117338915051454
q0 = 0.6331818230771687
CG = 8.60603
p0  q0  s0  s1  CG  SBA  Filterbank  

R=f 6bit 
11/64 0.171875 
36/64 0.5625 
91/64 1.421875 
85/64 1.328125 
8.63473 
22.0331 

R=f 5bit 
5/32 0.15625 
18/32 0.5625 
46/32 1.4375 
42/32 1.3125 
8.63409 
22.5715 
64px 
R=t,D=f  12/64 0.1875 
41/64 0.640625 
92/64 1.4375 
1093/768 1.423177 
8.60486 
20.0573 

R=t,D=t max CG 
16/64 0.25 
41/64 0.640625 
92/64 1.4375 
93/64 1.453125 
8.59886 
18.9411 

R=t,D=t 5bit 
8/32 0.25 
19/32 0.59375 
52/32 1.625 
47/32 1.46875 
8.56068 
20.3279 
64px 
R=t,D=t max SBA 
8/64 0.125 
30/64 0.46875 
136/64 2.125 
91/64 1.421875 
8.23230 
25.1934 
64px 
8x16
Optimal realvalued coefficients for V:
p0 = 0.39460731547057293
p1 = 0.33002212811740816
p2 = 0.12391270981321137
q0 = 0.822154737511288
q1 = 0.632488694485779
q2 = 0.40214668677553894
CG = 9.56867
16x32
Bestknown realvalued coefficients for V (R=t):
p0 = 0.42111473798940136
p1 = 0.4121736499899753
p2 = 0.3350240707669929
p3 = 0.3224547931861314
p4 = 0.25883387978005545
p5 = 0.20951913473498104
p6 = 0.0598657149803332
q0 = 0.9107782439906195
q1 = 0.8109855829278226
q2 = 0.715846584586721
q3 = 0.6135951570714172
q4 = 0.49846644853347627
q5 = 0.3945215834922529
q6 = 0.21822275136248082
CG = 9.81157
TypeIV Coding Gain
4x8  4x8 Ramp  8x16  8x16 Ramp  16x32  16x32 Ramp  

Real Valued  8.6349  8.60603  9.6005  9.56867  9.9057  9.81157 
Dyadic (6bit)  8.63473  8.59886  9.60021  9.56161  9.89338  9.78294 
Loss  0.00017  0.00717  0.00029  0.00706  0.01232  0.02863 
Dyadic (5bit)  8.63409  8.56068  
Loss  0.00081  0.04535 
8x16 TypeIII
p0  p1  p2  q0  q1  q2  s0  s1  s2  s3  CG  Filterbank  

R=f  25/64 0.390625 
20/64 0.3125 
7/64 0.109375 
49/64 0.765625 
35/64 0.546875 
21/64 0.328125 
90/64 1.40625 
72/64 1.125 
73/64 1.140625 
76/64 1.1875 
9.6112 
TypeIII Coding Gain
4x8  8x16  16x32  

Real Valued  8.6349  9.6115  9.9496 
Approximate  8.63473  9.6112  
Loss  0.00017  0.00030 