https://wiki.xiph.org/api.php?action=feedcontributions&feedformat=atom&user=Unlord 2026-05-15T07:24:02Z User contributions MediaWiki 1.45.1 https://wiki.xiph.org/index.php?title=Summer_of_Code_2021&diff=16746 2021-03-03T00:43:53Z

<p>Unlord: </p> <hr /> <div>== Introduction ==<br /> <br /> This year Xiph.org is focusing on the rav1e AV1 encoder for its GSoC participation. Both video and still images are currently hot topics, especially with the recent support of AVIF within browsers.<br /> <br /> Below you'll find the description for the following GSoC project ideas around the rav1e project.<br /> <br /> If you want to know more about a particular idea, please get in touch with the people listed under &quot;possible mentors&quot;. While no guarantee, that the person will be the actual mentor for the task, they know it and will be happy to answer your questions.<br /> ----<br /> <br /> == Detailed Project Descriptions ==<br /> <br /> These ideas were suggested by various members of the developer community as projects that would be beneficial and which we feel we can mentor. Students should feel free to select one of these, develop a variation, or propose their own ideas. Here, ideally.<br /> <br /> ----<br /> <br /> === Grain synthesis implementation inside of the rav1e encoder ===<br /> <br /> Grain synthesis is using the idea of modeling noise temporally and spatially using noise estimation.<br /> <br /> ==== Problem / Intro ====<br /> <br /> Keeping high frequency detail and noise(dithering, camera noise, grain) using traditional encoder techniques is very expensive in terms of bitrate allocation, and some tools implemented to take care of that problem can create additional artifacts that are not pleasing to the general viewer experience, or are detrimental to the fidelity of the image.<br /> <br /> ==== Solution / Task ====<br /> <br /> Implementing grain synthesis that models the noise parameters of a video, and applies the generated noise parameters during the decoding process, saving very high amounts of bitrates and providing a very high subjective visual fidelity and appeal.<br /> <br /> Making it faster than other forms of grain synthesis via smarter algorithms and using various forms of threading to speed up its application, such as tile threading and integration with rav1e-by-gop, making it possible to use as part of any encoding workflow. This will make sure adoption of the technique becomes as widespread as possible.<br /> <br /> ==== Requirements ====<br /> <br /> The student should be familiar with Rust and C, and must have a light background in general visual media encoding, such as video and image compression.<br /> Complexity: Medium.<br /> <br /> ==== Possible Mentors ====<br /> <br /> [[User:Lu_zero]] &amp;&amp; XX<br /> <br /> ----<br /> === Adaptive quantization ===<br /> <br /> Adaptive quantization is the process of an algorithm trying to efficiently allocate bitrate among the various macroblocks found in a frame by varying the quantizer across each of them according to different visual targets.<br /> <br /> ==== Problem / Intro ====<br /> <br /> Often times, an encoder does not know about the best way to allocate the bitrate budget across a frame, and may overspend a considerable amount of bitrate to regions that might not benefit from a low quantizer(low amounts of distortion, so less compression) while not giving enough bitrate to zones that might actually need it. This can even cause issues temporally, as bitrate allocation within a group of frames(GOPs) may be skewed towards more complex and high motion frames, while leaving other frames with barely any bitrate to work with, creating visual artifacts such as blocking and banding.<br /> <br /> ==== Solution / Task ====<br /> <br /> Implementing 1-3 forms on adaptive quantization either based on variance, complexity and/or variance variance with a bias in low contrast frames in rav1e. This will make bitrate allocation more efficient, and avoid bitrate overspending in areas which either need the lower quantizer to avoid the presence of lower quality frames that might detract from the viewer experience, and make objective/subjective quality targets easier to achieved. The combination of powerful adaptive quantization and grain synthesis would allow for a higher subjective quality viewing experience at lower bitrates while potentially lowering computational complexity by a good margin.<br /> <br /> Making a smart adaptive AQ mode in which the encoder chooses which adaptive quantization algorithm to use depending on the scene featured in a GOP. Potentially difficult.<br /> <br /> ==== Requirements ====<br /> <br /> The student should be familiar with Rust and C, and having a background in general visual media encoding, such as video and image compression is recommended.<br /> <br /> Difficulty: Medium depending on which targets the student chooses to follow<br /> <br /> ==== Possible Mentors ====<br /> <br /> [[User:Lu_zero]] &amp;&amp; XX<br /> <br /> ----<br /> === Integrate rav1e-by-gop into rav1e encoder ===<br /> <br /> rav1e-by-gop is an extended command line encoder that provides additional encoding strategies such as by-gop parallel encoding across multiple machines.<br /> <br /> ==== Problem / Intro ====<br /> <br /> rav1e-by-gop is currently a command line program, some users might want to enjoy on the extended features from other programs even if they do not always belong to an encoder.<br /> <br /> ==== Solution / Task ====<br /> <br /> make rav1e-by-gop expose the same API of the normal rav1e to make easy to use the multiple machine encoding features from other programs.<br /> <br /> ==== Requirements ====<br /> <br /> The student should be familiar with Rust or C programming.&lt;br&gt;<br /> Video knowledge is not strictly necessary, however a basic understanding of the concepts is vastly beneficial.<br /> <br /> ==== Possible Mentors ====<br /> <br /> [[User:Tdaede]] [[User:Lu_zero]]<br /> <br /> ----<br /> === Visual metric targeting in rav1e-by-gop ===<br /> <br /> Objective metrics are used to evaluate an encoder's performance in a diverse set of scenarios. Different metrics such as PSNR, SSIM, DSSIM, VMAF and some closed no-reference metrics are used in the field to record encoder performance changes across versions trying to correlate closely with human perception.<br /> <br /> ==== Problem / Intro ====<br /> <br /> Classical methods of rate control such as ABR(Average BitRate), fixed quantizers and even CRF(Constant Rate Factor) have the issue of not targeting a certain quality level. This can result in starved encodes where the bitrate budget has to be kept low to stay watchable by the viewer without interruption, leading to scenes that have exceptionally good visual targets by overspending bitrate, and scenes that have very poor visual appeal by having too little bitrate, detracting from the viewer experience entirely. More advanced forms of rate control like CRF help somewhat, but they still have the issue of having to overshoot so the lower quality scenes do not suffer, and do not adapt to the different type of content encoded, resulting in variable quality encodes.<br /> <br /> ==== Solution / Task ====<br /> <br /> Implementing visual metric targeting based on VMAF(mainly used for video) and butteraugli(mainly used for images) as part of rav1e-by-gop as a secondary rate control option. <br /> <br /> The application of visual metric targeting in rav1e-by-gop would take advantage of its adaptive keyframe placement and smart scene detection to its fullest. <br /> This would allow for the best rate control possible, as short scenes in the length of 1-15s are where visual metrics such as VMAF shine the most. The idea is to encode first with a very fast speed preset in the encoder to gauge the quality at a prefixed quantizer. If the visual metric target set is not achieved, the encoder tries again once or twice until it gets the right result. <br /> With this method, instead of targeting an average of bitrate, you would target a visual score, getting higher efficiency and higher subjective quality. This would also be advantageous in terms of encoding time spent, as encoder complexity could be dialed back while keeping overall efficiency the same or higher, with efficiency being a function of both encoder efficiency and rate control.<br /> <br /> Implementing butteraugli quality targeting as an option using rav1e for AVIF images. Since visual quality requirements are considerably higher for intra only(image only) media, keeping high visual fidelity is even more important than video compression. Quality targeting iterations would also be quite useful here.<br /> <br /> ==== Requirements ====<br /> <br /> The student should be familiar with Rust and C. General interest in image and video coding is recommended<br /> <br /> Difficulty: Low-medium.<br /> <br /> ==== Possible Mentors ====<br /> <br /> [[User:Lu_zero]] &amp;&amp; XX<br /> ----<br /> === Improved cluster support for Icecast ===<br /> Icecast servers deliver streams to million of users simultaneous worldwide. Each instance can handle many thousand clients at the same time. However redundancy, scalability, hardware requirement, and most importantly network connectivity often requires to use several instances in a professional deployment.<br /> ==== Problem / Intro ====<br /> Icecast is designed as a standalone application. While basic support exists (such as master-slave mode) support for clusters can be improved. At this point a cluster level mangement instance seems to be missing.<br /> ==== Solution / Task ====<br /> A solution for cluster management should be developed. A cluster controller should be implemented as well as support within Icecast. The focus is on Icecast itself at this point. However the controler should at least demonstrate all features implemented in Icecast.<br /> Possible features:<br /> * Automatic master-slave, and relay configuration.<br /> * Load distribution.<br /> * Statistic data collection.<br /> * Log collection.<br /> * Node monitoring.<br /> * Signalling of cluster state to external components (e.g. for automatic cluster scaling)<br /> ==== Requirements ====<br /> The student should be familiar with C. A basic understanding of HTTP, as well as other web technologies is helpful. Knowledge of visualisation technologies is '''not''' required.<br /> <br /> Complexity: Medium to high.<br /> ==== Possible Mentors ====<br /> phschafft (teamed with someone else)<br /> ----<br /> === Development of advanced content navigation in Icecast ===<br /> Icecast currently supports navigation of listeners between different streams. This was developed mostly for fallbacks (providing alternative content if the primary source fails). This support should be improved to provide better interaction with contents.<br /> ==== Problem / Intro ====<br /> The current implementation is designed to work very robust for source side events (such as fallbacks). However it fails for two requirements:<br /> * Listener initiated interaction such as adaptive streaming.<br /> * Exact timing.<br /> ==== Solution / Task ====<br /> The current concept is capable of being extended to the new requirements. Code should be written to add the new features on the existing (and well proven) infrastructure. The following additional features would be needed:<br /> Detection and matching of features within and between streams. This is required for any kind of synchronisation.<br /> * Executing operations exactly at detached features.<br /> * Adding ways to communicate features and operations between listener and Icecast, source and Icecast, and between multiple * * Icecast instances of the same cluster.<br /> ==== Requirements ====<br /> The student should be familiar with C. A basic understanding of HTTP, Ogg, and Matroska/WebM is helpful.<br /> <br /> Complexity: Medium to high.<br /> ==== Possible Mentors ====<br /> phschafft (teamed with someone else)<br /> ----<br /> === Uniform return channel for Icecast ===<br /> Icecast supports broadcasting media to several thousand listeners per instance. In a classic setup this is a one way process from the source (such as a radio or TV studio) to the consumer. However it is sometimes useful to provide a return channel, such as for implementing polls.<br /> ==== Problem / Intro ====<br /> Returning information from the listener to the source is part of classic media. The need has become more relevant with the development of more interactive ways of the web. Several technologies have been used to implement this including asking the listeners to call in, send e-mails, or comment on a web page.<br /> <br /> Classic ways to implement feedback include a media break and are only loosely bound to the forward channel.<br /> ==== Solution / Task ====<br /> A uniform return channel should be implemented that allows several types of data to be send from the listener to the source. This includes three major parts:<br /> * Improved session handling (both for listeners and for sources)<br /> * Implementing a return channel for listeners.<br /> * Implementing a return channel for sources.<br /> ==== Requirements ====<br /> The student should be familiar with C and HTTP.<br /> <br /> Complexity: Medium.<br /> ==== Possible Mentors ====<br /> phschafft (teamed with someone else)<br /> ----<br /> === (Live) listener statistics for Icecast ===<br /> Icecast supports writing a basic access.log that includes client information as well as connection time. In addition a playlist log is supported, and live statistic data via the STATS interface.<br /> <br /> A standard solution to use this data for detailed listener statistics is missing.<br /> ==== Problem / Intro ====<br /> In the current solution there is no off the shelf solution to process the statistic data provided by Icecast. The best available solutions are standard access log analysers. A solution for live statistics is completely missing. Statistics taking content into account is also absent.<br /> ==== Solution / Task ====<br /> To improve the situation two major steps must be accomplished:<br /> * The statistic interface of Icecast must be enhanced to provide the required information.<br /> * A solution that analyses this data must be developed. The focus is on this part.<br /> This project allows for a wide range of ideas from the participants to be incorporated. There is not yet and specific technical direction set. Evaluating different options is the first part of the project.<br /> ==== Requirements ====<br /> The student should be familiar with C. A basic understanding of log analysis, and monitoring and/or data collecting systems is helpful.<br /> <br /> Difficulty: Medium depending on which targets the student chooses to follow.<br /> ==== Possible Mentors ====<br /> phschafft (teamed with someone else)<br /> ----<br /> <br /> === Support WebAssembly SIMD in rav1e ===<br /> <br /> rav1e supports the [https://wasi.dev/ WASI] platform and it has its javascript API bindings relying on it.<br /> <br /> ==== Problem / Intro ====<br /> <br /> The WebAssembly SIMD is getting [https://github.com/WebAssembly/simd closer] to be available, we should support it.<br /> <br /> <br /> ==== Solution / Task ====<br /> <br /> * Implement the dispatch logic for WASM SIMD as done already for x86_64 and aarch64.<br /> * Implement the Sum of absolute difference (SAD) and Sum of absolute transformed differences (SATD)<br /> * Implement the inverse transforms (idct, iadst, identity, ...)<br /> * Implement the motion compensation.<br /> <br /> ==== Requirements ====<br /> <br /> The student should be familiar with Rust, WASM, wasmtime and related tools. Knowledge of x86 or arm assembly is not needed but will help.<br /> <br /> ==== Possible Mentors ====<br /> <br /> [[User:Lu_zero]]<br /> ----<br /> <br /> === Implement butteraugli in av-metrics ===<br /> <br /> av-metrics is a collection of video quality metrics, [https://github.com/google/butteraugli butteraugli] is a promising psychovisual similarity metric. <br /> <br /> ==== Problem / Intro ====<br /> <br /> Currently the implementation of butteraugli exists as [https://github.com/google/butteraugli stand-alone] codebase. The code is readable, but it could be faster. <br /> <br /> ==== Solution / Task ====<br /> <br /> * Implement rust bindings to the reference butteraugli.<br /> * Implement butteraugli in pure rust within av-metrics.<br /> * Write integration and unit tests to make sure the implementation does not diverge<br /> * Write criterion benchmarks<br /> * Implement x86_64 or aarch64 optimizations for it, using intrinsics or plain ASM.<br /> <br /> ==== Requirements ====<br /> <br /> The student should be familiar with Rust, C and C++. Knowledge of x86 or arm assembly is welcome.<br /> <br /> ==== Possible Mentors ====<br /> <br /> [[User:Lu_zero]]</div>

Unlord https://wiki.xiph.org/index.php?title=Summer_of_Code_2021&diff=16745 2021-03-03T00:39:44Z

<p>Unlord: </p> <hr /> <div>== Introduction ==<br /> <br /> This year Xiph.org is focusing on the rav1e AV1 encoder for its GSoC participation. Both video and still images are currently hot topics, especially with the recent support of AVIF within browsers.<br /> <br /> Below you'll find the description for the following GSoC project ideas around the rav1e project.<br /> <br /> If you want to know more about a particular idea, please get in touch with the people listed under &quot;possible mentors&quot;. While no guarantee, that the person will be the actual mentor for the task, they know it and will be happy to answer your questions.<br /> ----<br /> <br /> == Detailed Project Descriptions ==<br /> <br /> These ideas were suggested by various members of the developer community as projects that would be beneficial and which we feel we can mentor. Students should feel free to select one of these, develop a variation, or propose their own ideas. Here, ideally.<br /> <br /> ----<br /> === Support WebAssembly SIMD ===<br /> <br /> rav1e supports the [https://wasi.dev/ WASI] platform and it has its javascript API bindings relying on it.<br /> <br /> ==== Problem / Intro ====<br /> <br /> The WebAssembly SIMD is getting [https://github.com/WebAssembly/simd closer] to be available, we should support it.<br /> <br /> <br /> ==== Solution / Task ====<br /> <br /> * Implement the dispatch logic for WASM SIMD as done already for x86_64 and aarch64.<br /> * Implement the Sum of absolute difference (SAD) and Sum of absolute transformed differences (SATD)<br /> * Implement the inverse transforms (idct, iadst, identity, ...)<br /> * Implement the motion compensation.<br /> <br /> ==== Requirements ====<br /> <br /> The student should be familiar with Rust, WASM, wasmtime and related tools. Knowledge of x86 or arm assembly is not needed but will help.<br /> <br /> ==== Possible Mentors ====<br /> <br /> [[User:Lu_zero]]<br /> ----<br /> <br /> === Implement butteraugli in av-metrics ===<br /> <br /> av-metrics is a collection of video quality metrics, [https://github.com/google/butteraugli butteraugli] is a promising psychovisual similarity metric. <br /> <br /> ==== Problem / Intro ====<br /> <br /> Currently the implementation of butteraugli exists as [https://github.com/google/butteraugli stand-alone] codebase. The code is readable, but it could be faster. <br /> <br /> ==== Solution / Task ====<br /> <br /> * Implement rust bindings to the reference butteraugli.<br /> * Implement butteraugli in pure rust within av-metrics.<br /> * Write integration and unit tests to make sure the implementation does not diverge<br /> * Write criterion benchmarks<br /> * Implement x86_64 or aarch64 optimizations for it, using intrinsics or plain ASM.<br /> <br /> ==== Requirements ====<br /> <br /> The student should be familiar with Rust, C and C++. Knowledge of x86 or arm assembly is welcome.<br /> <br /> ==== Possible Mentors ====<br /> <br /> [[User:Lu_zero]]<br /> ----<br /> <br /> === Grain synthesis implementation inside of the rav1e encoder ===<br /> <br /> Grain synthesis is using the idea of modeling noise temporally and spatially using noise estimation.<br /> <br /> ==== Problem / Intro ====<br /> <br /> Keeping high frequency detail and noise(dithering, camera noise, grain) using traditional encoder techniques is very expensive in terms of bitrate allocation, and some tools implemented to take care of that problem can create additional artifacts that are not pleasing to the general viewer experience, or are detrimental to the fidelity of the image.<br /> <br /> ==== Solution / Task ====<br /> <br /> Implementing grain synthesis that models the noise parameters of a video, and applies the generated noise parameters during the decoding process, saving very high amounts of bitrates and providing a very high subjective visual fidelity and appeal.<br /> <br /> Making it faster than other forms of grain synthesis via smarter algorithms and using various forms of threading to speed up its application, such as tile threading and integration with rav1e-by-gop, making it possible to use as part of any encoding workflow. This will make sure adoption of the technique becomes as widespread as possible.<br /> <br /> ==== Requirements ====<br /> <br /> The student should be familiar with Rust and C, and must have a light background in general visual media encoding, such as video and image compression.<br /> Complexity: Medium.<br /> <br /> ==== Possible Mentors ====<br /> <br /> [[User:Lu_zero]] &amp;&amp; XX<br /> <br /> ----<br /> === Adaptive quantization ===<br /> <br /> Adaptive quantization is the process of an algorithm trying to efficiently allocate bitrate among the various macroblocks found in a frame by varying the quantizer across each of them according to different visual targets.<br /> <br /> ==== Problem / Intro ====<br /> <br /> Often times, an encoder does not know about the best way to allocate the bitrate budget across a frame, and may overspend a considerable amount of bitrate to regions that might not benefit from a low quantizer(low amounts of distortion, so less compression) while not giving enough bitrate to zones that might actually need it. This can even cause issues temporally, as bitrate allocation within a group of frames(GOPs) may be skewed towards more complex and high motion frames, while leaving other frames with barely any bitrate to work with, creating visual artifacts such as blocking and banding.<br /> <br /> ==== Solution / Task ====<br /> <br /> Implementing 1-3 forms on adaptive quantization either based on variance, complexity and/or variance variance with a bias in low contrast frames in rav1e. This will make bitrate allocation more efficient, and avoid bitrate overspending in areas which either need the lower quantizer to avoid the presence of lower quality frames that might detract from the viewer experience, and make objective/subjective quality targets easier to achieved. The combination of powerful adaptive quantization and grain synthesis would allow for a higher subjective quality viewing experience at lower bitrates while potentially lowering computational complexity by a good margin.<br /> <br /> Making a smart adaptive AQ mode in which the encoder chooses which adaptive quantization algorithm to use depending on the scene featured in a GOP. Potentially difficult.<br /> <br /> ==== Requirements ====<br /> <br /> The student should be familiar with Rust and C, and having a background in general visual media encoding, such as video and image compression is recommended.<br /> <br /> Difficulty: Medium depending on which targets the student chooses to follow<br /> <br /> ==== Possible Mentors ====<br /> <br /> [[User:Lu_zero]] &amp;&amp; XX<br /> <br /> ----<br /> === Integrate rav1e-by-gop into rav1e encoder ===<br /> <br /> rav1e-by-gop is an extended command line encoder that provides additional encoding strategies such as by-gop parallel encoding across multiple machines.<br /> <br /> ==== Problem / Intro ====<br /> <br /> rav1e-by-gop is currently a command line program, some users might want to enjoy on the extended features from other programs even if they do not always belong to an encoder.<br /> <br /> ==== Solution / Task ====<br /> <br /> make rav1e-by-gop expose the same API of the normal rav1e to make easy to use the multiple machine encoding features from other programs.<br /> <br /> ==== Requirements ====<br /> <br /> The student should be familiar with Rust or C programming.&lt;br&gt;<br /> Video knowledge is not strictly necessary, however a basic understanding of the concepts is vastly beneficial.<br /> <br /> ==== Possible Mentors ====<br /> <br /> [[User:Tdaede]] [[User:Lu_zero]]<br /> <br /> ----<br /> === Visual metric targeting in rav1e-by-gop ===<br /> <br /> Objective metrics are used to evaluate an encoder's performance in a diverse set of scenarios. Different metrics such as PSNR, SSIM, DSSIM, VMAF and some closed no-reference metrics are used in the field to record encoder performance changes across versions trying to correlate closely with human perception.<br /> <br /> ==== Problem / Intro ====<br /> <br /> Classical methods of rate control such as ABR(Average BitRate), fixed quantizers and even CRF(Constant Rate Factor) have the issue of not targeting a certain quality level. This can result in starved encodes where the bitrate budget has to be kept low to stay watchable by the viewer without interruption, leading to scenes that have exceptionally good visual targets by overspending bitrate, and scenes that have very poor visual appeal by having too little bitrate, detracting from the viewer experience entirely. More advanced forms of rate control like CRF help somewhat, but they still have the issue of having to overshoot so the lower quality scenes do not suffer, and do not adapt to the different type of content encoded, resulting in variable quality encodes.<br /> <br /> ==== Solution / Task ====<br /> <br /> Implementing visual metric targeting based on VMAF(mainly used for video) and butteraugli(mainly used for images) as part of rav1e-by-gop as a secondary rate control option. <br /> <br /> The application of visual metric targeting in rav1e-by-gop would take advantage of its adaptive keyframe placement and smart scene detection to its fullest. <br /> This would allow for the best rate control possible, as short scenes in the length of 1-15s are where visual metrics such as VMAF shine the most. The idea is to encode first with a very fast speed preset in the encoder to gauge the quality at a prefixed quantizer. If the visual metric target set is not achieved, the encoder tries again once or twice until it gets the right result. <br /> With this method, instead of targeting an average of bitrate, you would target a visual score, getting higher efficiency and higher subjective quality. This would also be advantageous in terms of encoding time spent, as encoder complexity could be dialed back while keeping overall efficiency the same or higher, with efficiency being a function of both encoder efficiency and rate control.<br /> <br /> Implementing butteraugli quality targeting as an option using rav1e for AVIF images. Since visual quality requirements are considerably higher for intra only(image only) media, keeping high visual fidelity is even more important than video compression. Quality targeting iterations would also be quite useful here.<br /> <br /> ==== Requirements ====<br /> <br /> The student should be familiar with Rust and C. General interest in image and video coding is recommended<br /> <br /> Difficulty: Low-medium.<br /> <br /> ==== Possible Mentors ====<br /> <br /> [[User:Lu_zero]] &amp;&amp; XX<br /> ----<br /> === Improved cluster support for Icecast ===<br /> Icecast servers deliver streams to million of users simultaneous worldwide. Each instance can handle many thousand clients at the same time. However redundancy, scalability, hardware requirement, and most importantly network connectivity often requires to use several instances in a professional deployment.<br /> ==== Problem / Intro ====<br /> Icecast is designed as a standalone application. While basic support exists (such as master-slave mode) support for clusters can be improved. At this point a cluster level mangement instance seems to be missing.<br /> ==== Solution / Task ====<br /> A solution for cluster management should be developed. A cluster controller should be implemented as well as support within Icecast. The focus is on Icecast itself at this point. However the controler should at least demonstrate all features implemented in Icecast.<br /> Possible features:<br /> * Automatic master-slave, and relay configuration.<br /> * Load distribution.<br /> * Statistic data collection.<br /> * Log collection.<br /> * Node monitoring.<br /> * Signalling of cluster state to external components (e.g. for automatic cluster scaling)<br /> ==== Requirements ====<br /> The student should be familiar with C. A basic understanding of HTTP, as well as other web technologies is helpful. Knowledge of visualisation technologies is '''not''' required.<br /> <br /> Complexity: Medium to high.<br /> ==== Possible Mentors ====<br /> phschafft (teamed with someone else)<br /> ----<br /> === Development of advanced content navigation in Icecast ===<br /> Icecast currently supports navigation of listeners between different streams. This was developed mostly for fallbacks (providing alternative content if the primary source fails). This support should be improved to provide better interaction with contents.<br /> ==== Problem / Intro ====<br /> The current implementation is designed to work very robust for source side events (such as fallbacks). However it fails for two requirements:<br /> * Listener initiated interaction such as adaptive streaming.<br /> * Exact timing.<br /> ==== Solution / Task ====<br /> The current concept is capable of being extended to the new requirements. Code should be written to add the new features on the existing (and well proven) infrastructure. The following additional features would be needed:<br /> Detection and matching of features within and between streams. This is required for any kind of synchronisation.<br /> * Executing operations exactly at detached features.<br /> * Adding ways to communicate features and operations between listener and Icecast, source and Icecast, and between multiple * * Icecast instances of the same cluster.<br /> ==== Requirements ====<br /> The student should be familiar with C. A basic understanding of HTTP, Ogg, and Matroska/WebM is helpful.<br /> <br /> Complexity: Medium to high.<br /> ==== Possible Mentors ====<br /> phschafft (teamed with someone else)<br /> ----<br /> === Uniform return channel for Icecast ===<br /> Icecast supports broadcasting media to several thousand listeners per instance. In a classic setup this is a one way process from the source (such as a radio or TV studio) to the consumer. However it is sometimes useful to provide a return channel, such as for implementing polls.<br /> ==== Problem / Intro ====<br /> Returning information from the listener to the source is part of classic media. The need has become more relevant with the development of more interactive ways of the web. Several technologies have been used to implement this including asking the listeners to call in, send e-mails, or comment on a web page.<br /> <br /> Classic ways to implement feedback include a media break and are only loosely bound to the forward channel.<br /> ==== Solution / Task ====<br /> A uniform return channel should be implemented that allows several types of data to be send from the listener to the source. This includes three major parts:<br /> * Improved session handling (both for listeners and for sources)<br /> * Implementing a return channel for listeners.<br /> * Implementing a return channel for sources.<br /> ==== Requirements ====<br /> The student should be familiar with C and HTTP.<br /> <br /> Complexity: Medium.<br /> ==== Possible Mentors ====<br /> phschafft (teamed with someone else)<br /> ----<br /> === (Live) listener statistics for Icecast ===<br /> Icecast supports writing a basic access.log that includes client information as well as connection time. In addition a playlist log is supported, and live statistic data via the STATS interface.<br /> <br /> A standard solution to use this data for detailed listener statistics is missing.<br /> ==== Problem / Intro ====<br /> In the current solution there is no off the shelf solution to process the statistic data provided by Icecast. The best available solutions are standard access log analysers. A solution for live statistics is completely missing. Statistics taking content into account is also absent.<br /> ==== Solution / Task ====<br /> To improve the situation two major steps must be accomplished:<br /> * The statistic interface of Icecast must be enhanced to provide the required information.<br /> * A solution that analyses this data must be developed. The focus is on this part.<br /> This project allows for a wide range of ideas from the participants to be incorporated. There is not yet and specific technical direction set. Evaluating different options is the first part of the project.<br /> ==== Requirements ====<br /> The student should be familiar with C. A basic understanding of log analysis, and monitoring and/or data collecting systems is helpful.<br /> <br /> Difficulty: Medium depending on which targets the student chooses to follow.<br /> ==== Possible Mentors ====<br /> phschafft (teamed with someone else)<br /> ----</div>

Unlord https://wiki.xiph.org/index.php?title=X264_features_in_rav1e&diff=16693 2019-02-12T21:19:04Z

<p>Unlord: Unlord moved page X264 features in rav1e to Unimplemented x264 features in rav1e</p> <hr /> <div>#REDIRECT [[Unimplemented x264 features in rav1e]]</div>

Unlord https://wiki.xiph.org/index.php?title=Unimplemented_x264_features_in_rav1e&diff=16692 2019-02-12T21:19:04Z

<p>Unlord: Unlord moved page X264 features in rav1e to Unimplemented x264 features in rav1e</p> <hr /> <div>This is the list of unimplemented x264 features, but put in the context of rav1e instead: [https://wiki.videolan.org/index.php?title=X264_TODO]<br /> <br /> === Motion Estimation ===<br /> <br /> *Sequential elimination (SEA), used for exhaustive search, might be more generally applicable to algorithms like UMH, by letting us skip a lot of SADs. The downside is we won't be able to use SAD_X4 anymore. <br /> **rav1e: This would be nice to have. SAD_X4 is not applicable.<br /> *(T)ESA is currently wrong for motion searches done on weightp duplicates. This effect is miniscule, but it still should be fixed. <br /> **rav1e: doesn't even have TESA yet.<br /> *Hierarchical motion estimation might be a useful way to catch very long motion vectors without the cost of UMH or ESA. It might also help regularize motion.<br /> **rav1e: already has this, in the form of half-resolution and quarter-resolution search. It's enormously helpful for video game sequences.<br /> *Somehow take into account the effect of motion vector decision on future blocks. <br /> **rav1e: daala did this to enormous gain, should be doable in rav1e too.<br /> *We don't need to check all 11 predictors all the time for 16x16 fullpel motion search. <br /> **But how do we know which ones we can afford to skip, and when? <br /> **Xvid and libtheora have algorithms for this, but the former's is almost surely 100% useless and the latter doesn't seem impressive either. <br /> **rav1e: checking predictors is so cheap that this is a waste.<br /> *libtheora does fullpel motion estimation on the source pixels instead of decoded pixels. Does this give a better starting point for the subpel search and discourage &quot;weird&quot; MVs? <br /> **rav1e: worth trying but probably only for halfres and quarterres<br /> *With extremely fast encoding settings (subme 0), can we rip off lookahead MVs instead of doing a real search? <br /> **This seems to be awful from my testing, but maybe there's something we can do?<br /> **rav1e: sounds like a waste, if you're going this fast you shouldn't be doing lookahead<br /> *Try sub-8x8 partitions in B-frames. Is it at all useful? <br /> **rav1e: already does this. might be worth investigating not doing this<br /> *Try bidir motion estimation for fullpel. That is, considering L1's MV when doing L0 (or vice versa). Xvid does this. How much does it help? <br /> **rav1e: probably worth doing.<br /> *Fullpel chroma ME? <br /> **rav1e: already done?<br /> <br /> === Intra Analysis ===<br /> <br /> *Make the early terminations smarter. Currently they're just hacks -- some statistical analysis might be useful.<br /> **rav1e early termination is more straightforward but still needs tuning.<br /> <br /> === Mode Decision ===<br /> <br /> *Can we find more ways to skip more motion searches in multiref?<br /> **A while back, I tried using weaker motion searches on older refs. This helped a bit for speed-vs-compression, but is ironically the opposite of what one wants; older refs will be harder to find good MVs in, and therefore really need better searches.<br /> *On extremely fast encoding settings, fast skip is actually kind of slow. But anything dumber (e.g. SAD) is completely useless. Is there some better balance that can be achieved here?<br /> **Can we do something smart by analyzing fenc? It's impossible to tell whether a block is motionless by looking at fdec, but looking at the source pixels is useful. There's still complexity such as lower-QP-than-reference though.<br /> *See the TODOs for deblock-aware RD in common/deblock.c.<br /> **I tried correcting weightp references for deblock RDO, but it didn't help.<br /> **I tried chroma, too, and again, it didn't help measurably.<br /> *Is there a faster way than SA8D/SATD to do 8x8dct vs 4x4dct mode decision? At very fast settings, the time this uses is nontrivial.<br /> **Doing a merged 4x4/8x8 SATD would help here, but would require new asm.<br /> **rav1e: this speedup is probably less important<br /> *Is there a faster way than RD to do 8x8dct vs 4x4dct mode decision that's still better than SATD? RD takes over an order of magnitude more time than SATD, so it might be useful to have something in the middle. <br /> **rav1e: even more critical than in x264, see fast rdo patches<br /> *Is there some value to swapping the mode decision metric from SATD to SA8D if we think that the macroblock will use 8x8dct? [http://akuvian.org/src/x264/x264_dct8_guess.diff This has been tried before], but only helped if our guess was extremely good (better than we could get in reality).<br /> **rav1e: only uses one metric at a time.<br /> <br /> === Psy ===<br /> <br /> *Psy-RD is a hack. It works, but it's a hack. If you apply QNS with Psy-RD as the metric, it goes way overboard and gives terrible results. This means that Psy-RD only works because normal mode decision is limited in the way it can modify the image to better suit the metric. Is there a way to make it better?<br /> **rav1e: see cdef-dist.<br /> *Should RD be linear at all? Perhaps we should weight more heavily against low quality blocks and also try to ignore minuscule distortion that viewers can't see. <br /> **rav1e: see cdef-dist.<br /> *Psy-trellis (and maybe psy-RD?) are too strong at very high QPs.<br /> **rav1e: see cdef-dist.<br /> *Psy-trellis should be merged with Psy-RD. There are patches for this, but they probably won't be committed until psy-RD itself is fixed.<br /> **rav1e: see cdef-dist.<br /> *RD should take into account local variance. <br /> **rav1e: see cdef-dist.<br /> *Lambda should be varied on a per-DCT-block basis instead of a per-macroblock basis. <br /> **rav1e: should already do this<br /> *Lambda should be picked independent of quantizer (i.e. with greater precision).<br /> **rav1e: already done<br /> *Classic problem: a block is mostly high complexity but has a small area of low complexity. How do we judge whether that area is important? Good example: sharp text on background with film grain; grain gets blurred out because of the text. <br /> **If we think it's important all the time, we ruin the quality of many clips that rely on raising complexity on edges (Touhou). <br /> *Should motion estimation lambda be as high as it is at very high quantizers? There's some value to capturing &quot;true motion&quot;... <br /> *Macroblock tree correlates pretty well with visual perception in that its concept of a &quot;high complexity&quot; matches well with the visual concept. Except for local illumination changes. Talk to Dark Shikari for a patch.<br /> **rav1e still needs mbtree<br /> <br /> === Lookahead ===<br /> <br /> *Temporal MV predictors in lookahead? There's a patch for these somewhere, but they biased heavily in favor of B-frames, likely by improving the motion search. <br /> **rav1e: lookahead should totally use temporal candidates if available<br /> *Should lookahead use variable lambda based on quantizer (esp. due to adaptive quant)? If so, should it take into account estimated ratecontrol quantizer, too? If so, how?<br /> **rav1e: should already do this, file a bug if it doesn't<br /> <br /> === Quantization ===<br /> <br /> *There's room for something between trellis and deadzone in terms of complexity. libvpx has a good example -- it biases towards zero-runs in its &quot;medium speed&quot; quantizer. This can't be SIMD'd easily, but is still vastly faster than trellis. A nonlinear quantizer (be more likely to round up larger coefficients) might also be useful.<br /> **How useful is this with an entropy coder that doesn't really bias towards zero-runs, as in CABAC?<br /> **rav1e: same concern, with lv-map instead<br /> *Floyd-Steinberg for quantization? Try pushing quantization error to nearby DCT coefficients. Should this go from high to low or low to high? <br /> **rav1e: eeeeeeeeeeeeh<br /> *Energy-preserving quantizer -- maintain L1 (or maybe L2? I'm not sure) energy. Should we maintain it in the spatial domain (post-iDCT) or residual domain? Probably the former. <br /> **See [https://github.com/saintdev/x264-devel/compare/enquant-base...energy-quant saintdev's github] for one attempt at this.<br /> **rav1e: this is pretty hard with just the quantizer, do delta q's first<br /> <br /> === Transform ===<br /> <br /> *Analyze the error characteristics of the fDCT. Is there any way to make it more accurate without much speed loss? Particularly at extremely low quantizers, this might help. <br /> **rav1e: Using the daala fDCT will accomplish this as well as improved speed (the AV1 fDCT is much more accurate than the H.264 one)<br /> *Before forward transform, run a &quot;blocking filter&quot; that acts as the approximate inverse of the deblock filter. See [http://akuvian.org/src/x264/Shwang_loopfilter_thesis.pdf this paper].<br /> **rav1e: even more applicable to AV1 due to the many postfilters.</div>

Unlord https://wiki.xiph.org/index.php?title=Daala&diff=16571 2017-04-19T20:01:49Z

<p>Unlord: </p> <hr /> <div>Daala is the codename for a new video compression technology.<br /> <br /> The effort is a collaboration between the [https://www.mozilla.org/en-US/research/ Mozilla Foundation], the [https://www.xiph.org/ Xiph.Org Foundation] and any other contributors that wish to help.<br /> <br /> The goal of the project is to provide a video format that's free to implement, use and distribute, and a reference implementation with technical performance superior to [https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding H.265]. <br /> <br /> Please see the links below or the [https://www.xiph.org/daala/ main page] for more information.<br /> <br /> == Wiki Pages ==<br /> * [[Daala Quickstart|Daala Quickstart (Linux/MacOS)]]<br /> * [[Daala Quickstart Windows|Daala Quickstart (Windows)]]<br /> * [[Daala MinGW64 Environment]]<br /> <br /> * [[Daala Weekly Meetings|Daala Weekly Meetings]]<br /> <br /> * [[AreWeCompressedYet]]<br /> * [[RD Curve Data Format]]<br /> <br /> * [[DaalaTodo|Daala To-do List]]<br /> * [[DaalaRoadmap|Daala Roadmap]]<br /> <br /> * [[Intra|Intra-prediction within Daala]]<br /> <br /> * [[Videos|Digital Primers]] - educational videos about audio/video technology<br /> <br /> == Communication ==<br /> You are '''encouraged''' to join the<br /> * [irc://irc.freenode.net/daala '''#daala''' IRC channel at freenode.net] - if you don't have an IRC client, you can use Freenode's '''[https://webchat.freenode.net/?channels=%23daala webchat]''' instead.<br /> * [http://lists.xiph.org/mailman/listinfo/daala Daala Email List]<br /> <br /> === Weekly Meetings ===<br /> You are also welcome to attend the public [[Daala Weekly Meetings|weekly progress meetings]] by installing and using [http://wiki.mumble.info Mumble].&lt;br /&gt;<br /> The address is '''mf4.xiph.org''' and the port is '''64738''' (you can run '''mumble://mf4.xiph.org:64738''' within your browser as a shortcut).&lt;br /&gt;<br /> The meetings occur on '''Tuesdays''' at '''[http://www.timeanddate.com/worldclock/fixedtime.html?msg=Daala+Weekly+Meeting&amp;iso=20150428T09&amp;p1=1241 9AM Pacific Time]''' (5PM UTC/GMT).<br /> The meeting agenda used to be available at '''[https://daala.etherpad.mozilla.org/weekly-meeting this Etherpad]''', the October 13, 2015 meeting is available on [https://docs.google.com/document/d/1JP_Ko3wPuyDWhooZcp_m9kndyfZ75xN5YOi5yIMCW0s/edit?pli=1 Google Docs] and, following the migration to Etherpad Lite, the meeting agenda and minutes are now available at [https://public.etherpad-mozilla.org/p/daala-weekly-meeting this Etherpad].<br /> <br /> === Other ===<br /> * [http://forum.doom9.org/showthread.php?t=168004 Doom9 Forum discussion] - generic forum thread regarding Daala<br /> * &lt;del&gt;[https://daala.etherpad.mozilla.org/ep/padlist/all-pads Daala Etherpads] - you can [https://daala.etherpad.mozilla.org/ep/account/request-account request a free account] to view these. You should receive access within a few days.&lt;/del&gt; Mozilla are transitioning to Etherpad Lite.<br /> * [http://benjamin.smedbergs.us/weekly-updates.fcgi/project/daala Daala Project Status Board] - what Daala bits the Mozilla people are working on<br /> <br /> == Coding ==<br /> You can get a copy of the latest Daala Source Code from [https://git.xiph.org/?p=daala.git;a=summary '''git.xiph.org'''] or [https://github.com/xiph/daala '''GitHub''']. Please stick to the '''[https://git.xiph.org/?p=daala.git;a=blob_plain;f=doc/coding_style.html Coding Style Guide]'''.<br /> <br /> * [https://review.xiph.org/all?limit=100 Xiph Code Reviews] - there is a proposal on the review process '''[[DaalaReview|here]]'''<br /> * [https://github.com/xiph/daala/issues Daala's issues] - Issue/bug tracker on Github<br /> * [https://mf4.xiph.org/jenkins/view/daala/ Continuous Integration Tests] - these run every time a new commit is made to the Daala git master, to make sure the new code hasn't broken existing functionality.<br /> * [[OggDaala]] - definitions for embedding Daala video within an [[Ogg]] container.<br /> <br /> == Demos ==<br /> * [https://people.xiph.org/~xiphmont/demo/daala/player-demo.shtml Daala Video Player] - an example implementation of a Daala decoder and player, ported to Javascript using [https://github.com/kripken/emscripten Emscripten].<br /> <br /> === Codec Techniques ===<br /> * [https://people.xiph.org/~xiphmont/demo/ Demo Articles] - explanations on certain techniques used in Daala (and other Xiph.Org projects)<br /> * [http://exp.martres.me/edi/ Edge-Directed Interpolation] ([https://github.com/smarter/edi source code])<br /> * [https://people.xiph.org/~ds/edi/info.html More Edge-Directed Interpolation]<br /> * [https://people.xiph.org/~unlord/demo/intra.html Intra-prediction]<br /> * [https://people.xiph.org/~unlord/zigzags.html Macroblock Coefficient Zigzag Graph] - HTML page generated using [https://github.com/xiph/daala/blob/master/tools/draw_zigzags.c tools/draw_zigzags.c] from the Daala source code.<br /> <br /> == Documents ==<br /> * [https://people.xiph.org/~unlord/spie_cfl.pdf Chroma from Luma (CfL)]<br /> * [http://jmvalin.ca/papers/spie_pvq.pdf Perceptual Vector Quantisation (PVQ)] - see also [https://people.xiph.org/~yushin/ietf/draft-cho-netvc-applypvq.html Applying PVQ Outside Daala]<br /> * [https://people.xiph.org/~tterribe/daala/vbsobmc.pdf Overlapped Block Motion Compensation (OBMC)]<br /> * [https://mf4.xiph.org/jenkins/job/daala-autotools/ws/doc/html/index.html C API Documentation]<br /> * [https://people.xiph.org/~yushin/tmp__/yushin_phd_thesis.pdf Image Coding Thesis] by Yushin Cho<br /> * [http://arxiv.org/pdf/1411.4290v1.pdf Maximising Coding Efficiency Through Block Rotation] and why it [http://lists.xiph.org/pipermail/daala/2015-January/000054.html won't work well within Daala]<br /> * [http://jmvalin.ca/video/theoretical_results.pdf JMSpeex' Journal of Dubious Theoretical Results] - &quot;take with an entire shaker-full of salt&quot;<br /> * [https://people.xiph.org/~unlord/pcs_daala.pdf Using Daala Intra Frames for Still Picture Coding]<br /> * [http://arxiv.org/abs/1602.05975 The Daala Directional Deringing Filter]<br /> * [https://people.xiph.org/~unlord/icip2016.pdf Daala: A Perceptually-Driven Still Image Codec](draft) - submitted at [http://2016.ieeeicip.org/ ICIP 2016].<br /> * [http://people.xiph.org/~tterribe/daala/neon_tutorial.pdf SIMD Assembly Tutorial: ARM NEON]<br /> <br /> === IETF Drafts ===<br /> * [https://tools.ietf.org/html/draft-egge-videocodec-tdlt Time-Domain Lapped Transforms (TDLT)] - documents the Lapped Transform pre- and post-filters used for block-edge decorrelation<br /> * [https://tools.ietf.org/html/draft-valin-videocodec-pvq Perceptual Vector Quantisation (PVQ)] - <br /> * [https://tools.ietf.org/html/draft-terriberry-codingtools Coding Tools] - documents Entropy Coding, Integer Transforms and other techniques<br /> * [https://tools.ietf.org/html/draft-moffitt-netvc-requirements Internet Video Codec (NetVC) Requirements] - explains what requirements and use cases Daala is trying to cater for<br /> * [https://tools.ietf.org/html/draft-daede-netvc-testing Internet Video Codec (NetVC) Testing and Quality Measurement]<br /> * [https://tools.ietf.org/html/draft-terriberry-ipr-license Example IPR Licence Terms]<br /> <br /> Additional drafts can be found at the [https://datatracker.ietf.org/wg/netvc/documents/ IETF DataTracker].<br /> <br /> == Presentations ==<br /> For a more in-depth look at the IETF's NetVC Meetings, use the [https://datatracker.ietf.org/wg/netvc/meetings/ IETF DataTracker].<br /> <br /> * 2017-02-05 - FOSDEM 2017 - [https://video.fosdem.org/2017/K.3.401/om_av1.vp8.webm Video] - [https://fosdem.org/2017/schedule/event/om_av1/attachments/slides/1795/export/events/attachments/om_av1/slides/1795/av1_update.pdf Slides]<br /> * 2017-01-17 - Linux Conf AU - [https://www.youtube.com/watch?v=lzPaldsmJbk Video] - [https://people.xiph.org/~tterribe/pubs/lca2017/aom.pdf Slides]<br /> -------<br /> * 2016-11-15 - IETF 97 - [https://datatracker.ietf.org/meeting/97/agenda/netvc/ Agenda] - [http://recs.conf.meetecho.com/Playout/watch.jsp?recording=IETF97_NETVC&amp;chapter=chapter_1 Video and Chat] - [https://datatracker.ietf.org/meeting/97/session/netvc/ Slides]<br /> * 2016-09-04 - VideoLAN Dev Days 2016 - [https://www.youtube.co.uk/watch?v=AOssZFJ0EdI Video] - [http://people.xiph.org/~tterribe/daala/vdd2016.pdf Slides]<br /> * 2016-08-31 - SPIE Royalty-free Video - [http://spie.org/OPO/conferencedetails/digital-image-processing#session-7 Schedule and Abstracts] - [https://www.youtube.co.uk/watch?v=wi1BefrfTos&amp;t=41m40s Video] - [http://people.xiph.org/~tterribe/daala/daala-spie-adip2016-slides.pdf Slides]<br /> * 2016-07-18 - IETF 96 - [https://datatracker.ietf.org/meeting/96/agenda/netvc/ Agenda] - [http://recs.conf.meetecho.com/Playout/watch.jsp?recording=IETF96_NETVC&amp;chapter=chapter_1 Video and Chat] - [https://datatracker.ietf.org/meeting/96/session/netvc/ Slides]<br /> * 2016-06-27 - Daala's Entropy Coder - [http://people.xiph.org/~tterribe/daala/daala_ec.pdf Slides]<br /> * 2016-04-07 - IETF 95 - [https://datatracker.ietf.org/meeting/95/agenda/netvc/ Agenda] - [http://recs.conf.meetecho.com/Playout/watch.jsp?recording=IETF95_NETVC&amp;chapter=chapter_1 Video and Chat] - [https://www.ietf.org/proceedings/95/slides/slides-95-netvc-2.pdf Slides] - [https://datatracker.ietf.org/meeting/95/session/netvc/ Other Materials]<br /> * 2016-04-01 - DCC 2016 - [https://people.xiph.org/~unlord/Daala-DCC2016.pdf Slides] - [https://people.xiph.org/~unlord/1853a466.pdf Paper]<br /> * 2016-01-30 - FOSDEM 2016 - [https://fosdem.org/2016/schedule/event/daala/ Summary] - [https://video.fosdem.org/2016/h2214/implementing-a-native-daala-decoder-in-ffmpeg.mp4 Video (MP4, 57MB)] - [https://people.xiph.org/~unlord/Xmi.pdf Slides]<br /> -------<br /> * 2015-11-02 - IETF 94 - [https://datatracker.ietf.org/meeting/94/agenda/netvc/ Agenda] - [http://www.meetecho.com/ietf94/netvc Video and Chat] - [https://datatracker.ietf.org/meeting/94/session/netvc/ Other Materials]<br /> * 2015-10-24 - LinuxDay 24 (Turin) - [https://people.xiph.org/~tterribe/daala/linuxday24.pdf Slides]<br /> * 2015-10-21 - MPEG 113 - Future Video Coding Workshop - [https://people.xiph.org/~tterribe/daala/mpeg113.pdf Slides]<br /> * 2015-09-19 - VideoLAN Dev Days - [https://www.youtube.com/playlist?list=PLQLpBN3oI7E44HIdTOovThc1MNHLchgHE YouTube Playlist] - [https://people.xiph.org/~tterribe/daala/vdd2015.pdf Daala Slides]<br /> * 2015-07-22 - IETF 93 - [http://recordings.conf.meetecho.com/Playout/watch.jsp?recording=IETF93_NETVC_II&amp;chapter=chapter_1 NetVC Session 2/2] - [https://datatracker.ietf.org/meeting/93/agenda/netvc/ Agenda] - [http://www.meetecho.com/ietf93/netvc_II Video and Chat] - [https://www.ietf.org/proceedings/93/slides/slides-93-netvc-4.pdf Slides] - [https://www.ietf.org/jabber/logs/netvc/2015-07-22.html Jabber Log] - [https://datatracker.ietf.org/meeting/93/session/netvc/ Other Materials]<br /> * 2015-07-20 - IETF 93 - [http://recordings.conf.meetecho.com/Playout/watch.jsp?recording=IETF93_NETVC&amp;chapter=chapter_1 NetVC Session 1/2] - [https://datatracker.ietf.org/meeting/93/agenda/netvc/ Agenda] - [http://www.meetecho.com/ietf93/netvc Video and Chat] - [https://www.ietf.org/proceedings/93/slides/slides-93-netvc-3.pdf Slides] - [https://www.ietf.org/jabber/logs/netvc/2015-07-20.html Jabber Log]<br /> * 2015-03-24 - IETF 92 - [http://recordings.conf.meetecho.com/Playout/watch.jsp?recording=IETF92_NETVC&amp;chapter=chapter_0 NetVC Session] - Audio as [https://people.xiph.org/~tdaede/audio/ietf92-venetian-20150324-0900-am1.opus Opus] (29MB) or [https://www.ietf.org/audio/ietf92/ietf92-venetian-20150324-0900-am1.mp3 MP3] (119MB, action starts at 14:50) - [https://www.ietf.org/proceedings/92/slides/slides-92-netvc-0.pdf Slides] - [https://www.ietf.org/mail-archive/web/video-codec/current/msg00235.html Notes] - [https://www.ietf.org/jabber/logs/netvc/2015-03-24.html Jabber Log]<br /> * 2015-02-11 - SPIE talks:<br /> &lt;!-- Originals of these 3 videos can be found at http://people.xiph.org/~unlord/ --&gt;<br /> &lt;!-- Someday, should do a better encode... --&gt;<br /> ** [http://people.xiph.org/~tdaede/video/SPIE_Nathan.webm Chroma from Luma (CfL)] - [https://people.xiph.org/~unlord/SPIE-2015-CfL.pdf Slides] - [https://people.xiph.org/~unlord/spie_cfl.pdf Paper]<br /> ** [http://people.xiph.org/~tdaede/video/SPIE_PVQ.webm Perceptual Vector Quantisation (PVQ)] - [http://people.xiph.org/~tterribe/daala/spie_pvq_slides.pdf Slides] - [http://jmvalin.ca/papers/spie_pvq.pdf Paper]<br /> ** [http://people.xiph.org/~tdaede/video/SPIE_Tim.webm Adaptive Motion Compensation Without Blocking Artifacts] - [http://people.xiph.org/~tterribe/daala/spie_obmc_slides.pdf Slides] - [https://people.xiph.org/~tterribe/daala/vbsobmc.pdf Paper]<br /> * 2015-01-31 - [http://ftp.osuosl.org/pub/fosdem/2015/devroom-open_media/daala.mp4 Daala Project Update at FOSDEM 2015] - [https://fosdem.org/2015/schedule/event/daala/ summary] - [https://fosdem.org/2015/schedule/event/daala/attachments/slides/569/export/events/attachments/daala/slides/569/Daala_FOSDEM_2015.pdf Slides]<br /> * 2015-01-14 - [https://www.youtube.co.uk/watch?v=Dmho4gcRvQ4 Linux Conf 2015] - [http://lca2015.linux.org.au/schedule/30187/view_talk presentation summary] - [https://people.xiph.org/~tterribe/pubs/lca2015/daala.pdf Slides]<br /> -------<br /> * 2014-09-16 - [https://air.mozilla.org/daala-are-we-compressed-yet/ Daala: Are We Compressed Yet?]<br /> * 2014-06-25 - [https://air.mozilla.org/sparsity-induced-prediction-for-images-and-video/ Sparsity Induced Prediction for Images and Video]<br /> * 2014-06-06 - VP9 Summit (no video available) - [https://people.xiph.org/~xiphmont/demo/daala/daala-vp9summit-20140606.pdf Slides]<br /> -------<br /> * 2013-10-23 - [https://people.xiph.org/~xiphmont/video/Free_Codecs_Update_Opus_and_Daala.ogv Opus and Daala: State of the Art Royalty-free Codecs] - [https://people.xiph.org/~greg/gstreamer-daala-opus.pdf Slides]<br /> * 2013-09-30 - [https://people.xiph.org/~tterribe/daala/coding_party2/?C=M;O=A Daala Coding Party 2] - [https://people.xiph.org/~unlord/Daala-Intra.pdf Slides]<br /> * 2013-05-02 - [https://people.xiph.org/~xiphmont/tim-terriberry-presents-daala/ Tim Terriberry Presents Daala]<br /> -------<br /> * 2012-01-24 - [https://media.basilgohar.com/derf-talks/?C=M;O=A Introduction to Video Coding] - [https://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf Slides] (no video for slides 1-50)<br /> <br /> == Other Websites ==<br /> * [https://www.youtube.com/playlist?list=PLEeMksZoEQ1xQEuLF50w0RwDwLgDGwSG- Daala Presentations on YouTube]<br /> * [https://www.youtube.com/playlist?list=PLOU2XLYxmsIJGErt5rrCqaSGTMyyqNt2H Google's Compressor Head videos] - a beginner's introduction to the world of data compression<br /> * [https://www.zazzle.com/daala_tee_shirt-235139149596175944 Daala T-shirts] - if you'd like a free one, help out with the project and ask the Mozilla guys nicely for one :-)<br /> * [https://www.xiph.org/donate/ Donate to Xiph.Org] <br /> * [[Daala_on_Wheels|Historical Daala wiki page]]<br /> <br /> [[Category:Daala]]</div>

Unlord https://wiki.xiph.org/index.php?title=Daala&diff=16467 2016-08-20T17:38:20Z

<p>Unlord: /* Presentations */</p> <hr /> <div>Daala is the codename for a new video compression technology.<br /> <br /> The effort is a collaboration between the [https://www.mozilla.org/en-US/research/ Mozilla Foundation], the [https://www.xiph.org/ Xiph.Org Foundation] and any other contributors that wish to help.<br /> <br /> The goal of the project is to provide a video format that's free to implement, use and distribute, and a reference implementation with technical performance superior to [https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding H.265]. <br /> <br /> Please see the links below or the [https://www.xiph.org/daala/ main page] for more information.<br /> <br /> == Wiki Pages ==<br /> * [[Daala Quickstart|Daala Quickstart (Linux/MacOS)]]<br /> * [[Daala Quickstart Windows|Daala Quickstart (Windows)]]<br /> * [[Daala MinGW64 Environment]]<br /> <br /> * [[Daala Weekly Meetings|Daala Weekly Meetings]]<br /> <br /> * [[AreWeCompressedYet]]<br /> * [[RD Curve Data Format]]<br /> <br /> * [[DaalaTodo|Daala To-do List]]<br /> * [[DaalaRoadmap|Daala Roadmap]]<br /> <br /> * [[Intra|Intra-prediction within Daala]]<br /> <br /> * [[Videos|Digital Primers]] - educational videos about audio/video technology<br /> <br /> == Communication ==<br /> You are '''encouraged''' to join the<br /> * [irc://irc.freenode.net/daala '''#daala''' IRC channel at freenode.net] - if you don't have an IRC client, you can use Freenode's '''[https://webchat.freenode.net/?channels=%23daala webchat]''' instead.<br /> * [http://lists.xiph.org/mailman/listinfo/daala Daala Email List]<br /> <br /> === Weekly Meetings ===<br /> You are also welcome to attend the public [[Daala Weekly Meetings|weekly progress meetings]] by installing and using [http://wiki.mumble.info Mumble].&lt;br /&gt;<br /> The address is '''mf4.xiph.org''' and the port is '''64738''' (you can run '''mumble://mf4.xiph.org:64738''' within your browser as a shortcut).&lt;br /&gt;<br /> The meetings occur on '''Tuesdays''' at '''[http://www.timeanddate.com/worldclock/fixedtime.html?msg=Daala+Weekly+Meeting&amp;iso=20150428T09&amp;p1=1241 9AM Pacific Time]''' (5PM UTC/GMT).<br /> The meeting agenda used to be available at '''[https://daala.etherpad.mozilla.org/weekly-meeting this Etherpad]''', the October 13, 2015 meeting is available on [https://docs.google.com/document/d/1JP_Ko3wPuyDWhooZcp_m9kndyfZ75xN5YOi5yIMCW0s/edit?pli=1 Google Docs] and, following the migration to Etherpad Lite, the meeting agenda and minutes are now available at [https://public.etherpad-mozilla.org/p/daala-weekly-meeting this Etherpad].<br /> <br /> === Other ===<br /> * [http://forum.doom9.org/showthread.php?t=168004 Doom9 Forum discussion] - generic forum thread regarding Daala<br /> * &lt;del&gt;[https://daala.etherpad.mozilla.org/ep/padlist/all-pads Daala Etherpads] - you can [https://daala.etherpad.mozilla.org/ep/account/request-account request a free account] to view these. You should receive access within a few days.&lt;/del&gt; Mozilla are transitioning to Etherpad Lite.<br /> * [http://benjamin.smedbergs.us/weekly-updates.fcgi/project/daala Daala Project Status Board] - what Daala bits the Mozilla people are working on<br /> <br /> == Coding ==<br /> You can get a copy of the latest Daala Source Code from [https://git.xiph.org/?p=daala.git;a=summary '''git.xiph.org'''] or [https://github.com/xiph/daala '''GitHub''']. Please stick to the '''[https://git.xiph.org/?p=daala.git;a=blob_plain;f=doc/coding_style.html Coding Style Guide]'''.<br /> <br /> * [https://review.xiph.org/all?limit=100 Xiph Code Reviews] - there is a proposal on the review process '''[[DaalaReview|here]]'''<br /> * [https://github.com/xiph/daala/issues Daala's issues] - Issue/bug tracker on Github<br /> * [https://mf4.xiph.org/jenkins/view/daala/ Continuous Integration Tests] - these run every time a new commit is made to the Daala git master, to make sure the new code hasn't broken existing functionality.<br /> * [[OggDaala]] - definitions for embedding Daala video within an [[Ogg]] container.<br /> <br /> == Demos ==<br /> * [https://people.xiph.org/~xiphmont/demo/daala/player-demo.shtml Daala Video Player] - an example implementation of a Daala decoder and player, ported to Javascript using [https://github.com/kripken/emscripten Emscripten].<br /> <br /> === Codec Techniques ===<br /> * [https://people.xiph.org/~xiphmont/demo/ Demo Articles] - explanations on certain techniques used in Daala (and other Xiph.Org projects)<br /> * [http://exp.martres.me/edi/ Edge-Directed Interpolation] ([https://github.com/smarter/edi source code])<br /> * [https://people.xiph.org/~ds/edi/info.html More Edge-Directed Interpolation]<br /> * [https://people.xiph.org/~unlord/demo/intra.html Intra-prediction]<br /> * [https://people.xiph.org/~unlord/zigzags.html Macroblock Coefficient Zigzag Graph] - HTML page generated using [https://github.com/xiph/daala/blob/master/tools/draw_zigzags.c tools/draw_zigzags.c] from the Daala source code.<br /> <br /> == Documents ==<br /> * [https://people.xiph.org/~unlord/spie_cfl.pdf Chroma from Luma (CfL)]<br /> * [http://jmvalin.ca/papers/spie_pvq.pdf Perceptual Vector Quantisation (PVQ)]<br /> * [https://people.xiph.org/~tterribe/daala/vbsobmc.pdf Overlapped Block Motion Compensation (OBMC)]<br /> * [https://mf4.xiph.org/jenkins/job/daala-autotools/ws/doc/html/index.html C API Documentation]<br /> * [https://people.xiph.org/~yushin/tmp__/yushin_phd_thesis.pdf Image Coding Thesis] by Yushin Cho<br /> * [http://arxiv.org/pdf/1411.4290v1.pdf Maximising Coding Efficiency Through Block Rotation] and why it [http://lists.xiph.org/pipermail/daala/2015-January/000054.html won't work well within Daala]<br /> * [http://jmvalin.ca/video/theoretical_results.pdf JMSpeex' Journal of Dubious Theoretical Results] - &quot;take with an entire shaker-full of salt&quot;<br /> * [https://people.xiph.org/~unlord/pcs_daala.pdf Using Daala Intra Frames for Still Picture Coding]<br /> * [http://arxiv.org/abs/1602.05975 The Daala Directional Deringing Filter]<br /> * [https://people.xiph.org/~unlord/icip2016.pdf Daala: A Perceptually-Driven Still Image Codec](draft) - submitted at [http://2016.ieeeicip.org/ ICIP 2016].<br /> * [http://people.xiph.org/~tterribe/daala/neon_tutorial.pdf SIMD Assembly Tutorial: ARM NEON]<br /> <br /> === IETF Drafts ===<br /> * [https://tools.ietf.org/html/draft-egge-videocodec-tdlt Time-Domain Lapped Transforms (TDLT)] - documents the Lapped Transform pre- and post-filters used for block-edge decorrelation<br /> * [https://tools.ietf.org/html/draft-valin-videocodec-pvq Perceptual Vector Quantisation (PVQ)] - <br /> * [https://tools.ietf.org/html/draft-terriberry-codingtools Coding Tools] - documents Entropy Coding, Integer Transforms and other techniques<br /> * [https://tools.ietf.org/html/draft-moffitt-netvc-requirements Internet Video Codec (NetVC) Requirements] - explains what requirements and use cases Daala is trying to cater for<br /> * [https://tools.ietf.org/html/draft-daede-netvc-testing Internet Video Codec (NetVC) Testing and Quality Measurement]<br /> * [https://tools.ietf.org/html/draft-terriberry-ipr-license Example IPR Licence Terms]<br /> <br /> Additional drafts can be found at the [https://datatracker.ietf.org/wg/netvc/documents/ IETF DataTracker].<br /> <br /> == Presentations ==<br /> For a more in-depth look at the IETF's NetVC Meetings, use the [https://datatracker.ietf.org/wg/netvc/meetings/ IETF DataTracker].<br /> <br /> * 2016-06-27 - Daala EC - [http://people.xiph.org/~tterribe/daala/daala_ec.pdf Slides]<br /> * 2016-07-18 - IETF 96 - [https://datatracker.ietf.org/meeting/96/agenda/netvc/ Agenda] - [http://recs.conf.meetecho.com/Playout/watch.jsp?recording=IETF96_NETVC&amp;chapter=chapter_1 Video and Chat] - [https://datatracker.ietf.org/meeting/96/session/netvc/ Slides]<br /> * 2016-04-07 - IETF 95 - [https://datatracker.ietf.org/meeting/95/agenda/netvc/ Agenda] - [http://recs.conf.meetecho.com/Playout/watch.jsp?recording=IETF95_NETVC&amp;chapter=chapter_1 Video and Chat] - [https://www.ietf.org/proceedings/95/slides/slides-95-netvc-2.pdf Slides] - [https://datatracker.ietf.org/meeting/95/session/netvc/ Other Materials]<br /> * 2016-04-01 - DCC 2016 - [https://people.xiph.org/~unlord/Daala-DCC2016.pdf Slides] - [https://people.xiph.org/~unlord/1853a466.pdf Paper]<br /> * 2016-01-30 - FOSDEM 2016 - [https://fosdem.org/2016/schedule/event/daala/ Summary] - [https://video.fosdem.org/2016/h2214/implementing-a-native-daala-decoder-in-ffmpeg.mp4 Video (MP4, 57MB)] - [https://people.xiph.org/~unlord/Xmi.pdf Slides]<br /> -------<br /> * 2015-11-02 - IETF 94 - [https://datatracker.ietf.org/meeting/94/agenda/netvc/ Agenda] - [http://www.meetecho.com/ietf94/netvc Video and Chat] - [https://datatracker.ietf.org/meeting/94/session/netvc/ Other Materials]<br /> * 2015-10-24 - LinuxDay 24 (Turin) - [https://people.xiph.org/~tterribe/daala/linuxday24.pdf Slides]<br /> * 2015-10-21 - MPEG 113 - Future Video Coding Workshop - [https://people.xiph.org/~tterribe/daala/mpeg113.pdf Slides]<br /> * 2015-09-19 - VideoLAN Dev Days - [https://www.youtube.com/playlist?list=PLQLpBN3oI7E44HIdTOovThc1MNHLchgHE YouTube Playlist] - [https://people.xiph.org/~tterribe/daala/vdd2015.pdf Daala Slides]<br /> * 2015-07-22 - IETF 93 - [http://recordings.conf.meetecho.com/Playout/watch.jsp?recording=IETF93_NETVC_II&amp;chapter=chapter_1 NetVC Session 2/2] - [https://datatracker.ietf.org/meeting/93/agenda/netvc/ Agenda] - [http://www.meetecho.com/ietf93/netvc_II Video and Chat] - [https://www.ietf.org/proceedings/93/slides/slides-93-netvc-4.pdf Slides] - [https://www.ietf.org/jabber/logs/netvc/2015-07-22.html Jabber Log] - [https://datatracker.ietf.org/meeting/93/session/netvc/ Other Materials]<br /> * 2015-07-20 - IETF 93 - [http://recordings.conf.meetecho.com/Playout/watch.jsp?recording=IETF93_NETVC&amp;chapter=chapter_1 NetVC Session 1/2] - [https://datatracker.ietf.org/meeting/93/agenda/netvc/ Agenda] - [http://www.meetecho.com/ietf93/netvc Video and Chat] - [https://www.ietf.org/proceedings/93/slides/slides-93-netvc-3.pdf Slides] - [https://www.ietf.org/jabber/logs/netvc/2015-07-20.html Jabber Log]<br /> * 2015-03-24 - IETF 92 - [http://recordings.conf.meetecho.com/Playout/watch.jsp?recording=IETF92_NETVC&amp;chapter=chapter_0 NetVC Session] - Audio as [https://people.xiph.org/~tdaede/audio/ietf92-venetian-20150324-0900-am1.opus Opus] (29MB) or [https://www.ietf.org/audio/ietf92/ietf92-venetian-20150324-0900-am1.mp3 MP3] (119MB, action starts at 14:50) - [https://www.ietf.org/proceedings/92/slides/slides-92-netvc-0.pdf Slides] - [https://www.ietf.org/mail-archive/web/video-codec/current/msg00235.html Notes] - [https://www.ietf.org/jabber/logs/netvc/2015-03-24.html Jabber Log]<br /> * 2015-02-11 - SPIE talks:<br /> &lt;!-- Originals of these 3 videos can be found at http://people.xiph.org/~unlord/ --&gt;<br /> &lt;!-- Someday, should do a better encode... --&gt;<br /> ** [http://people.xiph.org/~tdaede/video/SPIE_Nathan.webm Chroma from Luma (CfL)] - [https://people.xiph.org/~unlord/SPIE-2015-CfL.pdf Slides] - [https://people.xiph.org/~unlord/spie_cfl.pdf Paper]<br /> ** [http://people.xiph.org/~tdaede/video/SPIE_PVQ.webm Perceptual Vector Quantisation (PVQ)] - [http://people.xiph.org/~tterribe/daala/spie_pvq_slides.pdf Slides] - [http://jmvalin.ca/papers/spie_pvq.pdf Paper]<br /> ** [http://people.xiph.org/~tdaede/video/SPIE_Tim.webm Overlapped Block Motion Compensation (OBMC)] - [http://people.xiph.org/~tterribe/daala/spie_obmc_slides.pdf Slides] - [https://people.xiph.org/~tterribe/daala/vbsobmc.pdf Paper]<br /> * 2015-01-31 - [http://ftp.osuosl.org/pub/fosdem/2015/devroom-open_media/daala.mp4 Daala Project Update at FOSDEM 2015] - [https://fosdem.org/2015/schedule/event/daala/ summary] - [https://fosdem.org/2015/schedule/event/daala/attachments/slides/569/export/events/attachments/daala/slides/569/Daala_FOSDEM_2015.pdf Slides]<br /> * 2015-01-14 - [https://www.youtube.co.uk/watch?v=Dmho4gcRvQ4 Linux Conf 2015] - [http://lca2015.linux.org.au/schedule/30187/view_talk presentation summary] - [https://people.xiph.org/~tterribe/pubs/lca2015/daala.pdf Slides]<br /> -------<br /> * 2014-09-16 - [https://air.mozilla.org/daala-are-we-compressed-yet/ Daala: Are We Compressed Yet?]<br /> * 2014-06-25 - [https://air.mozilla.org/sparsity-induced-prediction-for-images-and-video/ Sparsity Induced Prediction for Images and Video]<br /> * 2014-06-06 - VP9 Summit (no video available) - [https://people.xiph.org/~xiphmont/demo/daala/daala-vp9summit-20140606.pdf Slides]<br /> -------<br /> * 2013-10-23 - [https://people.xiph.org/~xiphmont/video/Free_Codecs_Update_Opus_and_Daala.ogv Opus and Daala: State of the Art Royalty-free Codecs] - [https://people.xiph.org/~greg/gstreamer-daala-opus.pdf Slides]<br /> * 2013-09-30 - [https://people.xiph.org/~tterribe/daala/coding_party2/?C=M;O=A Daala Coding Party 2] - [https://people.xiph.org/~unlord/Daala-Intra.pdf Slides]<br /> * 2013-05-02 - [https://people.xiph.org/~xiphmont/tim-terriberry-presents-daala/ Tim Terriberry Presents Daala]<br /> -------<br /> * 2012-01-24 - [https://media.basilgohar.com/derf-talks/?C=M;O=A Introduction to Video Coding] - [https://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf Slides] (no video for slides 1-50)<br /> <br /> == Other Websites ==<br /> * [https://www.youtube.com/playlist?list=PLEeMksZoEQ1xQEuLF50w0RwDwLgDGwSG- Daala Presentations on YouTube]<br /> * [https://www.youtube.com/playlist?list=PLOU2XLYxmsIJGErt5rrCqaSGTMyyqNt2H Google's Compressor Head videos] - a beginner's introduction to the world of data compression<br /> * [https://www.zazzle.com/daala_tee_shirt-235139149596175944 Daala T-shirts] - if you'd like a free one, help out with the project and ask the Mozilla guys nicely for one :-)<br /> * [https://www.xiph.org/donate/ Donate to Xiph.Org] <br /> * [[Daala_on_Wheels|Historical Daala wiki page]]<br /> <br /> [[Category:Daala]]</div>

Unlord https://wiki.xiph.org/index.php?title=Daala&diff=16314 2016-04-06T18:00:46Z

<p>Unlord: </p> <hr /> <div>Daala is the codename for a new video compression technology.<br /> <br /> The effort is a collaboration between the [https://www.mozilla.org/en-US/research/ Mozilla Foundation], the [https://www.xiph.org/ Xiph.Org Foundation] and any other contributors that wish to help.<br /> <br /> The goal of the project is to provide a video format that's free to implement, use and distribute, and a reference implementation with technical performance superior to [https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding H.265]. <br /> <br /> Please see the links below or the [https://www.xiph.org/daala/ main page] for more information.<br /> <br /> == Wiki Pages ==<br /> * [[Daala Quickstart|Daala Quickstart (Linux/MacOS)]]<br /> * [[Daala Quickstart Windows|Daala Quickstart (Windows)]]<br /> * [[Daala MinGW64 Environment]]<br /> <br /> * [[Daala Weekly Meetings|Daala Weekly Meetings]]<br /> <br /> * [[AreWeCompressedYet]]<br /> * [[RD Curve Data Format]]<br /> <br /> * [[DaalaTodo|Daala To-do List]]<br /> * [[DaalaRoadmap|Daala Roadmap]]<br /> <br /> * [[Intra|Intra-prediction within Daala]]<br /> <br /> * [[Videos|Digital Primers]] - educational videos about audio/video technology<br /> <br /> == Communication ==<br /> You are '''encouraged''' to join the<br /> * [irc://irc.freenode.net/daala '''#daala''' IRC channel at freenode.net] - if you don't have an IRC client, you can use Freenode's '''[https://webchat.freenode.net/?channels=%23daala webchat]''' instead.<br /> * [http://lists.xiph.org/mailman/listinfo/daala Daala Email List]<br /> <br /> === Weekly Meetings ===<br /> You are also welcome to attend the public [[Daala Weekly Meetings|weekly progress meetings]] by installing and using [http://wiki.mumble.info Mumble].&lt;br /&gt;<br /> The address is '''mf4.xiph.org''' and the port is '''64738''' (you can run '''mumble://mf4.xiph.org:64738''' within your browser as a shortcut).&lt;br /&gt;<br /> The meetings occur on '''Tuesdays''' at '''[http://www.timeanddate.com/worldclock/fixedtime.html?msg=Daala+Weekly+Meeting&amp;iso=20150428T09&amp;p1=1241 9AM Pacific Time]''' (5PM UTC/GMT).<br /> The meeting agenda used to be available at '''[https://daala.etherpad.mozilla.org/weekly-meeting this Etherpad]''', the October 13, 2015 meeting is available on [https://docs.google.com/document/d/1JP_Ko3wPuyDWhooZcp_m9kndyfZ75xN5YOi5yIMCW0s/edit?pli=1 Google Docs] and, following the migration to Etherpad Lite, the meeting agenda and minutes are now available at [https://public.etherpad-mozilla.org/p/daala-weekly-meeting this Etherpad].<br /> <br /> === Other ===<br /> * [http://forum.doom9.org/showthread.php?t=168004 Doom9 Forum discussion] - generic forum thread regarding Daala<br /> * &lt;del&gt;[https://daala.etherpad.mozilla.org/ep/padlist/all-pads Daala Etherpads] - you can [https://daala.etherpad.mozilla.org/ep/account/request-account request a free account] to view these. You should receive access within a few days.&lt;/del&gt; Mozilla are transitioning to Etherpad Lite.<br /> * [http://benjamin.smedbergs.us/weekly-updates.fcgi/project/daala Daala Project Status Board] - what Daala bits the Mozilla people are working on<br /> <br /> == Coding ==<br /> You can get a copy of the latest Daala Source Code from [https://git.xiph.org/?p=daala.git;a=summary '''git.xiph.org'''] or [https://github.com/xiph/daala '''GitHub''']. Please stick to the '''[https://git.xiph.org/?p=daala.git;a=blob_plain;f=doc/coding_style.html Coding Style Guide]'''.<br /> <br /> * [https://review.xiph.org/all?limit=100 Xiph Code Reviews] - there is a proposal on the review process '''[[DaalaReview|here]]'''<br /> * [https://github.com/xiph/daala/issues Daala's issues] - Issue/bug tracker on Github<br /> * [https://mf4.xiph.org/jenkins/view/daala/ Continuous Integration Tests] - these run every time a new commit is made to the Daala git master, to make sure the new code hasn't broken existing functionality.<br /> * [[OggDaala]] - definitions for embedding Daala video within an [[Ogg]] container.<br /> <br /> == Demos ==<br /> * [https://people.xiph.org/~xiphmont/demo/daala/player-demo.shtml Daala Video Player] - an example implementation of a Daala decoder and player, ported to Javascript using [https://github.com/kripken/emscripten Emscripten].<br /> <br /> === Codec Techniques ===<br /> * [https://people.xiph.org/~xiphmont/demo/ Demo Articles] - explanations on certain techniques used in Daala (and other Xiph.Org projects)<br /> * [http://exp.martres.me/edi/ Edge-Directed Interpolation] ([https://github.com/smarter/edi source code])<br /> * [https://people.xiph.org/~ds/edi/info.html More Edge-Directed Interpolation]<br /> * [https://people.xiph.org/~unlord/demo/intra.html Intra-prediction]<br /> * [https://people.xiph.org/~unlord/zigzags.html Macroblock Coefficient Zigzag Graph] - HTML page generated using [https://github.com/xiph/daala/blob/master/tools/draw_zigzags.c tools/draw_zigzags.c] from the Daala source code.<br /> <br /> == Documents ==<br /> * [https://people.xiph.org/~unlord/spie_cfl.pdf Chroma from Luma (CfL)]<br /> * [http://jmvalin.ca/papers/spie_pvq.pdf Perceptual Vector Quantisation (PVQ)]<br /> * [https://people.xiph.org/~tterribe/daala/vbsobmc.pdf Overlapped Block Motion Compensation (OBMC)]<br /> * [https://mf4.xiph.org/jenkins/job/daala-autotools/ws/doc/html/index.html C API Documentation]<br /> * [https://people.xiph.org/~yushin/tmp__/yushin_phd_thesis.pdf Image Coding Thesis] by Yushin Cho<br /> * [http://arxiv.org/pdf/1411.4290v1.pdf Maximising Coding Efficiency Through Block Rotation] and why it [http://lists.xiph.org/pipermail/daala/2015-January/000054.html won't work well within Daala]<br /> * [http://jmvalin.ca/video/theoretical_results.pdf JMSpeex' Journal of Dubious Theoretical Results] - &quot;take with an entire shaker-full of salt&quot;<br /> * [https://people.xiph.org/~unlord/pcs_daala.pdf Using Daala Intra Frames for Still Picture Coding]<br /> * [http://arxiv.org/abs/1602.05975 The Daala Directional Deringing Filter]<br /> <br /> === IETF Drafts ===<br /> * [https://tools.ietf.org/html/draft-egge-videocodec-tdlt Time-Domain Lapped Transforms (TDLT)] - documents the Lapped Transform pre- and post-filters used for block-edge decorrelation<br /> * [https://tools.ietf.org/html/draft-valin-videocodec-pvq Perceptual Vector Quantisation (PVQ)] - <br /> * [https://tools.ietf.org/html/draft-terriberry-codingtools Coding Tools] - documents Entropy Coding, Integer Transforms and other techniques<br /> * [https://tools.ietf.org/html/draft-moffitt-netvc-requirements Internet Video Codec (NetVC) Requirements] - explains what requirements and use cases Daala is trying to cater for<br /> * [https://tools.ietf.org/html/draft-daede-netvc-testing Internet Video Codec (NetVC) Testing and Quality Measurement]<br /> * [https://tools.ietf.org/html/draft-terriberry-ipr-license Example IPR Licence Terms]<br /> <br /> Additional drafts can be found at the [https://datatracker.ietf.org/wg/netvc/documents/ IETF DataTracker].<br /> <br /> == Presentations ==<br /> * 2016-04-01 DCC 2016 - [https://people.xiph.org/~unlord/Daala-DCC2016.pdf Slides] - [https://people.xiph.org/~unlord/1853a466.pdf Paper]<br /> * 2016-01-30 FOSDEM 2016 - [https://fosdem.org/2016/schedule/event/daala/ Summary] - [https://video.fosdem.org/2016/h2214/implementing-a-native-daala-decoder-in-ffmpeg.mp4 Video (MP4, 57MB)] - [https://people.xiph.org/~unlord/Xmi.pdf Slides]<br /> -------<br /> * 2015-10-24 LinuxDay 24 (Turin) - [https://people.xiph.org/~tterribe/daala/linuxday24.pdf Slides]<br /> * 2015-10-21 MPEG 113 - Future Video Coding Workshop - [https://people.xiph.org/~tterribe/daala/mpeg113.pdf Slides]<br /> * 2015-09-19 VideoLAN Dev Days - [https://www.youtube.com/playlist?list=PLQLpBN3oI7E44HIdTOovThc1MNHLchgHE YouTube Playlist] - [https://people.xiph.org/~tterribe/daala/vdd2015.pdf Daala Slides]<br /> * 2015-07-22 IETF 93 - [http://recordings.conf.meetecho.com/Playout/watch.jsp?recording=IETF93_NETVC_II&amp;chapter=chapter_1 NetVC Session 2/2] - [https://datatracker.ietf.org/meeting/93/agenda/netvc/ Agenda] - [http://www.meetecho.com/ietf93/netvc_II Video and Chat] - [https://www.ietf.org/proceedings/93/slides/slides-93-netvc-4.pdf Slides] - [https://www.ietf.org/jabber/logs/netvc/2015-07-22.html Jabber Log]<br /> * 2015-07-20 IETF 93 - [http://recordings.conf.meetecho.com/Playout/watch.jsp?recording=IETF93_NETVC&amp;chapter=chapter_1 NetVC Session 1/2] - [https://datatracker.ietf.org/meeting/93/agenda/netvc/ Agenda] - [http://www.meetecho.com/ietf93/netvc Video and Chat] - [https://www.ietf.org/proceedings/93/slides/slides-93-netvc-3.pdf Slides] - [https://www.ietf.org/jabber/logs/netvc/2015-07-20.html Jabber Log]<br /> * 2015-03-24 IETF 92 - [http://recordings.conf.meetecho.com/Playout/watch.jsp?recording=IETF92_NETVC&amp;chapter=chapter_0 NetVC Session] - Audio as [https://people.xiph.org/~tdaede/audio/ietf92-venetian-20150324-0900-am1.opus Opus] (29MB) or [https://www.ietf.org/audio/ietf92/ietf92-venetian-20150324-0900-am1.mp3 MP3] (119MB, action starts at 14:50) - [https://www.ietf.org/proceedings/92/slides/slides-92-netvc-0.pdf Slides] - [https://www.ietf.org/mail-archive/web/video-codec/current/msg00235.html Notes] - [https://www.ietf.org/jabber/logs/netvc/2015-03-24.html Jabber Log]<br /> * 2015-02-11 SPIE talks:<br /> &lt;!-- Originals of these 3 videos can be found at http://people.xiph.org/~unlord/ --&gt;<br /> &lt;!-- Someday, do a better encode --&gt;<br /> ** [http://people.xiph.org/~tdaede/video/SPIE_Nathan.webm Chroma from Luma (CfL)] - [https://people.xiph.org/~unlord/SPIE-2015-CfL.pdf Slides] - [https://people.xiph.org/~unlord/spie_cfl.pdf Paper]<br /> ** [http://people.xiph.org/~tdaede/video/SPIE_PVQ.webm Perceptual Vector Quantisation (PVQ)] - [http://people.xiph.org/~tterribe/daala/spie_pvq_slides.pdf Slides] - [http://jmvalin.ca/papers/spie_pvq.pdf Paper]<br /> ** [http://people.xiph.org/~tdaede/video/SPIE_Tim.webm Overlapped Block Motion Compensation (OBMC)] - [http://people.xiph.org/~tterribe/daala/spie_obmc_slides.pdf Slides] - [https://people.xiph.org/~tterribe/daala/vbsobmc.pdf Paper]<br /> * 2015-01-31 [http://ftp.osuosl.org/pub/fosdem/2015/devroom-open_media/daala.mp4 Daala Project Update at FOSDEM 2015] - [https://fosdem.org/2015/schedule/event/daala/ summary] - [https://fosdem.org/2015/schedule/event/daala/attachments/slides/569/export/events/attachments/daala/slides/569/Daala_FOSDEM_2015.pdf Slides]<br /> * 2015-01-14 [https://www.youtube.co.uk/watch?v=Dmho4gcRvQ4 Linux Conf 2015] - [http://lca2015.linux.org.au/schedule/30187/view_talk presentation summary] - [https://people.xiph.org/~tterribe/pubs/lca2015/daala.pdf Slides]<br /> -------<br /> * 2014-09-16 [https://air.mozilla.org/daala-are-we-compressed-yet/ Daala: Are We Compressed Yet?]<br /> * 2014-06-25 [https://air.mozilla.org/sparsity-induced-prediction-for-images-and-video/ Sparsity Induced Prediction for Images and Video]<br /> * 2014-06-06 VP9 Summit (no video available) - [https://people.xiph.org/~xiphmont/demo/daala/daala-vp9summit-20140606.pdf Slides]<br /> -------<br /> * 2013-10-23 [https://people.xiph.org/~xiphmont/video/Free_Codecs_Update_Opus_and_Daala.ogv Opus and Daala: State of the Art Royalty-free Codecs] - [https://people.xiph.org/~greg/gstreamer-daala-opus.pdf Slides]<br /> * 2013-09-30 [https://people.xiph.org/~tterribe/daala/coding_party2/?C=M;O=A Daala Coding Party 2] - [https://people.xiph.org/~unlord/Daala-Intra.pdf Slides]<br /> * 2013-05-02 [https://people.xiph.org/~xiphmont/tim-terriberry-presents-daala/ Tim Terriberry Presents Daala]<br /> -------<br /> * 2012-01-24 [https://media.basilgohar.com/derf-talks/?C=M;O=A Introduction to Video Coding] - [https://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf Slides] (no video for slides 1-50)<br /> <br /> == Other Websites ==<br /> * [https://www.youtube.com/playlist?list=PLEeMksZoEQ1xQEuLF50w0RwDwLgDGwSG- Daala Presentations on YouTube]<br /> * [https://www.youtube.com/playlist?list=PLOU2XLYxmsIJGErt5rrCqaSGTMyyqNt2H Google's Compressor Head videos] - a beginner's introduction to the world of data compression<br /> * [https://www.zazzle.com/daala_tee_shirt-235139149596175944 Daala T-shirts] - if you'd like a free one, help out with the project and ask the Mozilla guys nicely for one :-)<br /> * [https://www.xiph.org/donate/ Donate to Xiph.Org] <br /> * [[Daala_on_Wheels|Historical Daala wiki page]]<br /> <br /> [[Category:Daala]]</div>

Unlord https://wiki.xiph.org/index.php?title=Daala&diff=16309 2016-03-30T17:55:04Z

<p>Unlord: /* Presentations */</p> <hr /> <div>Daala is the codename for a new video compression technology.<br /> <br /> The effort is a collaboration between the [https://www.mozilla.org/en-US/research/ Mozilla Foundation], the [https://www.xiph.org/ Xiph.Org Foundation] and any other contributors that wish to help.<br /> <br /> The goal of the project is to provide a video format that's free to implement, use and distribute, and a reference implementation with technical performance superior to [https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding H.265]. <br /> <br /> Please see the links below or the [https://www.xiph.org/daala/ main page] for more information.<br /> <br /> == Wiki Pages ==<br /> * [[Daala Quickstart|Daala Quickstart (Linux/MacOS)]]<br /> * [[Daala Quickstart Windows|Daala Quickstart (Windows)]]<br /> * [[Daala MinGW64 Environment]]<br /> <br /> * [[Daala Weekly Meetings|Daala Weekly Meetings]]<br /> <br /> * [[AreWeCompressedYet]]<br /> * [[RD Curve Data Format]]<br /> <br /> * [[DaalaTodo|Daala To-do List]]<br /> * [[DaalaRoadmap|Daala Roadmap]]<br /> <br /> * [[Intra|Intra-prediction within Daala]]<br /> <br /> * [[Videos|Digital Primers]] - educational videos about audio/video technology<br /> <br /> == Communication ==<br /> You are '''encouraged''' to join the<br /> * [irc://irc.freenode.net/daala '''#daala''' IRC channel at freenode.net] - if you don't have an IRC client, you can use Freenode's '''[https://webchat.freenode.net/?channels=%23daala webchat]''' instead.<br /> * [http://lists.xiph.org/mailman/listinfo/daala Daala Email List]<br /> <br /> === Weekly Meetings ===<br /> You are also welcome to attend the public [[Daala Weekly Meetings|weekly progress meetings]] by installing and using [http://wiki.mumble.info Mumble].&lt;br /&gt;<br /> The address is '''mf4.xiph.org''' and the port is '''64738''' (you can run '''mumble://mf4.xiph.org:64738''' within your browser as a shortcut).&lt;br /&gt;<br /> The meetings occur on '''Tuesdays''' at '''[http://www.timeanddate.com/worldclock/fixedtime.html?msg=Daala+Weekly+Meeting&amp;iso=20150428T09&amp;p1=1241 9AM Pacific Time]''' (5PM UTC/GMT).<br /> The meeting agenda used to be available at '''[https://daala.etherpad.mozilla.org/weekly-meeting this Etherpad]''', the October 13, 2015 meeting is available on [https://docs.google.com/document/d/1JP_Ko3wPuyDWhooZcp_m9kndyfZ75xN5YOi5yIMCW0s/edit?pli=1 Google Docs] and, following the migration to Etherpad Lite, the meeting agenda and minutes are now available at [https://public.etherpad-mozilla.org/p/daala-weekly-meeting this Etherpad].<br /> <br /> === Other ===<br /> * [http://forum.doom9.org/showthread.php?t=168004 Doom9 Forum discussion] - generic forum thread regarding Daala<br /> * &lt;del&gt;[https://daala.etherpad.mozilla.org/ep/padlist/all-pads Daala Etherpads] - you can [https://daala.etherpad.mozilla.org/ep/account/request-account request a free account] to view these. You should receive access within a few days.&lt;/del&gt; Mozilla are transitioning to Etherpad Lite.<br /> * [http://benjamin.smedbergs.us/weekly-updates.fcgi/project/daala Daala Project Status Board] - what Daala bits the Mozilla people are working on<br /> <br /> == Coding ==<br /> You can get a copy of the latest Daala Source Code from [https://git.xiph.org/?p=daala.git;a=summary '''git.xiph.org'''] or [https://github.com/xiph/daala '''GitHub''']. Please stick to the '''[https://git.xiph.org/?p=daala.git;a=blob_plain;f=doc/coding_style.html Coding Style Guide]'''.<br /> <br /> * [https://review.xiph.org/all?limit=100 Xiph Code Reviews] - there is a proposal on the review process '''[[DaalaReview|here]]'''<br /> * [https://github.com/xiph/daala/issues Daala's issues] - Issue/bug tracker on Github<br /> * [https://mf4.xiph.org/jenkins/view/daala/ Continuous Integration Tests] - these run every time a new commit is made to the Daala git master, to make sure the new code hasn't broken existing functionality.<br /> * [[OggDaala]] - definitions for embedding Daala video within an [[Ogg]] container.<br /> <br /> == Demos ==<br /> * [https://people.xiph.org/~xiphmont/demo/daala/player-demo.shtml Daala Video Player] - an example implementation of a Daala decoder and player, ported to Javascript using [https://github.com/kripken/emscripten Emscripten].<br /> <br /> === Codec Techniques ===<br /> * [https://people.xiph.org/~xiphmont/demo/ Demo Articles] - explanations on certain techniques used in Daala (and other Xiph.Org projects)<br /> * [http://exp.martres.me/edi/ Edge-Directed Interpolation] ([https://github.com/smarter/edi source code])<br /> * [https://people.xiph.org/~ds/edi/info.html More Edge-Directed Interpolation]<br /> * [https://people.xiph.org/~unlord/demo/intra.html Intra-prediction]<br /> * [https://people.xiph.org/~unlord/zigzags.html Macroblock Coefficient Zigzag Graph] - HTML page generated using [https://github.com/xiph/daala/blob/master/tools/draw_zigzags.c tools/draw_zigzags.c] from the Daala source code.<br /> <br /> == Documents ==<br /> * [https://people.xiph.org/~unlord/spie_cfl.pdf Chroma from Luma (CfL)]<br /> * [http://jmvalin.ca/papers/spie_pvq.pdf Perceptual Vector Quantisation (PVQ)]<br /> * [https://people.xiph.org/~tterribe/daala/vbsobmc.pdf Overlapped Block Motion Compensation (OBMC)]<br /> * [https://mf4.xiph.org/jenkins/job/daala-autotools/ws/doc/html/index.html C API Documentation]<br /> * [https://people.xiph.org/~yushin/tmp__/yushin_phd_thesis.pdf Image Coding Thesis] by Yushin Cho<br /> * [http://arxiv.org/pdf/1411.4290v1.pdf Maximising Coding Efficiency Through Block Rotation] and why it [http://lists.xiph.org/pipermail/daala/2015-January/000054.html won't work well within Daala]<br /> * [http://jmvalin.ca/video/theoretical_results.pdf JMSpeex' Journal of Dubious Theoretical Results] - &quot;take with an entire shaker-full of salt&quot;<br /> * [https://people.xiph.org/~unlord/pcs_daala.pdf Using Daala Intra Frames for Still Picture Coding]<br /> * [http://arxiv.org/abs/1602.05975 The Daala Directional Deringing Filter]<br /> <br /> === IETF Drafts ===<br /> * [https://tools.ietf.org/html/draft-egge-videocodec-tdlt Time-Domain Lapped Transforms (TDLT)] - documents the Lapped Transform pre- and post-filters used for block-edge decorrelation<br /> * [https://tools.ietf.org/html/draft-valin-videocodec-pvq Perceptual Vector Quantisation (PVQ)] - <br /> * [https://tools.ietf.org/html/draft-terriberry-codingtools Coding Tools] - documents Entropy Coding, Integer Transforms and other techniques<br /> * [https://tools.ietf.org/html/draft-moffitt-netvc-requirements Internet Video Codec (NetVC) Requirements] - explains what requirements and use cases Daala is trying to cater for<br /> * [https://tools.ietf.org/html/draft-daede-netvc-testing Internet Video Codec (NetVC) Testing and Quality Measurement]<br /> * [https://tools.ietf.org/html/draft-terriberry-ipr-license Example IPR Licence Terms]<br /> <br /> Additional drafts can be found at the [https://datatracker.ietf.org/wg/netvc/documents/ IETF DataTracker].<br /> <br /> == Presentations ==<br /> * 2016-01-30 FOSDEM 2016 - [https://fosdem.org/2016/schedule/event/daala/ Summary] - [https://video.fosdem.org/2016/h2214/implementing-a-native-daala-decoder-in-ffmpeg.mp4 Video (MP4, 57MB)] - [https://people.xiph.org/~unlord/Xmi.pdf Slides]<br /> -------<br /> * 2015-10-24 LinuxDay 24 (Turin) - [https://people.xiph.org/~tterribe/daala/linuxday24.pdf Slides]<br /> * 2015-10-21 MPEG 113 - Future Video Coding Workshop - [https://people.xiph.org/~tterribe/daala/mpeg113.pdf Slides]<br /> * 2015-09-19 VideoLAN Dev Days - [https://www.youtube.com/playlist?list=PLQLpBN3oI7E44HIdTOovThc1MNHLchgHE YouTube Playlist] - [https://people.xiph.org/~tterribe/daala/vdd2015.pdf Daala Slides]<br /> * 2015-07-22 IETF 93 - [http://recordings.conf.meetecho.com/Playout/watch.jsp?recording=IETF93_NETVC_II&amp;chapter=chapter_1 NetVC Session 2/2] - [https://datatracker.ietf.org/meeting/93/agenda/netvc/ Agenda] - [http://www.meetecho.com/ietf93/netvc_II Video and Chat] - [https://www.ietf.org/proceedings/93/slides/slides-93-netvc-4.pdf Slides] - [https://www.ietf.org/jabber/logs/netvc/2015-07-22.html Jabber Log]<br /> * 2015-07-20 IETF 93 - [http://recordings.conf.meetecho.com/Playout/watch.jsp?recording=IETF93_NETVC&amp;chapter=chapter_1 NetVC Session 1/2] - [https://datatracker.ietf.org/meeting/93/agenda/netvc/ Agenda] - [http://www.meetecho.com/ietf93/netvc Video and Chat] - [https://www.ietf.org/proceedings/93/slides/slides-93-netvc-3.pdf Slides] - [https://www.ietf.org/jabber/logs/netvc/2015-07-20.html Jabber Log]<br /> * 2015-03-24 IETF 92 - [http://recordings.conf.meetecho.com/Playout/watch.jsp?recording=IETF92_NETVC&amp;chapter=chapter_0 NetVC Session] - Audio as [https://people.xiph.org/~tdaede/audio/ietf92-venetian-20150324-0900-am1.opus Opus] (29MB) or [https://www.ietf.org/audio/ietf92/ietf92-venetian-20150324-0900-am1.mp3 MP3] (119MB, action starts at 14:50) - [https://www.ietf.org/proceedings/92/slides/slides-92-netvc-0.pdf Slides] - [https://www.ietf.org/mail-archive/web/video-codec/current/msg00235.html Notes] - [https://www.ietf.org/jabber/logs/netvc/2015-03-24.html Jabber Log]<br /> * 2015-02-11 SPIE talks:<br /> &lt;!-- Originals of these 3 videos can be found at http://people.xiph.org/~unlord/ --&gt;<br /> &lt;!-- Someday, do a better encode --&gt;<br /> ** [http://people.xiph.org/~tdaede/video/SPIE_Nathan.webm Chroma from Luma (CfL)] - [https://people.xiph.org/~unlord/SPIE-2015-CfL.pdf Slides] - [https://people.xiph.org/~unlord/spie_cfl.pdf Paper]<br /> ** [http://people.xiph.org/~tdaede/video/SPIE_PVQ.webm Perceptual Vector Quantisation (PVQ)] - [http://people.xiph.org/~tterribe/daala/spie_pvq_slides.pdf Slides] - [http://jmvalin.ca/papers/spie_pvq.pdf Paper]<br /> ** [http://people.xiph.org/~tdaede/video/SPIE_Tim.webm Overlapped Block Motion Compensation (OBMC)] - [http://people.xiph.org/~tterribe/daala/spie_obmc_slides.pdf Slides] - [https://people.xiph.org/~tterribe/daala/vbsobmc.pdf Paper]<br /> * 2015-01-31 [http://ftp.osuosl.org/pub/fosdem/2015/devroom-open_media/daala.mp4 Daala Project Update at FOSDEM 2015] - [https://fosdem.org/2015/schedule/event/daala/ summary] - [https://fosdem.org/2015/schedule/event/daala/attachments/slides/569/export/events/attachments/daala/slides/569/Daala_FOSDEM_2015.pdf Slides]<br /> * 2015-01-14 [https://www.youtube.co.uk/watch?v=Dmho4gcRvQ4 Linux Conf 2015] - [http://lca2015.linux.org.au/schedule/30187/view_talk presentation summary] - [https://people.xiph.org/~tterribe/pubs/lca2015/daala.pdf Slides]<br /> -------<br /> * 2014-09-16 [https://air.mozilla.org/daala-are-we-compressed-yet/ Daala: Are We Compressed Yet?]<br /> * 2014-06-25 [https://air.mozilla.org/sparsity-induced-prediction-for-images-and-video/ Sparsity Induced Prediction for Images and Video]<br /> * 2014-06-06 VP9 Summit (no video available) - [https://people.xiph.org/~xiphmont/demo/daala/daala-vp9summit-20140606.pdf Slides]<br /> -------<br /> * 2013-10-23 [https://people.xiph.org/~xiphmont/video/Free_Codecs_Update_Opus_and_Daala.ogv Opus and Daala: State of the Art Royalty-free Codecs] - [https://people.xiph.org/~greg/gstreamer-daala-opus.pdf Slides]<br /> * 2013-09-30 [https://people.xiph.org/~tterribe/daala/coding_party2/?C=M;O=A Daala Coding Party 2] - [https://people.xiph.org/~unlord/Daala-Intra.pdf Slides]<br /> * 2013-05-02 [https://people.xiph.org/~xiphmont/tim-terriberry-presents-daala/ Tim Terriberry Presents Daala]<br /> -------<br /> * 2012-01-24 [https://media.basilgohar.com/derf-talks/?C=M;O=A Introduction to Video Coding] - [https://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf Slides] (no video for slides 1-50)<br /> <br /> == Other Websites ==<br /> * [https://www.zazzle.com/daala_tee_shirt-235139149596175944 Daala T-shirts] - if you'd like a free one, help out with the project and ask the Mozilla guys nicely for one :-)<br /> * [https://www.xiph.org/donate/ Donate to Xiph.Org] <br /> * [[Daala_on_Wheels|Historical Daala wiki page]]<br /> <br /> [[Category:Daala]]</div>

Unlord https://wiki.xiph.org/index.php?title=RD_Curve_Data_Format&diff=15813 2015-05-08T14:44:17Z

<p>Unlord: </p> <hr /> <div>RD curves produced by rd_collect.sh and [[AreWeCompressedYet]] are text files with one data point per line, separated by spaces.<br /> <br /> The file headers are:<br /> <br /> &lt;-v quantizer value&gt; &lt;number of pixels per clip&gt; &lt;file size in bytes&gt; &lt;PSNR&gt; &lt;PSNR-HVS&gt; &lt;SSim&gt; &lt;FastSSim&gt;<br /> <br /> The units for all the metric results are in '''dB'''.&lt;br /&gt;<br /> See the tools directory in the Daala source code for how they are calculated.<br /> <br /> [[Category: Daala]]</div>

Unlord https://wiki.xiph.org/index.php?title=DaalaTodo&diff=15039 2014-10-20T17:42:32Z

<p>Unlord: </p> <hr /> <div>= Simple Things =<br /> * Overflow checking for allocations in od_state_ref_imgs_init().<br /> <br /> = Tuning =<br /> * Quantization matrix<br /> * Beta (activity masking)<br /> * Inter band masking<br /> * PVQ RDO<br /> * Better MV cost estimates<br /> * Better MC distortion metrics (SATD, some MSE/SATD hybrid, no-ref-aware, maybe table driven Theora-style, etc.)<br /> * Better MV split flag rate estimates<br /> <br /> = Known broken/suboptimal =<br /> * Block size decision<br /> ** Never tuned on inter<br /> ** Not considering chroma<br /> * Code quantizers on a log scale<br /> * Re-order bitstream (e.g., don't code all MVs at the front, etc.)<br /> * Investigate bias introduced by coefficient scaling<br /> <br /> = New work =<br /> <br /> * VQ for inter frames, with codebook selected based on dim m<br /> * Motion vector &quot;mask&quot; based on previous frame<br /> * Use coeff magnitude correlation for modelling<br /> * Per SB/MB/block/something quantizer changes<br /> * Variable Framerate support<br /> * Dynamic frame size changes (without keyframes)<br /> * SPIHT as k-tokenizer<br /> <br /> = Entropy coding =<br /> <br /> * Make the Laplace vector encoder (aka k-tokenizer) faster<br /> * Add &quot;skip-all remaining bands&quot; flag<br /> * Better encoding of the CfL sign<br /> * Add SIMD to the decoder search<br /> <br /> = Visualization =<br /> <br /> * Add ability to display decoder side information to player_example<br /> ** Block size split decision<br /> ** Cost in bits per block (use log scale)<br /> ** No-ref and skip flags<br /> ** Mode information (if we signal intra)<br /> ** Motion vectors<br /> ** Display prediction residual<br /> ** Show only one plane (Y, Cb, Cr)<br /> <br /> = Infrastructure =<br /> <br /> * Need a bugzilla instance for issue tracking</div>

Unlord https://wiki.xiph.org/index.php?title=DaalaTodo&diff=15028 2014-10-14T10:46:47Z

<p>Unlord: </p> <hr /> <div>= Simple Things =<br /> * Overflow checking for allocations in od_state_ref_imgs_init().<br /> <br /> = Tuning =<br /> * Quantization matrix<br /> * Beta<br /> * Inter band masking<br /> * PVQ RDO<br /> * Better MV cost estimates<br /> * Better MC distortion metrics (SATD, some MSE/SATD hybrid, no-ref-aware, maybe table driven Theora-style, etc.)<br /> * Better MV split flag rate estimates<br /> <br /> = Known broken/suboptimal =<br /> * Block size decision<br /> ** Never tuned on inter<br /> ** Not considering chroma<br /> * Code quantizers on a log scale<br /> * Re-order bitstream (e.g., don't code all MVs at the front, etc.)<br /> * Investigate bias introduced by coefficient scaling<br /> <br /> = New work =<br /> <br /> * VQ for inter frames, with codebook selected based on dim m<br /> * Motion vector &quot;mask&quot; based on previous frame<br /> * Use coeff magnitude correlation for modelling<br /> * Per SB/MB/block/something quantizer changes<br /> * Variable Framerate support<br /> * Dynamic frame size changes (without keyframes)<br /> * SPIHT as k-tokenizer<br /> <br /> = Entropy coding =<br /> <br /> * Make the Laplace vector encoder (aka k-tokenizer) faster<br /> * Add &quot;skip-all remaining bands&quot; flag<br /> * Better encoding of the CfL sign<br /> * Add SIMD to the decoder search<br /> <br /> = Visualization =<br /> <br /> * Add ability to display decoder side information to player_example<br /> ** Block size split decision<br /> ** Cost in bits per block (use log scale)<br /> ** No-ref and skip flags<br /> ** Mode information (if we signal intra)<br /> ** Motion vectors<br /> ** Display prediction residual<br /> ** Show only one plane (Y, Cb, Cr)</div>

Unlord https://wiki.xiph.org/index.php?title=Intra&diff=13861 2013-02-08T17:14:31Z

<p>Unlord: Adding results from 4x4 and 8x8 intra predictor training</p> <hr /> <div>In Daala, intra-frame predictors are coefficients that are used to predict the contents of a block based on its three neighboring blocks. Consider a 4x4 block structure:<br /> <br /> &lt;math&gt;<br /> \left(<br /> \begin{array}{cccc|cccc}<br /> x_{1}&amp;x_{2}&amp;x_{3}&amp;x_{4}&amp;x_{5}&amp;x_{6}&amp;x_{7}&amp;x_{8}\\<br /> x_{9}&amp;x_{10}&amp;x_{11}&amp;x_{12}&amp;x_{13}&amp;x_{14}&amp;x_{15}&amp;x_{16}\\<br /> x_{17}&amp;x_{18}&amp;x_{19}&amp;x_{20}&amp;x_{21}&amp;x_{22}&amp;x_{23}&amp;x_{24}\\<br /> x_{25}&amp;x_{26}&amp;x_{27}&amp;x_{28}&amp;x_{29}&amp;x_{30}&amp;x_{31}&amp;x_{32}\\ \hline<br /> x_{33}&amp;x_{34}&amp;x_{35}&amp;x_{36}&amp;y_{1}&amp;y_{2}&amp;y_{3}&amp;y_{4}\\<br /> x_{37}&amp;x_{38}&amp;x_{39}&amp;x_{40}&amp;y_{5}&amp;y_{6}&amp;y_{7}&amp;y_{8}\\<br /> x_{41}&amp;x_{42}&amp;x_{43}&amp;x_{44}&amp;y_{9}&amp;y_{10}&amp;y_{11}&amp;y_{12}\\<br /> x_{45}&amp;x_{46}&amp;x_{47}&amp;x_{48}&amp;y_{13}&amp;y_{14}&amp;y_{15}&amp;y_{16}\\<br /> \end{array}<br /> \right)<br /> &lt;/math&gt;<br /> <br /> Assuming a linear predictor, we would like to find the 768 coefficients that best predict &lt;math&gt;y_i&lt;/math&gt; from the neighboring &lt;math&gt;x_i&lt;/math&gt;. That is, for a given block with coefficients &lt;math&gt;\hat x&lt;/math&gt; and &lt;math&gt;\hat y&lt;/math&gt;, we would like a 48x16 matrix &lt;math&gt;\beta&lt;/math&gt; such that the residual &lt;math&gt;\hat e=\hat y-\hat x \beta&lt;/math&gt; is minimized (for some &lt;math&gt;p&lt;/math&gt;-norm). Because it is this residual &lt;math&gt;\hat e&lt;/math&gt; that is quantized and coded in the bitstream, we would prefer that most of its coefficients are zero. This will be discussed further when comparing the different &lt;math&gt;p&lt;/math&gt;-norms.<br /> <br /> == Block Modes ==<br /> <br /> These predictors often follow certain geometry in the time-domain image space. Imagine a vertical edge that runs through &lt;math&gt;y&lt;/math&gt;. A natural predictor might be to simply copy the values &lt;math&gt;\begin{array}{cccc}x_{29}&amp;x_{30}&amp;x_{31}&amp;x_{32}\end{array}&lt;/math&gt; down into each row of the 4x4 block &lt;math&gt;y&lt;/math&gt;. In order to account for the different possible geometries, each block is assigned a &lt;i&gt;mode&lt;/i&gt; which indicates what set of prediction coefficients to use (the &lt;math&gt;\beta&lt;/math&gt;). When encoding a block &lt;math&gt;y&lt;/math&gt;, the encoder chooses a mode, computes &lt;math&gt;\hat e&lt;/math&gt; and writes both the block mode and the residual &lt;math&gt;\hat e&lt;/math&gt;.<br /> <br /> Because the decoder must know the coefficients for each of the block modes, there is a tradeoff between how well we can predict the values in &lt;math&gt;y&lt;/math&gt; (number of block modes) with decoder complexity. Our hypothesis is that the optimal number of block modes will correspond to how closely we can fit different geometries inside the block. For 4x4 blocks, this might correspond to a mode for each of the 8 cardinal directions (ask Jason to clarify) with larger blocks potentially supporting a larger number of directions. In addition, we would like to support the DC and True Motion modes from WebM/VP8 as they are often the best fit. For common video sequences, anywhere from 20% to 45% of the intra frames use the True Motion mode [1].<br /> <br /> This is slightly complicated by the fact that our coefficients are not in the time-domain, but are the result of a lapped transform, see [[TDLT]]. In practice all of these modes have an equivalent in the frequency domain. As a starting point, we are using the 10 modes from Theora to classify the blocks from a set of sample images. Each category of blocks will be used to construct a set of predictors &lt;math&gt;\beta&lt;/math&gt; which is then used to reclassify the blocks based on Sum of Absolute Transform Differences (SATD). &lt;i&gt;Tim suggested weighting each blocks contribution based on SATD_bestfit-SATD_nearest&lt;/i&gt;.<br /> <br /> == &lt;math&gt;L^2&lt;/math&gt;-norm ==<br /> <br /> Suppose for some mode we have &lt;math&gt;n&lt;/math&gt; blocks. We can then construct the matrices &lt;math&gt;X&lt;/math&gt; and &lt;math&gt;Y&lt;/math&gt; where the ith row of each contains the zero-mean values from the ith block.<br /> <br /> &lt;math&gt;<br /> X=\begin{pmatrix}<br /> x_{1,1}-\bar{x_1} &amp; x_{1,2}-\bar{x_2} &amp; \cdots &amp; x_{1,m}-\bar{x_m}\\<br /> x_{2,1}-\bar{x_1} &amp; x_{2,2}-\bar{x_2} &amp; \cdots &amp; x_{2,m}-\bar{x_m}\\<br /> \vdots &amp; \vdots &amp; \ddots &amp; \vdots \\<br /> x_{n,1}-\bar{x_1} &amp; x_{n,2}-\bar{x_2} &amp; \cdots &amp; x_{n,m}-\bar{x_m}<br /> \end{pmatrix}<br /> &lt;/math&gt;<br /> <br /> &lt;math&gt;<br /> Y=\begin{pmatrix}<br /> y_{1,1}-\bar{y_1} &amp; y_{1,2}-\bar{y_2} &amp; \cdots &amp; y_{1,l}-\bar{y_l}\\<br /> y_{2,1}-\bar{y_1} &amp; y_{2,2}-\bar{y_2} &amp; \cdots &amp; y_{2,l}-\bar{y_l}\\<br /> \vdots &amp; \vdots &amp; \ddots &amp; \vdots \\<br /> y_{n,1}-\bar{y_1} &amp; y_{n,2}-\bar{y_2} &amp; \cdots &amp; y_{n,l}-\bar{y_l}<br /> \end{pmatrix}<br /> &lt;/math&gt;<br /> <br /> In the case of 4x4 blocks, &lt;math&gt;m=48&lt;/math&gt; and &lt;math&gt;l=16&lt;/math&gt;. Let &lt;math&gt;C=X^T X&lt;/math&gt; and &lt;math&gt;D=X^T Y&lt;/math&gt;. Then &lt;math&gt;\beta=C^{-1} D&lt;/math&gt;.<br /> <br /> == 4x4 Intra Predictors ==<br /> <br /> Using the set of 50 images in subset1-y4m to train sparse (4*4*4 = 64 multiplies per block) intra predictors:<br /> <br /> &lt;pre&gt;<br /> VP8 Intra Predictors<br /> Mode 0 Blocks 379965 SATD 1030.8 Bits 136.827 Mean 1725.16 Var 591085 CgRef 12.9338 CgPred 14.4075 Pg 1.47375<br /> Mode 1 Blocks 328826 SATD 1410.93 Bits 140.398 Mean 1755.67 Var 615743 CgRef 10.7723 CgPred 14.4396 Pg 3.66727<br /> Mode 2 Blocks 364320 SATD 1318.99 Bits 140.038 Mean 1771.33 Var 613037 CgRef 12.2597 CgPred 14.321 Pg 2.0613<br /> Mode 3 Blocks 461738 SATD 1266.24 Bits 138.956 Mean 1799.99 Var 581067 CgRef 12.4228 CgPred 14.4985 Pg 2.07569<br /> Mode 4 Blocks 317707 SATD 1320.78 Bits 140.709 Mean 1738.33 Var 615735 CgRef 11.4334 CgPred 13.989 Pg 2.55553<br /> Mode 5 Blocks 276650 SATD 1222.78 Bits 139.462 Mean 1691.97 Var 612193 CgRef 11.5441 CgPred 14.2428 Pg 2.69867<br /> Mode 6 Blocks 264186 SATD 1238.89 Bits 139.503 Mean 1664.18 Var 603871 CgRef 11.7692 CgPred 14.2162 Pg 2.44696<br /> Mode 7 Blocks 354682 SATD 1293.8 Bits 139.885 Mean 1654 Var 603002 CgRef 12.1736 CgPred 14.2243 Pg 2.05075<br /> Mode 8 Blocks 278080 SATD 1322.45 Bits 140.806 Mean 1706.37 Var 634358 CgRef 11.6018 CgPred 14.0524 Pg 2.4506<br /> Mode 9 Blocks 253209 SATD 1321.22 Bits 140.924 Mean 1722.95 Var 625202 CgRef 11.4638 CgPred 13.7727 Pg 2.30894<br /> Pooled Blocks 3279363 SATD 1270.74 Bits 139.871 Mean 1727.99 Var 619488 CgRef 11.4416 CgPred 14.2047 Pg 2.76305<br /> &lt;/pre&gt;<br /> <br /> &lt;pre&gt;<br /> Daala Intra Predictors<br /> Mode 0 Blocks 472906 SATD 1017.46 Bits 138.832 Mean 1728.18 Var 553398 CgRef 12.9443 CgPred 14.6488 Pg 1.7045<br /> Mode 1 Blocks 261910 SATD 1519.87 Bits 146.67 Mean 1779.45 Var 471328 CgRef 9.70601 CgPred 12.4896 Pg 2.78355<br /> Mode 2 Blocks 198050 SATD 1818.89 Bits 149.906 Mean 1778.28 Var 440574 CgRef 10.0855 CgPred 11.5142 Pg 1.42869<br /> Mode 3 Blocks 400658 SATD 1091.01 Bits 138.022 Mean 1765.68 Var 699810 CgRef 14.9146 CgPred 17.0632 Pg 2.14857<br /> Mode 4 Blocks 440888 SATD 1056.23 Bits 138.926 Mean 1768.35 Var 669611 CgRef 13.8652 CgPred 15.8732 Pg 2.008<br /> Mode 5 Blocks 123873 SATD 2646.11 Bits 158.928 Mean 1761.66 Var 348360 CgRef 6.56612 CgPred 7.681 Pg 1.11488<br /> Mode 6 Blocks 389669 SATD 1128.95 Bits 140.725 Mean 1607.85 Var 604298 CgRef 12.8995 CgPred 14.7494 Pg 1.84991<br /> Mode 7 Blocks 364446 SATD 1132.17 Bits 139.413 Mean 1649.46 Var 674713 CgRef 14.0093 CgPred 16.1988 Pg 2.18954<br /> Mode 8 Blocks 373419 SATD 1116.84 Bits 140.019 Mean 1720.85 Var 707029 CgRef 13.9352 CgPred 15.8526 Pg 1.91734<br /> Mode 9 Blocks 253544 SATD 1571.98 Bits 147.453 Mean 1797.04 Var 502237 CgRef 10.7593 CgPred 12.3135 Pg 1.55419<br /> Pooled Blocks 3279363 SATD 1261.89 Bits 142.641 Mean 1727.99 Var 619488 CgRef 12.3321 CgPred 14.4918 Pg 2.15973<br /> &lt;/pre&gt;<br /> <br /> Using the set of 1000 images in subset3-y4m to test the sparse (4*4*4 = 64 multiplies per block) intra predictors:<br /> <br /> &lt;pre&gt;<br /> VP8 Intra Predictors<br /> Mode 0 Blocks 8440618 SATD 773.069 Bits 129.891 Mean 1987.82 Var 714490 CgRef 16.1033 CgPred 17.7858 Pg 1.68249<br /> Mode 1 Blocks 7177017 SATD 1097.06 Bits 133.975 Mean 1835.81 Var 682211 CgRef 13.2692 CgPred 17.2354 Pg 3.96624<br /> Mode 2 Blocks 7486531 SATD 1129.14 Bits 136.141 Mean 1849.6 Var 699452 CgRef 14.1344 CgPred 16.3106 Pg 2.17612<br /> Mode 3 Blocks 9420648 SATD 1030.04 Bits 133.978 Mean 1856.55 Var 667016 CgRef 14.7014 CgPred 16.9182 Pg 2.21677<br /> Mode 4 Blocks 6537593 SATD 1074.28 Bits 135.633 Mean 1877.16 Var 703018 CgRef 13.7951 CgPred 16.4451 Pg 2.65005<br /> Mode 5 Blocks 5764375 SATD 1037.61 Bits 135.162 Mean 1862.93 Var 714729 CgRef 13.7467 CgPred 16.5772 Pg 2.83053<br /> Mode 6 Blocks 5526915 SATD 1047.34 Bits 135.235 Mean 1853.98 Var 725463 CgRef 13.9811 CgPred 16.5951 Pg 2.61399<br /> Mode 7 Blocks 7493442 SATD 1061.93 Bits 134.918 Mean 1797.8 Var 701592 CgRef 14.467 CgPred 16.6788 Pg 2.21183<br /> Mode 8 Blocks 5635636 SATD 1077.08 Bits 135.666 Mean 1850.12 Var 724307 CgRef 13.9346 CgPred 16.4812 Pg 2.54658<br /> Mode 9 Blocks 5148581 SATD 1100.52 Bits 136.283 Mean 1895.9 Var 726050 CgRef 13.7549 CgPred 16.1284 Pg 2.37353<br /> Pooled Blocks 68631356 SATD 1035.13 Bits 134.722 Mean 1868.07 Var 721805 CgRef 13.8553 CgPred 16.77 Pg 2.91471<br /> &lt;/pre&gt;<br /> <br /> &lt;pre&gt;<br /> Daala Intra Predictors<br /> Mode 0 Blocks 9675144 SATD 841.841 Bits 133.86 Mean 1877.93 Var 662343 CgRef 15.701 CgPred 17.4209 Pg 1.71986<br /> Mode 1 Blocks 4809122 SATD 1246.09 Bits 141.563 Mean 1764.73 Var 466747 CgRef 12.3822 CgPred 14.2618 Pg 1.8796<br /> Mode 2 Blocks 3895161 SATD 1620.32 Bits 146.753 Mean 1763.23 Var 408176 CgRef 10.9493 CgPred 12.402 Pg 1.45271<br /> Mode 3 Blocks 8512887 SATD 895.992 Bits 133.096 Mean 1838.33 Var 781431 CgRef 17.0033 CgPred 19.2762 Pg 2.27292<br /> Mode 4 Blocks 10224556 SATD 788.516 Bits 131.961 Mean 2073.27 Var 777684 CgRef 16.8937 CgPred 18.964 Pg 2.07024<br /> Mode 5 Blocks 2492854 SATD 2306.98 Bits 155.1 Mean 1781.56 Var 306540 CgRef 7.42632 CgPred 8.54354 Pg 1.11722<br /> Mode 6 Blocks 7839240 SATD 960.763 Bits 136.271 Mean 1723.23 Var 684064 CgRef 15.2766 CgPred 17.1139 Pg 1.83731<br /> Mode 7 Blocks 8127384 SATD 913.37 Bits 133.886 Mean 1865.64 Var 797013 CgRef 16.528 CgPred 18.8681 Pg 2.34004<br /> Mode 8 Blocks 8289385 SATD 856.357 Bits 133.506 Mean 1963.06 Var 814308 CgRef 16.8012 CgPred 18.6876 Pg 1.88641<br /> Mode 9 Blocks 4765623 SATD 1420.7 Bits 144.479 Mean 1773.33 Var 478656 CgRef 11.868 CgPred 13.3845 Pg 1.51648<br /> Pooled Blocks 68631356 SATD 1030.34 Bits 137.446 Mean 1868.07 Var 721805 CgRef 14.964 CgPred 17.0421 Pg 2.07809<br /> &lt;/pre&gt;<br /> <br /> == 8x8 Intra Predictors ==<br /> <br /> Using the set of 50 images in subset1-y4m to train sparse (4*8*8 = 256 multiplies per block) intra predictors:<br /> <br /> &lt;pre&gt;<br /> VP8 Intra Predictors<br /> Mode 0 Blocks 153189 SATD 6082.74 Bits 633.298 Mean 1720.67 Var 559647 CgRef 12.0272 CgPred 12.3392 Pg 0.311931<br /> Mode 1 Blocks 49244 SATD 6482.51 Bits 631.517 Mean 1752.33 Var 634379 CgRef 12.983 CgPred 14.5119 Pg 1.52891<br /> Mode 2 Blocks 81386 SATD 6317.27 Bits 633.888 Mean 1772.67 Var 617045 CgRef 13.2505 CgPred 13.8038 Pg 0.55328<br /> Mode 3 Blocks 149817 SATD 6142.2 Bits 629.138 Mean 1821.6 Var 577836 CgRef 13.4642 CgPred 14.0662 Pg 0.601936<br /> Mode 4 Blocks 59095 SATD 6488.09 Bits 639.403 Mean 1738.64 Var 635175 CgRef 12.8858 CgPred 13.546 Pg 0.660174<br /> Mode 5 Blocks 51138 SATD 6321.64 Bits 636.812 Mean 1680.52 Var 614762 CgRef 12.5191 CgPred 13.2898 Pg 0.770625<br /> Mode 6 Blocks 46860 SATD 6214.38 Bits 635.479 Mean 1636.45 Var 606974 CgRef 13.0011 CgPred 13.5919 Pg 0.590785<br /> Mode 7 Blocks 101899 SATD 6579.86 Bits 637.096 Mean 1621 Var 597451 CgRef 12.9896 CgPred 13.5883 Pg 0.598636<br /> Mode 8 Blocks 55425 SATD 6946.74 Bits 644.887 Mean 1714.81 Var 646845 CgRef 12.5421 CgPred 13.1933 Pg 0.651179<br /> Mode 9 Blocks 43833 SATD 6717.76 Bits 640.445 Mean 1738.18 Var 634206 CgRef 12.5208 CgPred 13.1502 Pg 0.629351<br /> Pooled Blocks 791886 SATD 6356.01 Bits 635.537 Mean 1728.58 Var 619153 CgRef 12.4832 CgPred 13.326 Pg 0.842891<br /> &lt;/pre&gt;<br /> <br /> &lt;pre&gt;Daala Intra Predictors<br /> Mode 0 Blocks 62290 SATD 8842.88 Bits 695.702 Mean 1762.94 Var 546310 CgRef 10.527 CgPred 11.2558 Pg 0.728816<br /> Mode 1 Blocks 51719 SATD 8525.04 Bits 687.553 Mean 1814.55 Var 507208 CgRef 10.9023 CgPred 11.7365 Pg 0.834129<br /> Mode 2 Blocks 36010 SATD 12451.8 Bits 728.279 Mean 1791.12 Var 384211 CgRef 7.39062 CgPred 7.865 Pg 0.474379<br /> Mode 3 Blocks 112574 SATD 5202.09 Bits 631.108 Mean 1783.42 Var 690506 CgRef 15.7576 CgPred 16.3784 Pg 0.620821<br /> Mode 4 Blocks 115633 SATD 5215.31 Bits 633.79 Mean 1769.83 Var 629411 CgRef 14.1115 CgPred 14.8214 Pg 0.709915<br /> Mode 5 Blocks 54821 SATD 9203.44 Bits 696.669 Mean 1744.5 Var 463757 CgRef 9.45701 CgPred 10.0708 Pg 0.613816<br /> Mode 6 Blocks 87676 SATD 5900.82 Bits 646.94 Mean 1578.28 Var 591176 CgRef 13.1847 CgPred 13.8014 Pg 0.616685<br /> Mode 7 Blocks 92133 SATD 5419.48 Bits 636.372 Mean 1634.71 Var 692905 CgRef 15.1441 CgPred 15.8044 Pg 0.660312<br /> Mode 8 Blocks 76551 SATD 6501.37 Bits 658.66 Mean 1736.97 Var 612091 CgRef 12.6443 CgPred 13.4359 Pg 0.791668<br /> Mode 9 Blocks 102479 SATD 5847.21 Bits 646.934 Mean 1733.7 Var 570790 CgRef 12.9057 CgPred 13.583 Pg 0.677333<br /> Pooled Blocks 791886 SATD 6625.85 Bits 659.058 Mean 1728.58 Var 619153 CgRef 12.6445 CgPred 13.4208 Pg 0.776311<br /> &lt;/pre&gt;<br /> <br /> Using the set of 1000 images in subset3-y4m to test the sparse (4*8*8 = 256 multiplies per block) intra predictors.<br /> <br /> &lt;pre&gt;<br /> VP8 Intra Predictors<br /> Mode 0 Blocks 3285321 SATD 4831.25 Bits 609.544 Mean 1942.13 Var 681922 CgRef 15.1468 CgPred 15.5036 Pg 0.356785<br /> Mode 1 Blocks 1097934 SATD 4966.25 Bits 600.994 Mean 1847.45 Var 682361 CgRef 16.1138 CgPred 17.8094 Pg 1.69564<br /> Mode 2 Blocks 1708487 SATD 5380.91 Bits 616.147 Mean 1869.28 Var 707993 CgRef 15.3892 CgPred 15.978 Pg 0.58875<br /> Mode 3 Blocks 3106373 SATD 5029.6 Bits 608.275 Mean 1879.34 Var 663156 CgRef 15.8358 CgPred 16.4784 Pg 0.642574<br /> Mode 4 Blocks 1208481 SATD 5328.85 Bits 618.814 Mean 1882.27 Var 709893 CgRef 15.2482 CgPred 15.8695 Pg 0.621321<br /> Mode 5 Blocks 1049455 SATD 5420.14 Bits 619.549 Mean 1855.61 Var 719825 CgRef 14.9456 CgPred 15.7082 Pg 0.762665<br /> Mode 6 Blocks 959802 SATD 5356.04 Bits 618.585 Mean 1832.64 Var 732730 CgRef 15.2935 CgPred 15.878 Pg 0.584477<br /> Mode 7 Blocks 2216886 SATD 5374.46 Bits 613.848 Mean 1782.23 Var 703311 CgRef 15.4919 CgPred 16.1339 Pg 0.64197<br /> Mode 8 Blocks 1089024 SATD 5614.08 Bits 620.635 Mean 1830.07 Var 723888 CgRef 15.031 CgPred 15.6673 Pg 0.636324<br /> Mode 9 Blocks 870162 SATD 5664.15 Bits 621.395 Mean 1877.86 Var 725235 CgRef 14.8465 CgPred 15.4287 Pg 0.582189<br /> Pooled Blocks 16591925 SATD 5205.41 Bits 613.921 Mean 1868.35 Var 720473 CgRef 15.1122 CgPred 15.9821 Pg 0.869956<br /> &lt;/pre&gt;<br /> <br /> &lt;pre&gt;<br /> Daala Intra Predictors<br /> Mode 0 Blocks 1179905 SATD 7866.76 Bits 682.077 Mean 1726.99 Var 519986 CgRef 12.0984 CgPred 12.4692 Pg 0.370783<br /> Mode 1 Blocks 1008452 SATD 7410.96 Bits 669.185 Mean 1815.4 Var 486451 CgRef 12.7369 CgPred 13.1166 Pg 0.379717<br /> Mode 2 Blocks 676403 SATD 11047.9 Bits 713.149 Mean 1806.12 Var 353773 CgRef 8.53864 CgPred 9.00691 Pg 0.46827<br /> Mode 3 Blocks 2540750 SATD 4183.79 Bits 607.011 Mean 1934.57 Var 794608 CgRef 18.4004 CgPred 19.0367 Pg 0.636302<br /> Mode 4 Blocks 2656446 SATD 3991.2 Bits 605.507 Mean 2030 Var 728302 CgRef 17.1992 CgPred 17.8889 Pg 0.68974<br /> Mode 5 Blocks 1011671 SATD 8495.58 Bits 687.746 Mean 1744.3 Var 449801 CgRef 10.8417 CgPred 11.4205 Pg 0.578879<br /> Mode 6 Blocks 1710619 SATD 5227.87 Bits 632.837 Mean 1648.81 Var 645320 CgRef 15.2445 CgPred 15.8622 Pg 0.617689<br /> Mode 7 Blocks 2352651 SATD 3996.86 Bits 601.981 Mean 2013.93 Var 842396 CgRef 18.4642 CgPred 19.1573 Pg 0.693167<br /> Mode 8 Blocks 1482503 SATD 5447.2 Bits 637.809 Mean 1810.08 Var 641515 CgRef 14.9565 CgPred 15.5448 Pg 0.588273<br /> Mode 9 Blocks 1972525 SATD 5184.71 Bits 633.59 Mean 1822.5 Var 633855 CgRef 14.9489 CgPred 15.5997 Pg 0.650804<br /> Pooled Blocks 16591925 SATD 5466.77 Bits 636.877 Mean 1868.35 Var 720473 CgRef 15.3899 CgPred 16.0271 Pg 0.63723<br /> &lt;/pre&gt;<br /> <br /> [1] http://blog.webmproject.org/2010/07/inside-webm-technology-vp8-intra-and.html</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13798 2012-12-17T10:15:12Z

<p>Unlord: Adding 8-bit 8x16 Type-IV dyadic</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -5/32&lt;br&gt;-0.15625<br /> | 18/32&lt;br&gt;0.5625<br /> | 46/32&lt;br&gt;1.4375<br /> | 42/32&lt;br&gt;1.3125<br /> | &amp;nbsp;&lt;br&gt;8.63409<br /> | &amp;nbsp;&lt;br&gt;22.5715<br /> | [[Image:4x8_5bit.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;8-bit<br /> | -32/256&lt;br&gt;-0.125<br /> | 162/256&lt;br&gt;0.6328125<br /> | 376/256&lt;br&gt;1.46875<br /> | 357/256&lt;br&gt;1.39453125<br /> | &amp;nbsp;&lt;br&gt;8.60104<br /> | &amp;nbsp;&lt;br&gt;21.4037<br /> | [[Image:4x8rd_8bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;7-bit<br /> | -32/128&lt;br&gt;-0.25<br /> | 82/128&lt;br&gt;0.640625<br /> | 184/128&lt;br&gt;1.4375<br /> | 186/128&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd_7bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -8/32&lt;br&gt;-0.25<br /> | 19/32&lt;br&gt;0.59375<br /> | 52/32&lt;br&gt;1.625<br /> | 47/32&lt;br&gt;1.46875<br /> | &amp;nbsp;&lt;br&gt;8.56068<br /> | &amp;nbsp;&lt;br&gt;20.3279<br /> | [[Image:4x8rd_5bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -12/32&lt;br&gt;-0.375<br /> | -9/32&lt;br&gt;-0.28125<br /> | -4/32&lt;br&gt;-0.125<br /> | 24/32&lt;br&gt;0.75<br /> | 17/32&lt;br&gt;0.53125<br /> | 10/32&lt;br&gt;<br /> | 45/32&lt;br&gt;1.40625<br /> | 37/32&lt;br&gt;1.15625<br /> | 36/32&lt;br&gt;<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.59946<br /> | [[Image:8x16_5bit.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;8-bit<br /> | -96/256&lt;br&gt;-0.375<br /> | -83/256&lt;br&gt;-0.32421875<br /> | -36/256&lt;br&gt;-0.140625<br /> | 210/256&lt;br&gt;0.8203125<br /> | 160/256&lt;br&gt;0.625<br /> | 104/256&lt;br&gt;0.40625<br /> | 368/256&lt;br&gt;1.4375<br /> | 302/256&lt;br&gt;1.1796875<br /> | 293/256&lt;br&gt;1.14453125<br /> | 317/256&lt;br&gt;1.23828125<br /> | &amp;nbsp;&lt;br&gt;9.56761<br /> | [[Image:8x16rd_8bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;7-bit<br /> | -48/128&lt;br&gt;-0.375<br /> | -45/128&lt;br&gt;-0.3515625<br /> | -16/128&lt;br&gt;-0.125<br /> | 105/128&lt;br&gt;0.8203125<br /> | 80/128&lt;br&gt;0.625<br /> | 53/128&lt;br&gt;0.4140625<br /> | 184/128&lt;br&gt;1.4375<br /> | 151/128&lt;br&gt;1.1796875<br /> | 147/128&lt;br&gt;1.1484375<br /> | 157/128&lt;br&gt;1.2265625<br /> | &amp;nbsp;&lt;br&gt;9.56672<br /> | [[Image:8x16rd_7bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -12/32&lt;br&gt;-0.375<br /> | -10/32&lt;br&gt;-0.3125<br /> | -2/32&lt;br&gt;-0.0625<br /> | 26/32&lt;br&gt;0.8125<br /> | 20/32&lt;br&gt;0.625<br /> | 12/32&lt;br&gt;0.375<br /> | 48/32&lt;br&gt;1.5<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.5596<br /> | [[Image:8x16rd_5bit.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -32/64&lt;br&gt;-0.5<br /> | -28/64&lt;br&gt;-0.4375<br /> | -24/64&lt;br&gt;-0.375<br /> | -32/64&lt;br&gt;-0.5<br /> | -24/64&lt;br&gt;-0.375<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 59/64&lt;br&gt;0.921875<br /> | 53/64&lt;br&gt;0.828125<br /> | 46/64&lt;br&gt;0.71875<br /> | 41/64&lt;br&gt;0.640625<br /> | 35/64&lt;br&gt;0.546875<br /> | 24/64&lt;br&gt;0.375<br /> | 12/64&lt;br&gt;0.1875<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | 74/64&lt;br&gt;1.15625<br /> | 74/64&lt;br&gt;1.15625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.78294<br /> | [[Image:16x32rd.png|64px]]<br /> |}<br /> <br /> == Type-IV Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !4x8 Ramp<br /> !8x16<br /> !8x16 Ramp<br /> !16x32<br /> !16x32 Ramp<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |8.60603<br /> |9.6005<br /> |9.56867<br /> |9.9057<br /> |9.81157<br /> |-<br /> |Dyadic (8-bit)<br /> |<br /> |8.60104<br /> |<br /> |9.56761<br /> |<br /> |<br /> |-<br /> |Loss<br /> |<br /> |0.00499<br /> |<br /> |0.00105<br /> |<br /> |<br /> |-<br /> |Dyadic (7-bit)<br /> |<br /> |8.59886<br /> |<br /> |9.56672<br /> |<br /> |<br /> |-<br /> |Loss<br /> |<br /> |0.00717<br /> |<br /> |0.00195<br /> |<br /> |<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |8.59886<br /> |9.60021<br /> |9.56161<br /> |9.89338<br /> |9.78294<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00717<br /> |0.00029<br /> |0.00706<br /> |0.01232<br /> |0.02863<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |8.56068<br /> |9.59946<br /> |9.5596<br /> |<br /> |<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.04535<br /> |0.00104<br /> |0.00907<br /> |<br /> |<br /> |}<br /> <br /> == 8x16 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -25/64&lt;br&gt;-0.390625<br /> | -20/64&lt;br&gt;-0.3125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 49/64&lt;br&gt;0.765625<br /> | 35/64&lt;br&gt;0.546875<br /> | 21/64&lt;br&gt;0.328125<br /> | 90/64&lt;br&gt;1.40625<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.6112<br /> | [[Image:8x16_type3.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -13/32&lt;br&gt;-0.40625<br /> | -11/32&lt;br&gt;-0.34375<br /> | -4/32&lt;br&gt;-0.125<br /> | 25/32&lt;br&gt;0.78125<br /> | 18/32&lt;br&gt;0.5625<br /> | 11/32&lt;br&gt;0.34375<br /> | 45/32&lt;br&gt;1.40625<br /> | 36/32&lt;br&gt;1.125<br /> | 36/32&lt;br&gt;1.125<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.61048<br /> | [[Image:8x16_type3_5bit.png|64px]]<br /> |}<br /> <br /> == 16x32 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -30/64&lt;br&gt;-0.46875<br /> | -35/64&lt;br&gt;-0.546875<br /> | -31/64&lt;br&gt;-0.484375<br /> | -29/64&lt;br&gt;-0.453125<br /> | -25/64&lt;br&gt;-0.390625<br /> | -19/64&lt;br&gt;-0.296875<br /> | -10/64&lt;br&gt;-0.15625<br /> | 54/64&lt;br&gt;0.84375<br /> | 45/64&lt;br&gt;0.703125<br /> | 40/64&lt;br&gt;0.625<br /> | 36/64&lt;br&gt;0.5625<br /> | 32/64&lt;br&gt;0.5<br /> | 25/64&lt;br&gt;0.390625<br /> | 17/64&lt;br&gt;0.265625<br /> | 90/64&lt;br&gt;1.40625<br /> | 70/64&lt;br&gt;1.09375<br /> | 69/64&lt;br&gt;1.078125<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 68/64&lt;br&gt;1.0625<br /> | 74/64&lt;br&gt;1.15625<br /> | &amp;nbsp;&lt;br&gt;9.94127<br /> | [[Image:16x32_type3_6bit.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -15/32&lt;br&gt;-0.46875<br /> | -17/32&lt;br&gt;-0.53125<br /> | -14/32&lt;br&gt;-0.4375<br /> | -12/32&lt;br&gt;-0.375<br /> | -9/32&lt;br&gt;-0.28125<br /> | -5/32&lt;br&gt;-0.15625<br /> | 0/32&lt;br&gt;0.0<br /> | 27/32&lt;br&gt;0.84375<br /> | 23/32&lt;br&gt;0.71875<br /> | 20/32&lt;br&gt;0.625<br /> | 17/32&lt;br&gt;0.53125<br /> | 14/32&lt;br&gt;0.4375<br /> | 9/32&lt;br&gt;0.28125<br /> | 3/32&lt;br&gt;0.09375<br /> | 45/32&lt;br&gt;1.40625<br /> | 35/32&lt;br&gt;1.09375<br /> | 35/32&lt;br&gt;1.09375<br /> | 34/32&lt;br&gt;1.0625<br /> | 34/32&lt;br&gt;1.0625<br /> | 35/32&lt;br&gt;1.09375<br /> | 36/32&lt;br&gt;1.125<br /> | 33/32&lt;br&gt;1.03125<br /> | &amp;nbsp;&lt;br&gt;9.93998<br /> | [[Image:16x32_type3_5bit.png|64px]]<br /> |}<br /> <br /> == Type-III Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !8x16<br /> !16x32<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |9.6115<br /> |9.9496<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |9.6112<br /> |9.94127<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00030<br /> |0.00833<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |9.61048<br /> |9.93998<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.00102<br /> |0.00962<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13780 2012-11-26T21:42:27Z

<p>Unlord: Updated 5-bit 8x16 Type-III</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -5/32&lt;br&gt;-0.15625<br /> | 18/32&lt;br&gt;0.5625<br /> | 46/32&lt;br&gt;1.4375<br /> | 42/32&lt;br&gt;1.3125<br /> | &amp;nbsp;&lt;br&gt;8.63409<br /> | &amp;nbsp;&lt;br&gt;22.5715<br /> | [[Image:4x8_5bit.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;8-bit<br /> | -32/256&lt;br&gt;-0.125<br /> | 162/256&lt;br&gt;0.6328125<br /> | 376/256&lt;br&gt;1.46875<br /> | 357/256&lt;br&gt;1.39453125<br /> | &amp;nbsp;&lt;br&gt;8.60104<br /> | &amp;nbsp;&lt;br&gt;21.4037<br /> | [[Image:4x8rd_8bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;7-bit<br /> | -32/128&lt;br&gt;-0.25<br /> | 82/128&lt;br&gt;0.640625<br /> | 184/128&lt;br&gt;1.4375<br /> | 186/128&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd_7bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -8/32&lt;br&gt;-0.25<br /> | 19/32&lt;br&gt;0.59375<br /> | 52/32&lt;br&gt;1.625<br /> | 47/32&lt;br&gt;1.46875<br /> | &amp;nbsp;&lt;br&gt;8.56068<br /> | &amp;nbsp;&lt;br&gt;20.3279<br /> | [[Image:4x8rd_5bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -12/32&lt;br&gt;-0.375<br /> | -9/32&lt;br&gt;-0.28125<br /> | -4/32&lt;br&gt;-0.125<br /> | 24/32&lt;br&gt;0.75<br /> | 17/32&lt;br&gt;0.53125<br /> | 10/32&lt;br&gt;<br /> | 45/32&lt;br&gt;1.40625<br /> | 37/32&lt;br&gt;1.15625<br /> | 36/32&lt;br&gt;<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.59946<br /> | [[Image:8x16_5bit.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;7-bit<br /> | -48/128&lt;br&gt;-0.375<br /> | -45/128&lt;br&gt;-0.3515625<br /> | -16/128&lt;br&gt;-0.125<br /> | 105/128&lt;br&gt;0.8203125<br /> | 80/128&lt;br&gt;0.625<br /> | 53/128&lt;br&gt;0.4140625<br /> | 184/128&lt;br&gt;1.4375<br /> | 151/128&lt;br&gt;1.1796875<br /> | 147/128&lt;br&gt;1.1484375<br /> | 157/128&lt;br&gt;1.2265625<br /> | &amp;nbsp;&lt;br&gt;9.56672<br /> | [[Image:8x16rd_7bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -12/32&lt;br&gt;-0.375<br /> | -10/32&lt;br&gt;-0.3125<br /> | -2/32&lt;br&gt;-0.0625<br /> | 26/32&lt;br&gt;0.8125<br /> | 20/32&lt;br&gt;0.625<br /> | 12/32&lt;br&gt;0.375<br /> | 48/32&lt;br&gt;1.5<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.5596<br /> | [[Image:8x16rd_5bit.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -32/64&lt;br&gt;-0.5<br /> | -28/64&lt;br&gt;-0.4375<br /> | -24/64&lt;br&gt;-0.375<br /> | -32/64&lt;br&gt;-0.5<br /> | -24/64&lt;br&gt;-0.375<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 59/64&lt;br&gt;0.921875<br /> | 53/64&lt;br&gt;0.828125<br /> | 46/64&lt;br&gt;0.71875<br /> | 41/64&lt;br&gt;0.640625<br /> | 35/64&lt;br&gt;0.546875<br /> | 24/64&lt;br&gt;0.375<br /> | 12/64&lt;br&gt;0.1875<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | 74/64&lt;br&gt;1.15625<br /> | 74/64&lt;br&gt;1.15625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.78294<br /> | [[Image:16x32rd.png|64px]]<br /> |}<br /> <br /> == Type-IV Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !4x8 Ramp<br /> !8x16<br /> !8x16 Ramp<br /> !16x32<br /> !16x32 Ramp<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |8.60603<br /> |9.6005<br /> |9.56867<br /> |9.9057<br /> |9.81157<br /> |-<br /> |Dyadic (8-bit)<br /> |<br /> |8.60104<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Loss<br /> |<br /> |0.00499<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Dyadic (7-bit)<br /> |<br /> |8.59886<br /> |<br /> |9.56672<br /> |<br /> |<br /> |-<br /> |Loss<br /> |<br /> |0.00717<br /> |<br /> |0.00195<br /> |<br /> |<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |8.59886<br /> |9.60021<br /> |9.56161<br /> |9.89338<br /> |9.78294<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00717<br /> |0.00029<br /> |0.00706<br /> |0.01232<br /> |0.02863<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |8.56068<br /> |9.59946<br /> |9.5596<br /> |<br /> |<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.04535<br /> |0.00104<br /> |0.00907<br /> |<br /> |<br /> |}<br /> <br /> == 8x16 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -25/64&lt;br&gt;-0.390625<br /> | -20/64&lt;br&gt;-0.3125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 49/64&lt;br&gt;0.765625<br /> | 35/64&lt;br&gt;0.546875<br /> | 21/64&lt;br&gt;0.328125<br /> | 90/64&lt;br&gt;1.40625<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.6112<br /> | [[Image:8x16_type3.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -13/32&lt;br&gt;-0.40625<br /> | -11/32&lt;br&gt;-0.34375<br /> | -4/32&lt;br&gt;-0.125<br /> | 25/32&lt;br&gt;0.78125<br /> | 18/32&lt;br&gt;0.5625<br /> | 11/32&lt;br&gt;0.34375<br /> | 45/32&lt;br&gt;1.40625<br /> | 36/32&lt;br&gt;1.125<br /> | 36/32&lt;br&gt;1.125<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.61048<br /> | [[Image:8x16_type3_5bit.png|64px]]<br /> |}<br /> <br /> == 16x32 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -30/64&lt;br&gt;-0.46875<br /> | -35/64&lt;br&gt;-0.546875<br /> | -31/64&lt;br&gt;-0.484375<br /> | -29/64&lt;br&gt;-0.453125<br /> | -25/64&lt;br&gt;-0.390625<br /> | -19/64&lt;br&gt;-0.296875<br /> | -10/64&lt;br&gt;-0.15625<br /> | 54/64&lt;br&gt;0.84375<br /> | 45/64&lt;br&gt;0.703125<br /> | 40/64&lt;br&gt;0.625<br /> | 36/64&lt;br&gt;0.5625<br /> | 32/64&lt;br&gt;0.5<br /> | 25/64&lt;br&gt;0.390625<br /> | 17/64&lt;br&gt;0.265625<br /> | 90/64&lt;br&gt;1.40625<br /> | 70/64&lt;br&gt;1.09375<br /> | 69/64&lt;br&gt;1.078125<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 68/64&lt;br&gt;1.0625<br /> | 74/64&lt;br&gt;1.15625<br /> | &amp;nbsp;&lt;br&gt;9.94127<br /> | [[Image:16x32_type3_6bit.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -15/32&lt;br&gt;-0.46875<br /> | -17/32&lt;br&gt;-0.53125<br /> | -14/32&lt;br&gt;-0.4375<br /> | -12/32&lt;br&gt;-0.375<br /> | -9/32&lt;br&gt;-0.28125<br /> | -5/32&lt;br&gt;-0.15625<br /> | 0/32&lt;br&gt;0.0<br /> | 27/32&lt;br&gt;0.84375<br /> | 23/32&lt;br&gt;0.71875<br /> | 20/32&lt;br&gt;0.625<br /> | 17/32&lt;br&gt;0.53125<br /> | 14/32&lt;br&gt;0.4375<br /> | 9/32&lt;br&gt;0.28125<br /> | 3/32&lt;br&gt;0.09375<br /> | 45/32&lt;br&gt;1.40625<br /> | 35/32&lt;br&gt;1.09375<br /> | 35/32&lt;br&gt;1.09375<br /> | 34/32&lt;br&gt;1.0625<br /> | 34/32&lt;br&gt;1.0625<br /> | 35/32&lt;br&gt;1.09375<br /> | 36/32&lt;br&gt;1.125<br /> | 33/32&lt;br&gt;1.03125<br /> | &amp;nbsp;&lt;br&gt;9.93998<br /> | [[Image:16x32_type3_5bit.png|64px]]<br /> |}<br /> <br /> == Type-III Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !8x16<br /> !16x32<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |9.6115<br /> |9.9496<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |9.6112<br /> |9.94127<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00030<br /> |0.00833<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |9.61048<br /> |9.93998<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.00102<br /> |0.00962<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13773 2012-11-17T20:05:51Z

<p>Unlord: Updated Type-III 16x32 5-bit</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -5/32&lt;br&gt;-0.15625<br /> | 18/32&lt;br&gt;0.5625<br /> | 46/32&lt;br&gt;1.4375<br /> | 42/32&lt;br&gt;1.3125<br /> | &amp;nbsp;&lt;br&gt;8.63409<br /> | &amp;nbsp;&lt;br&gt;22.5715<br /> | [[Image:4x8_5bit.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;8-bit<br /> | -32/256&lt;br&gt;-0.125<br /> | 162/256&lt;br&gt;0.6328125<br /> | 376/256&lt;br&gt;1.46875<br /> | 357/256&lt;br&gt;1.39453125<br /> | &amp;nbsp;&lt;br&gt;8.60104<br /> | &amp;nbsp;&lt;br&gt;21.4037<br /> | [[Image:4x8rd_8bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;7-bit<br /> | -32/128&lt;br&gt;-0.25<br /> | 82/128&lt;br&gt;0.640625<br /> | 184/128&lt;br&gt;1.4375<br /> | 186/128&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd_7bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -8/32&lt;br&gt;-0.25<br /> | 19/32&lt;br&gt;0.59375<br /> | 52/32&lt;br&gt;1.625<br /> | 47/32&lt;br&gt;1.46875<br /> | &amp;nbsp;&lt;br&gt;8.56068<br /> | &amp;nbsp;&lt;br&gt;20.3279<br /> | [[Image:4x8rd_5bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -12/32&lt;br&gt;-0.375<br /> | -9/32&lt;br&gt;-0.28125<br /> | -4/32&lt;br&gt;-0.125<br /> | 24/32&lt;br&gt;0.75<br /> | 17/32&lt;br&gt;0.53125<br /> | 10/32&lt;br&gt;<br /> | 45/32&lt;br&gt;1.40625<br /> | 37/32&lt;br&gt;1.15625<br /> | 36/32&lt;br&gt;<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.59946<br /> | [[Image:8x16_5bit.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;7-bit<br /> | -48/128&lt;br&gt;-0.375<br /> | -45/128&lt;br&gt;-0.3515625<br /> | -16/128&lt;br&gt;-0.125<br /> | 105/128&lt;br&gt;0.8203125<br /> | 80/128&lt;br&gt;0.625<br /> | 53/128&lt;br&gt;0.4140625<br /> | 184/128&lt;br&gt;1.4375<br /> | 151/128&lt;br&gt;1.1796875<br /> | 147/128&lt;br&gt;1.1484375<br /> | 157/128&lt;br&gt;1.2265625<br /> | &amp;nbsp;&lt;br&gt;9.56672<br /> | [[Image:8x16rd_7bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -12/32&lt;br&gt;-0.375<br /> | -10/32&lt;br&gt;-0.3125<br /> | -2/32&lt;br&gt;-0.0625<br /> | 26/32&lt;br&gt;0.8125<br /> | 20/32&lt;br&gt;0.625<br /> | 12/32&lt;br&gt;0.375<br /> | 48/32&lt;br&gt;1.5<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.5596<br /> | [[Image:8x16rd_5bit.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -32/64&lt;br&gt;-0.5<br /> | -28/64&lt;br&gt;-0.4375<br /> | -24/64&lt;br&gt;-0.375<br /> | -32/64&lt;br&gt;-0.5<br /> | -24/64&lt;br&gt;-0.375<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 59/64&lt;br&gt;0.921875<br /> | 53/64&lt;br&gt;0.828125<br /> | 46/64&lt;br&gt;0.71875<br /> | 41/64&lt;br&gt;0.640625<br /> | 35/64&lt;br&gt;0.546875<br /> | 24/64&lt;br&gt;0.375<br /> | 12/64&lt;br&gt;0.1875<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | 74/64&lt;br&gt;1.15625<br /> | 74/64&lt;br&gt;1.15625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.78294<br /> | [[Image:16x32rd.png|64px]]<br /> |}<br /> <br /> == Type-IV Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !4x8 Ramp<br /> !8x16<br /> !8x16 Ramp<br /> !16x32<br /> !16x32 Ramp<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |8.60603<br /> |9.6005<br /> |9.56867<br /> |9.9057<br /> |9.81157<br /> |-<br /> |Dyadic (8-bit)<br /> |<br /> |8.60104<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Loss<br /> |<br /> |0.00499<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Dyadic (7-bit)<br /> |<br /> |8.59886<br /> |<br /> |9.56672<br /> |<br /> |<br /> |-<br /> |Loss<br /> |<br /> |0.00717<br /> |<br /> |0.00195<br /> |<br /> |<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |8.59886<br /> |9.60021<br /> |9.56161<br /> |9.89338<br /> |9.78294<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00717<br /> |0.00029<br /> |0.00706<br /> |0.01232<br /> |0.02863<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |8.56068<br /> |9.59946<br /> |9.5596<br /> |<br /> |<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.04535<br /> |0.00104<br /> |0.00907<br /> |<br /> |<br /> |}<br /> <br /> == 8x16 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -25/64&lt;br&gt;-0.390625<br /> | -20/64&lt;br&gt;-0.3125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 49/64&lt;br&gt;0.765625<br /> | 35/64&lt;br&gt;0.546875<br /> | 21/64&lt;br&gt;0.328125<br /> | 90/64&lt;br&gt;1.40625<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.6112<br /> | [[Image:8x16_type3.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -13/32&lt;br&gt;-0.40625<br /> | -11/32&lt;br&gt;-0.34375<br /> | -4/32&lt;br&gt;-0.125<br /> | 25/32&lt;br&gt;0.78125<br /> | 18/32&lt;br&gt;0.5625<br /> | 11/32&lt;br&gt;0.34375<br /> | 45/32&lt;br&gt;1.40625<br /> | 36/32&lt;br&gt;1.125<br /> | 36/32&lt;br&gt;1.125<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.61048<br /> | [[Image:8x16_type3_5bit.png|64px]]<br /> |}<br /> <br /> == 16x32 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -30/64&lt;br&gt;-0.46875<br /> | -35/64&lt;br&gt;-0.546875<br /> | -31/64&lt;br&gt;-0.484375<br /> | -29/64&lt;br&gt;-0.453125<br /> | -25/64&lt;br&gt;-0.390625<br /> | -19/64&lt;br&gt;-0.296875<br /> | -10/64&lt;br&gt;-0.15625<br /> | 54/64&lt;br&gt;0.84375<br /> | 45/64&lt;br&gt;0.703125<br /> | 40/64&lt;br&gt;0.625<br /> | 36/64&lt;br&gt;0.5625<br /> | 32/64&lt;br&gt;0.5<br /> | 25/64&lt;br&gt;0.390625<br /> | 17/64&lt;br&gt;0.265625<br /> | 90/64&lt;br&gt;1.40625<br /> | 70/64&lt;br&gt;1.09375<br /> | 69/64&lt;br&gt;1.078125<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 68/64&lt;br&gt;1.0625<br /> | 74/64&lt;br&gt;1.15625<br /> | &amp;nbsp;&lt;br&gt;9.94127<br /> | [[Image:16x32_type3_6bit.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -15/32&lt;br&gt;-0.46875<br /> | -17/32&lt;br&gt;-0.53125<br /> | -14/32&lt;br&gt;-0.4375<br /> | -12/32&lt;br&gt;-0.375<br /> | -9/32&lt;br&gt;-0.28125<br /> | -4/32&lt;br&gt;-0.125<br /> | 0/32&lt;br&gt;0.0<br /> | 27/32&lt;br&gt;0.84375<br /> | 23/32&lt;br&gt;0.71875<br /> | 20/32&lt;br&gt;0.625<br /> | 17/32&lt;br&gt;0.53125<br /> | 14/32&lt;br&gt;0.4375<br /> | 9/32&lt;br&gt;0.28125<br /> | 2/32&lt;br&gt;0.0625<br /> | 45/32&lt;br&gt;1.40625<br /> | 35/32&lt;br&gt;1.09375<br /> | 35/32&lt;br&gt;1.09375<br /> | 34/32&lt;br&gt;1.0625<br /> | 34/32&lt;br&gt;1.0625<br /> | 35/32&lt;br&gt;1.09375<br /> | 36/32&lt;br&gt;1.125<br /> | 33/32&lt;br&gt;1.03125<br /> | &amp;nbsp;&lt;br&gt;9.93815<br /> | [[Image:16x32_type3_5bit.png|64px]]<br /> |}<br /> <br /> == Type-III Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !8x16<br /> !16x32<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |9.6115<br /> |9.9496<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |9.6112<br /> |9.94127<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00030<br /> |0.00833<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |9.61048<br /> |9.93815<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.00102<br /> |0.01145<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13772 2012-11-17T19:39:23Z

<p>Unlord: Updating Type-IV 8x16 5-bit</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -5/32&lt;br&gt;-0.15625<br /> | 18/32&lt;br&gt;0.5625<br /> | 46/32&lt;br&gt;1.4375<br /> | 42/32&lt;br&gt;1.3125<br /> | &amp;nbsp;&lt;br&gt;8.63409<br /> | &amp;nbsp;&lt;br&gt;22.5715<br /> | [[Image:4x8_5bit.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;8-bit<br /> | -32/256&lt;br&gt;-0.125<br /> | 162/256&lt;br&gt;0.6328125<br /> | 376/256&lt;br&gt;1.46875<br /> | 357/256&lt;br&gt;1.39453125<br /> | &amp;nbsp;&lt;br&gt;8.60104<br /> | &amp;nbsp;&lt;br&gt;21.4037<br /> | [[Image:4x8rd_8bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;7-bit<br /> | -32/128&lt;br&gt;-0.25<br /> | 82/128&lt;br&gt;0.640625<br /> | 184/128&lt;br&gt;1.4375<br /> | 186/128&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd_7bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -8/32&lt;br&gt;-0.25<br /> | 19/32&lt;br&gt;0.59375<br /> | 52/32&lt;br&gt;1.625<br /> | 47/32&lt;br&gt;1.46875<br /> | &amp;nbsp;&lt;br&gt;8.56068<br /> | &amp;nbsp;&lt;br&gt;20.3279<br /> | [[Image:4x8rd_5bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -12/32&lt;br&gt;-0.375<br /> | -9/32&lt;br&gt;-0.28125<br /> | -4/32&lt;br&gt;-0.125<br /> | 24/32&lt;br&gt;0.75<br /> | 17/32&lt;br&gt;0.53125<br /> | 10/32&lt;br&gt;<br /> | 45/32&lt;br&gt;1.40625<br /> | 37/32&lt;br&gt;1.15625<br /> | 36/32&lt;br&gt;<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.59946<br /> | [[Image:8x16_5bit.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;7-bit<br /> | -48/128&lt;br&gt;-0.375<br /> | -45/128&lt;br&gt;-0.3515625<br /> | -16/128&lt;br&gt;-0.125<br /> | 105/128&lt;br&gt;0.8203125<br /> | 80/128&lt;br&gt;0.625<br /> | 53/128&lt;br&gt;0.4140625<br /> | 184/128&lt;br&gt;1.4375<br /> | 151/128&lt;br&gt;1.1796875<br /> | 147/128&lt;br&gt;1.1484375<br /> | 157/128&lt;br&gt;1.2265625<br /> | &amp;nbsp;&lt;br&gt;9.56672<br /> | [[Image:8x16rd_7bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -12/32&lt;br&gt;-0.375<br /> | -10/32&lt;br&gt;-0.3125<br /> | -2/32&lt;br&gt;-0.0625<br /> | 26/32&lt;br&gt;0.8125<br /> | 20/32&lt;br&gt;0.625<br /> | 12/32&lt;br&gt;0.375<br /> | 48/32&lt;br&gt;1.5<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.5596<br /> | [[Image:8x16rd_5bit.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -32/64&lt;br&gt;-0.5<br /> | -28/64&lt;br&gt;-0.4375<br /> | -24/64&lt;br&gt;-0.375<br /> | -32/64&lt;br&gt;-0.5<br /> | -24/64&lt;br&gt;-0.375<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 59/64&lt;br&gt;0.921875<br /> | 53/64&lt;br&gt;0.828125<br /> | 46/64&lt;br&gt;0.71875<br /> | 41/64&lt;br&gt;0.640625<br /> | 35/64&lt;br&gt;0.546875<br /> | 24/64&lt;br&gt;0.375<br /> | 12/64&lt;br&gt;0.1875<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | 74/64&lt;br&gt;1.15625<br /> | 74/64&lt;br&gt;1.15625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.78294<br /> | [[Image:16x32rd.png|64px]]<br /> |}<br /> <br /> == Type-IV Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !4x8 Ramp<br /> !8x16<br /> !8x16 Ramp<br /> !16x32<br /> !16x32 Ramp<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |8.60603<br /> |9.6005<br /> |9.56867<br /> |9.9057<br /> |9.81157<br /> |-<br /> |Dyadic (8-bit)<br /> |<br /> |8.60104<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Loss<br /> |<br /> |0.00499<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Dyadic (7-bit)<br /> |<br /> |8.59886<br /> |<br /> |9.56672<br /> |<br /> |<br /> |-<br /> |Loss<br /> |<br /> |0.00717<br /> |<br /> |0.00195<br /> |<br /> |<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |8.59886<br /> |9.60021<br /> |9.56161<br /> |9.89338<br /> |9.78294<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00717<br /> |0.00029<br /> |0.00706<br /> |0.01232<br /> |0.02863<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |8.56068<br /> |9.59946<br /> |9.5596<br /> |<br /> |<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.04535<br /> |0.00104<br /> |0.00907<br /> |<br /> |<br /> |}<br /> <br /> == 8x16 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -25/64&lt;br&gt;-0.390625<br /> | -20/64&lt;br&gt;-0.3125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 49/64&lt;br&gt;0.765625<br /> | 35/64&lt;br&gt;0.546875<br /> | 21/64&lt;br&gt;0.328125<br /> | 90/64&lt;br&gt;1.40625<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.6112<br /> | [[Image:8x16_type3.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -13/32&lt;br&gt;-0.40625<br /> | -11/32&lt;br&gt;-0.34375<br /> | -4/32&lt;br&gt;-0.125<br /> | 25/32&lt;br&gt;0.78125<br /> | 18/32&lt;br&gt;0.5625<br /> | 11/32&lt;br&gt;0.34375<br /> | 45/32&lt;br&gt;1.40625<br /> | 36/32&lt;br&gt;1.125<br /> | 36/32&lt;br&gt;1.125<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.61048<br /> | [[Image:8x16_type3_5bit.png|64px]]<br /> |}<br /> <br /> == 16x32 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -30/64&lt;br&gt;-0.46875<br /> | -35/64&lt;br&gt;-0.546875<br /> | -31/64&lt;br&gt;-0.484375<br /> | -29/64&lt;br&gt;-0.453125<br /> | -25/64&lt;br&gt;-0.390625<br /> | -19/64&lt;br&gt;-0.296875<br /> | -10/64&lt;br&gt;-0.15625<br /> | 54/64&lt;br&gt;0.84375<br /> | 45/64&lt;br&gt;0.703125<br /> | 40/64&lt;br&gt;0.625<br /> | 36/64&lt;br&gt;0.5625<br /> | 32/64&lt;br&gt;0.5<br /> | 25/64&lt;br&gt;0.390625<br /> | 17/64&lt;br&gt;0.265625<br /> | 90/64&lt;br&gt;1.40625<br /> | 70/64&lt;br&gt;1.09375<br /> | 69/64&lt;br&gt;1.078125<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 68/64&lt;br&gt;1.0625<br /> | 74/64&lt;br&gt;1.15625<br /> | &amp;nbsp;&lt;br&gt;9.94127<br /> | [[Image:16x32_type3_6bit.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -14/32&lt;br&gt;-0.4375<br /> | -17/32&lt;br&gt;-0.53125<br /> | -14/32&lt;br&gt;-0.4375<br /> | -12/32&lt;br&gt;-0.375<br /> | -9/32&lt;br&gt;-0.28125<br /> | -4/32&lt;br&gt;-0.125<br /> | 1/32&lt;br&gt;0.03125<br /> | 27/32&lt;br&gt;0.84375<br /> | 23/32&lt;br&gt;0.71875<br /> | 20/32&lt;br&gt;0.625<br /> | 17/32&lt;br&gt;0.53125<br /> | 14/32&lt;br&gt;0.4375<br /> | 9/32&lt;br&gt;0.28125<br /> | 2/32&lt;br&gt;0.0625<br /> | 46/32&lt;br&gt;1.4375<br /> | 35/32&lt;br&gt;1.09375<br /> | 35/32&lt;br&gt;1.09375<br /> | 34/32&lt;br&gt;1.0625<br /> | 34/32&lt;br&gt;1.0625<br /> | 35/32&lt;br&gt;1.09375<br /> | 36/32&lt;br&gt;1.125<br /> | 33/32&lt;br&gt;1.03125<br /> | &amp;nbsp;&lt;br&gt;9.93502<br /> | [[Image:16x32_type3_5bit.png|64px]]<br /> |}<br /> <br /> == Type-III Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !8x16<br /> !16x32<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |9.6115<br /> |9.9496<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |9.6112<br /> |9.94127<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00030<br /> |0.00833<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |9.61048<br /> |9.93502<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.00102<br /> |0.01458<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13768 2012-11-15T19:47:41Z

<p>Unlord: Updating 5-bit 16x32 Type-III parameters</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -5/32&lt;br&gt;-0.15625<br /> | 18/32&lt;br&gt;0.5625<br /> | 46/32&lt;br&gt;1.4375<br /> | 42/32&lt;br&gt;1.3125<br /> | &amp;nbsp;&lt;br&gt;8.63409<br /> | &amp;nbsp;&lt;br&gt;22.5715<br /> | [[Image:4x8_5bit.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;8-bit<br /> | -32/256&lt;br&gt;-0.125<br /> | 162/256&lt;br&gt;0.6328125<br /> | 376/256&lt;br&gt;1.46875<br /> | 357/256&lt;br&gt;1.39453125<br /> | &amp;nbsp;&lt;br&gt;8.60104<br /> | &amp;nbsp;&lt;br&gt;21.4037<br /> | [[Image:4x8rd_8bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;7-bit<br /> | -32/128&lt;br&gt;-0.25<br /> | 82/128&lt;br&gt;0.640625<br /> | 184/128&lt;br&gt;1.4375<br /> | 186/128&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd_7bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -8/32&lt;br&gt;-0.25<br /> | 19/32&lt;br&gt;0.59375<br /> | 52/32&lt;br&gt;1.625<br /> | 47/32&lt;br&gt;1.46875<br /> | &amp;nbsp;&lt;br&gt;8.56068<br /> | &amp;nbsp;&lt;br&gt;20.3279<br /> | [[Image:4x8rd_5bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -12/32&lt;br&gt;-0.375<br /> | -9/32&lt;br&gt;-0.28125<br /> | -4/32&lt;br&gt;-0.125<br /> | 24/32&lt;br&gt;0.75<br /> | 17/32&lt;br&gt;0.53125<br /> | 11/32&lt;br&gt;0.34375<br /> | 45/32&lt;br&gt;1.40625<br /> | 37/32&lt;br&gt;1.15625<br /> | 35/32&lt;br&gt;1.09375<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.59741<br /> | [[Image:8x16_5bit.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;7-bit<br /> | -48/128&lt;br&gt;-0.375<br /> | -45/128&lt;br&gt;-0.3515625<br /> | -16/128&lt;br&gt;-0.125<br /> | 105/128&lt;br&gt;0.8203125<br /> | 80/128&lt;br&gt;0.625<br /> | 53/128&lt;br&gt;0.4140625<br /> | 184/128&lt;br&gt;1.4375<br /> | 151/128&lt;br&gt;1.1796875<br /> | 147/128&lt;br&gt;1.1484375<br /> | 157/128&lt;br&gt;1.2265625<br /> | &amp;nbsp;&lt;br&gt;9.56672<br /> | [[Image:8x16rd_7bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -12/32&lt;br&gt;-0.375<br /> | -10/32&lt;br&gt;-0.3125<br /> | -2/32&lt;br&gt;-0.0625<br /> | 26/32&lt;br&gt;0.8125<br /> | 20/32&lt;br&gt;0.625<br /> | 12/32&lt;br&gt;0.375<br /> | 48/32&lt;br&gt;1.5<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.5596<br /> | [[Image:8x16rd_5bit.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -32/64&lt;br&gt;-0.5<br /> | -28/64&lt;br&gt;-0.4375<br /> | -24/64&lt;br&gt;-0.375<br /> | -32/64&lt;br&gt;-0.5<br /> | -24/64&lt;br&gt;-0.375<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 59/64&lt;br&gt;0.921875<br /> | 53/64&lt;br&gt;0.828125<br /> | 46/64&lt;br&gt;0.71875<br /> | 41/64&lt;br&gt;0.640625<br /> | 35/64&lt;br&gt;0.546875<br /> | 24/64&lt;br&gt;0.375<br /> | 12/64&lt;br&gt;0.1875<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | 74/64&lt;br&gt;1.15625<br /> | 74/64&lt;br&gt;1.15625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.78294<br /> | [[Image:16x32rd.png|64px]]<br /> |}<br /> <br /> == Type-IV Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !4x8 Ramp<br /> !8x16<br /> !8x16 Ramp<br /> !16x32<br /> !16x32 Ramp<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |8.60603<br /> |9.6005<br /> |9.56867<br /> |9.9057<br /> |9.81157<br /> |-<br /> |Dyadic (8-bit)<br /> |<br /> |8.60104<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Loss<br /> |<br /> |0.00499<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Dyadic (7-bit)<br /> |<br /> |8.59886<br /> |<br /> |9.56672<br /> |<br /> |<br /> |-<br /> |Loss<br /> |<br /> |0.00717<br /> |<br /> |0.00195<br /> |<br /> |<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |8.59886<br /> |9.60021<br /> |9.56161<br /> |9.89338<br /> |9.78294<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00717<br /> |0.00029<br /> |0.00706<br /> |0.01232<br /> |0.02863<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |8.56068<br /> |9.59741<br /> |9.5596<br /> |<br /> |<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.04535<br /> |0.00309<br /> |0.00907<br /> |<br /> |<br /> |}<br /> <br /> == 8x16 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -25/64&lt;br&gt;-0.390625<br /> | -20/64&lt;br&gt;-0.3125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 49/64&lt;br&gt;0.765625<br /> | 35/64&lt;br&gt;0.546875<br /> | 21/64&lt;br&gt;0.328125<br /> | 90/64&lt;br&gt;1.40625<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.6112<br /> | [[Image:8x16_type3.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -13/32&lt;br&gt;-0.40625<br /> | -11/32&lt;br&gt;-0.34375<br /> | -4/32&lt;br&gt;-0.125<br /> | 25/32&lt;br&gt;0.78125<br /> | 18/32&lt;br&gt;0.5625<br /> | 11/32&lt;br&gt;0.34375<br /> | 45/32&lt;br&gt;1.40625<br /> | 36/32&lt;br&gt;1.125<br /> | 36/32&lt;br&gt;1.125<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.61048<br /> | [[Image:8x16_type3_5bit.png|64px]]<br /> |}<br /> <br /> == 16x32 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -30/64&lt;br&gt;-0.46875<br /> | -35/64&lt;br&gt;-0.546875<br /> | -31/64&lt;br&gt;-0.484375<br /> | -29/64&lt;br&gt;-0.453125<br /> | -25/64&lt;br&gt;-0.390625<br /> | -19/64&lt;br&gt;-0.296875<br /> | -10/64&lt;br&gt;-0.15625<br /> | 54/64&lt;br&gt;0.84375<br /> | 45/64&lt;br&gt;0.703125<br /> | 40/64&lt;br&gt;0.625<br /> | 36/64&lt;br&gt;0.5625<br /> | 32/64&lt;br&gt;0.5<br /> | 25/64&lt;br&gt;0.390625<br /> | 17/64&lt;br&gt;0.265625<br /> | 90/64&lt;br&gt;1.40625<br /> | 70/64&lt;br&gt;1.09375<br /> | 69/64&lt;br&gt;1.078125<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 68/64&lt;br&gt;1.0625<br /> | 74/64&lt;br&gt;1.15625<br /> | &amp;nbsp;&lt;br&gt;9.94127<br /> | [[Image:16x32_type3_6bit.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -14/32&lt;br&gt;-0.4375<br /> | -17/32&lt;br&gt;-0.53125<br /> | -14/32&lt;br&gt;-0.4375<br /> | -12/32&lt;br&gt;-0.375<br /> | -9/32&lt;br&gt;-0.28125<br /> | -4/32&lt;br&gt;-0.125<br /> | 1/32&lt;br&gt;0.03125<br /> | 27/32&lt;br&gt;0.84375<br /> | 23/32&lt;br&gt;0.71875<br /> | 20/32&lt;br&gt;0.625<br /> | 17/32&lt;br&gt;0.53125<br /> | 14/32&lt;br&gt;0.4375<br /> | 9/32&lt;br&gt;0.28125<br /> | 2/32&lt;br&gt;0.0625<br /> | 46/32&lt;br&gt;1.4375<br /> | 35/32&lt;br&gt;1.09375<br /> | 35/32&lt;br&gt;1.09375<br /> | 34/32&lt;br&gt;1.0625<br /> | 34/32&lt;br&gt;1.0625<br /> | 35/32&lt;br&gt;1.09375<br /> | 36/32&lt;br&gt;1.125<br /> | 33/32&lt;br&gt;1.03125<br /> | &amp;nbsp;&lt;br&gt;9.93502<br /> | [[Image:16x32_type3_5bit.png|64px]]<br /> |}<br /> <br /> == Type-III Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !8x16<br /> !16x32<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |9.6115<br /> |9.9496<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |9.6112<br /> |9.94127<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00030<br /> |0.00833<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |9.61048<br /> |9.93502<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.00102<br /> |0.01458<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13759 2012-11-12T15:24:37Z

<p>Unlord: Adding best known 16x32 Type-III</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -5/32&lt;br&gt;-0.15625<br /> | 18/32&lt;br&gt;0.5625<br /> | 46/32&lt;br&gt;1.4375<br /> | 42/32&lt;br&gt;1.3125<br /> | &amp;nbsp;&lt;br&gt;8.63409<br /> | &amp;nbsp;&lt;br&gt;22.5715<br /> | [[Image:4x8_5bit.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;8-bit<br /> | -32/256&lt;br&gt;-0.125<br /> | 162/256&lt;br&gt;0.6328125<br /> | 376/256&lt;br&gt;1.46875<br /> | 357/256&lt;br&gt;1.39453125<br /> | &amp;nbsp;&lt;br&gt;8.60104<br /> | &amp;nbsp;&lt;br&gt;21.4037<br /> | [[Image:4x8rd_8bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;7-bit<br /> | -32/128&lt;br&gt;-0.25<br /> | 82/128&lt;br&gt;0.640625<br /> | 184/128&lt;br&gt;1.4375<br /> | 186/128&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd_7bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -8/32&lt;br&gt;-0.25<br /> | 19/32&lt;br&gt;0.59375<br /> | 52/32&lt;br&gt;1.625<br /> | 47/32&lt;br&gt;1.46875<br /> | &amp;nbsp;&lt;br&gt;8.56068<br /> | &amp;nbsp;&lt;br&gt;20.3279<br /> | [[Image:4x8rd_5bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -12/32&lt;br&gt;-0.375<br /> | -9/32&lt;br&gt;-0.28125<br /> | -4/32&lt;br&gt;-0.125<br /> | 24/32&lt;br&gt;0.75<br /> | 17/32&lt;br&gt;0.53125<br /> | 11/32&lt;br&gt;0.34375<br /> | 45/32&lt;br&gt;1.40625<br /> | 37/32&lt;br&gt;1.15625<br /> | 35/32&lt;br&gt;1.09375<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.59741<br /> | [[Image:8x16_5bit.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;7-bit<br /> | -48/128&lt;br&gt;-0.375<br /> | -45/128&lt;br&gt;-0.3515625<br /> | -16/128&lt;br&gt;-0.125<br /> | 105/128&lt;br&gt;0.8203125<br /> | 80/128&lt;br&gt;0.625<br /> | 53/128&lt;br&gt;0.4140625<br /> | 184/128&lt;br&gt;1.4375<br /> | 151/128&lt;br&gt;1.1796875<br /> | 147/128&lt;br&gt;1.1484375<br /> | 157/128&lt;br&gt;1.2265625<br /> | &amp;nbsp;&lt;br&gt;9.56672<br /> | [[Image:8x16rd_7bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -12/32&lt;br&gt;-0.375<br /> | -10/32&lt;br&gt;-0.3125<br /> | -2/32&lt;br&gt;-0.0625<br /> | 26/32&lt;br&gt;0.8125<br /> | 20/32&lt;br&gt;0.625<br /> | 12/32&lt;br&gt;0.375<br /> | 48/32&lt;br&gt;1.5<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.5596<br /> | [[Image:8x16rd_5bit.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -32/64&lt;br&gt;-0.5<br /> | -28/64&lt;br&gt;-0.4375<br /> | -24/64&lt;br&gt;-0.375<br /> | -32/64&lt;br&gt;-0.5<br /> | -24/64&lt;br&gt;-0.375<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 59/64&lt;br&gt;0.921875<br /> | 53/64&lt;br&gt;0.828125<br /> | 46/64&lt;br&gt;0.71875<br /> | 41/64&lt;br&gt;0.640625<br /> | 35/64&lt;br&gt;0.546875<br /> | 24/64&lt;br&gt;0.375<br /> | 12/64&lt;br&gt;0.1875<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | 74/64&lt;br&gt;1.15625<br /> | 74/64&lt;br&gt;1.15625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.78294<br /> | [[Image:16x32rd.png|64px]]<br /> |}<br /> <br /> == Type-IV Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !4x8 Ramp<br /> !8x16<br /> !8x16 Ramp<br /> !16x32<br /> !16x32 Ramp<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |8.60603<br /> |9.6005<br /> |9.56867<br /> |9.9057<br /> |9.81157<br /> |-<br /> |Dyadic (8-bit)<br /> |<br /> |8.60104<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Loss<br /> |<br /> |0.00499<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Dyadic (7-bit)<br /> |<br /> |8.59886<br /> |<br /> |9.56672<br /> |<br /> |<br /> |-<br /> |Loss<br /> |<br /> |0.00717<br /> |<br /> |0.00195<br /> |<br /> |<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |8.59886<br /> |9.60021<br /> |9.56161<br /> |9.89338<br /> |9.78294<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00717<br /> |0.00029<br /> |0.00706<br /> |0.01232<br /> |0.02863<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |8.56068<br /> |9.59741<br /> |9.5596<br /> |<br /> |<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.04535<br /> |0.00309<br /> |0.00907<br /> |<br /> |<br /> |}<br /> <br /> == 8x16 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -25/64&lt;br&gt;-0.390625<br /> | -20/64&lt;br&gt;-0.3125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 49/64&lt;br&gt;0.765625<br /> | 35/64&lt;br&gt;0.546875<br /> | 21/64&lt;br&gt;0.328125<br /> | 90/64&lt;br&gt;1.40625<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.6112<br /> | [[Image:8x16_type3.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -13/32&lt;br&gt;-0.40625<br /> | -11/32&lt;br&gt;-0.34375<br /> | -4/32&lt;br&gt;-0.125<br /> | 25/32&lt;br&gt;0.78125<br /> | 18/32&lt;br&gt;0.5625<br /> | 11/32&lt;br&gt;0.34375<br /> | 45/32&lt;br&gt;1.40625<br /> | 36/32&lt;br&gt;1.125<br /> | 36/32&lt;br&gt;1.125<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.61048<br /> | [[Image:8x16_type3_5bit.png|64px]]<br /> |}<br /> <br /> == 16x32 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -30/64&lt;br&gt;-0.46875<br /> | -35/64&lt;br&gt;-0.546875<br /> | -31/64&lt;br&gt;-0.484375<br /> | -29/64&lt;br&gt;-0.453125<br /> | -25/64&lt;br&gt;-0.390625<br /> | -19/64&lt;br&gt;-0.296875<br /> | -10/64&lt;br&gt;-0.15625<br /> | 54/64&lt;br&gt;0.84375<br /> | 45/64&lt;br&gt;0.703125<br /> | 40/64&lt;br&gt;0.625<br /> | 36/64&lt;br&gt;0.5625<br /> | 32/64&lt;br&gt;0.5<br /> | 25/64&lt;br&gt;0.390625<br /> | 17/64&lt;br&gt;0.265625<br /> | 90/64&lt;br&gt;1.40625<br /> | 70/64&lt;br&gt;1.09375<br /> | 69/64&lt;br&gt;1.078125<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 68/64&lt;br&gt;1.0625<br /> | 74/64&lt;br&gt;1.15625<br /> | &amp;nbsp;&lt;br&gt;9.94127<br /> | [[Image:16x32_type3_6bit.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -15/32&lt;br&gt;-0.46875<br /> | -18/32&lt;br&gt;-0.5625<br /> | -14/32&lt;br&gt;-0.4375<br /> | -10/32&lt;br&gt;-0.3125<br /> | -10/32&lt;br&gt;-0.3125<br /> | -6/32&lt;br&gt;-0.1875<br /> | -2/32&lt;br&gt;-0.0625<br /> | 28/32&lt;br&gt;0.875<br /> | 23/32&lt;br&gt;0.71875<br /> | 20/32&lt;br&gt;0.625<br /> | 16/32&lt;br&gt;0.5<br /> | 12/32&lt;br&gt;0.375<br /> | 10/32&lt;br&gt;0.3125<br /> | 6/32&lt;br&gt;0.1875<br /> | 44/32&lt;br&gt;1.375<br /> | 36/32&lt;br&gt;1.125<br /> | 35/32&lt;br&gt;1.09375<br /> | 35/32&lt;br&gt;1.09375<br /> | 34/32&lt;br&gt;1.0625<br /> | 33/32&lt;br&gt;1.03125<br /> | 33/32&lt;br&gt;1.03125<br /> | 35/32&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.92855<br /> | [[Image:16x32_type3_5bit.png|64px]]<br /> |}<br /> <br /> == Type-III Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !8x16<br /> !16x32<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |9.6115<br /> |9.9496<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |9.6112<br /> |9.94127<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00030<br /> |0.00833<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |9.61048<br /> |9.92855<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.00102<br /> |0.02105<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13754 2012-11-04T17:31:07Z

<p>Unlord: Adding 8x16 Type-IV 5-bit R=F</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -5/32&lt;br&gt;-0.15625<br /> | 18/32&lt;br&gt;0.5625<br /> | 46/32&lt;br&gt;1.4375<br /> | 42/32&lt;br&gt;1.3125<br /> | &amp;nbsp;&lt;br&gt;8.63409<br /> | &amp;nbsp;&lt;br&gt;22.5715<br /> | [[Image:4x8_5bit.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;8-bit<br /> | -32/256&lt;br&gt;-0.125<br /> | 162/256&lt;br&gt;0.6328125<br /> | 376/256&lt;br&gt;1.46875<br /> | 357/256&lt;br&gt;1.39453125<br /> | &amp;nbsp;&lt;br&gt;8.60104<br /> | &amp;nbsp;&lt;br&gt;21.4037<br /> | [[Image:4x8rd_8bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;7-bit<br /> | -32/128&lt;br&gt;-0.25<br /> | 82/128&lt;br&gt;0.640625<br /> | 184/128&lt;br&gt;1.4375<br /> | 186/128&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd_7bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -8/32&lt;br&gt;-0.25<br /> | 19/32&lt;br&gt;0.59375<br /> | 52/32&lt;br&gt;1.625<br /> | 47/32&lt;br&gt;1.46875<br /> | &amp;nbsp;&lt;br&gt;8.56068<br /> | &amp;nbsp;&lt;br&gt;20.3279<br /> | [[Image:4x8rd_5bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -12/32&lt;br&gt;-0.375<br /> | -9/32&lt;br&gt;-0.28125<br /> | -4/32&lt;br&gt;-0.125<br /> | 24/32&lt;br&gt;0.75<br /> | 17/32&lt;br&gt;0.53125<br /> | 11/32&lt;br&gt;0.34375<br /> | 45/32&lt;br&gt;1.40625<br /> | 37/32&lt;br&gt;1.15625<br /> | 35/32&lt;br&gt;1.09375<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.59741<br /> | [[Image:8x16_5bit.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;7-bit<br /> | -48/128&lt;br&gt;-0.375<br /> | -45/128&lt;br&gt;-0.3515625<br /> | -16/128&lt;br&gt;-0.125<br /> | 105/128&lt;br&gt;0.8203125<br /> | 80/128&lt;br&gt;0.625<br /> | 53/128&lt;br&gt;0.4140625<br /> | 184/128&lt;br&gt;1.4375<br /> | 151/128&lt;br&gt;1.1796875<br /> | 147/128&lt;br&gt;1.1484375<br /> | 157/128&lt;br&gt;1.2265625<br /> | &amp;nbsp;&lt;br&gt;9.56672<br /> | [[Image:8x16rd_7bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -12/32&lt;br&gt;-0.375<br /> | -10/32&lt;br&gt;-0.3125<br /> | -2/32&lt;br&gt;-0.0625<br /> | 26/32&lt;br&gt;0.8125<br /> | 20/32&lt;br&gt;0.625<br /> | 12/32&lt;br&gt;0.375<br /> | 48/32&lt;br&gt;1.5<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.5596<br /> | [[Image:8x16rd_5bit.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -32/64&lt;br&gt;-0.5<br /> | -28/64&lt;br&gt;-0.4375<br /> | -24/64&lt;br&gt;-0.375<br /> | -32/64&lt;br&gt;-0.5<br /> | -24/64&lt;br&gt;-0.375<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 59/64&lt;br&gt;0.921875<br /> | 53/64&lt;br&gt;0.828125<br /> | 46/64&lt;br&gt;0.71875<br /> | 41/64&lt;br&gt;0.640625<br /> | 35/64&lt;br&gt;0.546875<br /> | 24/64&lt;br&gt;0.375<br /> | 12/64&lt;br&gt;0.1875<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | 74/64&lt;br&gt;1.15625<br /> | 74/64&lt;br&gt;1.15625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.78294<br /> | [[Image:16x32rd.png|64px]]<br /> |}<br /> <br /> == Type-IV Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !4x8 Ramp<br /> !8x16<br /> !8x16 Ramp<br /> !16x32<br /> !16x32 Ramp<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |8.60603<br /> |9.6005<br /> |9.56867<br /> |9.9057<br /> |9.81157<br /> |-<br /> |Dyadic (8-bit)<br /> |<br /> |8.60104<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Loss<br /> |<br /> |0.00499<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Dyadic (7-bit)<br /> |<br /> |8.59886<br /> |<br /> |9.56672<br /> |<br /> |<br /> |-<br /> |Loss<br /> |<br /> |0.00717<br /> |<br /> |0.00195<br /> |<br /> |<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |8.59886<br /> |9.60021<br /> |9.56161<br /> |9.89338<br /> |9.78294<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00717<br /> |0.00029<br /> |0.00706<br /> |0.01232<br /> |0.02863<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |8.56068<br /> |9.59741<br /> |9.5596<br /> |<br /> |<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.04535<br /> |0.00309<br /> |0.00907<br /> |<br /> |<br /> |}<br /> <br /> == 8x16 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -25/64&lt;br&gt;-0.390625<br /> | -20/64&lt;br&gt;-0.3125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 49/64&lt;br&gt;0.765625<br /> | 35/64&lt;br&gt;0.546875<br /> | 21/64&lt;br&gt;0.328125<br /> | 90/64&lt;br&gt;1.40625<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.6112<br /> | [[Image:8x16_type3.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -13/32&lt;br&gt;-0.40625<br /> | -11/32&lt;br&gt;-0.34375<br /> | -4/32&lt;br&gt;-0.125<br /> | 25/32&lt;br&gt;0.78125<br /> | 18/32&lt;br&gt;0.5625<br /> | 11/32&lt;br&gt;0.34375<br /> | 45/32&lt;br&gt;1.40625<br /> | 36/32&lt;br&gt;1.125<br /> | 36/32&lt;br&gt;1.125<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.61048<br /> | [[Image:8x16_type3_5bit.png|64px]]<br /> |}<br /> <br /> == Type-III Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !8x16<br /> !16x32<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |9.6115<br /> |9.9496<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |9.6112<br /> |<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00030<br /> |<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |9.61048<br /> |<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.00102<br /> |<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13753 2012-11-04T16:46:59Z

<p>Unlord: Adding 8x16 Type-IV 7-bit R=T</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -5/32&lt;br&gt;-0.15625<br /> | 18/32&lt;br&gt;0.5625<br /> | 46/32&lt;br&gt;1.4375<br /> | 42/32&lt;br&gt;1.3125<br /> | &amp;nbsp;&lt;br&gt;8.63409<br /> | &amp;nbsp;&lt;br&gt;22.5715<br /> | [[Image:4x8_5bit.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;8-bit<br /> | -32/256&lt;br&gt;-0.125<br /> | 162/256&lt;br&gt;0.6328125<br /> | 376/256&lt;br&gt;1.46875<br /> | 357/256&lt;br&gt;1.39453125<br /> | &amp;nbsp;&lt;br&gt;8.60104<br /> | &amp;nbsp;&lt;br&gt;21.4037<br /> | [[Image:4x8rd_8bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;7-bit<br /> | -32/128&lt;br&gt;-0.25<br /> | 82/128&lt;br&gt;0.640625<br /> | 184/128&lt;br&gt;1.4375<br /> | 186/128&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd_7bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -8/32&lt;br&gt;-0.25<br /> | 19/32&lt;br&gt;0.59375<br /> | 52/32&lt;br&gt;1.625<br /> | 47/32&lt;br&gt;1.46875<br /> | &amp;nbsp;&lt;br&gt;8.56068<br /> | &amp;nbsp;&lt;br&gt;20.3279<br /> | [[Image:4x8rd_5bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;7-bit<br /> | -48/128&lt;br&gt;-0.375<br /> | -45/128&lt;br&gt;-0.3515625<br /> | -16/128&lt;br&gt;-0.125<br /> | 105/128&lt;br&gt;0.8203125<br /> | 80/128&lt;br&gt;0.625<br /> | 53/128&lt;br&gt;0.4140625<br /> | 184/128&lt;br&gt;1.4375<br /> | 151/128&lt;br&gt;1.1796875<br /> | 147/128&lt;br&gt;1.1484375<br /> | 157/128&lt;br&gt;1.2265625<br /> | &amp;nbsp;&lt;br&gt;9.56672<br /> | [[Image:8x16rd_7bit.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -12/32&lt;br&gt;-0.375<br /> | -10/32&lt;br&gt;-0.3125<br /> | -2/32&lt;br&gt;-0.0625<br /> | 26/32&lt;br&gt;0.8125<br /> | 20/32&lt;br&gt;0.625<br /> | 12/32&lt;br&gt;0.375<br /> | 48/32&lt;br&gt;1.5<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.5596<br /> | [[Image:8x16rd_5bit.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -32/64&lt;br&gt;-0.5<br /> | -28/64&lt;br&gt;-0.4375<br /> | -24/64&lt;br&gt;-0.375<br /> | -32/64&lt;br&gt;-0.5<br /> | -24/64&lt;br&gt;-0.375<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 59/64&lt;br&gt;0.921875<br /> | 53/64&lt;br&gt;0.828125<br /> | 46/64&lt;br&gt;0.71875<br /> | 41/64&lt;br&gt;0.640625<br /> | 35/64&lt;br&gt;0.546875<br /> | 24/64&lt;br&gt;0.375<br /> | 12/64&lt;br&gt;0.1875<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | 74/64&lt;br&gt;1.15625<br /> | 74/64&lt;br&gt;1.15625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.78294<br /> | [[Image:16x32rd.png|64px]]<br /> |}<br /> <br /> == Type-IV Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !4x8 Ramp<br /> !8x16<br /> !8x16 Ramp<br /> !16x32<br /> !16x32 Ramp<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |8.60603<br /> |9.6005<br /> |9.56867<br /> |9.9057<br /> |9.81157<br /> |-<br /> |Dyadic (8-bit)<br /> |<br /> |8.60104<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Loss<br /> |<br /> |0.00499<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Dyadic (7-bit)<br /> |<br /> |8.59886<br /> |<br /> |9.56672<br /> |<br /> |<br /> |-<br /> |Loss<br /> |<br /> |0.00717<br /> |<br /> |0.00195<br /> |<br /> |<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |8.59886<br /> |9.60021<br /> |9.56161<br /> |9.89338<br /> |9.78294<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00717<br /> |0.00029<br /> |0.00706<br /> |0.01232<br /> |0.02863<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |8.56068<br /> |<br /> |9.5596<br /> |<br /> |<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.04535<br /> |<br /> |0.00907<br /> |<br /> |<br /> |}<br /> <br /> == 8x16 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -25/64&lt;br&gt;-0.390625<br /> | -20/64&lt;br&gt;-0.3125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 49/64&lt;br&gt;0.765625<br /> | 35/64&lt;br&gt;0.546875<br /> | 21/64&lt;br&gt;0.328125<br /> | 90/64&lt;br&gt;1.40625<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.6112<br /> | [[Image:8x16_type3.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -13/32&lt;br&gt;-0.40625<br /> | -11/32&lt;br&gt;-0.34375<br /> | -4/32&lt;br&gt;-0.125<br /> | 25/32&lt;br&gt;0.78125<br /> | 18/32&lt;br&gt;0.5625<br /> | 11/32&lt;br&gt;0.34375<br /> | 45/32&lt;br&gt;1.40625<br /> | 36/32&lt;br&gt;1.125<br /> | 36/32&lt;br&gt;1.125<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.61048<br /> | [[Image:8x16_type3_5bit.png|64px]]<br /> |}<br /> <br /> == Type-III Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !8x16<br /> !16x32<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |9.6115<br /> |9.9496<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |9.6112<br /> |<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00030<br /> |<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |9.61048<br /> |<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.00102<br /> |<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13752 2012-11-04T13:42:23Z

<p>Unlord: Adding 8x16 Type-III 5bit</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -5/32&lt;br&gt;-0.15625<br /> | 18/32&lt;br&gt;0.5625<br /> | 46/32&lt;br&gt;1.4375<br /> | 42/32&lt;br&gt;1.3125<br /> | &amp;nbsp;&lt;br&gt;8.63409<br /> | &amp;nbsp;&lt;br&gt;22.5715<br /> | [[Image:4x8_5bits.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;8-bit<br /> | -32/256&lt;br&gt;-0.125<br /> | 162/256&lt;br&gt;0.6328125<br /> | 376/256&lt;br&gt;1.46875<br /> | 357/256&lt;br&gt;1.39453125<br /> | &amp;nbsp;&lt;br&gt;8.60104<br /> | &amp;nbsp;&lt;br&gt;21.4037<br /> | [[Image:4x8rd_8bits.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;7-bit<br /> | -32/128&lt;br&gt;-0.25<br /> | 82/128&lt;br&gt;0.640625<br /> | 184/128&lt;br&gt;1.4375<br /> | 186/128&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd_7bits.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -8/32&lt;br&gt;-0.25<br /> | 19/32&lt;br&gt;0.59375<br /> | 52/32&lt;br&gt;1.625<br /> | 47/32&lt;br&gt;1.46875<br /> | &amp;nbsp;&lt;br&gt;8.56068<br /> | &amp;nbsp;&lt;br&gt;20.3279<br /> | [[Image:4x8rd_5bits.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -12/32&lt;br&gt;-0.375<br /> | -10/32&lt;br&gt;-0.3125<br /> | -2/32&lt;br&gt;-0.0625<br /> | 26/32&lt;br&gt;0.8125<br /> | 20/32&lt;br&gt;0.625<br /> | 12/32&lt;br&gt;0.375<br /> | 48/32&lt;br&gt;1.5<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.5596<br /> | [[Image:8x16rd_5bits.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -32/64&lt;br&gt;-0.5<br /> | -28/64&lt;br&gt;-0.4375<br /> | -24/64&lt;br&gt;-0.375<br /> | -32/64&lt;br&gt;-0.5<br /> | -24/64&lt;br&gt;-0.375<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 59/64&lt;br&gt;0.921875<br /> | 53/64&lt;br&gt;0.828125<br /> | 46/64&lt;br&gt;0.71875<br /> | 41/64&lt;br&gt;0.640625<br /> | 35/64&lt;br&gt;0.546875<br /> | 24/64&lt;br&gt;0.375<br /> | 12/64&lt;br&gt;0.1875<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | 74/64&lt;br&gt;1.15625<br /> | 74/64&lt;br&gt;1.15625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.78294<br /> | [[Image:16x32rd.png|64px]]<br /> |}<br /> <br /> == Type-IV Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !4x8 Ramp<br /> !8x16<br /> !8x16 Ramp<br /> !16x32<br /> !16x32 Ramp<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |8.60603<br /> |9.6005<br /> |9.56867<br /> |9.9057<br /> |9.81157<br /> |-<br /> |Dyadic (8-bit)<br /> |<br /> |8.60104<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Loss<br /> |<br /> |0.00499<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Dyadic (7-bit)<br /> |<br /> |8.59886<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Loss<br /> |<br /> |0.00717<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |8.59886<br /> |9.60021<br /> |9.56161<br /> |9.89338<br /> |9.78294<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00717<br /> |0.00029<br /> |0.00706<br /> |0.01232<br /> |0.02863<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |8.56068<br /> |<br /> |9.5596<br /> |<br /> |<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.04535<br /> |<br /> |0.00907<br /> |<br /> |<br /> |}<br /> <br /> == 8x16 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -25/64&lt;br&gt;-0.390625<br /> | -20/64&lt;br&gt;-0.3125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 49/64&lt;br&gt;0.765625<br /> | 35/64&lt;br&gt;0.546875<br /> | 21/64&lt;br&gt;0.328125<br /> | 90/64&lt;br&gt;1.40625<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.6112<br /> | [[Image:8x16_type3.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -13/32&lt;br&gt;-0.40625<br /> | -11/32&lt;br&gt;-0.34375<br /> | -4/32&lt;br&gt;-0.125<br /> | 25/32&lt;br&gt;0.78125<br /> | 18/32&lt;br&gt;0.5625<br /> | 11/32&lt;br&gt;0.34375<br /> | 45/32&lt;br&gt;1.40625<br /> | 36/32&lt;br&gt;1.125<br /> | 36/32&lt;br&gt;1.125<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.61048<br /> | [[Image:8x16_type3_5bit.png|64px]]<br /> |}<br /> <br /> == Type-III Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !8x16<br /> !16x32<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |9.6115<br /> |9.9496<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |9.6112<br /> |<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00030<br /> |<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |9.61048<br /> |<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.00102<br /> |<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13750 2012-11-04T08:42:52Z

<p>Unlord: /* Type-IV Coding Gain */</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -5/32&lt;br&gt;-0.15625<br /> | 18/32&lt;br&gt;0.5625<br /> | 46/32&lt;br&gt;1.4375<br /> | 42/32&lt;br&gt;1.3125<br /> | &amp;nbsp;&lt;br&gt;8.63409<br /> | &amp;nbsp;&lt;br&gt;22.5715<br /> | [[Image:4x8_5bits.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;8-bit<br /> | -32/256&lt;br&gt;-0.125<br /> | 162/256&lt;br&gt;0.6328125<br /> | 376/256&lt;br&gt;1.46875<br /> | 357/256&lt;br&gt;1.39453125<br /> | &amp;nbsp;&lt;br&gt;8.60104<br /> | &amp;nbsp;&lt;br&gt;21.4037<br /> | [[Image:4x8rd_8bits.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;7-bit<br /> | -32/128&lt;br&gt;-0.25<br /> | 82/128&lt;br&gt;0.640625<br /> | 184/128&lt;br&gt;1.4375<br /> | 186/128&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd_7bits.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -8/32&lt;br&gt;-0.25<br /> | 19/32&lt;br&gt;0.59375<br /> | 52/32&lt;br&gt;1.625<br /> | 47/32&lt;br&gt;1.46875<br /> | &amp;nbsp;&lt;br&gt;8.56068<br /> | &amp;nbsp;&lt;br&gt;20.3279<br /> | [[Image:4x8rd_5bits.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -12/32&lt;br&gt;-0.375<br /> | -10/32&lt;br&gt;-0.3125<br /> | -2/32&lt;br&gt;-0.0625<br /> | 26/32&lt;br&gt;0.8125<br /> | 20/32&lt;br&gt;0.625<br /> | 12/32&lt;br&gt;0.375<br /> | 48/32&lt;br&gt;1.5<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.5596<br /> | [[Image:8x16rd_5bits.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -32/64&lt;br&gt;-0.5<br /> | -28/64&lt;br&gt;-0.4375<br /> | -24/64&lt;br&gt;-0.375<br /> | -32/64&lt;br&gt;-0.5<br /> | -24/64&lt;br&gt;-0.375<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 59/64&lt;br&gt;0.921875<br /> | 53/64&lt;br&gt;0.828125<br /> | 46/64&lt;br&gt;0.71875<br /> | 41/64&lt;br&gt;0.640625<br /> | 35/64&lt;br&gt;0.546875<br /> | 24/64&lt;br&gt;0.375<br /> | 12/64&lt;br&gt;0.1875<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | 74/64&lt;br&gt;1.15625<br /> | 74/64&lt;br&gt;1.15625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.78294<br /> | [[Image:16x32rd.png|64px]]<br /> |}<br /> <br /> == Type-IV Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !4x8 Ramp<br /> !8x16<br /> !8x16 Ramp<br /> !16x32<br /> !16x32 Ramp<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |8.60603<br /> |9.6005<br /> |9.56867<br /> |9.9057<br /> |9.81157<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |8.59886<br /> |9.60021<br /> |9.56161<br /> |9.89338<br /> |9.78294<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00717<br /> |0.00029<br /> |0.00706<br /> |0.01232<br /> |0.02863<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |8.56068<br /> |<br /> |9.5596<br /> |<br /> |<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.04535<br /> |<br /> |0.00907<br /> |<br /> |<br /> |}<br /> <br /> == 8x16 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -25/64&lt;br&gt;-0.390625<br /> | -20/64&lt;br&gt;-0.3125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 49/64&lt;br&gt;0.765625<br /> | 35/64&lt;br&gt;0.546875<br /> | 21/64&lt;br&gt;0.328125<br /> | 90/64&lt;br&gt;1.40625<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.6112<br /> | [[Image:8x16_type3.png|64px]]<br /> |}<br /> <br /> == Type-III Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !8x16<br /> !16x32<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |9.6115<br /> |9.9496<br /> |-<br /> |Approximate<br /> |8.63473<br /> |9.6112<br /> |<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00030<br /> |<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13749 2012-11-04T08:38:11Z

<p>Unlord: /* 8x16 */</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -5/32&lt;br&gt;-0.15625<br /> | 18/32&lt;br&gt;0.5625<br /> | 46/32&lt;br&gt;1.4375<br /> | 42/32&lt;br&gt;1.3125<br /> | &amp;nbsp;&lt;br&gt;8.63409<br /> | &amp;nbsp;&lt;br&gt;22.5715<br /> | [[Image:4x8_5bits.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;8-bit<br /> | -32/256&lt;br&gt;-0.125<br /> | 162/256&lt;br&gt;0.6328125<br /> | 376/256&lt;br&gt;1.46875<br /> | 357/256&lt;br&gt;1.39453125<br /> | &amp;nbsp;&lt;br&gt;8.60104<br /> | &amp;nbsp;&lt;br&gt;21.4037<br /> | [[Image:4x8rd_8bits.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;7-bit<br /> | -32/128&lt;br&gt;-0.25<br /> | 82/128&lt;br&gt;0.640625<br /> | 184/128&lt;br&gt;1.4375<br /> | 186/128&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd_7bits.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -8/32&lt;br&gt;-0.25<br /> | 19/32&lt;br&gt;0.59375<br /> | 52/32&lt;br&gt;1.625<br /> | 47/32&lt;br&gt;1.46875<br /> | &amp;nbsp;&lt;br&gt;8.56068<br /> | &amp;nbsp;&lt;br&gt;20.3279<br /> | [[Image:4x8rd_5bits.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -12/32&lt;br&gt;-0.375<br /> | -10/32&lt;br&gt;-0.3125<br /> | -2/32&lt;br&gt;-0.0625<br /> | 26/32&lt;br&gt;0.8125<br /> | 20/32&lt;br&gt;0.625<br /> | 12/32&lt;br&gt;0.375<br /> | 48/32&lt;br&gt;1.5<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | 38/32&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.5596<br /> | [[Image:8x16rd_5bits.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -32/64&lt;br&gt;-0.5<br /> | -28/64&lt;br&gt;-0.4375<br /> | -24/64&lt;br&gt;-0.375<br /> | -32/64&lt;br&gt;-0.5<br /> | -24/64&lt;br&gt;-0.375<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 59/64&lt;br&gt;0.921875<br /> | 53/64&lt;br&gt;0.828125<br /> | 46/64&lt;br&gt;0.71875<br /> | 41/64&lt;br&gt;0.640625<br /> | 35/64&lt;br&gt;0.546875<br /> | 24/64&lt;br&gt;0.375<br /> | 12/64&lt;br&gt;0.1875<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | 74/64&lt;br&gt;1.15625<br /> | 74/64&lt;br&gt;1.15625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.78294<br /> | [[Image:16x32rd.png|64px]]<br /> |}<br /> <br /> == Type-IV Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !4x8 Ramp<br /> !8x16<br /> !8x16 Ramp<br /> !16x32<br /> !16x32 Ramp<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |8.60603<br /> |9.6005<br /> |9.56867<br /> |9.9057<br /> |9.81157<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |8.59886<br /> |9.60021<br /> |9.56161<br /> |9.89338<br /> |9.78294<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00717<br /> |0.00029<br /> |0.00706<br /> |0.01232<br /> |0.02863<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |8.56068<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.04535<br /> |<br /> |<br /> |<br /> |<br /> |}<br /> <br /> == 8x16 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -25/64&lt;br&gt;-0.390625<br /> | -20/64&lt;br&gt;-0.3125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 49/64&lt;br&gt;0.765625<br /> | 35/64&lt;br&gt;0.546875<br /> | 21/64&lt;br&gt;0.328125<br /> | 90/64&lt;br&gt;1.40625<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.6112<br /> | [[Image:8x16_type3.png|64px]]<br /> |}<br /> <br /> == Type-III Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !8x16<br /> !16x32<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |9.6115<br /> |9.9496<br /> |-<br /> |Approximate<br /> |8.63473<br /> |9.6112<br /> |<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00030<br /> |<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13747 2012-11-04T07:29:27Z

<p>Unlord: /* 4x8 */</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -5/32&lt;br&gt;-0.15625<br /> | 18/32&lt;br&gt;0.5625<br /> | 46/32&lt;br&gt;1.4375<br /> | 42/32&lt;br&gt;1.3125<br /> | &amp;nbsp;&lt;br&gt;8.63409<br /> | &amp;nbsp;&lt;br&gt;22.5715<br /> | [[Image:4x8_5bits.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;8-bit<br /> | -32/256&lt;br&gt;-0.125<br /> | 162/256&lt;br&gt;0.6328125<br /> | 376/256&lt;br&gt;1.46875<br /> | 357/256&lt;br&gt;1.39453125<br /> | &amp;nbsp;&lt;br&gt;8.60104<br /> | &amp;nbsp;&lt;br&gt;21.4037<br /> | [[Image:4x8rd_8bits.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;7-bit<br /> | -32/128&lt;br&gt;-0.25<br /> | 82/128&lt;br&gt;0.640625<br /> | 184/128&lt;br&gt;1.4375<br /> | 186/128&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd_7bits.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;6-bit<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -8/32&lt;br&gt;-0.25<br /> | 19/32&lt;br&gt;0.59375<br /> | 52/32&lt;br&gt;1.625<br /> | 47/32&lt;br&gt;1.46875<br /> | &amp;nbsp;&lt;br&gt;8.56068<br /> | &amp;nbsp;&lt;br&gt;20.3279<br /> | [[Image:4x8rd_5bits.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -32/64&lt;br&gt;-0.5<br /> | -28/64&lt;br&gt;-0.4375<br /> | -24/64&lt;br&gt;-0.375<br /> | -32/64&lt;br&gt;-0.5<br /> | -24/64&lt;br&gt;-0.375<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 59/64&lt;br&gt;0.921875<br /> | 53/64&lt;br&gt;0.828125<br /> | 46/64&lt;br&gt;0.71875<br /> | 41/64&lt;br&gt;0.640625<br /> | 35/64&lt;br&gt;0.546875<br /> | 24/64&lt;br&gt;0.375<br /> | 12/64&lt;br&gt;0.1875<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | 74/64&lt;br&gt;1.15625<br /> | 74/64&lt;br&gt;1.15625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.78294<br /> | [[Image:16x32rd.png|64px]]<br /> |}<br /> <br /> == Type-IV Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !4x8 Ramp<br /> !8x16<br /> !8x16 Ramp<br /> !16x32<br /> !16x32 Ramp<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |8.60603<br /> |9.6005<br /> |9.56867<br /> |9.9057<br /> |9.81157<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |8.59886<br /> |9.60021<br /> |9.56161<br /> |9.89338<br /> |9.78294<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00717<br /> |0.00029<br /> |0.00706<br /> |0.01232<br /> |0.02863<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |8.56068<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.04535<br /> |<br /> |<br /> |<br /> |<br /> |}<br /> <br /> == 8x16 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -25/64&lt;br&gt;-0.390625<br /> | -20/64&lt;br&gt;-0.3125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 49/64&lt;br&gt;0.765625<br /> | 35/64&lt;br&gt;0.546875<br /> | 21/64&lt;br&gt;0.328125<br /> | 90/64&lt;br&gt;1.40625<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.6112<br /> | [[Image:8x16_type3.png|64px]]<br /> |}<br /> <br /> == Type-III Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !8x16<br /> !16x32<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |9.6115<br /> |9.9496<br /> |-<br /> |Approximate<br /> |8.63473<br /> |9.6112<br /> |<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00030<br /> |<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13746 2012-11-04T06:48:47Z

<p>Unlord: /* Type-IV Coding Gain */</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -5/32&lt;br&gt;-0.15625<br /> | 18/32&lt;br&gt;0.5625<br /> | 46/32&lt;br&gt;1.4375<br /> | 42/32&lt;br&gt;1.3125<br /> | &amp;nbsp;&lt;br&gt;8.63409<br /> | &amp;nbsp;&lt;br&gt;22.5715<br /> | [[Image:4x8_5bits.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max CG<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -8/32&lt;br&gt;-0.25<br /> | 19/32&lt;br&gt;0.59375<br /> | 52/32&lt;br&gt;1.625<br /> | 47/32&lt;br&gt;1.46875<br /> | &amp;nbsp;&lt;br&gt;8.56068<br /> | &amp;nbsp;&lt;br&gt;20.3279<br /> | [[Image:4x8rd_5bits.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -32/64&lt;br&gt;-0.5<br /> | -28/64&lt;br&gt;-0.4375<br /> | -24/64&lt;br&gt;-0.375<br /> | -32/64&lt;br&gt;-0.5<br /> | -24/64&lt;br&gt;-0.375<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 59/64&lt;br&gt;0.921875<br /> | 53/64&lt;br&gt;0.828125<br /> | 46/64&lt;br&gt;0.71875<br /> | 41/64&lt;br&gt;0.640625<br /> | 35/64&lt;br&gt;0.546875<br /> | 24/64&lt;br&gt;0.375<br /> | 12/64&lt;br&gt;0.1875<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | 74/64&lt;br&gt;1.15625<br /> | 74/64&lt;br&gt;1.15625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.78294<br /> | [[Image:16x32rd.png|64px]]<br /> |}<br /> <br /> == Type-IV Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !4x8 Ramp<br /> !8x16<br /> !8x16 Ramp<br /> !16x32<br /> !16x32 Ramp<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |8.60603<br /> |9.6005<br /> |9.56867<br /> |9.9057<br /> |9.81157<br /> |-<br /> |Dyadic (6-bit)<br /> |8.63473<br /> |8.59886<br /> |9.60021<br /> |9.56161<br /> |9.89338<br /> |9.78294<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00717<br /> |0.00029<br /> |0.00706<br /> |0.01232<br /> |0.02863<br /> |-<br /> |Dyadic (5-bit)<br /> |8.63409<br /> |8.56068<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Loss<br /> |0.00081<br /> |0.04535<br /> |<br /> |<br /> |<br /> |<br /> |}<br /> <br /> == 8x16 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -25/64&lt;br&gt;-0.390625<br /> | -20/64&lt;br&gt;-0.3125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 49/64&lt;br&gt;0.765625<br /> | 35/64&lt;br&gt;0.546875<br /> | 21/64&lt;br&gt;0.328125<br /> | 90/64&lt;br&gt;1.40625<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.6112<br /> | [[Image:8x16_type3.png|64px]]<br /> |}<br /> <br /> == Type-III Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !8x16<br /> !16x32<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |9.6115<br /> |9.9496<br /> |-<br /> |Approximate<br /> |8.63473<br /> |9.6112<br /> |<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00030<br /> |<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13745 2012-11-04T06:30:47Z

<p>Unlord: /* 4x8 */</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f&lt;br&gt;6-bit<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=f&lt;br&gt;5-bit<br /> | -5/32&lt;br&gt;-0.15625<br /> | 18/32&lt;br&gt;0.5625<br /> | 46/32&lt;br&gt;1.4375<br /> | 42/32&lt;br&gt;1.3125<br /> | &amp;nbsp;&lt;br&gt;8.63409<br /> | &amp;nbsp;&lt;br&gt;22.5715<br /> | [[Image:4x8_5bits.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max CG<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;5-bit<br /> | -8/32&lt;br&gt;-0.25<br /> | 19/32&lt;br&gt;0.59375<br /> | 52/32&lt;br&gt;1.625<br /> | 47/32&lt;br&gt;1.46875<br /> | &amp;nbsp;&lt;br&gt;8.56068<br /> | &amp;nbsp;&lt;br&gt;20.3279<br /> | [[Image:4x8rd_5bits.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -32/64&lt;br&gt;-0.5<br /> | -28/64&lt;br&gt;-0.4375<br /> | -24/64&lt;br&gt;-0.375<br /> | -32/64&lt;br&gt;-0.5<br /> | -24/64&lt;br&gt;-0.375<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 59/64&lt;br&gt;0.921875<br /> | 53/64&lt;br&gt;0.828125<br /> | 46/64&lt;br&gt;0.71875<br /> | 41/64&lt;br&gt;0.640625<br /> | 35/64&lt;br&gt;0.546875<br /> | 24/64&lt;br&gt;0.375<br /> | 12/64&lt;br&gt;0.1875<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | 74/64&lt;br&gt;1.15625<br /> | 74/64&lt;br&gt;1.15625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.78294<br /> | [[Image:16x32rd.png|64px]]<br /> |}<br /> <br /> == Type-IV Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !8x16<br /> !16x32<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |9.6005<br /> |9.9057<br /> |-<br /> |Approximate<br /> |8.63473<br /> |9.60021<br /> |9.89338<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00029<br /> |0.01232<br /> |}<br /> <br /> == 8x16 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -25/64&lt;br&gt;-0.390625<br /> | -20/64&lt;br&gt;-0.3125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 49/64&lt;br&gt;0.765625<br /> | 35/64&lt;br&gt;0.546875<br /> | 21/64&lt;br&gt;0.328125<br /> | 90/64&lt;br&gt;1.40625<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.6112<br /> | [[Image:8x16_type3.png|64px]]<br /> |}<br /> <br /> == Type-III Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !8x16<br /> !16x32<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |9.6115<br /> |9.9496<br /> |-<br /> |Approximate<br /> |8.63473<br /> |9.6112<br /> |<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00030<br /> |<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13743 2012-11-03T17:58:03Z

<p>Unlord: Adding Type-III comparison table</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max CG<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -32/64&lt;br&gt;-0.5<br /> | -28/64&lt;br&gt;-0.4375<br /> | -24/64&lt;br&gt;-0.375<br /> | -32/64&lt;br&gt;-0.5<br /> | -24/64&lt;br&gt;-0.375<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 59/64&lt;br&gt;0.921875<br /> | 53/64&lt;br&gt;0.828125<br /> | 46/64&lt;br&gt;0.71875<br /> | 41/64&lt;br&gt;0.640625<br /> | 35/64&lt;br&gt;0.546875<br /> | 24/64&lt;br&gt;0.375<br /> | 12/64&lt;br&gt;0.1875<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | 74/64&lt;br&gt;1.15625<br /> | 74/64&lt;br&gt;1.15625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.78294<br /> | [[Image:16x32rd.png|64px]]<br /> |}<br /> <br /> == Type-IV Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !8x16<br /> !16x32<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |9.6005<br /> |9.9057<br /> |-<br /> |Approximate<br /> |8.63473<br /> |9.60021<br /> |9.89338<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00029<br /> |0.01232<br /> |}<br /> <br /> == 8x16 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -25/64&lt;br&gt;-0.390625<br /> | -20/64&lt;br&gt;-0.3125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 49/64&lt;br&gt;0.765625<br /> | 35/64&lt;br&gt;0.546875<br /> | 21/64&lt;br&gt;0.328125<br /> | 90/64&lt;br&gt;1.40625<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.6112<br /> | [[Image:8x16_type3.png|64px]]<br /> |}<br /> <br /> == Type-III Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !8x16<br /> !16x32<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |9.6115<br /> |9.9496<br /> |-<br /> |Approximate<br /> |8.63473<br /> |9.6112<br /> |<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00030<br /> |<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13742 2012-11-03T17:51:25Z

<p>Unlord: /* Type-IV Coding Gain */</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max CG<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -32/64&lt;br&gt;-0.5<br /> | -28/64&lt;br&gt;-0.4375<br /> | -24/64&lt;br&gt;-0.375<br /> | -32/64&lt;br&gt;-0.5<br /> | -24/64&lt;br&gt;-0.375<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 59/64&lt;br&gt;0.921875<br /> | 53/64&lt;br&gt;0.828125<br /> | 46/64&lt;br&gt;0.71875<br /> | 41/64&lt;br&gt;0.640625<br /> | 35/64&lt;br&gt;0.546875<br /> | 24/64&lt;br&gt;0.375<br /> | 12/64&lt;br&gt;0.1875<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | 74/64&lt;br&gt;1.15625<br /> | 74/64&lt;br&gt;1.15625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.78294<br /> | [[Image:16x32rd.png|64px]]<br /> |}<br /> <br /> == Type-IV Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !8x16<br /> !16x32<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |9.6005<br /> |9.9057<br /> |-<br /> |Approximate<br /> |8.63473<br /> |9.60021<br /> |9.89338<br /> |-<br /> |Loss<br /> |0.00017<br /> |0.00029<br /> |0.01232<br /> |}<br /> <br /> == 8x16 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -25/64&lt;br&gt;-0.390625<br /> | -20/64&lt;br&gt;-0.3125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 49/64&lt;br&gt;0.765625<br /> | 35/64&lt;br&gt;0.546875<br /> | 21/64&lt;br&gt;0.328125<br /> | 90/64&lt;br&gt;1.40625<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.6112<br /> | [[Image:8x16_type3.png|64px]]<br /> |</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13741 2012-11-03T17:47:51Z

<p>Unlord: </p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max CG<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -32/64&lt;br&gt;-0.5<br /> | -28/64&lt;br&gt;-0.4375<br /> | -24/64&lt;br&gt;-0.375<br /> | -32/64&lt;br&gt;-0.5<br /> | -24/64&lt;br&gt;-0.375<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 59/64&lt;br&gt;0.921875<br /> | 53/64&lt;br&gt;0.828125<br /> | 46/64&lt;br&gt;0.71875<br /> | 41/64&lt;br&gt;0.640625<br /> | 35/64&lt;br&gt;0.546875<br /> | 24/64&lt;br&gt;0.375<br /> | 12/64&lt;br&gt;0.1875<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | 74/64&lt;br&gt;1.15625<br /> | 74/64&lt;br&gt;1.15625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.78294<br /> | [[Image:16x32rd.png|64px]]<br /> |}<br /> <br /> == Type-IV Coding Gain ==<br /> <br /> {|<br /> !<br /> !4x8<br /> !8x16<br /> !16x32<br /> |-<br /> |Real Valued<br /> |8.6349<br /> |9.6005<br /> |9.9057<br /> |-<br /> |Approximate<br /> |<br /> |}<br /> <br /> == 8x16 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -25/64&lt;br&gt;-0.390625<br /> | -20/64&lt;br&gt;-0.3125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 49/64&lt;br&gt;0.765625<br /> | 35/64&lt;br&gt;0.546875<br /> | 21/64&lt;br&gt;0.328125<br /> | 90/64&lt;br&gt;1.40625<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.6112<br /> | [[Image:8x16_type3.png|64px]]<br /> |</div>

Unlord https://wiki.xiph.org/index.php?title=File:8x16_type3.png&diff=13740 2012-11-03T17:43:22Z

<p>Unlord: (8x16 Type-III CG=9.6112 SBA=-19.7585 BO=0.0139179)</p> <hr /> <div>(8x16 Type-III CG=9.6112 SBA=-19.7585 BO=0.0139179)</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13739 2012-11-03T17:39:43Z

<p>Unlord: 8x16 Type-III parameters</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max CG<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -32/64&lt;br&gt;-0.5<br /> | -28/64&lt;br&gt;-0.4375<br /> | -24/64&lt;br&gt;-0.375<br /> | -32/64&lt;br&gt;-0.5<br /> | -24/64&lt;br&gt;-0.375<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 59/64&lt;br&gt;0.921875<br /> | 53/64&lt;br&gt;0.828125<br /> | 46/64&lt;br&gt;0.71875<br /> | 41/64&lt;br&gt;0.640625<br /> | 35/64&lt;br&gt;0.546875<br /> | 24/64&lt;br&gt;0.375<br /> | 12/64&lt;br&gt;0.1875<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | 74/64&lt;br&gt;1.15625<br /> | 74/64&lt;br&gt;1.15625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.78294<br /> | [[Image:16x32rd.png|64px]]<br /> |}<br /> <br /> <br /> == 8x16 Type-III ==<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -25/64&lt;br&gt;-0.390625<br /> | -20/64&lt;br&gt;-0.3125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 49/64&lt;br&gt;0.765625<br /> | 35/64&lt;br&gt;0.546875<br /> | 21/64&lt;br&gt;0.328125<br /> | 90/64&lt;br&gt;1.40625<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.6112<br /> | [[Image:8x16_type3.png|64px]]<br /> |</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13556 2012-08-15T13:18:39Z

<p>Unlord: /* 16x32 */</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max CG<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -32/64&lt;br&gt;-0.5<br /> | -28/64&lt;br&gt;-0.4375<br /> | -24/64&lt;br&gt;-0.375<br /> | -32/64&lt;br&gt;-0.5<br /> | -24/64&lt;br&gt;-0.375<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 59/64&lt;br&gt;0.921875<br /> | 53/64&lt;br&gt;0.828125<br /> | 46/64&lt;br&gt;0.71875<br /> | 41/64&lt;br&gt;0.640625<br /> | 35/64&lt;br&gt;0.546875<br /> | 24/64&lt;br&gt;0.375<br /> | 12/64&lt;br&gt;0.1875<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | 74/64&lt;br&gt;1.15625<br /> | 74/64&lt;br&gt;1.15625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.78294<br /> | [[Image:16x32rd.png|64px]]<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=File:16x32rd.png&diff=13551 2012-08-04T18:18:24Z

<p>Unlord: uploaded a new version of &amp;quot;File:16x32rd.png&amp;quot;: 16x32 R=t,D=t CG=9.73109 SBA=-16.1398 BO=0.236722 W=20.7699</p> <hr /> <div></div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13550 2012-08-04T18:13:28Z

<p>Unlord: /* 16x32 */</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max CG<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -28/64&lt;br&gt;-0.4375<br /> | -34/64&lt;br&gt;-0.53125<br /> | -16/64&lt;br&gt;-0.25<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 59/64&lt;br&gt;0.921875<br /> | 54/64&lt;br&gt;0.84375<br /> | 49/64&lt;br&gt;0.765625<br /> | 38/64&lt;br&gt;0.59375<br /> | 30/64&lt;br&gt;0.46875<br /> | 24/64&lt;br&gt;0.375<br /> | 12/64&lt;br&gt;0.1875<br /> | 80/64&lt;br&gt;1.25<br /> | 65/64&lt;br&gt;1.015625<br /> | 65/64&lt;br&gt;1.015625<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | 69/64&lt;br&gt;1.078125<br /> | 74/64&lt;br&gt;1.15625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.73109<br /> | [[Image:16x32rd.png|64px]]<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=File:16x32.png&diff=13540 2012-07-27T05:43:44Z

<p>Unlord: uploaded a new version of &amp;quot;File:16x32.png&amp;quot;: 16x32 R=f CG=9.89338 SBA=-16.5126 BO=0.0705998 W=18.8955</p> <hr /> <div>16x32 R=f</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13539 2012-07-27T05:39:16Z

<p>Unlord: /* 16x32 */</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max CG<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.109375<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -16/64&lt;br&gt;-0.25<br /> | -16/64&lt;br&gt;-0.25<br /> | -8/64&lt;br&gt;-0.125<br /> | -17/64&lt;br&gt;-0.265625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 57/64&lt;br&gt;0.890625<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 32/64&lt;br&gt;0.5<br /> | 32/64&lt;br&gt;0.5<br /> | 28/64&lt;br&gt;0.4375<br /> | 12/64&lt;br&gt;0.1875<br /> | 112/64&lt;br&gt;1.75<br /> | 84/64&lt;br&gt;1.3125<br /> | 88/64&lt;br&gt;1.375<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.65884<br /> | [[Image:16x32rd.png|64px]]<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13538 2012-07-27T05:36:46Z

<p>Unlord: /* 16x32 */</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max CG<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -7/64&lt;br&gt;-0.125<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 71/64&lt;br&gt;1.109375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89338<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -16/64&lt;br&gt;-0.25<br /> | -16/64&lt;br&gt;-0.25<br /> | -8/64&lt;br&gt;-0.125<br /> | -17/64&lt;br&gt;-0.265625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 57/64&lt;br&gt;0.890625<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 32/64&lt;br&gt;0.5<br /> | 32/64&lt;br&gt;0.5<br /> | 28/64&lt;br&gt;0.4375<br /> | 12/64&lt;br&gt;0.1875<br /> | 112/64&lt;br&gt;1.75<br /> | 84/64&lt;br&gt;1.3125<br /> | 88/64&lt;br&gt;1.375<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.65884<br /> | [[Image:16x32rd.png|64px]]<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=File:16x32.png&diff=13532 2012-07-21T05:38:03Z

<p>Unlord: uploaded a new version of &amp;quot;File:16x32.png&amp;quot;: 16x32 R=f CG=9.89269 SBA=-16.5439 BO=0.072092 W=18.9268</p> <hr /> <div>16x32 R=f</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13531 2012-07-21T05:33:47Z

<p>Unlord: /* 16x32 */</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max CG<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -8/64&lt;br&gt;-0.125<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 15/64&lt;br&gt;0.234375<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89269<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -16/64&lt;br&gt;-0.25<br /> | -16/64&lt;br&gt;-0.25<br /> | -8/64&lt;br&gt;-0.125<br /> | -17/64&lt;br&gt;-0.265625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 57/64&lt;br&gt;0.890625<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 32/64&lt;br&gt;0.5<br /> | 32/64&lt;br&gt;0.5<br /> | 28/64&lt;br&gt;0.4375<br /> | 12/64&lt;br&gt;0.1875<br /> | 112/64&lt;br&gt;1.75<br /> | 84/64&lt;br&gt;1.3125<br /> | 88/64&lt;br&gt;1.375<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.65884<br /> | [[Image:16x32rd.png|64px]]<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=File:16x32.png&diff=13527 2012-07-19T22:36:43Z

<p>Unlord: uploaded a new version of &amp;quot;File:16x32.png&amp;quot;: 16x32 R=f CG=9.89242 SBA=-16.4861 BO=0.0707185 W=18.9345</p> <hr /> <div>16x32 R=f</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13526 2012-07-19T22:34:50Z

<p>Unlord: /* 16x32 */</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max CG<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -24/64&lt;br&gt;-0.375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -8/64&lt;br&gt;-0.125<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 31/64&lt;br&gt;0.484375<br /> | 22/64&lt;br&gt;0.34375<br /> | 18/64&lt;br&gt;0.28125<br /> | 16/64&lt;br&gt;0.25<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 74/64&lt;br&gt;1.15625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 68/64&lt;br&gt;1.0625<br /> | 72/64&lt;br&gt;1.125<br /> | &amp;nbsp;&lt;br&gt;9.89242<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -16/64&lt;br&gt;-0.25<br /> | -16/64&lt;br&gt;-0.25<br /> | -8/64&lt;br&gt;-0.125<br /> | -17/64&lt;br&gt;-0.265625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 57/64&lt;br&gt;0.890625<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 32/64&lt;br&gt;0.5<br /> | 32/64&lt;br&gt;0.5<br /> | 28/64&lt;br&gt;0.4375<br /> | 12/64&lt;br&gt;0.1875<br /> | 112/64&lt;br&gt;1.75<br /> | 84/64&lt;br&gt;1.3125<br /> | 88/64&lt;br&gt;1.375<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.65884<br /> | [[Image:16x32rd.png|64px]]<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=File:16x32.png&diff=13521 2012-07-16T16:09:28Z

<p>Unlord: uploaded a new version of &amp;quot;File:16x32.png&amp;quot;: 16x32 R=f CG=9.87871 SBA=-15.9328 BO=0.0969324 W=19.0401</p> <hr /> <div>16x32 R=f</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13520 2012-07-16T16:01:37Z

<p>Unlord: /* 16x32 */</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max CG<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -23/64&lt;br&gt;-0.359375<br /> | -23/64&lt;br&gt;-0.359375<br /> | -17/64&lt;br&gt;-0.265625<br /> | -12/64&lt;br&gt;-0.1875<br /> | -14/64&lt;br&gt;-0.21875<br /> | -15/64&lt;br&gt; -0.234375<br /> | -12/64&lt;br&gt;-0.1875<br /> | 50/64&lt;br&gt;0.78125<br /> | 41/64&lt;br&gt;0.640625<br /> | 30/64&lt;br&gt;0.46875<br /> | 20/64&lt;br&gt;0.3125<br /> | 16/64&lt;br&gt;0.25<br /> | 15/64&lt;br&gt;0.234375<br /> | 11/64&lt;br&gt;0.171875<br /> | 90/64&lt;br&gt;1.40625<br /> | 72/64&lt;br&gt;1.125<br /> | 74/64&lt;br&gt;1.15625<br /> | 72/64&lt;br&gt;1.125<br /> | 67/64&lt;br&gt;1.046875<br /> | 66/64&lt;br&gt;1.03125<br /> | 68/64&lt;br&gt;1.0625<br /> | 74/64&lt;br&gt;1.15625<br /> | &amp;nbsp;&lt;br&gt;9.87871<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -16/64&lt;br&gt;-0.25<br /> | -16/64&lt;br&gt;-0.25<br /> | -8/64&lt;br&gt;-0.125<br /> | -17/64&lt;br&gt;-0.265625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 57/64&lt;br&gt;0.890625<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 32/64&lt;br&gt;0.5<br /> | 32/64&lt;br&gt;0.5<br /> | 28/64&lt;br&gt;0.4375<br /> | 12/64&lt;br&gt;0.1875<br /> | 112/64&lt;br&gt;1.75<br /> | 84/64&lt;br&gt;1.3125<br /> | 88/64&lt;br&gt;1.375<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.65884<br /> | [[Image:16x32rd.png|64px]]<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=Intra&diff=13503 2012-06-27T17:35:57Z

<p>Unlord: </p> <hr /> <div>In Daala, intra-frame predictors are coefficients that are used to predict the contents of a block based on its three neighboring blocks. Consider a 4x4 block structure:<br /> <br /> &lt;math&gt;<br /> \left(<br /> \begin{array}{cccc|cccc}<br /> x_{1}&amp;x_{2}&amp;x_{3}&amp;x_{4}&amp;x_{5}&amp;x_{6}&amp;x_{7}&amp;x_{8}\\<br /> x_{9}&amp;x_{10}&amp;x_{11}&amp;x_{12}&amp;x_{13}&amp;x_{14}&amp;x_{15}&amp;x_{16}\\<br /> x_{17}&amp;x_{18}&amp;x_{19}&amp;x_{20}&amp;x_{21}&amp;x_{22}&amp;x_{23}&amp;x_{24}\\<br /> x_{25}&amp;x_{26}&amp;x_{27}&amp;x_{28}&amp;x_{29}&amp;x_{30}&amp;x_{31}&amp;x_{32}\\ \hline<br /> x_{33}&amp;x_{34}&amp;x_{35}&amp;x_{36}&amp;y_{1}&amp;y_{2}&amp;y_{3}&amp;y_{4}\\<br /> x_{37}&amp;x_{38}&amp;x_{39}&amp;x_{40}&amp;y_{5}&amp;y_{6}&amp;y_{7}&amp;y_{8}\\<br /> x_{41}&amp;x_{42}&amp;x_{43}&amp;x_{44}&amp;y_{9}&amp;y_{10}&amp;y_{11}&amp;y_{12}\\<br /> x_{45}&amp;x_{46}&amp;x_{47}&amp;x_{48}&amp;y_{13}&amp;y_{14}&amp;y_{15}&amp;y_{16}\\<br /> \end{array}<br /> \right)<br /> &lt;/math&gt;<br /> <br /> Assuming a linear predictor, we would like to find the 768 coefficients that best predict &lt;math&gt;y_i&lt;/math&gt; from the neighboring &lt;math&gt;x_i&lt;/math&gt;. That is, for a given block with coefficients &lt;math&gt;\hat x&lt;/math&gt; and &lt;math&gt;\hat y&lt;/math&gt;, we would like a 48x16 matrix &lt;math&gt;\beta&lt;/math&gt; such that the residual &lt;math&gt;\hat e=\hat y-\hat x \beta&lt;/math&gt; is minimized (for some &lt;math&gt;p&lt;/math&gt;-norm). Because it is this residual &lt;math&gt;\hat e&lt;/math&gt; that is quantized and coded in the bitstream, we would prefer that most of its coefficients are zero. This will be discussed further when comparing the different &lt;math&gt;p&lt;/math&gt;-norms.<br /> <br /> == Block Modes ==<br /> <br /> These predictors often follow certain geometry in the time-domain image space. Imagine a vertical edge that runs through &lt;math&gt;y&lt;/math&gt;. A natural predictor might be to simply copy the values &lt;math&gt;\begin{array}{cccc}x_{29}&amp;x_{30}&amp;x_{31}&amp;x_{32}\end{array}&lt;/math&gt; down into each row of the 4x4 block &lt;math&gt;y&lt;/math&gt;. In order to account for the different possible geometries, each block is assigned a &lt;i&gt;mode&lt;/i&gt; which indicates what set of prediction coefficients to use (the &lt;math&gt;\beta&lt;/math&gt;). When encoding a block &lt;math&gt;y&lt;/math&gt;, the encoder chooses a mode, computes &lt;math&gt;\hat e&lt;/math&gt; and writes both the block mode and the residual &lt;math&gt;\hat e&lt;/math&gt;.<br /> <br /> Because the decoder must know the coefficients for each of the block modes, there is a tradeoff between how well we can predict the values in &lt;math&gt;y&lt;/math&gt; (number of block modes) with decoder complexity. Our hypothesis is that the optimal number of block modes will correspond to how closely we can fit different geometries inside the block. For 4x4 blocks, this might correspond to a mode for each of the 8 cardinal directions (ask Jason to clarify) with larger blocks potentially supporting a larger number of directions. In addition, we would like to support the DC and True Motion modes from WebM/VP8 as they are often the best fit. For common video sequences, anywhere from 20% to 45% of the intra frames use the True Motion mode [1].<br /> <br /> This is slightly complicated by the fact that our coefficients are not in the time-domain, but are the result of a lapped transform, see [[TDLT]]. In practice all of these modes have an equivalent in the frequency domain. As a starting point, we are using the 10 modes from Theora to classify the blocks from a set of sample images. Each category of blocks will be used to construct a set of predictors &lt;math&gt;\beta&lt;/math&gt; which is then used to reclassify the blocks based on Sum of Absolute Transform Differences (SATD). &lt;i&gt;Tim suggested weighting each blocks contribution based on SATD_bestfit-SATD_nearest&lt;/i&gt;.<br /> <br /> == &lt;math&gt;L^2&lt;/math&gt;-norm ==<br /> <br /> Suppose for some mode we have &lt;math&gt;n&lt;/math&gt; blocks. We can then construct the matrices &lt;math&gt;X&lt;/math&gt; and &lt;math&gt;Y&lt;/math&gt; where the ith row of each contains the zero-mean values from the ith block.<br /> <br /> &lt;math&gt;<br /> X=\begin{pmatrix}<br /> x_{1,1}-\bar{x_1} &amp; x_{1,2}-\bar{x_2} &amp; \cdots &amp; x_{1,m}-\bar{x_m}\\<br /> x_{2,1}-\bar{x_1} &amp; x_{2,2}-\bar{x_2} &amp; \cdots &amp; x_{2,m}-\bar{x_m}\\<br /> \vdots &amp; \vdots &amp; \ddots &amp; \vdots \\<br /> x_{n,1}-\bar{x_1} &amp; x_{n,2}-\bar{x_2} &amp; \cdots &amp; x_{n,m}-\bar{x_m}<br /> \end{pmatrix}<br /> &lt;/math&gt;<br /> <br /> &lt;math&gt;<br /> Y=\begin{pmatrix}<br /> y_{1,1}-\bar{y_1} &amp; y_{1,2}-\bar{y_2} &amp; \cdots &amp; y_{1,l}-\bar{y_l}\\<br /> y_{2,1}-\bar{y_1} &amp; y_{2,2}-\bar{y_2} &amp; \cdots &amp; y_{2,l}-\bar{y_l}\\<br /> \vdots &amp; \vdots &amp; \ddots &amp; \vdots \\<br /> y_{n,1}-\bar{y_1} &amp; y_{n,2}-\bar{y_2} &amp; \cdots &amp; y_{n,l}-\bar{y_l}<br /> \end{pmatrix}<br /> &lt;/math&gt;<br /> <br /> In the case of 4x4 blocks, &lt;math&gt;m=48&lt;/math&gt; and &lt;math&gt;l=16&lt;/math&gt;. Let &lt;math&gt;C=X^T X&lt;/math&gt; and &lt;math&gt;D=X^T Y&lt;/math&gt;. Then &lt;math&gt;\beta=C^{-1} D&lt;/math&gt;.<br /> <br /> [1] http://blog.webmproject.org/2010/07/inside-webm-technology-vp8-intra-and.html</div>

Unlord https://wiki.xiph.org/index.php?title=Intra&diff=13502 2012-06-27T02:15:13Z

<p>Unlord: /* L^2-norm */ now subtracting the correct mean</p> <hr /> <div>In general, intra-frame predictors are coefficients that are used to predict the contents of a block based on its three neighboring blocks. Consider a 4x4 block structure:<br /> <br /> &lt;math&gt;<br /> \left(<br /> \begin{array}{cccc|cccc}<br /> x_{1}&amp;x_{2}&amp;x_{3}&amp;x_{4}&amp;x_{5}&amp;x_{6}&amp;x_{7}&amp;x_{8}\\<br /> x_{9}&amp;x_{10}&amp;x_{11}&amp;x_{12}&amp;x_{13}&amp;x_{14}&amp;x_{15}&amp;x_{16}\\<br /> x_{17}&amp;x_{18}&amp;x_{19}&amp;x_{20}&amp;x_{21}&amp;x_{22}&amp;x_{23}&amp;x_{24}\\<br /> x_{25}&amp;x_{26}&amp;x_{27}&amp;x_{28}&amp;x_{29}&amp;x_{30}&amp;x_{31}&amp;x_{32}\\ \hline<br /> x_{33}&amp;x_{34}&amp;x_{35}&amp;x_{36}&amp;y_{1}&amp;y_{2}&amp;y_{3}&amp;y_{4}\\<br /> x_{37}&amp;x_{38}&amp;x_{39}&amp;x_{40}&amp;y_{5}&amp;y_{6}&amp;y_{7}&amp;y_{8}\\<br /> x_{41}&amp;x_{42}&amp;x_{43}&amp;x_{44}&amp;y_{9}&amp;y_{10}&amp;y_{11}&amp;y_{12}\\<br /> x_{45}&amp;x_{46}&amp;x_{47}&amp;x_{48}&amp;y_{13}&amp;y_{14}&amp;y_{15}&amp;y_{16}\\<br /> \end{array}<br /> \right)<br /> &lt;/math&gt;<br /> <br /> Assuming a linear predictor, we would like to find the 768 coefficients that best predict &lt;math&gt;y_i&lt;/math&gt; from the neighboring &lt;math&gt;x_i&lt;/math&gt;. That is, for a given block with coefficients &lt;math&gt;\hat x&lt;/math&gt; and &lt;math&gt;\hat y&lt;/math&gt;, we would like a 48x16 matrix &lt;math&gt;\beta&lt;/math&gt; such that the residual &lt;math&gt;\hat e=\hat y-\hat x \beta&lt;/math&gt; is minimized (for some &lt;math&gt;p&lt;/math&gt;-norm). Because it is this residual &lt;math&gt;\hat e&lt;/math&gt; that is quantized and coded in the bitstream, we would prefer that most of its coefficients are zero. This will be discussed further when comparing the different &lt;math&gt;p&lt;/math&gt;-norms.<br /> <br /> == Block Modes ==<br /> <br /> These predictors often follow certain geometry in the time-domain image space. Imagine a vertical edge that runs through &lt;math&gt;y&lt;/math&gt;. A natural predictor might be to simply copy the values &lt;math&gt;\begin{array}{cccc}x_{29}&amp;x_{30}&amp;x_{31}&amp;x_{32}\end{array}&lt;/math&gt; down into each row of the 4x4 block &lt;math&gt;y&lt;/math&gt;. In order to account for the different possible geometries, each block is assigned a &lt;i&gt;mode&lt;/i&gt; which indicates what set of prediction coefficients to use (the &lt;math&gt;\beta&lt;/math&gt;). When encoding a block &lt;math&gt;y&lt;/math&gt;, the encoder chooses a mode, computes &lt;math&gt;\hat e&lt;/math&gt; and writes both the block mode and the residual &lt;math&gt;\hat e&lt;/math&gt;.<br /> <br /> Because the decoder must know the coefficients for each of the block modes, there is a tradeoff between how well we can predict the values in &lt;math&gt;y&lt;/math&gt; (number of block modes) with decoder complexity. Our hypothesis is that the optimal number of block modes will correspond to how closely we can fit different geometries inside the block. For 4x4 blocks, this might correspond to a mode for each of the 8 cardinal directions (ask Jason to clarify) with larger blocks potentially supporting a larger number of directions. In addition, we would like to support the DC and True Motion modes from Theora/VP3 and WebM/VP8 as they are often the best fit. For common video sequences, In the case of True Motion, anywhere from 20% to 45% of the intra frames use this mode are <br /> <br /> This is slightly complicated by the fact that our coefficients are not in the time-domain, but are the result of a lapped transform, see [[TDLT]]. In practice all of these modes have an equivalent in the frequency domain (even True Motion?). As a starting point, we are using the 10 modes from Theora to classify the blocks from a set of sample images. Each category of blocks will be used to construct a set of predictors &lt;math&gt;\beta&lt;/math&gt; which is then used to reclassify the blocks based on Sum of Absolute Transform Differences (SADT). &lt;i&gt;Tim suggested weighting each blocks contribution based on SADT_bestfit-SADT_nearest&lt;/i&gt;.<br /> <br /> == &lt;math&gt;L^2&lt;/math&gt;-norm ==<br /> <br /> Suppose for some mode we have &lt;math&gt;n&lt;/math&gt; blocks. We can then construct the matrices &lt;math&gt;X&lt;/math&gt; and &lt;math&gt;Y&lt;/math&gt; where the ith row of each contains the zero-mean values from the ith block.<br /> <br /> &lt;math&gt;<br /> X=\begin{pmatrix}<br /> x_{1,1}-\bar{x_1} &amp; x_{1,2}-\bar{x_2} &amp; \cdots &amp; x_{1,m}-\bar{x_m}\\<br /> x_{2,1}-\bar{x_1} &amp; x_{2,2}-\bar{x_2} &amp; \cdots &amp; x_{2,m}-\bar{x_m}\\<br /> \vdots &amp; \vdots &amp; \ddots &amp; \vdots \\<br /> x_{n,1}-\bar{x_1} &amp; x_{n,2}-\bar{x_2} &amp; \cdots &amp; x_{n,m}-\bar{x_m}<br /> \end{pmatrix}<br /> &lt;/math&gt;<br /> <br /> &lt;math&gt;<br /> Y=\begin{pmatrix}<br /> y_{1,1}-\bar{y_1} &amp; y_{1,2}-\bar{y_2} &amp; \cdots &amp; y_{1,l}-\bar{y_l}\\<br /> y_{2,1}-\bar{y_1} &amp; y_{2,2}-\bar{y_2} &amp; \cdots &amp; y_{2,l}-\bar{y_l}\\<br /> \vdots &amp; \vdots &amp; \ddots &amp; \vdots \\<br /> y_{n,1}-\bar{y_1} &amp; y_{n,2}-\bar{y_2} &amp; \cdots &amp; y_{n,l}-\bar{y_l}<br /> \end{pmatrix}<br /> &lt;/math&gt;<br /> <br /> In the case of 4x4 blocks, &lt;math&gt;m=48&lt;/math&gt; and &lt;math&gt;l=16&lt;/math&gt;. Let &lt;math&gt;C=X^T X&lt;/math&gt; and &lt;math&gt;D=X^T Y&lt;/math&gt;. Then &lt;math&gt;\beta=C^{-1} D&lt;/math&gt;.</div>

Unlord https://wiki.xiph.org/index.php?title=Intra&diff=13501 2012-06-27T01:21:40Z

<p>Unlord: Created page with &quot;In general, intra-frame predictors are coefficients that are used to predict the contents of a block based on its three neighboring blocks. Consider a 4x4 block structure: &lt;mat...&quot;</p> <hr /> <div>In general, intra-frame predictors are coefficients that are used to predict the contents of a block based on its three neighboring blocks. Consider a 4x4 block structure:<br /> <br /> &lt;math&gt;<br /> \left(<br /> \begin{array}{cccc|cccc}<br /> x_{1}&amp;x_{2}&amp;x_{3}&amp;x_{4}&amp;x_{5}&amp;x_{6}&amp;x_{7}&amp;x_{8}\\<br /> x_{9}&amp;x_{10}&amp;x_{11}&amp;x_{12}&amp;x_{13}&amp;x_{14}&amp;x_{15}&amp;x_{16}\\<br /> x_{17}&amp;x_{18}&amp;x_{19}&amp;x_{20}&amp;x_{21}&amp;x_{22}&amp;x_{23}&amp;x_{24}\\<br /> x_{25}&amp;x_{26}&amp;x_{27}&amp;x_{28}&amp;x_{29}&amp;x_{30}&amp;x_{31}&amp;x_{32}\\ \hline<br /> x_{33}&amp;x_{34}&amp;x_{35}&amp;x_{36}&amp;y_{1}&amp;y_{2}&amp;y_{3}&amp;y_{4}\\<br /> x_{37}&amp;x_{38}&amp;x_{39}&amp;x_{40}&amp;y_{5}&amp;y_{6}&amp;y_{7}&amp;y_{8}\\<br /> x_{41}&amp;x_{42}&amp;x_{43}&amp;x_{44}&amp;y_{9}&amp;y_{10}&amp;y_{11}&amp;y_{12}\\<br /> x_{45}&amp;x_{46}&amp;x_{47}&amp;x_{48}&amp;y_{13}&amp;y_{14}&amp;y_{15}&amp;y_{16}\\<br /> \end{array}<br /> \right)<br /> &lt;/math&gt;<br /> <br /> Assuming a linear predictor, we would like to find the 768 coefficients that best predict &lt;math&gt;y_i&lt;/math&gt; from the neighboring &lt;math&gt;x_i&lt;/math&gt;. That is, for a given block with coefficients &lt;math&gt;\hat x&lt;/math&gt; and &lt;math&gt;\hat y&lt;/math&gt;, we would like a 48x16 matrix &lt;math&gt;\beta&lt;/math&gt; such that the residual &lt;math&gt;\hat e=\hat y-\hat x \beta&lt;/math&gt; is minimized (for some &lt;math&gt;p&lt;/math&gt;-norm). Because it is this residual &lt;math&gt;\hat e&lt;/math&gt; that is quantized and coded in the bitstream, we would prefer that most of its coefficients are zero. This will be discussed further when comparing the different &lt;math&gt;p&lt;/math&gt;-norms.<br /> <br /> == Block Modes ==<br /> <br /> These predictors often follow certain geometry in the time-domain image space. Imagine a vertical edge that runs through &lt;math&gt;y&lt;/math&gt;. A natural predictor might be to simply copy the values &lt;math&gt;\begin{array}{cccc}x_{29}&amp;x_{30}&amp;x_{31}&amp;x_{32}\end{array}&lt;/math&gt; down into each row of the 4x4 block &lt;math&gt;y&lt;/math&gt;. In order to account for the different possible geometries, each block is assigned a &lt;i&gt;mode&lt;/i&gt; which indicates what set of prediction coefficients to use (the &lt;math&gt;\beta&lt;/math&gt;). When encoding a block &lt;math&gt;y&lt;/math&gt;, the encoder chooses a mode, computes &lt;math&gt;\hat e&lt;/math&gt; and writes both the block mode and the residual &lt;math&gt;\hat e&lt;/math&gt;.<br /> <br /> Because the decoder must know the coefficients for each of the block modes, there is a tradeoff between how well we can predict the values in &lt;math&gt;y&lt;/math&gt; (number of block modes) with decoder complexity. Our hypothesis is that the optimal number of block modes will correspond to how closely we can fit different geometries inside the block. For 4x4 blocks, this might correspond to a mode for each of the 8 cardinal directions (ask Jason to clarify) with larger blocks potentially supporting a larger number of directions. In addition, we would like to support the DC and True Motion modes from Theora/VP3 and WebM/VP8 as they are often the best fit. For common video sequences, In the case of True Motion, anywhere from 20% to 45% of the intra frames use this mode are <br /> <br /> This is slightly complicated by the fact that our coefficients are not in the time-domain, but are the result of a lapped transform, see [[TDLT]]. In practice all of these modes have an equivalent in the frequency domain (even True Motion?). As a starting point, we are using the 10 modes from Theora to classify the blocks from a set of sample images. Each category of blocks will be used to construct a set of predictors &lt;math&gt;\beta&lt;/math&gt; which is then used to reclassify the blocks based on Sum of Absolute Transform Differences (SADT). &lt;i&gt;Tim suggested weighting each blocks contribution based on SADT_bestfit-SADT_nearest&lt;/i&gt;.<br /> <br /> == &lt;math&gt;L^2&lt;/math&gt;-norm ==<br /> <br /> Suppose for some mode we have &lt;math&gt;n&lt;/math&gt; blocks. We can then construct the matrices &lt;math&gt;X&lt;/math&gt; and &lt;math&gt;Y&lt;/math&gt; where the ith row of each contains the zero-mean values from the ith block.<br /> <br /> &lt;math&gt;<br /> X=\begin{pmatrix}<br /> x_{1,1}-\bar{x_1} &amp; x_{1,2}-\bar{x_1} &amp; \cdots &amp; x_{1,m}-\bar{x_1}\\<br /> x_{2,1}-\bar{x_2} &amp; x_{2,2}-\bar{x_2} &amp; \cdots &amp; x_{2,m}-\bar{x_2}\\<br /> \vdots &amp; \vdots &amp; \ddots &amp; \vdots \\<br /> x_{n,1}-\bar{x_n} &amp; x_{n,2}-\bar{x_n} &amp; \cdots &amp; x_{n,m}-\bar{x_n}<br /> \end{pmatrix}<br /> &lt;/math&gt;<br /> <br /> &lt;math&gt;<br /> Y=\begin{pmatrix}<br /> y_{1,1}-\bar y_1 &amp; y_{1,2}-\bar y_1 &amp; \cdots &amp; y_{1,l}-\bar y_1\\<br /> y_{2,1}-\bar y_2 &amp; y_{2,2}-\bar y_2 &amp; \cdots &amp; y_{2,l}-\bar y_2\\<br /> \vdots &amp; \vdots &amp; \ddots &amp; \vdots \\<br /> y_{n,1}-\bar y_n &amp; y_{n,2}-\bar y_n &amp; \cdots &amp; y_{n,l}-\bar y_n <br /> \end{pmatrix}<br /> &lt;/math&gt;<br /> <br /> In the case of 4x4 blocks, &lt;math&gt;m=48&lt;/math&gt; and &lt;math&gt;l=16&lt;/math&gt;. Let &lt;math&gt;C=X^T X&lt;/math&gt; and &lt;math&gt;D=X^T Y&lt;/math&gt;. Then &lt;math&gt;\beta=C^{-1} D&lt;/math&gt;.</div>

Unlord https://wiki.xiph.org/index.php?title=File:16x32.png&diff=13487 2012-06-12T03:19:32Z

<p>Unlord: uploaded a new version of &amp;quot;File:16x32.png&amp;quot;: 16x32 R=f CG=9.85161 SBA=-16.5091 BO=0.12764 W=18.7201</p> <hr /> <div>16x32 R=f</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13486 2012-06-12T02:56:27Z

<p>Unlord: /* 16x32 */</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max CG<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -25/64&lt;br&gt;-0.390625<br /> | -39/64&lt;br&gt;-0.609375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -12/64&lt;br&gt;-0.1875<br /> | -10/64&lt;br&gt;-0.15625<br /> | -9/64&lt;br&gt;-0.140625<br /> | -6/64&lt;br&gt;-0.09375<br /> | 50/64&lt;br&gt;0.78125<br /> | 41/64&lt;br&gt;0.640625<br /> | 33/64&lt;br&gt;0.515625<br /> | 20/64&lt;br&gt;0.3125<br /> | 12/64&lt;br&gt;0.1875<br /> | 9/64&lt;br&gt;0.140625<br /> | 5/64&lt;br&gt;0.078125<br /> | 93/64&lt;br&gt;1.453125<br /> | 71/64&lt;br&gt;1.109375<br /> | 75/64&lt;br&gt;1.171875<br /> | 71/64&lt;br&gt;1.109375<br /> | 70/64&lt;br&gt;1.09375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 71/64&lt;br&gt;1.109375<br /> | &amp;nbsp;&lt;br&gt;9.85161<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -16/64&lt;br&gt;-0.25<br /> | -16/64&lt;br&gt;-0.25<br /> | -8/64&lt;br&gt;-0.125<br /> | -17/64&lt;br&gt;-0.265625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 57/64&lt;br&gt;0.890625<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 32/64&lt;br&gt;0.5<br /> | 32/64&lt;br&gt;0.5<br /> | 28/64&lt;br&gt;0.4375<br /> | 12/64&lt;br&gt;0.1875<br /> | 112/64&lt;br&gt;1.75<br /> | 84/64&lt;br&gt;1.3125<br /> | 88/64&lt;br&gt;1.375<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.65884<br /> | [[Image:16x32rd.png|64px]]<br /> |}</div>

Unlord https://wiki.xiph.org/index.php?title=File:4x8_-8,36.png&diff=13477 2012-06-05T07:44:38Z

<p>Unlord: 4x8 p=-8 q=36 CG=8.52182 SBA=23.80569257</p> <hr /> <div>4x8 p=-8 q=36 CG=8.52182 SBA=23.80569257</div>

Unlord https://wiki.xiph.org/index.php?title=TDLT&diff=13476 2012-06-05T02:25:22Z

<p>Unlord: /* 4x8 */</p> <hr /> <div>This page holds the results of Time Domain Lapped Transform (TDLT) optimization problems looking for integer transform coefficients that provide optimal coding gain. Wherever possible the assumptions are stated. Later we should include testing against actual image data to verify the results (see test data [http://people.xiph.org/~tterribe/tmp/subset1-y4m.tar.gz here]).<br /> <br /> The coding gain objective used as the objective is taken from slide 13 of Tim's presentation [http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf An Introduction to Video Coding]<br /> <br /> &lt;need figure with block matrix diagrams&gt;<br /> <br /> The free parameters are initially just the coefficients p_0,...,p_m,q_0,...,q_m where m=(n/2)-1. We limit these to being dyadic rationals, e.g., x/2^d with d=6, between [-1,1].<br /> <br /> Given p's and q's and assuming a linear ramp constrains the s's.<br /> <br /> == 4x8 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.18117338915051454<br /> <br /> q0 = 0.6331818230771687<br /> <br /> CG = 8.60603<br /> <br /> {|<br /> !<br /> !p0<br /> !q0<br /> !s0<br /> !s1<br /> !CG<br /> !SBA<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -11/64&lt;br&gt;-0.171875<br /> | 36/64&lt;br&gt;0.5625<br /> | 91/64&lt;br&gt;1.421875<br /> | 85/64&lt;br&gt;1.328125<br /> | &amp;nbsp;&lt;br&gt;8.63473<br /> | &amp;nbsp;&lt;br&gt;22.0331<br /> | [[Image:4x8.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -12/64&lt;br&gt;-0.1875<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 1093/768&lt;br&gt;1.423177<br /> | &amp;nbsp;&lt;br&gt;8.60486<br /> | &amp;nbsp;&lt;br&gt;20.0573<br /> | [[Image:4x8r.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max CG<br /> | -16/64&lt;br&gt;-0.25<br /> | 41/64&lt;br&gt;0.640625<br /> | 92/64&lt;br&gt;1.4375<br /> | 93/64&lt;br&gt;1.453125<br /> | &amp;nbsp;&lt;br&gt;8.59886<br /> | &amp;nbsp;&lt;br&gt;18.9411<br /> | [[Image:4x8rd.png|64px]]<br /> |-<br /> |R=t,D=t&lt;br&gt;max SBA<br /> | -8/64&lt;br&gt;-0.125<br /> | 30/64&lt;br&gt;0.46875<br /> | 136/64&lt;br&gt;2.125<br /> | 91/64&lt;br&gt;1.421875<br /> | &amp;nbsp;&lt;br&gt;8.23230<br /> | &amp;nbsp;&lt;br&gt;25.1934<br /> | [[Image:4x8rd_sba.png|64px]]<br /> |}<br /> <br /> == 8x16 ==<br /> <br /> Optimal real-valued coefficients for V:<br /> <br /> p0 = -0.39460731547057293<br /> <br /> p1 = -0.33002212811740816<br /> <br /> p2 = -0.12391270981321137<br /> <br /> q0 = 0.822154737511288<br /> <br /> q1 = 0.632488694485779<br /> <br /> q2 = 0.40214668677553894<br /> <br /> CG = 9.56867<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !q0<br /> !q1<br /> !q2<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -23/64&lt;br&gt;-0.359375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -6/64&lt;br&gt;-0.09375<br /> | 48/64&lt;br&gt;0.75<br /> | 34/64&lt;br&gt;0.53125<br /> | 20/64&lt;br&gt;0.3125<br /> | 90/64&lt;br&gt;1.40625<br /> | 73/64&lt;br&gt;1.140625<br /> | 72/64&lt;br&gt;1.125<br /> | 75/64&lt;br&gt;1.171875<br /> | &amp;nbsp;&lt;br&gt;9.60021<br /> | [[Image:8x16.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -22/64&lt;br&gt;-0.34375<br /> | -8/64&lt;br&gt;-0.125<br /> | 53/64&lt;br&gt;0.828125<br /> | 41/64&lt;br&gt;0.640625<br /> | 26/64&lt;br&gt;0.40625<br /> | 11/8&lt;br&gt;1.375<br /> | 879/768&lt;br&gt;1.14453125<br /> | 1469/1280&lt;br&gt;1.14765625<br /> | 275/224&lt;br&gt;1.2276785714285714<br /> | &amp;nbsp;&lt;br&gt;9.56627<br /> | [[Image:8x16r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -24/64&lt;br&gt;-0.375<br /> | -20/64&lt;br&gt;-0.3125<br /> | -4/64&lt;br&gt;-0.0625<br /> | 53/64&lt;br&gt;0.828125<br /> | 40/64&lt;br&gt;0.625<br /> | 24/64&lt;br&gt;0.375<br /> | 88/64&lt;br&gt;1.375<br /> | 75/64&lt;br&gt;1.171875<br /> | 76/64&lt;br&gt;1.1875<br /> | 76/64&lt;br&gt;1.1875<br /> | &amp;nbsp;&lt;br&gt;9.56161<br /> | [[Image:8x16rd.png|64px]]<br /> |}<br /> <br /> == 16x32 ==<br /> <br /> Best-known real-valued coefficients for V (R=t):<br /> <br /> p0 = -0.42111473798940136<br /> <br /> p1 = -0.4121736499899753<br /> <br /> p2 = -0.3350240707669929<br /> <br /> p3 = -0.3224547931861314<br /> <br /> p4 = -0.25883387978005545<br /> <br /> p5 = -0.20951913473498104<br /> <br /> p6 = -0.0598657149803332<br /> <br /> q0 = 0.9107782439906195<br /> <br /> q1 = 0.8109855829278226<br /> <br /> q2 = 0.715846584586721<br /> <br /> q3 = 0.6135951570714172<br /> <br /> q4 = 0.49846644853347627<br /> <br /> q5 = 0.3945215834922529<br /> <br /> q6 = 0.21822275136248082<br /> <br /> CG = 9.81157<br /> <br /> {|<br /> !<br /> !p0<br /> !p1<br /> !p2<br /> !p3<br /> !p4<br /> !p5<br /> !p6<br /> !q0<br /> !q1<br /> !q2<br /> !q3<br /> !q4<br /> !q5<br /> !q6<br /> !s0<br /> !s1<br /> !s2<br /> !s3<br /> !s4<br /> !s5<br /> !s6<br /> !s7<br /> !CG<br /> !Filterbank<br /> |-<br /> |R=f<br /> | -25/64&lt;br&gt;-0.390625<br /> | -39/64&lt;br&gt;-0.609375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -13/64&lt;br&gt;-0.203125<br /> | -11/64&lt;br&gt;-0.171875<br /> | -9/64&lt;br&gt;-0.140625<br /> | -9/64&lt;br&gt;-0.140625<br /> | 51/64&lt;br&gt;0.796875<br /> | 41/64&lt;br&gt;0.640625<br /> | 33/64&lt;br&gt;0.515625<br /> | 21/64&lt;br&gt;0.328125<br /> | 12/64&lt;br&gt;0.1875<br /> | 8/64&lt;br&gt;0.125<br /> | 5/64&lt;br&gt;0.078125<br /> | 93/64&lt;br&gt;1.453125<br /> | 71/64&lt;br&gt;1.109375<br /> | 75/64&lt;br&gt;1.171875<br /> | 71/64&lt;br&gt;1.109375<br /> | 70/64&lt;br&gt;1.09375<br /> | 67/64&lt;br&gt;1.046875<br /> | 67/64&lt;br&gt;1.046875<br /> | 71/64&lt;br&gt;1.109375<br /> | &amp;nbsp;&lt;br&gt;9.84702<br /> | [[Image:16x32.png|64px]]<br /> |-<br /> |R=t,D=f<br /> | -26/64&lt;br&gt;-0.40625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -22/64&lt;br&gt;-0.34375<br /> | -18/64&lt;br&gt;-0.28125<br /> | -16/64&lt;br&gt;-0.25<br /> | -14/64&lt;br&gt;-0.21875<br /> | -5/64&lt;br&gt;-0.078125<br /> | 58/64&lt;br&gt;0.90625<br /> | 52/64&lt;br&gt;0.8125<br /> | 45/64&lt;br&gt;0.703125<br /> | 36/64&lt;br&gt;0.5625<br /> | 31/64&lt;br&gt;0.484375<br /> | 23/64&lt;br&gt;0.359375<br /> | 13/64&lt;br&gt;0.203125<br /> | 3/2&lt;br&gt;1.5<br /> | 77/64&lt;br&gt;1.203125<br /> | 77/64&lt;br&gt;1.203125<br /> | 1105/896&lt;br&gt;1.23326<br /> | 218/192&lt;br&gt;1.135417<br /> | 197/176&lt;br&gt;1.119318<br /> | 1919/1664&lt;br&gt;1.153245<br /> | 4351/3840&lt;br&gt;1.133073<br /> | &amp;nbsp;&lt;br&gt;9.79398<br /> | [[Image:16x32r.png|64px]]<br /> |-<br /> |R=t,D=t<br /> | -16/64&lt;br&gt;-0.25<br /> | -16/64&lt;br&gt;-0.25<br /> | -8/64&lt;br&gt;-0.125<br /> | -17/64&lt;br&gt;-0.265625<br /> | -27/64&lt;br&gt;-0.421875<br /> | -13/64&lt;br&gt;-0.203125<br /> | -2/64&lt;br&gt;-0.03125<br /> | 57/64&lt;br&gt;0.890625<br /> | 50/64&lt;br&gt;0.78125<br /> | 40/64&lt;br&gt;0.625<br /> | 32/64&lt;br&gt;0.5<br /> | 32/64&lt;br&gt;0.5<br /> | 28/64&lt;br&gt;0.4375<br /> | 12/64&lt;br&gt;0.1875<br /> | 112/64&lt;br&gt;1.75<br /> | 84/64&lt;br&gt;1.3125<br /> | 88/64&lt;br&gt;1.375<br /> | 80/64&lt;br&gt;1.25<br /> | 72/64&lt;br&gt;1.125<br /> | 72/64&lt;br&gt;1.125<br /> | 73/64&lt;br&gt;1.140625<br /> | 70/64&lt;br&gt;1.09375<br /> | &amp;nbsp;&lt;br&gt;9.65884<br /> | [[Image:16x32rd.png|64px]]<br /> |}</div>

Unlord