Quantization of random orthonormal matrices with application to adaptive transform coding

This thesis explores the quantization of orthonormal random matrices, an important component in the adaptive transform coding of images and video. While vector quantization in Euclidean space has been extensively studied, applying these methods to orthonormal matrices is challenging due to the inherent orthogonality constraints. Directly solving the constrained optimization problem in Euclidean space is difficult. Therefore, this research investigates several novel constructive approaches to the quantization of orthonormal random matrices, specifically for application in video and image compression. These matrices serve as approximations to the Karhunen-Loève Transforms (KLTs) of pixel blocks in images. The main objective of this thesis is to design an optimal codebook of transform matrices that can be utilized in adaptive transform coding, thereby enhancing compression efficiency. This thesis first investigates solving the constrained optimization problem in Euclidean space as an unconstrained optimization problem on the orthonormal matrix manifold. The primary goal is to minimize the mean square error (MSE) of transform coding, and this thesis derives an objective function for minimization based on high-rate analysis. Since the minimization problem lacks a closed-form solution, this thesis proposes a coordinate descent algorithm, which is guaranteed to converge to a local minimum. The proposed algorithm can be used to design both separable and non-separable transforms from sample image data. Experimental results demonstrate that adaptive transform coding using codebooks designed by the proposed algorithm outperforms both non-adaptive coding based on the widely used two-dimensional discrete cosine transform (2D-DCT) and adaptive coding using transform codebooks designed by various recently reported methods. This thesis further investigates a model-based approach to transform matrix codebook learning by modeling natural image blocks as finite lattice non-causal homogeneous Gauss Markov random fields (GMRFs) with Neumann boundary conditions. The proposed method involves the estimation of GMRF parameters. While the standard approach for GMRF parameter estimation is maximum likelihood, this thesis introduces a novel method based on high-rate analysis of transform coding, which focuses on minimizing the MSE of transform coding. In the proposed codebook design approach, the quantization of an orthonormal matrix is carried out in the parameter space of GMRF, transforming the complex task of quantizing a large matrix with orthonormality constraints into a much simpler task of vector quantization in a reduced-dimensional space. A very important advantage of the proposed method is that it can be easily used to design transforms for variable block-size adaptive transform coding. Experimental results are presented for adaptive transform coding of still images which compares the proposed approach against various other alternatives recently reported in the literature. Finally, this thesis addresses the problem of predictive quantization of a random orthonormal matrix process. While predictive quantization is commonly used to quantize correlated processes in Euclidean space, these methods are not directly applicable to processes on a manifold. The approach proposed in this thesis views the prediction problem as one of tracking KLT matrices on the orthonormal matrix manifold. A definition for matrix prediction error on the manifold is proposed. Numerical results demonstrate the effectiveness of utilizing predictive quantization of transform matrices to improve transform coding gain.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.