UNIVERSITY OF CALIFORNIA, SANTA CRUZ

Kernel Regression for Image Processing and Reconstruction

Presented by Hiroyuki Takeda, Dr. Sina Farsiu, and Professor Peyman Milanfar

This page shows that the applicability of the kernel regression technique to a wide-class of problems:

The details of the kernel regression technique are described in here. The software package is also available here.

home

Gaussian Noise Removal

 Lena image 512 x 512 gray scale image Came from Playboy (more useful information is available here.) Noisy image Added white Gaussian noise (standard deviation 25) The noise is generated by MATLAB commands "randn" by initialized the seed with 0. RMSE (root mean square error) is around 25.

 Wavelet The state of art denoising method Bayes Least Square-Gaussian Scale Mixture Denoising in wavelet domain Proposed by Portilla and Simonceli in 2003 Codes available here Standard variation of noise is 25, which is the best parameter for the method. RMSE is 6.64 Iterative steering kernel regression The second order steering kernel regression with iterative filtering algorithm Gaussian kernel Global smoothing parameter is 2.4 The number of iterations is 7 RMSE is 6.66

Comment : Although the RMSE of our method is slightly worse than the one of the wavelet method, the ringing effect and some artifacts, which we can see in the denoised image by the wavelet method, are invisible in our result. We cannot tell which one is better than the other, and leave the judgement to everyone. However, we can say that, with only the assumption of zero mean noise (unlike the wavelet method assuming noise is Gaussian), the iterative steering kernel regression method did a good job.

top

Compression Artifact Reduction

 Pepper image 254 x 254 gray scale image Compressed image Compressed by MATLAB JPEG routine with a quality parameter 10 RMSE is 9.76

 Bilateral filter Equivalent to the zeroth order bilateral kernel regression Gaussian kernel for both spatial and radiometric kernels Spatial smoothing parameter is 2.0 Radiometric smoothing parameter is 4.1 Window size is 7 RMSE is 8.52 Iterative steering kernel regression The second order steering kernel regression with iterative filtering algorithm Gaussian kernel Global smoothing parameter is 2.0 The number of iterations is 3 RMSE is 8.49

top

Film Grain Reduction

 Noisy John F. Kennedy image 343 x 367 color image Real film grain noise (the statistics of noise is unknown) Wavelet The state of art denoising method Bayes Least Square-Gaussian Scale Mixture Denoising in wavelet domain Proposed by Portilla and Simonceli in 2003 Codes available here Standard variation of noise is assumed 7, which produced a visually good result

 Bilateral filter Equivalent to the zeroth order bilateral kernel regression Gaussian kernel for both spatial and radiometric kernels Spatial smoothing parameter is 2.0 Radiometric smoothing parameter is 3.5 Window size is 7 Iterative steering kernel regression The second order steering kernel regression with iterative filtering algorithm Gaussian kernel Global smoothing parameter is 2.0 The number of iterations is 3

 Wavelet Bilateral filter Iterative steering kernel regression

Residuals

• The absolute values of the residuals on the illuminance channel

Almost no objects are visible in the residual image of iterative steering kernel regression, therefore the iterative method removed noise the most effectively.

Note: We did this denoising experiment in the YCrCb channels.

top

Image Upscale

 Downsampled image Downsample the original Lena image with factor 3 for both vertical and horizontal directions Noiseless Spline smoother An interpolation technique regularizes on the second derivatives has some denoising effects Regularization parameter is 0.0006 RMSE is 7.92

 Lanczos interpolation A well-known linear interpolation method RMSE is 9.78 Bicubic interpolation A common linear intepolation method RMSE is 7.93

 Classic kernel regression Linear interpolation method Gaussian kernel The second order Global smoothing parameter is 1.0 RMSE is 8.03 Iterative steering kernel regression The second order steering kernel regression with iterative filtering algorithm Gaussian kernel Global smoothing parameter is 1.25 The number of iterations is 0 RMSE is 7.42

top

Image Reconstruction from an Irregularly Sampled Data Set

 Irregularly sampled data set Randomly delete 85% of pixels of Lena image The black pixels represent the place we do not have values Delaunay-spline smoother* Regularization parameter is 0.087 RMSE is 9.05

 Classic kernel regression Gaussian kernel The second order Global smoothing parameter is 2.25 RMSE is 9.69 Iterative steering kernel regression The second order steering kernel regression with iterative filtering algorithm Gaussian kernel Global smoothing parameter is 1.6 The number of iterations is 1 RMSE is 8.21
*To implement the Delaunay-spline smoother we used MATLAB's "griddata" function with "cubic" parameter to transform the irregularly sampled data set to a dense regularly sampled data set (Delaunay triangulation). The quality of the resulting image was further enhanced by applying MATLAB's spline smoother routine "csaps".

top

Image Fusion

Tank Sequence

 Tank sequence 64 x 64 video sequence 8 frames Taken by an infrared camera Courtesy of B. Yasuda and the FLIR research groupe in the Sensors Technology Branch, Wright Laboratory, WPAFB, OH Classic kernel regression Gaussian kernel The zeroth order Global smoothing parameter is 0.8 Resolution enhancement factor is 5 for both vertical and horizontal directions

 Classic kernel regression Gaussian kernel The second order Global smoothing parameter is 0.8 Resolution enhancement factor is 5 for both vertical and horizontal directions Deblurring result Deblurring on the image reconstructed by the second order classic kernel regression PSF is the 5 x 5 Gaussian with standard deviation 1.0 Used the bilateral total variation regularization

Emily Sequence

 Emily sequence 35 x 54 video sequence 53 frames Taken by a commercial webcam (3COM, Model No.3718) Single-frame Delaunay-spline smoother Single frame upscaling Resolution enhancement factor is 5 Regularization factor is 0.01

 Delaunay-spline smoother Multi frame upscaling Resolution enhancement factor is 5 Regularization factor is 0.067 Iterative steering kernel regression The second order steering kernel regression with iterative filtering algorithm Gaussian kernel Global smoothing parameter is 1.0 The number of iterations is 0

top

Super Resolution

 A real compressed color video sequence 90 x 90 color video sequence 10 frames Taken by a commercial video surveillance camera courtesy of Adyoron Intelligent Systems, Ltd., Tel Aviv, Israel Delaunay-spline smoother Multi frame upscaling Resolution enhancement factor is 5 Regularization parameters for Y, Cr, and Cb are 0.5, 1.0, and 1.0, respectively Deblurring PSF is the 5 x 5 disk kernel Bilateral total variation regularization

 Classic kernel regression Resolution enhancement factor is 5 Gaussian kernel The second order Global smoothing parameters for Y, Cr, and Cb are 2.0, 3.5, and 3.5, respectively Deblurring PSF is the 5 x 5 disk kernel Bilateral total variation regularization Iterative steering kernel regression Resolution enhancement factor is 5 Gaussian kernel The second order steering kernel regression with iterative filtering algorithm The number of iterations is 1 Global smoothing parameters for Y, Cr, and Cb are 4.0, 8.0, and 8.0, respectively Deblurring PSF is the 5 x 5 disk kernel Bilateral total variation regularization

top