UNIVERSITY OF CALIFORNIA,
SANTA CRUZ
 
 
 
 

Kernel Regression for Image Processing and Reconstruction

Presented by Hiroyuki Takeda, Dr. Sina Farsiu, and Professor Peyman Milanfar

This page shows that the applicability of the kernel regression technique to a wide-class of problems:

The details of the kernel regression technique are described in here. The software package is also available here.

  
 
home
  
 

Gaussian Noise Removal

Lena image
  • 512 x 512 gray scale image
  • Came from Playboy (more useful information is available here.)

Noisy image

  • Added white Gaussian noise (standard deviation 25)
  • The noise is generated by MATLAB commands "randn" by initialized the seed with 0.
  • RMSE (root mean square error) is around 25.



Wavelet

  • The state of art denoising method
    • Bayes Least Square-Gaussian Scale Mixture
    • Denoising in wavelet domain
    • Proposed by Portilla and Simonceli in 2003
    • Codes available here
    • Standard variation of noise is 25, which is the best parameter for the method.
  • RMSE is 6.64

Iterative steering kernel regression

  • The second order steering kernel regression with iterative filtering algorithm
  • Gaussian kernel
  • Global smoothing parameter is 2.4
  • The number of iterations is 7
  • RMSE is 6.66

Comment : Although the RMSE of our method is slightly worse than the one of the wavelet method, the ringing effect and some artifacts, which we can see in the denoised image by the wavelet method, are invisible in our result. We cannot tell which one is better than the other, and leave the judgement to everyone. However, we can say that, with only the assumption of zero mean noise (unlike the wavelet method assuming noise is Gaussian), the iterative steering kernel regression method did a good job.

  
 
top
  
 

Compression Artifact Reduction

Pepper image

  • 254 x 254 gray scale image

Compressed image

  • Compressed by MATLAB JPEG routine with a quality parameter 10
  • RMSE is 9.76

Bilateral filter

  • Equivalent to the zeroth order bilateral kernel regression
  • Gaussian kernel for both spatial and radiometric kernels
  • Spatial smoothing parameter is 2.0
  • Radiometric smoothing parameter is 4.1
  • Window size is 7
  • RMSE is 8.52

Iterative steering kernel regression

  • The second order steering kernel regression with iterative filtering algorithm
  • Gaussian kernel
  • Global smoothing parameter is 2.0
  • The number of iterations is 3
  • RMSE is 8.49
 
 
top
  
 

Film Grain Reduction

Noisy John F. Kennedy image

  • 343 x 367 color image
  • Real film grain noise (the statistics of noise is unknown)

Wavelet

  • The state of art denoising method
    • Bayes Least Square-Gaussian Scale Mixture
    • Denoising in wavelet domain
    • Proposed by Portilla and Simonceli in 2003
    • Codes available here
    • Standard variation of noise is assumed 7, which produced a visually good result

Bilateral filter

  • Equivalent to the zeroth order bilateral kernel regression
  • Gaussian kernel for both spatial and radiometric kernels
  • Spatial smoothing parameter is 2.0
  • Radiometric smoothing parameter is 3.5
  • Window size is 7

Iterative steering kernel regression

  • The second order steering kernel regression with iterative filtering algorithm
  • Gaussian kernel
  • Global smoothing parameter is 2.0
  • The number of iterations is 3


Wavelet
Bilateral filter
Iterative steering kernel regression

Residuals

  • The absolute values of the residuals on the illuminance channel

Almost no objects are visible in the residual image of iterative steering kernel regression, therefore the iterative method removed noise the most effectively.
Note: We did this denoising experiment in the YCrCb channels.
  
 
top
  
 

Image Upscale

Downsampled image

  • Downsample the original Lena image with factor 3 for both vertical and horizontal directions
  • Noiseless

Spline smoother

  • An interpolation technique
    • regularizes on the second derivatives
    • has some denoising effects
  • Regularization parameter is 0.0006
  • RMSE is 7.92

Lanczos interpolation

  • A well-known linear interpolation method
  • RMSE is 9.78

Bicubic interpolation

  • A common linear intepolation method
  • RMSE is 7.93

Classic kernel regression

  • Linear interpolation method
  • Gaussian kernel
  • The second order
  • Global smoothing parameter is 1.0
  • RMSE is 8.03

Iterative steering kernel regression

  • The second order steering kernel regression with iterative filtering algorithm
  • Gaussian kernel
  • Global smoothing parameter is 1.25
  • The number of iterations is 0
  • RMSE is 7.42
 
 
top
  
 

Image Reconstruction from an Irregularly Sampled Data Set

Irregularly sampled data set

  • Randomly delete 85% of pixels of Lena image
  • The black pixels represent the place we do not have values

Delaunay-spline smoother*

  • Regularization parameter is 0.087
  • RMSE is 9.05

Classic kernel regression

  • Gaussian kernel
  • The second order
  • Global smoothing parameter is 2.25
  • RMSE is 9.69

Iterative steering kernel regression

  • The second order steering kernel regression with iterative filtering algorithm
  • Gaussian kernel
  • Global smoothing parameter is 1.6
  • The number of iterations is 1
  • RMSE is 8.21
*To implement the Delaunay-spline smoother we used MATLAB's "griddata" function with "cubic" parameter to transform the irregularly sampled data set to a dense regularly sampled data set (Delaunay triangulation). The quality of the resulting image was further enhanced by applying MATLAB's spline smoother routine "csaps".
  
 
top
  
 

Image Fusion

Tank Sequence

Tank sequence

  • 64 x 64 video sequence
  • 8 frames
  • Taken by an infrared camera
  • Courtesy of B. Yasuda and the FLIR research groupe in the Sensors Technology Branch, Wright Laboratory, WPAFB, OH

Classic kernel regression

  • Gaussian kernel
  • The zeroth order
  • Global smoothing parameter is 0.8
  • Resolution enhancement factor is 5 for both vertical and horizontal directions

Classic kernel regression

  • Gaussian kernel
  • The second order
  • Global smoothing parameter is 0.8
  • Resolution enhancement factor is 5 for both vertical and horizontal directions

Deblurring result

  • Deblurring on the image reconstructed by the second order classic kernel regression
  • PSF is the 5 x 5 Gaussian with standard deviation 1.0
  • Used the bilateral total variation regularization

Emily Sequence

Emily sequence

  • 35 x 54 video sequence
  • 53 frames
  • Taken by a commercial webcam (3COM, Model No.3718)

Single-frame Delaunay-spline smoother

  • Single frame upscaling
  • Resolution enhancement factor is 5
  • Regularization factor is 0.01

Delaunay-spline smoother

  • Multi frame upscaling
  • Resolution enhancement factor is 5
  • Regularization factor is 0.067

Iterative steering kernel regression

  • The second order steering kernel regression with iterative filtering algorithm
  • Gaussian kernel
  • Global smoothing parameter is 1.0
  • The number of iterations is 0
 
 
top
  
 

Super Resolution

A real compressed color video sequence

  • 90 x 90 color video sequence
  • 10 frames
  • Taken by a commercial video surveillance camera
    • courtesy of Adyoron Intelligent Systems, Ltd., Tel Aviv, Israel

Delaunay-spline smoother

  • Multi frame upscaling
  • Resolution enhancement factor is 5
  • Regularization parameters for Y, Cr, and Cb are 0.5, 1.0, and 1.0, respectively
  • Deblurring
    • PSF is the 5 x 5 disk kernel
    • Bilateral total variation regularization

Classic kernel regression

  • Resolution enhancement factor is 5
  • Gaussian kernel
  • The second order
  • Global smoothing parameters for Y, Cr, and Cb are 2.0, 3.5, and 3.5, respectively
  • Deblurring
    • PSF is the 5 x 5 disk kernel
    • Bilateral total variation regularization

Iterative steering kernel regression

  • Resolution enhancement factor is 5
  • Gaussian kernel
  • The second order steering kernel regression with iterative filtering algorithm
  • The number of iterations is 1
  • Global smoothing parameters for Y, Cr, and Cb are 4.0, 8.0, and 8.0, respectively
  • Deblurring
    • PSF is the 5 x 5 disk kernel
    • Bilateral total variation regularization
 
 
top