PiCam: An Ultra-Thin High Performance Monolithic Camera Array

Kartik Venkataraman       Dan Lelescu       Jacques Duparré       Andrew McMahon       Gabriel Molina       Priyam Chatterjee       Robert Mullis
Pelican Imaging Corporation

Shree Nayar
Columbia University

ACM Transactions on Graphics (Proc. SIGGRAPH Asia), 32(5), 2013

Figure 1

Figure 1: From left to right - (a) The PiCam Camera Array Module (b) Raw 4×4 array images each 1000×750 pixels (c) Parallax corrected and superresolved high resolution 8MP Image; (d) A high resolution filtered 8MP depth map

Abstract

We present PiCam (Pelican Imaging Camera-Array), an ultra-thin high performance monolithic camera array, that captures light fields and synthesizes high resolution images along with a range image (scene depth) through integrated parallax detection and superresolution. The camera is passive, supporting both stills and video, low light capable, and small enough to be included in the next generation of mobile devices including smartphones. Prior works [Rander et al. 1997; Yang et al. 2002; Zhang and Chen 2004; Tanida et al. 2001; Tanida et al. 2003; Duparre ́ et al. 2004] in camera arrays have explored multiple facets of light field capture - from viewpoint synthesis, synthetic refocus, computing range images, high speed video, and micro-optical aspects of system miniaturization. However, none of these have addressed the modifications needed to achieve the strict form factor and image quality required to make array cameras practical for mobile devices. In our approach, we customize many aspects of the camera array including lenses, pixels, sensors, and software algorithms to achieve imaging performance and form factor comparable to existing mobile phone cameras.

Our contributions to the post-processing of images from camera arrays include a cost function for parallax detection that integrates across multiple color channels, and a regularized image restoration (superresolution) process that takes into account all the system degradations and adapts to a range of practical imaging conditions. The registration uncertainty from the parallax detection process is integrated into a Maximum-a-Posteriori formulation that synthesizes an estimate of the high resolution image and scene depth. We conclude with some examples of our array capabilities such as post-capture (still) refocus, video refocus, view synthesis to demonstrate motion parallax, 3D range images, and briefly address future work.


Video -- PiCam: An Ultra-Thin High Performance Monolithic Camera Array

CR Categories: I.3.7 [Computer Graphics]: Digitization and Image Capture—Applications I.4.4 [Image Processing and Computer Vision]: Restoration—Inverse filtering I.4.8 [Image Processing and Computer Vision]: Scene Analysis—Range data;

Keywords: plenoptic acquisition, computational camera, light field, array camera, parallax detection, superresolution, depth map

Links:

Images taken with the Pelican Imaging reference camera module.
The high resolution downloadable images are 3264 x 2448. The low resolution images from the camera array are 1000 x 750.

Image taken with Pelican camera -- lab

Image taken with Pelican camera -- low-res inset Image taken with Pelican camera -- high res inset

Images taken with the Pelican Imaging reference camera module, f/3.1, > 500 Lux. The low resolution image in the inset is scaled to reflect the same size. Download the original image in TIF format.


Images taken with the Pelican Imaging reference camera module.

Image taken with Pelican camera -- lab

Image taken with Pelican camera -- low-res inset Image taken with Pelican camera -- high res inset

Image taken with Pelican camera -- low-res inset Image taken with Pelican camera -- high res inset

Image taken with Pelican camera -- low-res inset Image taken with Pelican camera -- high res inset


Images taken with the Pelican Imaging reference camera module, f/3.1, > 500 Lux. The low resolution image in the inset is scaled to reflect the same size. Download the original high-res image in TIF format. Download an original low-res image in TIF format.

Image taken with Pelican camera -- lunch

Image taken with Pelican camera -- low res inset Image taken with Pelican camera -- high res inset

Images taken with the Pelican Imaging reference camera module, f/3.1, > 500 Lux. The low resolution image in the inset is scaled to reflect the same size. Download the original high-res image in TIF format. Download an original low-res image in TIF format.

Image taken with Pelican camera -- treetrunk

Images taken with the Pelican Imaging reference camera module, f/3.1, > 500 Lux.
Download the original image in TIF format.

Image taken with Pelican camera -- Mexicali Grill

Images taken with the Pelican Imaging reference camera module, f/3.1, > 500 Lux.
Download the original high-res image in TIF format. Download an original low-res image in TIF format.

Image taken with Pelican camera -- fountain

Images taken with the Pelican Imaging reference camera module, f/3.1, > 500 Lux.
Download the original high-res image in TIF format. Download an original low-res image in TIF format.

Image taken with Pelican camera -- winter tree

Images taken with the Pelican Imaging reference camera module, f/3.1, > 500 Lux.
Download the original high-res image in TIF format. Download an original low-res image in TIF format.

Image taken with Pelican camera -- flowers

Images taken with the Pelican Imaging reference camera module, f/3.1, > 500 Lux.
Download the original image in TIF format.

Image taken with Pelican camera -- flower bowl

Images taken with the Pelican Imaging reference camera module, f/3.1, > 500 Lux.
Download the original image in TIF format.

Image below taken in low light, 30 - 50 lux, 15fps.

Image taken with Pelican camera -- lab low light 30-50lux

Image taken with Pelican camera -- low-res inset Image taken with Pelican camera -- high res inset


Images above taken in low light, 30 - 50 lux, 15fps, with the Pelican Imaging reference camera module.
Download the original high-res image in TIF format. Download an original low-res image in TIF format.

Image below taken in low light, 5 - 10 lux, 7fps.

Image taken with Pelican camera -- lab low light 5-10lux

Image taken with Pelican camera -- low-res inset Image taken with Pelican camera -- high res inset


Images above taken in low light, 5 - 10 lux, 7fps, with the Pelican Imaging reference camera module. The low-res inset has been brightened slightly to show detail. Download the original high-res image in TIF format. Download an original low-res image in TIF format.


Image taken with Pelican camera Image taken with Pelican camera - depth map

Image left taken with the Pelican Imaging reference camera module. The regularized depth map is shown below the image.


Image taken with Pelican camera Image taken with Pelican camera - depth map

Image left taken with the Pelican Imaging reference camera module. The regularized depth map is shown below the image.