Synthesis of light-field raw data from RGB-D images

Image raw data captured by light-field cameras such as Lytro and Raytrix have been strictly protected by the manufacturers, which results in a serious hurdle to most academic researchers working in this area. In this paper, we make an attempt of developing an algorithm that can generate light-field raw data from a single input picture captured by a conventional camera together with its depth map. Our algorithm follows closely the imaging procedure of the light-field camera with micro lens, with certain simplification. Specifically, an input image (of size W × H) will be projected to a light-field data of size (M × P) × (N × P), assuming that the micro-lens array has size M × N micro lens and each micro lens is supported by P × P sensors. In our work, the main lens is partitioned into K × K cells, each pixel of the input picture is assumed to locate at certain distance from the main lens and regarded as a lighting source. The directional light emitted from each pixel goes through the main lens and the corresponding micro-lens to be recorded at a particular sensor in the CMOS array. We will derive the geometry of this imaging process as well as the intensity recorded at each sensor. We choose to run the refocusing algorithm to verify whether the light-field data generated by our algorithm is meaningful or not. Comparing the refocused images against the original input picture shows that our synthesis algorithm seems quite successful.