Stochastic Light Field Holography

ICCP 2023

Abstract: The Visual Turing Test is the ultimate goal to evaluate the realism of holographic displays. Previous studies have focused on addressing challenges such as limited étendue and image quality over a large focal volume, but they have not investigated the effect of pupil sampling on the viewing experience in full 3D holograms. In this work, we tackle this problem with a novel hologram generation algorithm motivated by matching the projection operators of incoherent (Light Field) and coherent (Wigner Function) light transport. To this end, we supervise hologram computation using synthesized photographs, which are rendered on-the-fly using Light Field refocusing from stochastically sampled pupil states during optimization. The proposed method produces holograms with correct parallax and focus cues, which are important for passing the Visual Turing Test. We validate that our approach compares favorably to state-of-the-art CGH algorithms that use Light Field and Focal Stack supervision. Our experiments demonstrate that our algorithm improves the viewing experience when evaluated under a large variety of different pupil states.

Stochastic Light Field Holography Florian Schiffers, Praneeth Chakravarthula, Nathan Matsuda, Grace Kuo, Ethan Tseng, Douglas Lanman, Felix Heide, Oliver Cossairt ICCP 2023 Paper Supplement

Fig. 2 Stochastic Light Field Holography Algorithm We wish to infer an SLM phase image φ that produces a hologram with viewable intensity matching the projections rendered from a target Light Field L. To do this, a series of pupil samples with randomly generated position, diameter, and focus, pi are passed to two equivalent image formation models: a Light Field projection operator, which produces individual target images 𝑇𝑖, and a Wigner projection operator, which produces individual output estimates 𝑇̂𝑖 for each of the pupil samples. These target and output estimates are compared using a photo-consistency loss function, which is in turn used to update the SLM phase image with a gradient descent step.

Fig. 2 Stochastic Light Field Holography Algorithm Simplified Diagram of the Proposed Setup: An SLM (illumination path not shown) diffracts a collimated beam according to an optimized pattern. The modulated beam passes through a 4-f system, with a pupil having programmable shift s and diameter d. The relay system produces an image of the SLM offset from the detector by a defocus distance z.

Experimental Results from Benchtop Prototype: The figures below presents a visual comparison of holograms generated with the two comparison methods and our proposed approach. Each hologram consists of eight temporal multiplexed SLM-patterns per color channel (i.e., 24 independent SLM-patterns per image ). Though subtle, the differences can be discerned between the results obtained from STFT and Focal Stack, which exhibit slight defocus artifacts and color fringing artifacts. These artifacts are not present in the results obtained from our proposed method. We see further potential for improvement in image quality with the application of a Neural-holography style calibration to reduce systematic error due to misalignment and modeling of other physical effects such as field-fringing.

Fig. 2 Experimental results (Deep Focus dataset): For each hologram, we sample the eyebox with a small pupil that is moved from left to right. We further evaluated it with a front and back focus. Our results demonstrate the existence of correct parallax and defocus cues when moving a small pupil around for each of the comparison methods.

Fig. 2 Experimental Results (Tnux Robot sene): For each hologram, we sample the eyebox with a small pupil that is moved from left to right. Our results demonstrate the existence of correct parallax and defocus cues when moving a small pupil around for each of the comparison methods. Overall, our proposed method shows the best image fidelity over the complete eyebox.

Related Work

Chakravarthula, Praneeth, et al.
Pupil-aware Holography. ACM Transactions on Graphics (TOG), 2022.

Shi, Liang, Donghun Ryu, and Wojciech Matusik.
Ergonomic-Centric Holography: Optimizing Realism, Immersion, and Comfort for Holographic Display. arXiv preprint arXiv:2306.08138, 2023.

Monin, Sagi, Aswin C. Sankaranarayanan, and Anat Levin.
Analyzing phase masks for wide étendue holographic displays. 2022 IEEE International Conference on Computational Photography (ICCP), IEEE, 2022.