$\textbf{C}^{3}$G-NeRF

Class-Continuous Conditional Generative Neural Radiance Field

Jiwook Kim, Minhyeok Lee
School of Electrical & Electronics Engineering, Chung-Ang University


Synthesized images of each class of AFHQ by our model. Each row and column indicate the class and a single object with different rotation input vectors, respectively.

Paper $|$ Code



Abstract

The 3D-aware image synthesis focuses on conserving spatial consistency besides generating high-resolution images with fine details. Recently, Neural Radiance Field (NeRF) has been introduced for synthesizing novel views with low computational cost and superior performance. While several works investigate a generative NeRF and show remarkable achievement, they cannot handle conditional and continuous feature manipulation in the generation procedure. In this work, we introduce a novel model, called Class-Continuous Conditional Generative NeRF ($\textbf{C}^{3}$G-NeRF), which can synthesize conditionally manipulated photorealistic 3D-consistent images by projecting conditional features to the generator and the discriminator. The proposed $\textbf{C}^{3}$G-NeRF is evaluated with three image datasets, AFHQ, CelebA, and Cars. As a result, our model shows strong 3D-consistency with fine details and smooth interpolation in conditional feature manipulation. For instance, $\textbf{C}^{3}$G-NeRF exhibits a Fréchet Inception Distance (FID) of 7.64 in 3D-aware face image synthesis with a $\text{128}^{2}$ resolution. Additionally, we provide FIDs of generated 3D-aware images of each class of the datasets as it is possible to synthesize class-conditional images with $\textbf{C}^{3}$G-NeRF.



Results

Controllable Generation

We can control the object in the 3D space. We perform rotation and positional translations which are the depth and the horizontal translation.


$\textbf{Rotation}$





$\textbf{Depth Translation}$


$\textbf{Horizontal Translation}$


Conditional generation

We can control class-continuous conditions of the object. We provide the results that interpolate each condition. We restore the changed condition to its former condition in the following videos.


$\textbf{Female to Man}$


$\textbf{Opening Mouth}$


$\textbf{Smiling}$


$\textbf{Chubby}$


$\textbf{Blonde}$


$\textbf{Pale Skin}$