Frequency-domain Acceleration for 3D Generative Adversarial Networks

3D generative adversarial networks (3D GANs) are widely utilized in model prediction and object generation. To address the challenges of massive computation and significant energy consumption in accelerating 3D GANs, a novel fast Fourier transform based frequency-domain accelerator (called FAG) is proposed. Firstly, FAG provides a frequency-domain hardware architecture, which utilizes the compact computation complexity and the zero repeat pattern in deconvolution, to reduce the computation overhead in 3D GANs accelerators. Secondly, FAG exploits the frequency-domain characteristic of Hermitian symmetry and the zero repeat pattern of deconvolution to significantly reduce data movements, and it utilizes the repeat pattern in deconvolution to significantly reduce data movements. Finally, comprehensive evaluations based on the ModelNet dataset and three 3D GAN models (3DGAN, 3D-IWGAN and 3D-PhysNet) show that, the performance and the energy efficiency can be improved by 76% and 141%, respectively, compared with the frequency-domain baseline; FAG achieves 6× higher performance and 46× better energy efficiency compared with the spatial baseline.