Coprocessor for Multi-mode High Precision Nonlinear Activation Function

The deployment of nonlinear activation functions on a chip is suffering from accuracy loss and hardware resource overhead. To address these problems, a multi-mode high-precision coprocessor design framework for nonlinear activation functions is proposed and which is based on the three-split exponential method. In the first stage, the approximation error and the operation workload of nonlinear activation functions on different approximation parameters are analyzed to guide the design. In the second stage, a modular hardware framework is designed to deploy several nonlinear activation functions at a low cost by reusing exponential, logarithmic, and sigmoid modules and combining them with floating-point computation units. The prototype of the proposed framework has been implemented on Xilinx Vertix series FPGA. The experimental results show that with only 32 additional lookup table entries than the two-split exponential methods, the approximation error of tanh and sigmoid is 65.02% and 69% of that using the two-split method, and the fitting range is extended by 60%. Compared with the high-precision piecewise linear approximation method, the design has led to an 82% reduction in approximation error with only 4% of the number of lookup tables in use.