高级检索
欧文辉, 王峥, 吴卓宇, 王伟伦, 甘志银. 多模态高精度非线性激活函数协处理器设计[J]. 计算机辅助设计与图形学学报.
引用本文: 欧文辉, 王峥, 吴卓宇, 王伟伦, 甘志银. 多模态高精度非线性激活函数协处理器设计[J]. 计算机辅助设计与图形学学报.
Wenhui Ou, Zheng Wang, Zhuoyu Wu, Weilun Wang, Zhiyin Gan. Coprocessor for Multi-mode High Precision Nonlinear Activation Function[J]. Journal of Computer-Aided Design & Computer Graphics.
Citation: Wenhui Ou, Zheng Wang, Zhuoyu Wu, Weilun Wang, Zhiyin Gan. Coprocessor for Multi-mode High Precision Nonlinear Activation Function[J]. Journal of Computer-Aided Design & Computer Graphics.

多模态高精度非线性激活函数协处理器设计

Coprocessor for Multi-mode High Precision Nonlinear Activation Function

  • 摘要: 针对片上部署非线性激活函数所产生的精度损失以及硬件资源开销大的问题, 提出一种基于三分法指数方法的多模态高精度非线性激活函数协处理器设计. 首先分析了激活函数在不同拟合参数下的近似误差以及运算量为设计提供指导; 然后设计一种模块化的硬件框架, 通过复用指数、对数、sigmoid模块并结合浮点计算单元, 能够以较低的面积开销部署多种激活函数; 在Xilinx的Vertix系列FPGA上完成原型测试, 实验结果表明, 在仅增加32个查找表的情况下, 该设计tanh和sigmoid的近似误差仅为2项拆分指数方法的65.02%和69%, 同时拟合范围扩大60%. 而相比于高精度分段线性逼近方法, 该设计在仅用了4%的查找表数量的同时就将近似误差缩小了82%.

     

    Abstract: The deployment of nonlinear activation functions on a chip is suffering from accuracy loss and hardware resource overhead. To address these problems, a multi-mode high-precision coprocessor design framework for nonlinear activation functions is proposed and which is based on the three-split exponential method. In the first stage, the approximation error and the operation workload of nonlinear activation functions on different approximation parameters are analyzed to guide the design. In the second stage, a modular hardware framework is designed to deploy several nonlinear activation functions at a low cost by reusing exponential, logarithmic, and sigmoid modules and combining them with floating-point computation units. The prototype of the proposed framework has been implemented on Xilinx Vertix series FPGA. The experimental results show that with only 32 additional lookup table entries than the two-split exponential methods, the approximation error of tanh and sigmoid is 65.02% and 69% of that using the two-split method, and the fitting range is extended by 60%. Compared with the high-precision piecewise linear approximation method, the design has led to an 82% reduction in approximation error with only 4% of the number of lookup tables in use.

     

/

返回文章
返回