Abstract: The architecture of image processing unit towards OpenGL 2.0 was designed for autonomy and controllability of graphics processing unit. Firstly, the structure and workflow of image processing unit were determined. Secondly, for each function being processed, a data path was designed for it. Then, functions each module need processing were concluded according to the data path of each function and the OpenGL specification. Finally, the architecture is implemented, and functions such as texture loading/acquisition, convolution, color table and histogram were validated both on virtual simulation platform and FPGA platform. Moreover, the peak performance of texture loading was validated, and the costs of texture loading were evaluated both for FPGA and ASIC. The results show that the structure can realize the series of functions specified by the image processing unit, and the performance meets the design requirements. In peak condition, 30 frames of 2 K?2 K images can be processed within a second, and the implementation cost is acceptable. Currently, the image processing unit has been integrated into a fully self-developed graphics processor.