Abstract: The transfer function selecting and setting process plays an important role in medical image visualization, and is affected by many factors, such as experience and subjective consciousness. The quality of transfer function is closely related to the performance of medical image visualization. In this paper, a medical image visualization method based on visual optimization is proposed. For the three-dimensional image rendering, the semi-automated transfer function based on the GPU-accelerated ray-cast volume rendering is given, which successfully figures out display problem between the global and the focus context of the three-dimensional image.For the two-dimensional image rendering, three image enhancement algorithms(histogram equalization, image fusion and total variation model) are adopted, which solve the problems, such as two-dimensional image noise,contrast blur and edge loss. Compared with the traditional methods, the proposed three-dimensional medical image visualization method is improved in drawing time and rendering effect. The peak signal to noise ratio and mean structural similarity of the image quality evaluation index are introduced to verify the effectiveness of the total variation model in two-dimensional medical image enhancement. Finally, a visual interactive system for medical images is designed and developed, which includes the three-dimensional visualization and interactive analysis of medical data.