A Dimension Reduction Algorithm Based on Divergence Balance

Most existing traditional supervised dimension reduction methods always maximize the between-class divergences to select discriminant subspace. This causes that some small divergences between two classes in original space are ignored easily because samples in these classes will mix together after they are projected in the subspace. To this end, this paper proposed a novel dimension reduction algorithm based on divergence balance which is called divergence balance projection（DBP）. This method utilizes symmetric KL divergence to measure divergences between classes and combines symmetric KL divergence with the concept of divergence balance. It pays more attention to small divergences while maintaining some large divergences, which achieves the goal to balance all divergences. In order to utilize abundant unlabeled samples in the real world, this paper utilized Laplacian graph and further proposed semi-supervised divergence balance projection（SDBP）. Various experiments on Soybean, Isolet and COIL20 have shown that our proposed method can achieve better performances to reduce dimensions of samples.