Abstract: We provide an effective way to interact with public large display at distance by taking advantages of the multimodal input and output of smart phone.We analyze the characteristics of the large display remote interaction,based on which a smart phone 3D interaction framework is presented;then according to the characteristics of the interaction process,we propose a real time,effective and accurate orientation estimation algorithm by using inertial sensors;in addition,two interaction techniques respectively for 2D and 3D interaction scenarios with large display are designed,and three application prototypes are demonstrated.Experiments revealed the rationality and effectiveness of interacting with large display at distance through smart phone.