Abstract: Detecting shadows from a single outdoor image has been one of research hotspots in image processing. To solve the problem that existing algorithms often focused on either shadow edges or shadow areas and ignored their dependence, specifically a layout information of “shadow area - shadow edge - non-shadow area”, this paper proposed a shadow detection algorithm: firstly, an outdoor image was segmented into regions;secondly, the layout information was modeled by constructing a high order energy function with the help of support vector machine;finally, shadows were determined by minimizing this function. This paper also rigorously proved the property of the proposed high order energy function under the local optimum, and introduced an optimal method of reducing itself into a second order energy function. Experiments on public image databases show that the proposed algorithm can effectively detect shadows from a single outdoor image.