Research on Heatmap for Big Data Based on Spark

It is important to provide data analysts with effective and efficient exploratory tools via web browsers. However, due to the characteristics of big data, current data visualization approaches can hardly display the whole datasets. This paper proposes a general-purpose visualization framework based on Spark and LOD. Firstly, we implement a tile-based parallel computing algorithm for layered datasets with Spark. Secondly, we store the temporary results on HDFS. Finally, with Ajax and geographic information, we provide all kinds of spatial-temporal analysis services via web. This paper resolves two problems:one is the mapping of data points from heat map to geographic map; the other is the correction of the marginal error in overlapping areas caused by parallel computing. The experiment results suggest that by separating the data display and manipulation from the data rendering and computing tasks, our method provides a new way for big data visualization via web browsers.