D*: A Data Storage and Retrieval System for Scientific Studies

D* is a novel system for data storage and retrieval appropriate for advanced scien- tific studies, as in high-energy physics, environmental sciences, and astronomy. The design of the D* system is based on certain principles of organizing and ac- cessing multi-dimensional data on storage, whose pursuit requires that the storage system acquire a greater knowledge about the data. This provides a basis for a tighter integration of data storage and data mining technologies. Through the ap- plication of innovative retrieval and clustering techniques for high-dimensional data, D* can support high-performance data access and provide data mining ap- plications useful insights into the data that can facilitate subsequent processes of data preparation and data mining. The basic processes of the D* system include data clustering, space partitioning, data loading, and data retrieval based on region queries and similarity searching. D* scales well with increasing data dimensional- ity and works well on incremental load of data.