Astro-DISC: Astronomy and Cosmology Applications of Data-Intensive Supercomputing
We are developing new algorithms, data structures, and software tools for the analysis of massive astronomical and cosmological datasets. The goal is to enable the efficient processing of massive astrophysics datasets, with sizes in the tens of terabytes, by scaling up standard algorithms and designing new ones. We are developing programming abstractions and tools to address these challenges and to extend the developed techniques to other fields of science.
RESEARCH AREAS
- Distributed algorithms and data structures
- Indexing and analysis of massive structured data
CHALLENGES
- Developing distributed versions of standard astronomy applications
- Creating efficient algorithms for new astronomical challenges, in collaboration with astronomers and cosmologists
- Building a general-purpose toolkit for the analysis of massive
astronomical observations and cosmological simulations
More details: Slide show illustrating motivation and challenges.
SUB-PROJECTS
- DISC-Distances: Analyzing the distribution of distances between galaxies
- DISC-Finder: Identifying clusters of astronomical objects
- DISC-Holes: Indexing black-hole simulations
- DISC-Index: Indexing cosmological simulations
- DISC-Quasars: Identifying distant quasars in sky surveys
- DISC-SkyMap: Indexing astronomical objects
Acknowledgements
Made possible by seed funding from
The Gordon and Betty Moore Foundation.
We thank the members and companies of the PDL Consortium: Amazon, Google, Hewlett Packard Enterprise, Hitachi Ltd., Intel Corporation, IBM, Meta, Microsoft Research, NetApp, Inc., Oracle Corporation, Pure Storage, Salesforce, Samsung Semiconductor Inc., Seagate Technology, Two Sigma, and Western Digital for their interest, insights, feedback, and support.