Utilizing (and Designing) Modern Hardware for Data-Intensive Computations: The Role of Abstraction

ABSTRACTModern information-intensive systems, including data management systems, operate on data that is mostly resident in RAM. As a result, the data management community has shifted focus from I/O optimization to addressing performance issues higher in the memory hierarchy. In this keynote, I will give a personal perspective of these developments, illustrated by work from my group at Columbia University. I will use the concept of abstraction as a lens through which various kinds of optimizations for modern hardware platforms can be understood and evaluated. Through this lens, some "cute implementation tricks" can be seen as much more than mere implementation details. I will discuss abstractions at various granularities, from single lines of code to whole programming/query languages. I will touch on software and hardware design for data-intensive computations. I will also discuss data processing in a conventional programming language, and how the data management community might contribute to the design of compilers.