On-Demand View Materialization and Indexing for Network Forensic Analysis

Roxana Geambasu, Tanya Bragin, Jaeyeon Jung, Magdalena Balazinska

Proceedings of the Third International Workshop on Networking Meets Databases (NetDB), April, 2007


Today, network intrusion detection systems (NIDSs) use custom solutions to log historical network flows and support forensic analysis by network administrators. These solutions are expensive, inefficient, and lack flexibility. In this paper, we investigate database support for interactive network forensic analysis. We show that an “out-of-the-box” relational database management system (RDBMS) can support moderate flow rates in a manner that ensures high query performance. To enable support for significantly higher data rates, we propose a technique based on on-demand view materialization and indexing. In our approach, when an event occurs, the system proactively extracts relevant historical data and indexes it in preparation for forensic queries over that data. We show that our approach significantly improves response times for a large class of queries, while maintaining high insert throughput.



Columbia University Department of Computer Science