It is becoming increasingly important for online businesses to be able to control the experience that their clients have when visiting their web sites. The most important aspect of the client experience is the client perceived response time. Billions of dollars in revenue are lost each year by web sites that are unable to measure and control client perceived response times. Clients get frustrated and leave web sites before completing a transaction, often never to return. Likewise, many web sites are over-provisioned, vastly increasing costs, without a quantitative understanding of why the additional resources are required.
We are developing ksniffer, a kernel-based traffic monitor capable of determining pageview response times as perceived by remote clients, in real-time at gigabit traffic rates. ksniffer builds on our previous work on Certes (CliEnt Response Time Estimated by the Server) that quantifies the effect that connection drops have on client perceived response time. ksniffer is based on novel, online mechanisms that take a “look once, then drop” approach to packet analysis to reconstruct TCP connections and learn client pageview activity. These mechanisms are designed to operate accurately with live network traffic even in the presence of packet loss. They are also designed to be efficiently implemented in kernel space, allowing ksniffer to perform analysis that exceeds the functionality of current traffic analyzers while doing so at high bandwidth rates. ksniffer can be used to determine client perceived response times for any web content, not just HTML. It only needs to perform passive monitoring of network traffic and does not require modifications to HTTP servers, HTTP browsers, or web pages. Our experimental results demonstrate that ksniffer can run on an inexpensive, commodity, Linux-based PC and provide accurate online pageview response time measurements, across a wide range of operating conditions. It can be effectively used by web service providers to verify compliance with service level objectives.