Grand Challenge: High Performance Stream Queries in Scala

Dantong Song, Kaiwen Zhang, Tilmann Rabl, Prashanth Menon, and Hans-Arno Jacobsen.

In DEBS, 2015.

Abstract

Traffic monitoring is an important stream processing application which is highly dynamic and requires aggregation of spatially colocated data. Inspired by this, the DEBS 2015 Grand Challenge uses publicly available taxi transportation information to compute online the most frequent routes and most profitable areas. In this paper, we describe our solution to the DEBS 2015 Grand Challenge written in Scala. Our large-scale solution employs Apache Spark, while our challenge implementation is highly specialized and can process events at a 10 ms latency and at a throughput of 114,000 events per second.

Download



Tags: grand challenge, spark, scala, taxi monitoring

Sorry, can't prepare a list of recommended papers at the moment.