ABOUT THE TALK

Many emerging Big Data problems are in fact "Fast Big Data problems" where data has to be accessed with very low response time. The rise of in-memory platforms like Apache Spark is an indication of this trend. However most existing distributed in memory platforms such as Apache Spark rely on the system layer that isn't built for high performance in-memory processing. In addition, the existing system software doesn’t optimally use the parallelism inherent in the new architectures where we have CPUs with many cores and SSDs with many flash channels. Other issues such as persistence and uniform access to a large memory space in the cluster also need a fundamental rethinking.