<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=420156728350215&amp;ev=PageView&amp;noscript=1">

Beyond Spark RDDs: Dataframes & Datasets for Mixing Functional & Relational Code (also vroom vroom!)

Holden Karau | IBM

ABOUT THE TALK

Apache Spark is one of the most popular open source big data systems, with APIs available for Scala, Java, Python, and R. Spark's core building block of RDDs allow you to write functional program that are automatically distributed, and the new Dataset API allows us to also intermix relational style programming. In addition to allowing multiple types of programming on the same data, the Dataset API brings a more thorough optimizer and is able to better understand our programs. This talk will introduce the Dataset API while also looking how the optimizer and formats differ and the best ways to take advantage of this. The talk will wrap up with looking at ways in which Spark Datasets' can sometimes fail us (not everything is magic), and ways to work around them.

Register for DataEngConf SF '17

holden karau

Principal Software Engineer | IBM

Holden Karau currently works on improvements to Apache Spark and help other developers contribute to Spark as a principal software engineer at IBM. She also frequently gives external facing talks on Apache Spark.
Holden_Karau_IBM_DataEngConf.png

Experience this and other insightful data science & engineering talks at DataEngConf SF '17

GET TICKETS