Apache: Big Data North America 2017 will be held at the Intercontinental Miami in Miami, Florida. 

Register now for the event taking place May 16-18, 2017. 
Back To Schedule
Tuesday, May 16 • 3:30pm - 4:20pm
Leveraging Smart Meter Data for Electric Utilities: Comparison of Spark SQL with Hive - Yusuke Furuyama, Hitachi

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
Hitachi has focused on social innovation business. It has constantly evolved to create sustainable business products and solutions to enhance the quality of life across the globe. Now we are leveraging smart meter data for electric utilities. To meet their needs, we compared the performance of batch processing for aggregating data from smart meters using Hadoop (MapReduce) and Spark 1.6 and Spark 2.0 with changing some parameters (the amount of input data, the logic of processing, input file format, etc.).In this session, we report the results of performance test above.

avatar for Yusuke Furuyama

Yusuke Furuyama

Senior Engineer, Hitachi, Ltd.
Yusuke Furuyama is a solution engineer at Hitachi and responsible for leveraging OSS related to bigdata and AI and working on offering progressive bigdata solutions to customers who are going to build enterprise system. Currently, he is focusing on Python and Python libraries for... Read More →

Tuesday May 16, 2017 3:30pm - 4:20pm EDT
  • Experience Level Any