Apache: Big Data North America 2017 will be held at the Intercontinental Miami in Miami, Florida. 

Register now for the event taking place May 16-18, 2017. 
Back To Schedule
Thursday, May 18 • 9:00am - 9:50am
Venturing into Large Hadoop Clusters - Varun Saxena & Naganarasimha Garla, Huawei Technologies

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
Hadoop clusters are continuously becoming larger with several thousand machines,running thousands of jobs concurrently on 1000-1500 queues divided by different tenants and crunching higher volume of data than before.Hence,maintaining good performance of such large clusters,ensuring fast recovery times,upgrading them and debugging them becomes a major challenge.With larger clusters,enterprises expect even more efficient cluster utilisation.The fact that jobs are in turn executed as part of a workflow adds to the complexity. As time progresses, clusters would become even larger, i.e. have several tens of thousands of machines.

In this talk, we plan to share issues we came across while handling large clusters and the optimizations we had to make to resolve them.We would also talk about a few upcoming features in Hadoop which aim to overcome challenges posed by clusters at gigantic scale.


Naganarasimha Garla

System Architect, Huawei Technologies
I am a Big Data Enthusiast and have experience in developing Big Data Hadoop applications and platforms since 5 years. I have 12 years of experience as a Java Software Developer.I have been actively contributing for Hadoop YARN and Map Reduce since 2.5 years and currently Apache Hadoop... Read More →
avatar for Varun Saxena

Varun Saxena

Senior Technical Leader, Huawei Technologies
I am currently working as a Senior Tech Lead in Huawei's Hadoop Team which provides big data solutions to multiple product lines in Huawei and contributes to Hadoop community. I am also an Apache Hadoop Committer and have been contributing to YARN for almost 2.5 years. Overall, I... Read More →

Thursday May 18, 2017 9:00am - 9:50am EDT
  Big Data