Apache: Big Data North America 2017

Amazon EMR is a cloud-based provider that allows companies, research centers and academic divisions to leverage managed clusters at massive scale. In order to maintain and achieve performance in the open-source world of big data processing, Amazon EMR built an automatic performance benchmarking pipeline to aid in validating a new release prior to release. Why do we need this performance benchmarking pipeline? Open source communities move fast; innovations and implementations often need multiple iterations in order to effectively work at massive scale. Amazon EMR aims to provide a stable service; historical performance metrics help us to preview and capture the issues of each product before releasing to the market, meanwhile Amazon EMR is following closely to the open source releases.