Apache: Big Data North America 2017

Apache Spark is one of the most popular Big Data platform. In this talk we will have a quick introduction of some of the high-level concepts in Spark and its various modules: SQL, Streaming, ML, Graph and Structured Streaming.

Then we will go through some of the current Best Practices to operationalize Spark for better performance in production, and tips to detect and avoid some of the most common issues.

And lastly we will explore how some enterprises are building solutions with Spark.

Are you interested in harnessing and analyzing the data that drives the Spark Web UI? Are you keen to use that data to tune your applications or understand fluctuations in runtime of your production applications? Do you want to understand the efficiency of your Spark executors and system resources?

This presentation will help you do that and more, by walking through the wealth of data in Spark application events. This data can be used as a foundation for a Spark profiler and advisor that analyzes application events in batch or real-time.

Big Data is all about being to access and process data in various formats, and from various sources. Apache Bahir provides extensions to distributed analytic platforms providing them access to different data sources. In this talk, we will introduce you to Apache Bahir and its various connectors that are available for Apache Spark and Apache Flink. We will also go over the details of how to build, test and deploy a Spark Application using the MQTT data source for the new Apache Spark Structure Streaming functionality.

Recommender Systems have changed the way companies and people interact with each other. Does your organisation need a 360° view of their customer? Today it is possible to recommend the right products to customers or potential customers. For example, a film based on their previous interests or a new accessory that fits their model of smartphone.

The technology behind recommender systems has evolved significantly over the past 20 years and with the explosion of Big Data technologies, there are tools that can create very powerful recommender systems. This introduction will explain how Recommender Systems work, describing their main functionalities, and providing some basic algorithms frequently used in such systems. We will look at how to create a Recommender System using technologies like Apache Spark and ElasticSearch.