Apache: Big Data North America 2017

Building an online real-estate marketplace is an extremely complex high touch business. The data that the business deals with varies from scanned PDFs and complex excel spread sheets to transactional RDBMSes(?) and click stream data. Data engineering at Ten-X has spent the last couple of years building a highly effective multi-model data platform that brings all of this data together and analyses it to help the business make better decisions and move faster. In this talk we will talk about how our data platform evolved, including the technology choices we made and why we made them. Our data lake is built as a multi-model platform on top of technologies including Hadoop, JanusGraph, Spark, Hive, Cassandra and HBase. We will also introduce you to some of the complex pattern matching algorithms and Natural Language Processing techniques we have implemented on our platform.