Apache: Big Data North America 2017 will be held at the Intercontinental Miami in Miami, Florida. 

Register now for the event taking place May 16-18, 2017. 
Back To Schedule
Tuesday, May 16 • 2:30pm - 3:20pm
What It Takes to Process a Trillion Events a Day: Case-Studies in Scaling Stream Processing at LinkedIn - Jagadish Venkatraman, LinkedIn

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
In this talk, we will present practical case-studies of large scale stream processing applications at LinkedIn. Example applications discussed will include:
  • LinkedIn’s real-time communication platform that delivers relevant content at massive scale to our 450M members. 
  • The LinkedIn feed that processes billions of events each day, and keeps track of what members viewed on their news feed. 
We will present the hard scalability problems we had to solve in each of these applications and the techniques used to address them. Problems include scaling ingestion of events, partitioned processing, highly performant data access and performing efficient remote I/O. We will explain how we leveraged and improved Apache Samza in addressing these problems and how we scaled to process over a trillion events every day.

avatar for Jagadish Venkatraman

Jagadish Venkatraman

Jagadish Venkatraman is an Apache Samza committer and a Senior Software Engineer in the Streams Infrastructure group at LinkedIn. He has been working on building, scaling and improving Apache Samza at LinkedIn. He has four years of experience working on practical problems at the intersection... Read More →

Tuesday May 16, 2017 2:30pm - 3:20pm EDT