Just a reminder and some more details of the coming meetup, cheers! Hi Kafka, Brooklin and Samza Users, The Streams Infra team invites you to attend the Streams Processing meetup <https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/251481797/> on July 18th 2018. This meetup focuses on Apache Kafka, Apache Samza, and related streaming technologies. We will host the actual event at LinkedIn Sunnyvale office, and in addition to that, we will also host a "*viewing room*" from San Francisco.This time we have Xinyu Liu <https://www.linkedin.com/in/xinyu-liu-b0b21648/> from the Samza team talking about Apache Beam <https://beam.apache.org/> runner for Samza <https://iwww.corp.linkedin.com/wiki/cf/display/ENGS/BEAM>. The Beam runner provides an ability to write-once but execute the same job in multiple environments (e.g. Hadoop for Batch Processing or Samza in Nearline). It also opens up possibilities for supporting different languages for stream processing (e.g. Python Applications on Samza). Our second speaker Hongliang Xu <https://www.linkedin.com/in/hongliangxu/> is from the Infrastructure team @Uber. His team recently built uReplicator to replicate data across Kafka clusters. You can find a blog <https://eng.uber.com/ureplicator/> about the original version of uReplicator here <https://eng.uber.com/ureplicator/> for reference. In his talk, Hongliang will focus on the new version of uReplicator, its architecture and share some of their learnings .Ajith Muralidharan <https://www.linkedin.com/in/ajithmuralidharan/> & Vivek Nelamangala <https://www.linkedin.com/in/viveknelamangala/> from LinkedIn will talk about how they built a near real time targeting and scoring platform (Concourse) for LinkedIn Notifications. Concourse is one of the largest Samza Jobs at LinkedIn and if you are building large scale streaming applications, this is the talk to attend.Below are some additional details about the talks. If you are interested to attend, Please RSVP via meetup.com <https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/251481797/>. You can also find additional details (streaming link, location, etc.) in the meetup link <https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/251481797/>. Hope to see you there!
*Location*: Main Event - Yosemite Conference Room, LinkedIn Corporate HQ in Sunnyvale. 2nd floor of 605 W Maude Ave, Sunnyvale, CA. Viewing Party - Lotta’s Fountain Conference Room, LinkedIn in San Francisco at 222 2nd Street, San Francisco, CA. Agenda:6PM: Doors open6-6:35 PM: Networking6:35-7:10 PM: Beam me up Samza: How we built a Samza Runner for Apache Beam (Speaker: Xinyu Liu, LinkedIn) Apache Beam provides an easy-to-use, and powerful model for state-of-the-art stream and batch processing, portability across a variety of languages, and the ability to converge offline and nearline data processing. At LinkedIn, we have developed a Samza Runner to leverage the cutting-edge features of Beam. This runner combines the large-scale streaming processing capabilities and first-class state support in Samza with the advancements in Beam data processing. In this talk, we will discuss the Beam API and its implementation in Samza and the benefits of Beam Runner to the Samza and Beam community. 7:15-7:50 PM: uReplicator: Uber Engineering’s Scalable Robust Kafka Replicator(Speaker: Hongliang Xu, Uber) At Uber, we operate 20+ Kafka clusters to collect system and application logs as well as event data from rider and driver apps. We need a Kafka replication solution to replicate data between Kafka clusters across multiple data centers for different purposes. This talk will introduce the history behind uReplicator and the high level architecture. As the original uReplicator ran into scalability challenges and operational overhead as the scale of Kafka clusters increased, we built the Federated uReplicator which addressed above issues and provide an extensible architecture for further scaling. 7:55-8:30 PM: Concourse - Near real time notifications platform at Linkedin (Speakers: Ajith Muralidharan & Vivek Nelamangala, LinkedIn) Concourse is LinkedIn’s first near-real-time targeting and scoring platform for notifications. In this talk, we will provide an in-depth overview of the design and discuss various scaling optimizations. We'll explain how Concourse can score millions of notifications per second, while supporting the use of feature-rich machine learning models based on terabytes of feature data. 8:30-9PM: Additional networking and Q&AThank you,Streams Infra @ LinkedIn On Wed, Jun 13, 2018 at 10:56 PM, Yi Pan <nickpa...@gmail.com> wrote: > Hi, all, > > We have planed for some super-exciting talks at our next Streams Meetup on > July 19. > > * Beam-Samza integration enabling new real time scenarios @ LinkedIn > * U-Replicator : Uber's multi-datacenter kafka mirroring service > * Concourse : LinkedIn’s near-real-time targeting and scoring platform > for notifications built on top of our ML and Stream Processing Infra. > > You can sign up at https://lnkd.in/gz3WcWb > > This time we will be open at both our Sunnyvale and San Francisco office. > Looking forward to see you! > > Best, > > -Yi > >