Hi Nemoers, Just wanted to share my trip to ApacheCon NA 2019. :) Overall it was great fun. I met and talked with many folks working on a number of different Apache big data projects. Several people who attended Wonwook's talk at Beam summit EU already knew about Nemo. Kudos to Wonwook for the excellent job!
People really liked how Nemo transparently integrates with existing programming libraries (Beam, Spark), and cluster resource managers (YARN, Mesos). This seems to validate the decision we made early on to make Nemo compatible with other Apache big data projects including REEF. People also liked the optimizations, and some of them were curious whether the optimization options can be configured automatically for them. I suppose this validates the "smart compiler" works some of us are currently working on. I did 2 talks at the conference. One was the main Nemo talk, and the other was a demo of running Nemo on Google DataProc in a Beam session. The main talk was primarily about how Nemo optimizes distributed execution. On the next day, Beam organizers generously let me do a demo talk to demonstrate how easy it is to spin up new cloud instances, and run Beam programs using the Nemo runner. After seeing the demo one of the attendees there suggested that I file a PR to a GCP repository ( https://github.com/GoogleCloudPlatform/dataproc-initialization-actions) to automate the installation process, and we can do a blog post on using Nemo on Google Dataproc. I'll work on this to make Nemo more accessible to GCP users. Thank Nemoers for all your work! Cheers, John
