Hi Beamers! It’s been six months today since Beam was accepted into incubation. It’s thrilling how far we’ve come since then!
I’d like to volunteer to put together a post on the Beam blog summarizing our progress since February. Here’s a starting point... What am I missing that we should include? What makes you proud? By the numbers: * 48,238 lines of preexisting code donated by Cloudera, dataArtisans, and Google. * 761 pull requests from 45 contributors. * 498 Jira issues opened and 245 resolved. * 1 incubating release (and another 1 in progress). * 4200 hours of automated tests. * 161 subscribers / 606 messages on user@. * 217 subscribers / 1205 messages on dev@. There’s been a lot of technical progress, including: * Refactoring of the entire codebase, examples, and tests to be truly runner-independent. * New functionality in the Apache Flink runner for timestamps/windows in batch and bounded sources and side inputs in streaming mode. * Work in progress to upgrade the Apache Spark runner to use Spark 2.0. * Several new runners from the wider Apache community -- Apache Gearpump has its own feature branch, Apache Apex has a PR, and conversations are starting on Apache Storm and others. * New SDKs/DSLs -- the Python SDK from Google is in, and there are plans to add the Scio DSL from Spotify. * Support for new IO connectors -- Apache Kafka and JMS are in, with Amazon Kinesis in PR. And community-wise, we’ve: * Started building a vibrant developer community, with detailed design discussions on features like DoFn reuse semantics, serialization technology, and an API for accessing state. * Started building a user community with improvements to the website and documentation. * Had multiple talks on Beam at venues including ApacheCon, Hadoop Summit, Kafka Summit, JBCN Barcelona, QCon, and Strata. * Presented at multiple existing meetups and are starting to organize some of our own. This half birthday is also a good chance to take a step back and reflect on our goals for this year -- TLP graduation and the first stable release. Where are we on this path? What can we do better to accomplish these high-level goals? Frances
