Hi David,
Generally speaking, having different fluent DSL on top of the Beam SDK is great.
I would like to take a look on your wordcount examples to give you a complete
feedback. I like the idea and a fluent Java DSL is valuable.
Let's wait feedback from others. If we have a consensus, then I would be more
than happy to help you for the donation (I worked on the Camel Java DSL while
ago, so I have some experience here).
Thanks !
Regards
JB
On 12/17/2017 07:00 PM, David Morávek wrote:
Hello,
First of all, thanks for the amazing work the Apache Beam community is doing!
In 2014, we've started development of the runtime independent Java 8 API, that
helps us to create unified big-data processing flows. It has been used as a core
building block of Seznam.cz web crawler data infrastructure every since. Its
design principles and execution model are very similar to Apache Beam.
This API was open sourced in 2016, under the name Euphoria API:
https://github.com/seznam/euphoria
As it is very similar to Apache Beam, we feel, that it is not worth of
duplicating effort in terms of development of new runtimes and fine-tuning of
current ones.
The main blocker for us to switch to Apache Beam is lack of the Java 8 API. *W*e
propose the integration of Euphoria API into Apache Beam as a Java 8 DSL, in
order to share our effort with the community.
Simple example of the Euphoria API usage, can be found here:
https://github.com/seznam/euphoria/tree/master/euphoria-examples/src/main/java/cz/seznam/euphoria/examples/wordcount
If you feel, that Beam community could leverage from our work, we would love to
start working on Euphoria integration into Apache Beam (we already have a
working POC, with few basic operators implemented).
I look forward to hearing from you,
David
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
