Re: [DISCUSS] support different versions of backends in an IO

2017-06-23 Thread Chamikara Jayalath
Probably this will be a common question from IO transform authors as Beam matures. Probably we should add a section on this to IO authoring guide [1][2] ? Thanks, Cham [1] https://beam.apache.org/documentation/io/authoring-overview/ [2] https://issues.apache.org/jira/browse/BEAM-1025 On Fri, Jun

Re: [DISCUSS] support different versions of backends in an IO

2017-06-23 Thread Jean-Baptiste Onofré
Hi, It's something we already discussed in the past (for Kafka by instance). For Kafka, we were able to use a single IO with spring-el to detect the version. That's certainly the preferred approach, but it would not be possible in all cases. I would suggest, if first approach doesn't work: *

[DISCUSS] support different versions of backends in an IO

2017-06-23 Thread Etienne Chauchot
Hi guys, I'm working on Elasticsearch 5.x support for Beam IO (it only supports Elasticsearch 2.x right now). I wanted to have your opinion on some points related to maintenance. In this ES case a big part of the code of the IO is common between ES v2.x and ES v5.x. Still, there are some dif