Hi Tim, regarding the IO, while ago (at the incubator time of the project), we discussed how to deal with different versions of the backend API and dependencies. I proposed to have a release cycle per IO and have a subproject per IO version, like for instance:
sdks/java/io/elasticsearch-5 sdks/java/io/elasticsearch-6 ... I'm still thinking that the best option, allowing to really leverage the backend version in the right way. Regarding the release, it's like what I'm doing in the ServiceMix Bundles: the IO can have their own release cycle. As we agreed on a periodical release cycle, not sure if it's still required, but it could be interesting (why not having a specific repository for IOs). Regards JB On 28/08/2018 10:43, Tim Robertson wrote: > Hi folks, > > I'd like to revisit the discussion around our versioning policy > specifically for the Hadoop ecosystem and make sure we are aware of the > implications. > > As an example our policy today would have us on HBase 2.1 and I have > reminders to address this. > > However, currently the versions of HBase in the major hadoop distros are: > > - Cloudera 5 on HBase 1.2 (Cloudera 6 is 2.1 but is only in beta) > - Hortonworks HDP3 on HBase 2.0 (only recently released so we can > assume is not widely adopted) > - AWS EMR HBase on 1.4 > > On the versioning I think we might need a more nuanced approach to > ensure that we target real communities of existing and potential users. > Enterprise users need to stick to the supported versions in the > distributions to maintain support contracts from the vendors. > > Should our versioning policy have more room to consider on a case by > case basis? > > For Hadoop might we benefit from a strategy on which community of users > Beam is targeting? > > (OT: I'm collecting some thoughts on what we might consider to target > enterprise hadoop users - kerberos on all relevant IO, performance, > leaking beyond encryption zones with temporary files etc) > > Thanks, > Tim
