I see thanks Jozsef, marking things as Experimental was discussed but we never agreed on doing this at the directory level. We can cover the same ground by putting an annotation in the classes (in particular the JetRunner and JetPipelineOptions classes which are the real public interface, or in the documentation (in particular website), I do not see how putting this in the directory name helps and if so we may need to put this in many other directories which is far from ideal. Any chance this can be fixed (jet-experimental -> jet) ?
On Fri, May 24, 2019 at 9:08 AM Jozsef Bartok <jo...@hazelcast.com> wrote: > > Hi Ismaël! > > Quoting Kenn (from PR-8410): "We discussed on list that it would be better to > have new things always start as experimental in a way that clearly > distinguishes them from the core." > > Rgds > > On Thu, May 23, 2019 at 10:44 PM Ismaël Mejía <ieme...@gmail.com> wrote: >> >> I saw that the runner was merged but I don’t get why the foler is >> called ‘runners/jet experimental’ and not simply ‘runners/jet’. Is it >> because the runner does not pass ValidatesRunner? Or because the >> contributors are few? I don’t really see any reason behind this >> suffix. And even if the status is not mature that’s not different from >> other already merged runners. >> >> On Fri, Apr 26, 2019 at 9:43 PM Kenneth Knowles <k...@apache.org> wrote: >> > >> > Nice! That is *way* more than the PR I was looking for. I just meant that >> > you could update the website/ directory. It is fine to keep the runner in >> > your own repository if you want. >> > >> > But I think it is great if you want to contribute it to Apache Beam (hence >> > donate it to the Apache Software Foundation). The benefits include: >> > low-latency testing, free updates when someone does a refactor. Things to >> > consider are: subject to ASF / Beam governance, PMC, commiters, subject to >> > Beam's release cadence (and we might exclude from Beam releases for a >> > little bit). Typically, we have kept runners on a branch until they are >> > somewhat stable. I don't feel strongly about this for disjoint codebases >> > that can easily be excluded from releases. We might want to suffix >> > `-experimental` to the artifacts for some time. >> > >> > I commented on the PR about the necessary i.p. clearance steps. >> > >> > Kenn >> > >> > On Fri, Apr 26, 2019 at 3:59 AM jo...@hazelcast.com <jo...@hazelcast.com> >> > wrote: >> >> >> >> Hi Kenn. >> >> >> >> It took me a while to migrate our code to the Beam repo, but I finally >> >> have been able to create the Pull Request you asked for, this is it: >> >> https://github.com/apache/beam/pull/8410 >> >> >> >> Looking forward to your feedback! >> >> >> >> Best regards, >> >> Jozsef >> >> >> >> On 2019/04/19 20:52:42, Kenneth Knowles <k...@apache.org> wrote: >> >> > The ValidatesRunner tests are the best source we have for knowing the >> >> > capabilities of a runner. Are there instructions for running the tests? >> >> > >> >> > Assuming we can check it out, then just open a PR to the website with >> >> > the >> >> > current capabilities and caveats. Since it is a big deal and could use >> >> > lots >> >> > of eyes, I would share the PR link on this thread. >> >> > >> >> > Kenn >> >> > >> >> > On Thu, Apr 18, 2019 at 11:53 AM Jozsef Bartok <jo...@hazelcast.com> >> >> > wrote: >> >> > >> >> > > Hi. We at Hazelcast Jet have been working for a while now to >> >> > > implement a >> >> > > Java Beam Runner (non-portable) based on Hazelcast Jet ( >> >> > > https://jet.hazelcast.org/). The process is still ongoing ( >> >> > > https://github.com/hazelcast/hazelcast-jet-beam-runner), but we are >> >> > > aiming for a fully functional, reliable Runner which can proudly join >> >> > > the >> >> > > Capability Matrix. For that purpose I would like to ask what’s your >> >> > > process >> >> > > of validating runners? We are already running the @ValidatesRunner >> >> > > tests >> >> > > and the Nexmark test suite, but beyond that what other steps do we >> >> > > need to >> >> > > take to get our Runner to the level it needs to be at? >> >> > > >> >> >
