I don't have the expertise on jdk 11 but would love know more about datasketches and specifically the java lib. Would be great if you can bring back the result of what you have discussed in the slack dev channel. This helps other contributors easily to search and get to know what happened in the community and get involved :)
Best Evans leerho <[email protected]> 於 2019年10月18日 週五 11:03 寫道： > Christopher, > > This would be fabulous. I have tons of questions, whenever you are ready. > I might be easier to interact via the ASF slack channel "datasketches" or > "datasketches-dev". > > Lee. > > On Thu, Oct 17, 2019 at 5:07 PM Jon Malkin <[email protected]> wrote: > > > The memory repository is the one that needs the most conversion help. > > Everything using Unsafe in datasketches-java runs through the memory > > package. > > > > I'm not sure off-hand if the characterization repo has low level tests of > > Memory (on my phone at the moment), but we'd be looking to minimize the > > speed impact in the process since it very directly impacts sketch speed. > > > > jon > > > > > > > > On Thu, Oct 17, 2019, 4:43 PM Christopher <[email protected]> wrote: > > > > > I've converted several other projects to using JDK 11, and can help > > > with this if there's still a need for help. I've actually been looking > > > at ways I can contribute to DataSketches since ApacheCon, and this > > > seems to be something I can probably do. > > > > > > On Wed, Oct 16, 2019 at 9:47 PM leerho <[email protected]> wrote: > > > > > > > > Gian, > > > > > > > > If I were to supply you with a link to a snapshot or a Release > > Candidate > > > > version of the datasketches jar (and Memory jar) could you test in > your > > > > environment under JDK 11 and see if it works? > > > > > > > > Lee > > > > > > > > > > > > On Mon, Oct 14, 2019 at 9:19 PM leerho <[email protected]> wrote: > > > > > > > > > Holy Cow! I have never seen such a complex travis config! > > > > > > > > > > On Mon, Oct 14, 2019 at 6:54 PM Gian Merlino <[email protected]> > > wrote: > > > > > > > > > >> I should have linked Druid's Travis config file for reference. > Here > > > it is: > > > > >> https://github.com/apache/incubator-druid/blob/master/.travis.yml > > > > >> > > > > >> On Mon, Oct 14, 2019 at 6:53 PM Gian Merlino <[email protected]> > > wrote: > > > > >> > > > > >> > > So what exactly is the requirement? Our DataSketches code > does > > > > >> compile > > > > >> > > under JDK8 and we have lots of customers and users using JDK > 8. > > > > >> > > > > > >> > It depends on who you ask, I guess :) > > > > >> > > > > > >> > From Druid's perspective, we aren't going to be compiling your > > code, > > > > >> only > > > > >> > running it. So the only requirement we would ask for is that the > > > code > > > > >> > _runs_ under a Java 11 runtime. Being able to compile under Java > > 11 > > > is > > > > >> good > > > > >> > for future-proofing your code, though. It is starting to become > > more > > > > >> > prevalent. > > > > >> > > > > > >> > > It sounds like (as far as Unsafe is concerned) converting all > > the > > > > >> direct > > > > >> > > unsafe.blah() calls to static MethodHandle.invoke calls is > what > > is > > > > >> > required. > > > > >> > > > > > > >> > > Is this correct? > > > > >> > > > > > >> > That is certainly part of it, I'm not 100% sure if that's all > > that's > > > > >> > needed. Btw, CI can help verify. In Druid's Travis config, we > run > > > our > > > > >> tests > > > > >> > on both JDK 8 and JDK 11. In order to make this automated > testing > > > > >> easier, > > > > >> > we chose to do the work needed to compile Druid on JDK 11, even > > > though > > > > >> we > > > > >> > still compile release builds using JDK 8 only. (If we hadn't > done > > > this > > > > >> > work, then we'd have to compile on 8 and test on 11, which > Travis > > > > >> doesn't > > > > >> > easily support.) > > > > >> > > > > > >> > On Mon, Oct 14, 2019 at 5:51 PM leerho <[email protected]> > wrote: > > > > >> > > > > > >> >> Thanks, Gian, this is VERY helpful! > > > > >> >> > > > > >> >> From my cursory glance, it appears that UnsafeUtil acquires > > Unsafe > > > and > > > > >> >> Unsafe-class the usual way using reflection (I didn't think > this > > > would > > > > >> >> work > > > > >> >> with 9+ ), and then defines some method-handles that the other > > > classes > > > > >> >> use. > > > > >> >> > > > > >> >> Will this approach allow code to compile under JDK11 and JDK8? > > > > >> >> > > > > >> >> It also seems that compiling with JDK11 is still a work in > > > progress as > > > > >> the > > > > >> >> pom.xml clearly specifies JDK8. > > > > >> >> > > > > >> >> So what exactly is the requirement? Our DataSketches code does > > > compile > > > > >> >> under JDK8 and we have lots of customers and users using JDK 8. > > > > >> >> > > > > >> >> It sounds like (as far as Unsafe is concerned) converting all > the > > > > >> direct > > > > >> >> unsafe.blah() calls to static MethodHandle.invoke calls is what > > is > > > > >> >> required. > > > > >> >> > > > > >> >> Is this correct? > > > > >> >> > > > > >> >> Lee. > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> On Mon, Oct 14, 2019 at 2:07 PM Gian Merlino <[email protected]> > > > wrote: > > > > >> >> > > > > >> >> > And nobody should be using the Oracle JDK without a > commercial > > > > >> >> relationship > > > > >> >> > with Oracle, its license is a nightmare. > > > > >> >> > > > > > >> >> > On Mon, Oct 14, 2019 at 2:04 PM Gian Merlino < > [email protected]> > > > > >> wrote: > > > > >> >> > > > > > >> >> > > Oracle and OpenJDK builds behave the same as each other > now. > > > The > > > > >> >> > > differences are in terms of licensing, length of support, > and > > > > >> release > > > > >> >> > > cadence. IMO most Java devs should be using one of the > > > third-party > > > > >> >> > OpenJDK > > > > >> >> > > distributions (e.g. Corretto, Zulu, AdoptOpenJDK) rather > than > > > > >> vanilla > > > > >> >> > > OpenJDK these days, because of the new OpenJDK policy that > > > public > > > > >> >> updates > > > > >> >> > > will cease for major versions after 6 months. > > > > >> >> > > > > > > >> >> > > On Mon, Oct 14, 2019 at 11:07 AM leerho <[email protected]> > > > wrote: > > > > >> >> > > > > > > >> >> > >> Thanks, Gian, > > > > >> >> > >> > > > > >> >> > >> Your references are helpful and I am studying them. > > > > >> >> > >> > > > > >> >> > >> I didn't see any references to OpenJDK vs Oracle's JDKs. > > > > >> >> > >> > > > > >> >> > >> Are there any differences in the way Unsafe calls (or > other > > > hidden > > > > >> >> > >> classes) > > > > >> >> > >> are handled by OpenJDK? Or is this a "don't care" as far > as > > > > >> Druid is > > > > >> >> > >> concerned? Are the version numbers between Oracle and > > OpenJDK > > > > >> always > > > > >> >> > >> aligned? > > > > >> >> > >> > > > > >> >> > >> > > > > >> >> > >> Lee. > > > > >> >> > >> > > > > >> >> > >> > > > > >> >> > >> > > > > >> >> > >> On Fri, Oct 11, 2019 at 5:45 PM Gian Merlino < > > [email protected] > > > > > > > > >> >> wrote: > > > > >> >> > >> > > > > >> >> > >> > You didn't address this message to me, but in Druid most > > of > > > the > > > > >> >> work > > > > >> >> > for > > > > >> >> > >> > Java 9+ compatibility is mentioned in this master issue, > > > which > > > > >> you > > > > >> >> > might > > > > >> >> > >> > find helpful: > > > > >> >> https://github.com/apache/incubator-druid/issues/5589. > > > > >> >> > >> > Skimming the list, these might be particularly relevant > to > > > > >> >> > DataSketches: > > > > >> >> > >> > > > > > >> >> > >> > - https://github.com/apache/incubator-druid/pull/7487 > > > (cleaner > > > > >> >> > >> operations) > > > > >> >> > >> > - https://github.com/apache/incubator-druid/pull/7466 > > > > >> (ByteBuffer > > > > >> >> > unmap > > > > >> >> > >> > operation) > > > > >> >> > >> > - https://github.com/apache/incubator-druid/pull/7576 > > > (Remove > > > > >> >> direct > > > > >> >> > >> > references to Unsafe) > > > > >> >> > >> > > > > > >> >> > >> > Btw, it would be better, I think, to address emails to > the > > > list > > > > >> at > > > > >> >> > >> large. > > > > >> >> > >> > It encourages more people to participate. If Roman is a > > > > >> subscriber > > > > >> >> he > > > > >> >> > >> would > > > > >> >> > >> > get a copy anyway. If not, you could encourage him to > > > subscribe. > > > > >> >> > >> > > > > > >> >> > >> > On Fri, Oct 11, 2019 at 3:13 PM leerho < > [email protected]> > > > > >> wrote: > > > > >> >> > >> > > > > > >> >> > >> > > Roman, > > > > >> >> > >> > > > > > > >> >> > >> > > I hope you are doing well. > > > > >> >> > >> > > > > > > >> >> > >> > > I really appreciate the contributions you made to our > > > Memory > > > > >> >> > >> component! > > > > >> >> > >> > > > > > > >> >> > >> > > Given that Druid has made its code base compatible > with > > > > >> OpenJDK > > > > >> >> 11, > > > > >> >> > we > > > > >> >> > >> > > could use your help in what changes do we need to make > > to > > > make > > > > >> >> that > > > > >> >> > >> > happen > > > > >> >> > >> > > for DataSketches. As I recall, our DataSketches > library > > > was > > > > >> not > > > > >> >> the > > > > >> >> > >> only > > > > >> >> > >> > > Druid dependency that took advantage of Unsafe. So I > > > assume > > > > >> you > > > > >> >> > have > > > > >> >> > >> > been > > > > >> >> > >> > > down this path :) > > > > >> >> > >> > > > > > > >> >> > >> > > Currently, I have a check in the static initializer in > > > > >> >> > >> Memory/UnsafeUtil: > > > > >> >> > >> > > parseJavaVersion(...) that checks the string returned > > from > > > > >> >> > >> > > System.getProperty("java.version"). If that string > does > > > not > > > > >> >> contain > > > > >> >> > >> > "1.8" > > > > >> >> > >> > > or "8" it will throw an error. Given that we have not > > > tested > > > > >> >> with > > > > >> >> > >> JDK 9, > > > > >> >> > >> > > 10, 11 or 12, I felt it was safer to explicitly throw > > > rather > > > > >> than > > > > >> >> > >> having > > > > >> >> > >> > > the users experience some weird failure later that may > > be > > > > >> >> difficult > > > > >> >> > to > > > > >> >> > >> > > debug. > > > > >> >> > >> > > > > > > >> >> > >> > > With only one minor exception (which we could easily > > fix) > > > the > > > > >> >> Memory > > > > >> >> > >> > > component is the only place where Unsafe is used. > > > However, > > > > >> we do > > > > >> >> > use > > > > >> >> > >> > > reflection to gain internal access to a number of > other > > > > >> classes > > > > >> >> > >> including > > > > >> >> > >> > > ByteBuffer, Cleaner, and Bits. > > > > >> >> > >> > > > > > > >> >> > >> > > Looking forward to hearing from you. > > > > >> >> > >> > > > > > > >> >> > >> > > Cheers, > > > > >> >> > >> > > > > > > >> >> > >> > > Lee. > > > > >> >> > >> > > > > > > >> >> > >> > > > > > >> >> > >> > > > > >> >> > > > > > > >> >> > > > > > >> >> > > > > >> > > > > > >> > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > > > > > >
