Holy Cow! I have never seen such a complex travis config! On Mon, Oct 14, 2019 at 6:54 PM Gian Merlino <[email protected]> wrote:
> I should have linked Druid's Travis config file for reference. Here it is: > https://github.com/apache/incubator-druid/blob/master/.travis.yml > > On Mon, Oct 14, 2019 at 6:53 PM Gian Merlino <[email protected]> wrote: > > > > So what exactly is the requirement? Our DataSketches code does compile > > > under JDK8 and we have lots of customers and users using JDK 8. > > > > It depends on who you ask, I guess :) > > > > From Druid's perspective, we aren't going to be compiling your code, only > > running it. So the only requirement we would ask for is that the code > > _runs_ under a Java 11 runtime. Being able to compile under Java 11 is > good > > for future-proofing your code, though. It is starting to become more > > prevalent. > > > > > It sounds like (as far as Unsafe is concerned) converting all the > direct > > > unsafe.blah() calls to static MethodHandle.invoke calls is what is > > required. > > > > > > Is this correct? > > > > That is certainly part of it, I'm not 100% sure if that's all that's > > needed. Btw, CI can help verify. In Druid's Travis config, we run our > tests > > on both JDK 8 and JDK 11. In order to make this automated testing easier, > > we chose to do the work needed to compile Druid on JDK 11, even though we > > still compile release builds using JDK 8 only. (If we hadn't done this > > work, then we'd have to compile on 8 and test on 11, which Travis doesn't > > easily support.) > > > > On Mon, Oct 14, 2019 at 5:51 PM leerho <[email protected]> wrote: > > > >> Thanks, Gian, this is VERY helpful! > >> > >> From my cursory glance, it appears that UnsafeUtil acquires Unsafe and > >> Unsafe-class the usual way using reflection (I didn't think this would > >> work > >> with 9+ ), and then defines some method-handles that the other classes > >> use. > >> > >> Will this approach allow code to compile under JDK11 and JDK8? > >> > >> It also seems that compiling with JDK11 is still a work in progress as > the > >> pom.xml clearly specifies JDK8. > >> > >> So what exactly is the requirement? Our DataSketches code does compile > >> under JDK8 and we have lots of customers and users using JDK 8. > >> > >> It sounds like (as far as Unsafe is concerned) converting all the direct > >> unsafe.blah() calls to static MethodHandle.invoke calls is what is > >> required. > >> > >> Is this correct? > >> > >> Lee. > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> On Mon, Oct 14, 2019 at 2:07 PM Gian Merlino <[email protected]> wrote: > >> > >> > And nobody should be using the Oracle JDK without a commercial > >> relationship > >> > with Oracle, its license is a nightmare. > >> > > >> > On Mon, Oct 14, 2019 at 2:04 PM Gian Merlino <[email protected]> wrote: > >> > > >> > > Oracle and OpenJDK builds behave the same as each other now. The > >> > > differences are in terms of licensing, length of support, and > release > >> > > cadence. IMO most Java devs should be using one of the third-party > >> > OpenJDK > >> > > distributions (e.g. Corretto, Zulu, AdoptOpenJDK) rather than > vanilla > >> > > OpenJDK these days, because of the new OpenJDK policy that public > >> updates > >> > > will cease for major versions after 6 months. > >> > > > >> > > On Mon, Oct 14, 2019 at 11:07 AM leerho <[email protected]> wrote: > >> > > > >> > >> Thanks, Gian, > >> > >> > >> > >> Your references are helpful and I am studying them. > >> > >> > >> > >> I didn't see any references to OpenJDK vs Oracle's JDKs. > >> > >> > >> > >> Are there any differences in the way Unsafe calls (or other hidden > >> > >> classes) > >> > >> are handled by OpenJDK? Or is this a "don't care" as far as Druid > is > >> > >> concerned? Are the version numbers between Oracle and OpenJDK > always > >> > >> aligned? > >> > >> > >> > >> > >> > >> Lee. > >> > >> > >> > >> > >> > >> > >> > >> On Fri, Oct 11, 2019 at 5:45 PM Gian Merlino <[email protected]> > >> wrote: > >> > >> > >> > >> > You didn't address this message to me, but in Druid most of the > >> work > >> > for > >> > >> > Java 9+ compatibility is mentioned in this master issue, which > you > >> > might > >> > >> > find helpful: > >> https://github.com/apache/incubator-druid/issues/5589. > >> > >> > Skimming the list, these might be particularly relevant to > >> > DataSketches: > >> > >> > > >> > >> > - https://github.com/apache/incubator-druid/pull/7487 (cleaner > >> > >> operations) > >> > >> > - https://github.com/apache/incubator-druid/pull/7466 > (ByteBuffer > >> > unmap > >> > >> > operation) > >> > >> > - https://github.com/apache/incubator-druid/pull/7576 (Remove > >> direct > >> > >> > references to Unsafe) > >> > >> > > >> > >> > Btw, it would be better, I think, to address emails to the list > at > >> > >> large. > >> > >> > It encourages more people to participate. If Roman is a > subscriber > >> he > >> > >> would > >> > >> > get a copy anyway. If not, you could encourage him to subscribe. > >> > >> > > >> > >> > On Fri, Oct 11, 2019 at 3:13 PM leerho <[email protected]> wrote: > >> > >> > > >> > >> > > Roman, > >> > >> > > > >> > >> > > I hope you are doing well. > >> > >> > > > >> > >> > > I really appreciate the contributions you made to our Memory > >> > >> component! > >> > >> > > > >> > >> > > Given that Druid has made its code base compatible with OpenJDK > >> 11, > >> > we > >> > >> > > could use your help in what changes do we need to make to make > >> that > >> > >> > happen > >> > >> > > for DataSketches. As I recall, our DataSketches library was > not > >> the > >> > >> only > >> > >> > > Druid dependency that took advantage of Unsafe. So I assume > you > >> > have > >> > >> > been > >> > >> > > down this path :) > >> > >> > > > >> > >> > > Currently, I have a check in the static initializer in > >> > >> Memory/UnsafeUtil: > >> > >> > > parseJavaVersion(...) that checks the string returned from > >> > >> > > System.getProperty("java.version"). If that string does not > >> contain > >> > >> > "1.8" > >> > >> > > or "8" it will throw an error. Given that we have not tested > >> with > >> > >> JDK 9, > >> > >> > > 10, 11 or 12, I felt it was safer to explicitly throw rather > than > >> > >> having > >> > >> > > the users experience some weird failure later that may be > >> difficult > >> > to > >> > >> > > debug. > >> > >> > > > >> > >> > > With only one minor exception (which we could easily fix) the > >> Memory > >> > >> > > component is the only place where Unsafe is used. However, we > do > >> > use > >> > >> > > reflection to gain internal access to a number of other classes > >> > >> including > >> > >> > > ByteBuffer, Cleaner, and Bits. > >> > >> > > > >> > >> > > Looking forward to hearing from you. > >> > >> > > > >> > >> > > Cheers, > >> > >> > > > >> > >> > > Lee. > >> > >> > > > >> > >> > > >> > >> > >> > > > >> > > >> > > >
