I wonder if there is a way to have slack auto-post to an email list? On Fri, Oct 18, 2019 at 5:13 AM Evans Ye <evan...@apache.org> wrote:
> I don't have the expertise on jdk 11 but would love know more about > datasketches and specifically the java lib. Would be great if you can bring > back the result of what you have discussed in the slack dev channel. This > helps other contributors easily to search and get to know what happened in > the community and get involved :) > > Best > Evans > > leerho <lee...@gmail.com> 於 2019年10月18日 週五 11:03 寫道： > > > Christopher, > > > > This would be fabulous. I have tons of questions, whenever you are > ready. > > I might be easier to interact via the ASF slack channel "datasketches" or > > "datasketches-dev". > > > > Lee. > > > > On Thu, Oct 17, 2019 at 5:07 PM Jon Malkin <jon.mal...@gmail.com> wrote: > > > > > The memory repository is the one that needs the most conversion help. > > > Everything using Unsafe in datasketches-java runs through the memory > > > package. > > > > > > I'm not sure off-hand if the characterization repo has low level tests > of > > > Memory (on my phone at the moment), but we'd be looking to minimize the > > > speed impact in the process since it very directly impacts sketch > speed. > > > > > > jon > > > > > > > > > > > > On Thu, Oct 17, 2019, 4:43 PM Christopher <ctubb...@apache.org> wrote: > > > > > > > I've converted several other projects to using JDK 11, and can help > > > > with this if there's still a need for help. I've actually been > looking > > > > at ways I can contribute to DataSketches since ApacheCon, and this > > > > seems to be something I can probably do. > > > > > > > > On Wed, Oct 16, 2019 at 9:47 PM leerho <lee...@gmail.com> wrote: > > > > > > > > > > Gian, > > > > > > > > > > If I were to supply you with a link to a snapshot or a Release > > > Candidate > > > > > version of the datasketches jar (and Memory jar) could you test in > > your > > > > > environment under JDK 11 and see if it works? > > > > > > > > > > Lee > > > > > > > > > > > > > > > On Mon, Oct 14, 2019 at 9:19 PM leerho <lee...@gmail.com> wrote: > > > > > > > > > > > Holy Cow! I have never seen such a complex travis config! > > > > > > > > > > > > On Mon, Oct 14, 2019 at 6:54 PM Gian Merlino <g...@apache.org> > > > wrote: > > > > > > > > > > > >> I should have linked Druid's Travis config file for reference. > > Here > > > > it is: > > > > > >> > https://github.com/apache/incubator-druid/blob/master/.travis.yml > > > > > >> > > > > > >> On Mon, Oct 14, 2019 at 6:53 PM Gian Merlino <g...@apache.org> > > > wrote: > > > > > >> > > > > > >> > > So what exactly is the requirement? Our DataSketches code > > does > > > > > >> compile > > > > > >> > > under JDK8 and we have lots of customers and users using JDK > > 8. > > > > > >> > > > > > > >> > It depends on who you ask, I guess :) > > > > > >> > > > > > > >> > From Druid's perspective, we aren't going to be compiling your > > > code, > > > > > >> only > > > > > >> > running it. So the only requirement we would ask for is that > the > > > > code > > > > > >> > _runs_ under a Java 11 runtime. Being able to compile under > Java > > > 11 > > > > is > > > > > >> good > > > > > >> > for future-proofing your code, though. It is starting to > become > > > more > > > > > >> > prevalent. > > > > > >> > > > > > > >> > > It sounds like (as far as Unsafe is concerned) converting > all > > > the > > > > > >> direct > > > > > >> > > unsafe.blah() calls to static MethodHandle.invoke calls is > > what > > > is > > > > > >> > required. > > > > > >> > > > > > > > >> > > Is this correct? > > > > > >> > > > > > > >> > That is certainly part of it, I'm not 100% sure if that's all > > > that's > > > > > >> > needed. Btw, CI can help verify. In Druid's Travis config, we > > run > > > > our > > > > > >> tests > > > > > >> > on both JDK 8 and JDK 11. In order to make this automated > > testing > > > > > >> easier, > > > > > >> > we chose to do the work needed to compile Druid on JDK 11, > even > > > > though > > > > > >> we > > > > > >> > still compile release builds using JDK 8 only. (If we hadn't > > done > > > > this > > > > > >> > work, then we'd have to compile on 8 and test on 11, which > > Travis > > > > > >> doesn't > > > > > >> > easily support.) > > > > > >> > > > > > > >> > On Mon, Oct 14, 2019 at 5:51 PM leerho <lee...@gmail.com> > > wrote: > > > > > >> > > > > > > >> >> Thanks, Gian, this is VERY helpful! > > > > > >> >> > > > > > >> >> From my cursory glance, it appears that UnsafeUtil acquires > > > Unsafe > > > > and > > > > > >> >> Unsafe-class the usual way using reflection (I didn't think > > this > > > > would > > > > > >> >> work > > > > > >> >> with 9+ ), and then defines some method-handles that the > other > > > > classes > > > > > >> >> use. > > > > > >> >> > > > > > >> >> Will this approach allow code to compile under JDK11 and > JDK8? > > > > > >> >> > > > > > >> >> It also seems that compiling with JDK11 is still a work in > > > > progress as > > > > > >> the > > > > > >> >> pom.xml clearly specifies JDK8. > > > > > >> >> > > > > > >> >> So what exactly is the requirement? Our DataSketches code > does > > > > compile > > > > > >> >> under JDK8 and we have lots of customers and users using JDK > 8. > > > > > >> >> > > > > > >> >> It sounds like (as far as Unsafe is concerned) converting all > > the > > > > > >> direct > > > > > >> >> unsafe.blah() calls to static MethodHandle.invoke calls is > what > > > is > > > > > >> >> required. > > > > > >> >> > > > > > >> >> Is this correct? > > > > > >> >> > > > > > >> >> Lee. > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> On Mon, Oct 14, 2019 at 2:07 PM Gian Merlino < > g...@apache.org> > > > > wrote: > > > > > >> >> > > > > > >> >> > And nobody should be using the Oracle JDK without a > > commercial > > > > > >> >> relationship > > > > > >> >> > with Oracle, its license is a nightmare. > > > > > >> >> > > > > > > >> >> > On Mon, Oct 14, 2019 at 2:04 PM Gian Merlino < > > g...@apache.org> > > > > > >> wrote: > > > > > >> >> > > > > > > >> >> > > Oracle and OpenJDK builds behave the same as each other > > now. > > > > The > > > > > >> >> > > differences are in terms of licensing, length of support, > > and > > > > > >> release > > > > > >> >> > > cadence. IMO most Java devs should be using one of the > > > > third-party > > > > > >> >> > OpenJDK > > > > > >> >> > > distributions (e.g. Corretto, Zulu, AdoptOpenJDK) rather > > than > > > > > >> vanilla > > > > > >> >> > > OpenJDK these days, because of the new OpenJDK policy > that > > > > public > > > > > >> >> updates > > > > > >> >> > > will cease for major versions after 6 months. > > > > > >> >> > > > > > > > >> >> > > On Mon, Oct 14, 2019 at 11:07 AM leerho < > lee...@gmail.com> > > > > wrote: > > > > > >> >> > > > > > > > >> >> > >> Thanks, Gian, > > > > > >> >> > >> > > > > > >> >> > >> Your references are helpful and I am studying them. > > > > > >> >> > >> > > > > > >> >> > >> I didn't see any references to OpenJDK vs Oracle's JDKs. > > > > > >> >> > >> > > > > > >> >> > >> Are there any differences in the way Unsafe calls (or > > other > > > > hidden > > > > > >> >> > >> classes) > > > > > >> >> > >> are handled by OpenJDK? Or is this a "don't care" as > far > > as > > > > > >> Druid is > > > > > >> >> > >> concerned? Are the version numbers between Oracle and > > > OpenJDK > > > > > >> always > > > > > >> >> > >> aligned? > > > > > >> >> > >> > > > > > >> >> > >> > > > > > >> >> > >> Lee. > > > > > >> >> > >> > > > > > >> >> > >> > > > > > >> >> > >> > > > > > >> >> > >> On Fri, Oct 11, 2019 at 5:45 PM Gian Merlino < > > > g...@apache.org > > > > > > > > > > >> >> wrote: > > > > > >> >> > >> > > > > > >> >> > >> > You didn't address this message to me, but in Druid > most > > > of > > > > the > > > > > >> >> work > > > > > >> >> > for > > > > > >> >> > >> > Java 9+ compatibility is mentioned in this master > issue, > > > > which > > > > > >> you > > > > > >> >> > might > > > > > >> >> > >> > find helpful: > > > > > >> >> https://github.com/apache/incubator-druid/issues/5589. > > > > > >> >> > >> > Skimming the list, these might be particularly > relevant > > to > > > > > >> >> > DataSketches: > > > > > >> >> > >> > > > > > > >> >> > >> > - https://github.com/apache/incubator-druid/pull/7487 > > > > (cleaner > > > > > >> >> > >> operations) > > > > > >> >> > >> > - https://github.com/apache/incubator-druid/pull/7466 > > > > > >> (ByteBuffer > > > > > >> >> > unmap > > > > > >> >> > >> > operation) > > > > > >> >> > >> > - https://github.com/apache/incubator-druid/pull/7576 > > > > (Remove > > > > > >> >> direct > > > > > >> >> > >> > references to Unsafe) > > > > > >> >> > >> > > > > > > >> >> > >> > Btw, it would be better, I think, to address emails to > > the > > > > list > > > > > >> at > > > > > >> >> > >> large. > > > > > >> >> > >> > It encourages more people to participate. If Roman is > a > > > > > >> subscriber > > > > > >> >> he > > > > > >> >> > >> would > > > > > >> >> > >> > get a copy anyway. If not, you could encourage him to > > > > subscribe. > > > > > >> >> > >> > > > > > > >> >> > >> > On Fri, Oct 11, 2019 at 3:13 PM leerho < > > lee...@gmail.com> > > > > > >> wrote: > > > > > >> >> > >> > > > > > > >> >> > >> > > Roman, > > > > > >> >> > >> > > > > > > > >> >> > >> > > I hope you are doing well. > > > > > >> >> > >> > > > > > > > >> >> > >> > > I really appreciate the contributions you made to > our > > > > Memory > > > > > >> >> > >> component! > > > > > >> >> > >> > > > > > > > >> >> > >> > > Given that Druid has made its code base compatible > > with > > > > > >> OpenJDK > > > > > >> >> 11, > > > > > >> >> > we > > > > > >> >> > >> > > could use your help in what changes do we need to > make > > > to > > > > make > > > > > >> >> that > > > > > >> >> > >> > happen > > > > > >> >> > >> > > for DataSketches. As I recall, our DataSketches > > library > > > > was > > > > > >> not > > > > > >> >> the > > > > > >> >> > >> only > > > > > >> >> > >> > > Druid dependency that took advantage of Unsafe. So > I > > > > assume > > > > > >> you > > > > > >> >> > have > > > > > >> >> > >> > been > > > > > >> >> > >> > > down this path :) > > > > > >> >> > >> > > > > > > > >> >> > >> > > Currently, I have a check in the static initializer > in > > > > > >> >> > >> Memory/UnsafeUtil: > > > > > >> >> > >> > > parseJavaVersion(...) that checks the string > returned > > > from > > > > > >> >> > >> > > System.getProperty("java.version"). If that string > > does > > > > not > > > > > >> >> contain > > > > > >> >> > >> > "1.8" > > > > > >> >> > >> > > or "8" it will throw an error. Given that we have > not > > > > tested > > > > > >> >> with > > > > > >> >> > >> JDK 9, > > > > > >> >> > >> > > 10, 11 or 12, I felt it was safer to explicitly > throw > > > > rather > > > > > >> than > > > > > >> >> > >> having > > > > > >> >> > >> > > the users experience some weird failure later that > may > > > be > > > > > >> >> difficult > > > > > >> >> > to > > > > > >> >> > >> > > debug. > > > > > >> >> > >> > > > > > > > >> >> > >> > > With only one minor exception (which we could easily > > > fix) > > > > the > > > > > >> >> Memory > > > > > >> >> > >> > > component is the only place where Unsafe is used. > > > > However, > > > > > >> we do > > > > > >> >> > use > > > > > >> >> > >> > > reflection to gain internal access to a number of > > other > > > > > >> classes > > > > > >> >> > >> including > > > > > >> >> > >> > > ByteBuffer, Cleaner, and Bits. > > > > > >> >> > >> > > > > > > > >> >> > >> > > Looking forward to hearing from you. > > > > > >> >> > >> > > > > > > > >> >> > >> > > Cheers, > > > > > >> >> > >> > > > > > > > >> >> > >> > > Lee. > > > > > >> >> > >> > > > > > > > >> >> > >> > > > > > > >> >> > >> > > > > > >> >> > > > > > > > >> >> > > > > > > >> >> > > > > > >> > > > > > > >> > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: dev-unsubscr...@datasketches.apache.org > > > > For additional commands, e-mail: dev-h...@datasketches.apache.org > > > > > > > > > > > > > > -- >From my cell phone.
