Meanwhile, back to the topic of Migration to OpenJDK11 ... Unfortunately, the Apache email lists do not accommodate images or files. So I am going to move this next discussion to our asf slack channel "datasketches-dev".
I have posted my study there. You can offer comments or suggestions either here or there. Cheers, Lee. On Fri, Oct 18, 2019 at 3:53 PM leerho <[email protected]> wrote: > I opened a ticket (DATASKETCHES-3) with infra to see if we could add this > capability to our slack channel. > > On Fri, Oct 18, 2019 at 12:29 PM Dave Fisher <[email protected]> wrote: > >> Hi - >> >> Apache Pulsar uses this: https://github.com/merlimat/slack-email-digest >> >> They don’t have their slack channels at the-asf.slack.com but it does >> work. You can always ask for infra’s help. >> >> Regards, >> Dave >> >> > On Oct 18, 2019, at 12:26 PM, leerho <[email protected]> wrote: >> > >> > I wonder if there is a way to have slack auto-post to an email list? >> > >> > On Fri, Oct 18, 2019 at 5:13 AM Evans Ye <[email protected]> wrote: >> > >> >> I don't have the expertise on jdk 11 but would love know more about >> >> datasketches and specifically the java lib. Would be great if you can >> bring >> >> back the result of what you have discussed in the slack dev channel. >> This >> >> helps other contributors easily to search and get to know what >> happened in >> >> the community and get involved :) >> >> >> >> Best >> >> Evans >> >> >> >> leerho <[email protected]> 於 2019年10月18日 週五 11:03 寫道： >> >> >> >>> Christopher, >> >>> >> >>> This would be fabulous. I have tons of questions, whenever you are >> >> ready. >> >>> I might be easier to interact via the ASF slack channel >> "datasketches" or >> >>> "datasketches-dev". >> >>> >> >>> Lee. >> >>> >> >>> On Thu, Oct 17, 2019 at 5:07 PM Jon Malkin <[email protected]> >> wrote: >> >>> >> >>>> The memory repository is the one that needs the most conversion help. >> >>>> Everything using Unsafe in datasketches-java runs through the memory >> >>>> package. >> >>>> >> >>>> I'm not sure off-hand if the characterization repo has low level >> tests >> >> of >> >>>> Memory (on my phone at the moment), but we'd be looking to minimize >> the >> >>>> speed impact in the process since it very directly impacts sketch >> >> speed. >> >>>> >> >>>> jon >> >>>> >> >>>> >> >>>> >> >>>> On Thu, Oct 17, 2019, 4:43 PM Christopher <[email protected]> >> wrote: >> >>>> >> >>>>> I've converted several other projects to using JDK 11, and can help >> >>>>> with this if there's still a need for help. I've actually been >> >> looking >> >>>>> at ways I can contribute to DataSketches since ApacheCon, and this >> >>>>> seems to be something I can probably do. >> >>>>> >> >>>>> On Wed, Oct 16, 2019 at 9:47 PM leerho <[email protected]> wrote: >> >>>>>> >> >>>>>> Gian, >> >>>>>> >> >>>>>> If I were to supply you with a link to a snapshot or a Release >> >>>> Candidate >> >>>>>> version of the datasketches jar (and Memory jar) could you test in >> >>> your >> >>>>>> environment under JDK 11 and see if it works? >> >>>>>> >> >>>>>> Lee >> >>>>>> >> >>>>>> >> >>>>>> On Mon, Oct 14, 2019 at 9:19 PM leerho <[email protected]> wrote: >> >>>>>> >> >>>>>>> Holy Cow! I have never seen such a complex travis config! >> >>>>>>> >> >>>>>>> On Mon, Oct 14, 2019 at 6:54 PM Gian Merlino <[email protected]> >> >>>> wrote: >> >>>>>>> >> >>>>>>>> I should have linked Druid's Travis config file for reference. >> >>> Here >> >>>>> it is: >> >>>>>>>> >> >> https://github.com/apache/incubator-druid/blob/master/.travis.yml >> >>>>>>>> >> >>>>>>>> On Mon, Oct 14, 2019 at 6:53 PM Gian Merlino <[email protected]> >> >>>> wrote: >> >>>>>>>> >> >>>>>>>>>> So what exactly is the requirement? Our DataSketches code >> >>> does >> >>>>>>>> compile >> >>>>>>>>>> under JDK8 and we have lots of customers and users using JDK >> >>> 8. >> >>>>>>>>> >> >>>>>>>>> It depends on who you ask, I guess :) >> >>>>>>>>> >> >>>>>>>>> From Druid's perspective, we aren't going to be compiling your >> >>>> code, >> >>>>>>>> only >> >>>>>>>>> running it. So the only requirement we would ask for is that >> >> the >> >>>>> code >> >>>>>>>>> _runs_ under a Java 11 runtime. Being able to compile under >> >> Java >> >>>> 11 >> >>>>> is >> >>>>>>>> good >> >>>>>>>>> for future-proofing your code, though. It is starting to >> >> become >> >>>> more >> >>>>>>>>> prevalent. >> >>>>>>>>> >> >>>>>>>>>> It sounds like (as far as Unsafe is concerned) converting >> >> all >> >>>> the >> >>>>>>>> direct >> >>>>>>>>>> unsafe.blah() calls to static MethodHandle.invoke calls is >> >>> what >> >>>> is >> >>>>>>>>> required. >> >>>>>>>>>> >> >>>>>>>>>> Is this correct? >> >>>>>>>>> >> >>>>>>>>> That is certainly part of it, I'm not 100% sure if that's all >> >>>> that's >> >>>>>>>>> needed. Btw, CI can help verify. In Druid's Travis config, we >> >>> run >> >>>>> our >> >>>>>>>> tests >> >>>>>>>>> on both JDK 8 and JDK 11. In order to make this automated >> >>> testing >> >>>>>>>> easier, >> >>>>>>>>> we chose to do the work needed to compile Druid on JDK 11, >> >> even >> >>>>> though >> >>>>>>>> we >> >>>>>>>>> still compile release builds using JDK 8 only. (If we hadn't >> >>> done >> >>>>> this >> >>>>>>>>> work, then we'd have to compile on 8 and test on 11, which >> >>> Travis >> >>>>>>>> doesn't >> >>>>>>>>> easily support.) >> >>>>>>>>> >> >>>>>>>>> On Mon, Oct 14, 2019 at 5:51 PM leerho <[email protected]> >> >>> wrote: >> >>>>>>>>> >> >>>>>>>>>> Thanks, Gian, this is VERY helpful! >> >>>>>>>>>> >> >>>>>>>>>> From my cursory glance, it appears that UnsafeUtil acquires >> >>>> Unsafe >> >>>>> and >> >>>>>>>>>> Unsafe-class the usual way using reflection (I didn't think >> >>> this >> >>>>> would >> >>>>>>>>>> work >> >>>>>>>>>> with 9+ ), and then defines some method-handles that the >> >> other >> >>>>> classes >> >>>>>>>>>> use. >> >>>>>>>>>> >> >>>>>>>>>> Will this approach allow code to compile under JDK11 and >> >> JDK8? >> >>>>>>>>>> >> >>>>>>>>>> It also seems that compiling with JDK11 is still a work in >> >>>>> progress as >> >>>>>>>> the >> >>>>>>>>>> pom.xml clearly specifies JDK8. >> >>>>>>>>>> >> >>>>>>>>>> So what exactly is the requirement? Our DataSketches code >> >> does >> >>>>> compile >> >>>>>>>>>> under JDK8 and we have lots of customers and users using JDK >> >> 8. >> >>>>>>>>>> >> >>>>>>>>>> It sounds like (as far as Unsafe is concerned) converting all >> >>> the >> >>>>>>>> direct >> >>>>>>>>>> unsafe.blah() calls to static MethodHandle.invoke calls is >> >> what >> >>>> is >> >>>>>>>>>> required. >> >>>>>>>>>> >> >>>>>>>>>> Is this correct? >> >>>>>>>>>> >> >>>>>>>>>> Lee. >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> On Mon, Oct 14, 2019 at 2:07 PM Gian Merlino < >> >> [email protected]> >> >>>>> wrote: >> >>>>>>>>>> >> >>>>>>>>>>> And nobody should be using the Oracle JDK without a >> >>> commercial >> >>>>>>>>>> relationship >> >>>>>>>>>>> with Oracle, its license is a nightmare. >> >>>>>>>>>>> >> >>>>>>>>>>> On Mon, Oct 14, 2019 at 2:04 PM Gian Merlino < >> >>> [email protected]> >> >>>>>>>> wrote: >> >>>>>>>>>>> >> >>>>>>>>>>>> Oracle and OpenJDK builds behave the same as each other >> >>> now. >> >>>>> The >> >>>>>>>>>>>> differences are in terms of licensing, length of support, >> >>> and >> >>>>>>>> release >> >>>>>>>>>>>> cadence. IMO most Java devs should be using one of the >> >>>>> third-party >> >>>>>>>>>>> OpenJDK >> >>>>>>>>>>>> distributions (e.g. Corretto, Zulu, AdoptOpenJDK) rather >> >>> than >> >>>>>>>> vanilla >> >>>>>>>>>>>> OpenJDK these days, because of the new OpenJDK policy >> >> that >> >>>>> public >> >>>>>>>>>> updates >> >>>>>>>>>>>> will cease for major versions after 6 months. >> >>>>>>>>>>>> >> >>>>>>>>>>>> On Mon, Oct 14, 2019 at 11:07 AM leerho < >> >> [email protected]> >> >>>>> wrote: >> >>>>>>>>>>>> >> >>>>>>>>>>>>> Thanks, Gian, >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Your references are helpful and I am studying them. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> I didn't see any references to OpenJDK vs Oracle's JDKs. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Are there any differences in the way Unsafe calls (or >> >>> other >> >>>>> hidden >> >>>>>>>>>>>>> classes) >> >>>>>>>>>>>>> are handled by OpenJDK? Or is this a "don't care" as >> >> far >> >>> as >> >>>>>>>> Druid is >> >>>>>>>>>>>>> concerned? Are the version numbers between Oracle and >> >>>> OpenJDK >> >>>>>>>> always >> >>>>>>>>>>>>> aligned? >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Lee. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> On Fri, Oct 11, 2019 at 5:45 PM Gian Merlino < >> >>>> [email protected] >> >>>>>> >> >>>>>>>>>> wrote: >> >>>>>>>>>>>>> >> >>>>>>>>>>>>>> You didn't address this message to me, but in Druid >> >> most >> >>>> of >> >>>>> the >> >>>>>>>>>> work >> >>>>>>>>>>> for >> >>>>>>>>>>>>>> Java 9+ compatibility is mentioned in this master >> >> issue, >> >>>>> which >> >>>>>>>> you >> >>>>>>>>>>> might >> >>>>>>>>>>>>>> find helpful: >> >>>>>>>>>> https://github.com/apache/incubator-druid/issues/5589. >> >>>>>>>>>>>>>> Skimming the list, these might be particularly >> >> relevant >> >>> to >> >>>>>>>>>>> DataSketches: >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7487 >> >>>>> (cleaner >> >>>>>>>>>>>>> operations) >> >>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7466 >> >>>>>>>> (ByteBuffer >> >>>>>>>>>>> unmap >> >>>>>>>>>>>>>> operation) >> >>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7576 >> >>>>> (Remove >> >>>>>>>>>> direct >> >>>>>>>>>>>>>> references to Unsafe) >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> Btw, it would be better, I think, to address emails to >> >>> the >> >>>>> list >> >>>>>>>> at >> >>>>>>>>>>>>> large. >> >>>>>>>>>>>>>> It encourages more people to participate. If Roman is >> >> a >> >>>>>>>> subscriber >> >>>>>>>>>> he >> >>>>>>>>>>>>> would >> >>>>>>>>>>>>>> get a copy anyway. If not, you could encourage him to >> >>>>> subscribe. >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> On Fri, Oct 11, 2019 at 3:13 PM leerho < >> >>> [email protected]> >> >>>>>>>> wrote: >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Roman, >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> I hope you are doing well. >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> I really appreciate the contributions you made to >> >> our >> >>>>> Memory >> >>>>>>>>>>>>> component! >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Given that Druid has made its code base compatible >> >>> with >> >>>>>>>> OpenJDK >> >>>>>>>>>> 11, >> >>>>>>>>>>> we >> >>>>>>>>>>>>>>> could use your help in what changes do we need to >> >> make >> >>>> to >> >>>>> make >> >>>>>>>>>> that >> >>>>>>>>>>>>>> happen >> >>>>>>>>>>>>>>> for DataSketches. As I recall, our DataSketches >> >>> library >> >>>>> was >> >>>>>>>> not >> >>>>>>>>>> the >> >>>>>>>>>>>>> only >> >>>>>>>>>>>>>>> Druid dependency that took advantage of Unsafe. So >> >> I >> >>>>> assume >> >>>>>>>> you >> >>>>>>>>>>> have >> >>>>>>>>>>>>>> been >> >>>>>>>>>>>>>>> down this path :) >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Currently, I have a check in the static initializer >> >> in >> >>>>>>>>>>>>> Memory/UnsafeUtil: >> >>>>>>>>>>>>>>> parseJavaVersion(...) that checks the string >> >> returned >> >>>> from >> >>>>>>>>>>>>>>> System.getProperty("java.version"). If that string >> >>> does >> >>>>> not >> >>>>>>>>>> contain >> >>>>>>>>>>>>>> "1.8" >> >>>>>>>>>>>>>>> or "8" it will throw an error. Given that we have >> >> not >> >>>>> tested >> >>>>>>>>>> with >> >>>>>>>>>>>>> JDK 9, >> >>>>>>>>>>>>>>> 10, 11 or 12, I felt it was safer to explicitly >> >> throw >> >>>>> rather >> >>>>>>>> than >> >>>>>>>>>>>>> having >> >>>>>>>>>>>>>>> the users experience some weird failure later that >> >> may >> >>>> be >> >>>>>>>>>> difficult >> >>>>>>>>>>> to >> >>>>>>>>>>>>>>> debug. >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> With only one minor exception (which we could easily >> >>>> fix) >> >>>>> the >> >>>>>>>>>> Memory >> >>>>>>>>>>>>>>> component is the only place where Unsafe is used. >> >>>>> However, >> >>>>>>>> we do >> >>>>>>>>>>> use >> >>>>>>>>>>>>>>> reflection to gain internal access to a number of >> >>> other >> >>>>>>>> classes >> >>>>>>>>>>>>> including >> >>>>>>>>>>>>>>> ByteBuffer, Cleaner, and Bits. >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Looking forward to hearing from you. >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Cheers, >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Lee. >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>> >> >>>>> >> --------------------------------------------------------------------- >> >>>>> To unsubscribe, e-mail: [email protected] >> >>>>> For additional commands, e-mail: [email protected] >> >>>>> >> >>>>> >> >>>> >> >>> >> >> >> > -- >> > From my cell phone. >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >>
