Hi Lee, > On Oct 18, 2019, at 5:01 PM, leerho <[email protected]> wrote: > > Meanwhile, back to the topic of Migration to OpenJDK11 ... > > Unfortunately, the Apache email lists do not accommodate images or files.
While true by default, Infra will enable images/files if the podling requests. Regards, Dave > So I am going to move this next discussion to our asf slack channel > "datasketches-dev". > > I have posted my study there. You can offer comments or suggestions > either here or there. > > Cheers, > > Lee. > > On Fri, Oct 18, 2019 at 3:53 PM leerho <[email protected]> wrote: > >> I opened a ticket (DATASKETCHES-3) with infra to see if we could add this >> capability to our slack channel. >> >> On Fri, Oct 18, 2019 at 12:29 PM Dave Fisher <[email protected]> wrote: >> >>> Hi - >>> >>> Apache Pulsar uses this: https://github.com/merlimat/slack-email-digest >>> >>> They don’t have their slack channels at the-asf.slack.com but it does >>> work. You can always ask for infra’s help. >>> >>> Regards, >>> Dave >>> >>>> On Oct 18, 2019, at 12:26 PM, leerho <[email protected]> wrote: >>>> >>>> I wonder if there is a way to have slack auto-post to an email list? >>>> >>>> On Fri, Oct 18, 2019 at 5:13 AM Evans Ye <[email protected]> wrote: >>>> >>>>> I don't have the expertise on jdk 11 but would love know more about >>>>> datasketches and specifically the java lib. Would be great if you can >>> bring >>>>> back the result of what you have discussed in the slack dev channel. >>> This >>>>> helps other contributors easily to search and get to know what >>> happened in >>>>> the community and get involved :) >>>>> >>>>> Best >>>>> Evans >>>>> >>>>> leerho <[email protected]> 於 2019年10月18日 週五 11:03 寫道： >>>>> >>>>>> Christopher, >>>>>> >>>>>> This would be fabulous. I have tons of questions, whenever you are >>>>> ready. >>>>>> I might be easier to interact via the ASF slack channel >>> "datasketches" or >>>>>> "datasketches-dev". >>>>>> >>>>>> Lee. >>>>>> >>>>>> On Thu, Oct 17, 2019 at 5:07 PM Jon Malkin <[email protected]> >>> wrote: >>>>>> >>>>>>> The memory repository is the one that needs the most conversion help. >>>>>>> Everything using Unsafe in datasketches-java runs through the memory >>>>>>> package. >>>>>>> >>>>>>> I'm not sure off-hand if the characterization repo has low level >>> tests >>>>> of >>>>>>> Memory (on my phone at the moment), but we'd be looking to minimize >>> the >>>>>>> speed impact in the process since it very directly impacts sketch >>>>> speed. >>>>>>> >>>>>>> jon >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Thu, Oct 17, 2019, 4:43 PM Christopher <[email protected]> >>> wrote: >>>>>>> >>>>>>>> I've converted several other projects to using JDK 11, and can help >>>>>>>> with this if there's still a need for help. I've actually been >>>>> looking >>>>>>>> at ways I can contribute to DataSketches since ApacheCon, and this >>>>>>>> seems to be something I can probably do. >>>>>>>> >>>>>>>> On Wed, Oct 16, 2019 at 9:47 PM leerho <[email protected]> wrote: >>>>>>>>> >>>>>>>>> Gian, >>>>>>>>> >>>>>>>>> If I were to supply you with a link to a snapshot or a Release >>>>>>> Candidate >>>>>>>>> version of the datasketches jar (and Memory jar) could you test in >>>>>> your >>>>>>>>> environment under JDK 11 and see if it works? >>>>>>>>> >>>>>>>>> Lee >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Oct 14, 2019 at 9:19 PM leerho <[email protected]> wrote: >>>>>>>>> >>>>>>>>>> Holy Cow! I have never seen such a complex travis config! >>>>>>>>>> >>>>>>>>>> On Mon, Oct 14, 2019 at 6:54 PM Gian Merlino <[email protected]> >>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> I should have linked Druid's Travis config file for reference. >>>>>> Here >>>>>>>> it is: >>>>>>>>>>> >>>>> https://github.com/apache/incubator-druid/blob/master/.travis.yml >>>>>>>>>>> >>>>>>>>>>> On Mon, Oct 14, 2019 at 6:53 PM Gian Merlino <[email protected]> >>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>>> So what exactly is the requirement? Our DataSketches code >>>>>> does >>>>>>>>>>> compile >>>>>>>>>>>>> under JDK8 and we have lots of customers and users using JDK >>>>>> 8. >>>>>>>>>>>> >>>>>>>>>>>> It depends on who you ask, I guess :) >>>>>>>>>>>> >>>>>>>>>>>> From Druid's perspective, we aren't going to be compiling your >>>>>>> code, >>>>>>>>>>> only >>>>>>>>>>>> running it. So the only requirement we would ask for is that >>>>> the >>>>>>>> code >>>>>>>>>>>> _runs_ under a Java 11 runtime. Being able to compile under >>>>> Java >>>>>>> 11 >>>>>>>> is >>>>>>>>>>> good >>>>>>>>>>>> for future-proofing your code, though. It is starting to >>>>> become >>>>>>> more >>>>>>>>>>>> prevalent. >>>>>>>>>>>> >>>>>>>>>>>>> It sounds like (as far as Unsafe is concerned) converting >>>>> all >>>>>>> the >>>>>>>>>>> direct >>>>>>>>>>>>> unsafe.blah() calls to static MethodHandle.invoke calls is >>>>>> what >>>>>>> is >>>>>>>>>>>> required. >>>>>>>>>>>>> >>>>>>>>>>>>> Is this correct? >>>>>>>>>>>> >>>>>>>>>>>> That is certainly part of it, I'm not 100% sure if that's all >>>>>>> that's >>>>>>>>>>>> needed. Btw, CI can help verify. In Druid's Travis config, we >>>>>> run >>>>>>>> our >>>>>>>>>>> tests >>>>>>>>>>>> on both JDK 8 and JDK 11. In order to make this automated >>>>>> testing >>>>>>>>>>> easier, >>>>>>>>>>>> we chose to do the work needed to compile Druid on JDK 11, >>>>> even >>>>>>>> though >>>>>>>>>>> we >>>>>>>>>>>> still compile release builds using JDK 8 only. (If we hadn't >>>>>> done >>>>>>>> this >>>>>>>>>>>> work, then we'd have to compile on 8 and test on 11, which >>>>>> Travis >>>>>>>>>>> doesn't >>>>>>>>>>>> easily support.) >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Oct 14, 2019 at 5:51 PM leerho <[email protected]> >>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Thanks, Gian, this is VERY helpful! >>>>>>>>>>>>> >>>>>>>>>>>>> From my cursory glance, it appears that UnsafeUtil acquires >>>>>>> Unsafe >>>>>>>> and >>>>>>>>>>>>> Unsafe-class the usual way using reflection (I didn't think >>>>>> this >>>>>>>> would >>>>>>>>>>>>> work >>>>>>>>>>>>> with 9+ ), and then defines some method-handles that the >>>>> other >>>>>>>> classes >>>>>>>>>>>>> use. >>>>>>>>>>>>> >>>>>>>>>>>>> Will this approach allow code to compile under JDK11 and >>>>> JDK8? >>>>>>>>>>>>> >>>>>>>>>>>>> It also seems that compiling with JDK11 is still a work in >>>>>>>> progress as >>>>>>>>>>> the >>>>>>>>>>>>> pom.xml clearly specifies JDK8. >>>>>>>>>>>>> >>>>>>>>>>>>> So what exactly is the requirement? Our DataSketches code >>>>> does >>>>>>>> compile >>>>>>>>>>>>> under JDK8 and we have lots of customers and users using JDK >>>>> 8. >>>>>>>>>>>>> >>>>>>>>>>>>> It sounds like (as far as Unsafe is concerned) converting all >>>>>> the >>>>>>>>>>> direct >>>>>>>>>>>>> unsafe.blah() calls to static MethodHandle.invoke calls is >>>>> what >>>>>>> is >>>>>>>>>>>>> required. >>>>>>>>>>>>> >>>>>>>>>>>>> Is this correct? >>>>>>>>>>>>> >>>>>>>>>>>>> Lee. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Mon, Oct 14, 2019 at 2:07 PM Gian Merlino < >>>>> [email protected]> >>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> And nobody should be using the Oracle JDK without a >>>>>> commercial >>>>>>>>>>>>> relationship >>>>>>>>>>>>>> with Oracle, its license is a nightmare. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Mon, Oct 14, 2019 at 2:04 PM Gian Merlino < >>>>>> [email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Oracle and OpenJDK builds behave the same as each other >>>>>> now. >>>>>>>> The >>>>>>>>>>>>>>> differences are in terms of licensing, length of support, >>>>>> and >>>>>>>>>>> release >>>>>>>>>>>>>>> cadence. IMO most Java devs should be using one of the >>>>>>>> third-party >>>>>>>>>>>>>> OpenJDK >>>>>>>>>>>>>>> distributions (e.g. Corretto, Zulu, AdoptOpenJDK) rather >>>>>> than >>>>>>>>>>> vanilla >>>>>>>>>>>>>>> OpenJDK these days, because of the new OpenJDK policy >>>>> that >>>>>>>> public >>>>>>>>>>>>> updates >>>>>>>>>>>>>>> will cease for major versions after 6 months. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Mon, Oct 14, 2019 at 11:07 AM leerho < >>>>> [email protected]> >>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, Gian, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Your references are helpful and I am studying them. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I didn't see any references to OpenJDK vs Oracle's JDKs. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Are there any differences in the way Unsafe calls (or >>>>>> other >>>>>>>> hidden >>>>>>>>>>>>>>>> classes) >>>>>>>>>>>>>>>> are handled by OpenJDK? Or is this a "don't care" as >>>>> far >>>>>> as >>>>>>>>>>> Druid is >>>>>>>>>>>>>>>> concerned? Are the version numbers between Oracle and >>>>>>> OpenJDK >>>>>>>>>>> always >>>>>>>>>>>>>>>> aligned? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Lee. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Fri, Oct 11, 2019 at 5:45 PM Gian Merlino < >>>>>>> [email protected] >>>>>>>>> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> You didn't address this message to me, but in Druid >>>>> most >>>>>>> of >>>>>>>> the >>>>>>>>>>>>> work >>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>> Java 9+ compatibility is mentioned in this master >>>>> issue, >>>>>>>> which >>>>>>>>>>> you >>>>>>>>>>>>>> might >>>>>>>>>>>>>>>>> find helpful: >>>>>>>>>>>>> https://github.com/apache/incubator-druid/issues/5589. >>>>>>>>>>>>>>>>> Skimming the list, these might be particularly >>>>> relevant >>>>>> to >>>>>>>>>>>>>> DataSketches: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7487 >>>>>>>> (cleaner >>>>>>>>>>>>>>>> operations) >>>>>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7466 >>>>>>>>>>> (ByteBuffer >>>>>>>>>>>>>> unmap >>>>>>>>>>>>>>>>> operation) >>>>>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7576 >>>>>>>> (Remove >>>>>>>>>>>>> direct >>>>>>>>>>>>>>>>> references to Unsafe) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Btw, it would be better, I think, to address emails to >>>>>> the >>>>>>>> list >>>>>>>>>>> at >>>>>>>>>>>>>>>> large. >>>>>>>>>>>>>>>>> It encourages more people to participate. If Roman is >>>>> a >>>>>>>>>>> subscriber >>>>>>>>>>>>> he >>>>>>>>>>>>>>>> would >>>>>>>>>>>>>>>>> get a copy anyway. If not, you could encourage him to >>>>>>>> subscribe. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Fri, Oct 11, 2019 at 3:13 PM leerho < >>>>>> [email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Roman, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I hope you are doing well. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I really appreciate the contributions you made to >>>>> our >>>>>>>> Memory >>>>>>>>>>>>>>>> component! >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Given that Druid has made its code base compatible >>>>>> with >>>>>>>>>>> OpenJDK >>>>>>>>>>>>> 11, >>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>>> could use your help in what changes do we need to >>>>> make >>>>>>> to >>>>>>>> make >>>>>>>>>>>>> that >>>>>>>>>>>>>>>>> happen >>>>>>>>>>>>>>>>>> for DataSketches. As I recall, our DataSketches >>>>>> library >>>>>>>> was >>>>>>>>>>> not >>>>>>>>>>>>> the >>>>>>>>>>>>>>>> only >>>>>>>>>>>>>>>>>> Druid dependency that took advantage of Unsafe. So >>>>> I >>>>>>>> assume >>>>>>>>>>> you >>>>>>>>>>>>>> have >>>>>>>>>>>>>>>>> been >>>>>>>>>>>>>>>>>> down this path :) >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Currently, I have a check in the static initializer >>>>> in >>>>>>>>>>>>>>>> Memory/UnsafeUtil: >>>>>>>>>>>>>>>>>> parseJavaVersion(...) that checks the string >>>>> returned >>>>>>> from >>>>>>>>>>>>>>>>>> System.getProperty("java.version"). If that string >>>>>> does >>>>>>>> not >>>>>>>>>>>>> contain >>>>>>>>>>>>>>>>> "1.8" >>>>>>>>>>>>>>>>>> or "8" it will throw an error. Given that we have >>>>> not >>>>>>>> tested >>>>>>>>>>>>> with >>>>>>>>>>>>>>>> JDK 9, >>>>>>>>>>>>>>>>>> 10, 11 or 12, I felt it was safer to explicitly >>>>> throw >>>>>>>> rather >>>>>>>>>>> than >>>>>>>>>>>>>>>> having >>>>>>>>>>>>>>>>>> the users experience some weird failure later that >>>>> may >>>>>>> be >>>>>>>>>>>>> difficult >>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>> debug. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> With only one minor exception (which we could easily >>>>>>> fix) >>>>>>>> the >>>>>>>>>>>>> Memory >>>>>>>>>>>>>>>>>> component is the only place where Unsafe is used. >>>>>>>> However, >>>>>>>>>>> we do >>>>>>>>>>>>>> use >>>>>>>>>>>>>>>>>> reflection to gain internal access to a number of >>>>>> other >>>>>>>>>>> classes >>>>>>>>>>>>>>>> including >>>>>>>>>>>>>>>>>> ByteBuffer, Cleaner, and Bits. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Looking forward to hearing from you. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Lee. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> >>> --------------------------------------------------------------------- >>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> -- >>>> From my cell phone. >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >>> --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
