Ah! Thank you! I will file a request with INFRA. Lee.
On Sat, Oct 19, 2019 at 9:23 AM Dave Fisher <[email protected]> wrote: > Hi Lee, > > > On Oct 18, 2019, at 5:01 PM, leerho <[email protected]> wrote: > > > > Meanwhile, back to the topic of Migration to OpenJDK11 ... > > > > Unfortunately, the Apache email lists do not accommodate images or files. > > While true by default, Infra will enable images/files if the podling > requests. > > Regards, > Dave > > > So I am going to move this next discussion to our asf slack channel > > "datasketches-dev". > > > > I have posted my study there. You can offer comments or suggestions > > either here or there. > > > > Cheers, > > > > Lee. > > > > On Fri, Oct 18, 2019 at 3:53 PM leerho <[email protected]> wrote: > > > >> I opened a ticket (DATASKETCHES-3) with infra to see if we could add > this > >> capability to our slack channel. > >> > >> On Fri, Oct 18, 2019 at 12:29 PM Dave Fisher <[email protected]> wrote: > >> > >>> Hi - > >>> > >>> Apache Pulsar uses this: > https://github.com/merlimat/slack-email-digest > >>> > >>> They don’t have their slack channels at the-asf.slack.com but it does > >>> work. You can always ask for infra’s help. > >>> > >>> Regards, > >>> Dave > >>> > >>>> On Oct 18, 2019, at 12:26 PM, leerho <[email protected]> wrote: > >>>> > >>>> I wonder if there is a way to have slack auto-post to an email list? > >>>> > >>>> On Fri, Oct 18, 2019 at 5:13 AM Evans Ye <[email protected]> wrote: > >>>> > >>>>> I don't have the expertise on jdk 11 but would love know more about > >>>>> datasketches and specifically the java lib. Would be great if you can > >>> bring > >>>>> back the result of what you have discussed in the slack dev channel. > >>> This > >>>>> helps other contributors easily to search and get to know what > >>> happened in > >>>>> the community and get involved :) > >>>>> > >>>>> Best > >>>>> Evans > >>>>> > >>>>> leerho <[email protected]> 於 2019年10月18日 週五 11:03 寫道： > >>>>> > >>>>>> Christopher, > >>>>>> > >>>>>> This would be fabulous. I have tons of questions, whenever you are > >>>>> ready. > >>>>>> I might be easier to interact via the ASF slack channel > >>> "datasketches" or > >>>>>> "datasketches-dev". > >>>>>> > >>>>>> Lee. > >>>>>> > >>>>>> On Thu, Oct 17, 2019 at 5:07 PM Jon Malkin <[email protected]> > >>> wrote: > >>>>>> > >>>>>>> The memory repository is the one that needs the most conversion > help. > >>>>>>> Everything using Unsafe in datasketches-java runs through the > memory > >>>>>>> package. > >>>>>>> > >>>>>>> I'm not sure off-hand if the characterization repo has low level > >>> tests > >>>>> of > >>>>>>> Memory (on my phone at the moment), but we'd be looking to minimize > >>> the > >>>>>>> speed impact in the process since it very directly impacts sketch > >>>>> speed. > >>>>>>> > >>>>>>> jon > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> On Thu, Oct 17, 2019, 4:43 PM Christopher <[email protected]> > >>> wrote: > >>>>>>> > >>>>>>>> I've converted several other projects to using JDK 11, and can > help > >>>>>>>> with this if there's still a need for help. I've actually been > >>>>> looking > >>>>>>>> at ways I can contribute to DataSketches since ApacheCon, and this > >>>>>>>> seems to be something I can probably do. > >>>>>>>> > >>>>>>>> On Wed, Oct 16, 2019 at 9:47 PM leerho <[email protected]> wrote: > >>>>>>>>> > >>>>>>>>> Gian, > >>>>>>>>> > >>>>>>>>> If I were to supply you with a link to a snapshot or a Release > >>>>>>> Candidate > >>>>>>>>> version of the datasketches jar (and Memory jar) could you test > in > >>>>>> your > >>>>>>>>> environment under JDK 11 and see if it works? > >>>>>>>>> > >>>>>>>>> Lee > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Mon, Oct 14, 2019 at 9:19 PM leerho <[email protected]> wrote: > >>>>>>>>> > >>>>>>>>>> Holy Cow! I have never seen such a complex travis config! > >>>>>>>>>> > >>>>>>>>>> On Mon, Oct 14, 2019 at 6:54 PM Gian Merlino <[email protected]> > >>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>>> I should have linked Druid's Travis config file for reference. > >>>>>> Here > >>>>>>>> it is: > >>>>>>>>>>> > >>>>> https://github.com/apache/incubator-druid/blob/master/.travis.yml > >>>>>>>>>>> > >>>>>>>>>>> On Mon, Oct 14, 2019 at 6:53 PM Gian Merlino <[email protected]> > >>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>>> So what exactly is the requirement? Our DataSketches code > >>>>>> does > >>>>>>>>>>> compile > >>>>>>>>>>>>> under JDK8 and we have lots of customers and users using JDK > >>>>>> 8. > >>>>>>>>>>>> > >>>>>>>>>>>> It depends on who you ask, I guess :) > >>>>>>>>>>>> > >>>>>>>>>>>> From Druid's perspective, we aren't going to be compiling your > >>>>>>> code, > >>>>>>>>>>> only > >>>>>>>>>>>> running it. So the only requirement we would ask for is that > >>>>> the > >>>>>>>> code > >>>>>>>>>>>> _runs_ under a Java 11 runtime. Being able to compile under > >>>>> Java > >>>>>>> 11 > >>>>>>>> is > >>>>>>>>>>> good > >>>>>>>>>>>> for future-proofing your code, though. It is starting to > >>>>> become > >>>>>>> more > >>>>>>>>>>>> prevalent. > >>>>>>>>>>>> > >>>>>>>>>>>>> It sounds like (as far as Unsafe is concerned) converting > >>>>> all > >>>>>>> the > >>>>>>>>>>> direct > >>>>>>>>>>>>> unsafe.blah() calls to static MethodHandle.invoke calls is > >>>>>> what > >>>>>>> is > >>>>>>>>>>>> required. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Is this correct? > >>>>>>>>>>>> > >>>>>>>>>>>> That is certainly part of it, I'm not 100% sure if that's all > >>>>>>> that's > >>>>>>>>>>>> needed. Btw, CI can help verify. In Druid's Travis config, we > >>>>>> run > >>>>>>>> our > >>>>>>>>>>> tests > >>>>>>>>>>>> on both JDK 8 and JDK 11. In order to make this automated > >>>>>> testing > >>>>>>>>>>> easier, > >>>>>>>>>>>> we chose to do the work needed to compile Druid on JDK 11, > >>>>> even > >>>>>>>> though > >>>>>>>>>>> we > >>>>>>>>>>>> still compile release builds using JDK 8 only. (If we hadn't > >>>>>> done > >>>>>>>> this > >>>>>>>>>>>> work, then we'd have to compile on 8 and test on 11, which > >>>>>> Travis > >>>>>>>>>>> doesn't > >>>>>>>>>>>> easily support.) > >>>>>>>>>>>> > >>>>>>>>>>>> On Mon, Oct 14, 2019 at 5:51 PM leerho <[email protected]> > >>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> Thanks, Gian, this is VERY helpful! > >>>>>>>>>>>>> > >>>>>>>>>>>>> From my cursory glance, it appears that UnsafeUtil acquires > >>>>>>> Unsafe > >>>>>>>> and > >>>>>>>>>>>>> Unsafe-class the usual way using reflection (I didn't think > >>>>>> this > >>>>>>>> would > >>>>>>>>>>>>> work > >>>>>>>>>>>>> with 9+ ), and then defines some method-handles that the > >>>>> other > >>>>>>>> classes > >>>>>>>>>>>>> use. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Will this approach allow code to compile under JDK11 and > >>>>> JDK8? > >>>>>>>>>>>>> > >>>>>>>>>>>>> It also seems that compiling with JDK11 is still a work in > >>>>>>>> progress as > >>>>>>>>>>> the > >>>>>>>>>>>>> pom.xml clearly specifies JDK8. > >>>>>>>>>>>>> > >>>>>>>>>>>>> So what exactly is the requirement? Our DataSketches code > >>>>> does > >>>>>>>> compile > >>>>>>>>>>>>> under JDK8 and we have lots of customers and users using JDK > >>>>> 8. > >>>>>>>>>>>>> > >>>>>>>>>>>>> It sounds like (as far as Unsafe is concerned) converting all > >>>>>> the > >>>>>>>>>>> direct > >>>>>>>>>>>>> unsafe.blah() calls to static MethodHandle.invoke calls is > >>>>> what > >>>>>>> is > >>>>>>>>>>>>> required. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Is this correct? > >>>>>>>>>>>>> > >>>>>>>>>>>>> Lee. > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Mon, Oct 14, 2019 at 2:07 PM Gian Merlino < > >>>>> [email protected]> > >>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> And nobody should be using the Oracle JDK without a > >>>>>> commercial > >>>>>>>>>>>>> relationship > >>>>>>>>>>>>>> with Oracle, its license is a nightmare. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Mon, Oct 14, 2019 at 2:04 PM Gian Merlino < > >>>>>> [email protected]> > >>>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Oracle and OpenJDK builds behave the same as each other > >>>>>> now. > >>>>>>>> The > >>>>>>>>>>>>>>> differences are in terms of licensing, length of support, > >>>>>> and > >>>>>>>>>>> release > >>>>>>>>>>>>>>> cadence. IMO most Java devs should be using one of the > >>>>>>>> third-party > >>>>>>>>>>>>>> OpenJDK > >>>>>>>>>>>>>>> distributions (e.g. Corretto, Zulu, AdoptOpenJDK) rather > >>>>>> than > >>>>>>>>>>> vanilla > >>>>>>>>>>>>>>> OpenJDK these days, because of the new OpenJDK policy > >>>>> that > >>>>>>>> public > >>>>>>>>>>>>> updates > >>>>>>>>>>>>>>> will cease for major versions after 6 months. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Mon, Oct 14, 2019 at 11:07 AM leerho < > >>>>> [email protected]> > >>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Thanks, Gian, > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Your references are helpful and I am studying them. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> I didn't see any references to OpenJDK vs Oracle's JDKs. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Are there any differences in the way Unsafe calls (or > >>>>>> other > >>>>>>>> hidden > >>>>>>>>>>>>>>>> classes) > >>>>>>>>>>>>>>>> are handled by OpenJDK? Or is this a "don't care" as > >>>>> far > >>>>>> as > >>>>>>>>>>> Druid is > >>>>>>>>>>>>>>>> concerned? Are the version numbers between Oracle and > >>>>>>> OpenJDK > >>>>>>>>>>> always > >>>>>>>>>>>>>>>> aligned? > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Lee. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Fri, Oct 11, 2019 at 5:45 PM Gian Merlino < > >>>>>>> [email protected] > >>>>>>>>> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> You didn't address this message to me, but in Druid > >>>>> most > >>>>>>> of > >>>>>>>> the > >>>>>>>>>>>>> work > >>>>>>>>>>>>>> for > >>>>>>>>>>>>>>>>> Java 9+ compatibility is mentioned in this master > >>>>> issue, > >>>>>>>> which > >>>>>>>>>>> you > >>>>>>>>>>>>>> might > >>>>>>>>>>>>>>>>> find helpful: > >>>>>>>>>>>>> https://github.com/apache/incubator-druid/issues/5589. > >>>>>>>>>>>>>>>>> Skimming the list, these might be particularly > >>>>> relevant > >>>>>> to > >>>>>>>>>>>>>> DataSketches: > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7487 > >>>>>>>> (cleaner > >>>>>>>>>>>>>>>> operations) > >>>>>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7466 > >>>>>>>>>>> (ByteBuffer > >>>>>>>>>>>>>> unmap > >>>>>>>>>>>>>>>>> operation) > >>>>>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7576 > >>>>>>>> (Remove > >>>>>>>>>>>>> direct > >>>>>>>>>>>>>>>>> references to Unsafe) > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Btw, it would be better, I think, to address emails to > >>>>>> the > >>>>>>>> list > >>>>>>>>>>> at > >>>>>>>>>>>>>>>> large. > >>>>>>>>>>>>>>>>> It encourages more people to participate. If Roman is > >>>>> a > >>>>>>>>>>> subscriber > >>>>>>>>>>>>> he > >>>>>>>>>>>>>>>> would > >>>>>>>>>>>>>>>>> get a copy anyway. If not, you could encourage him to > >>>>>>>> subscribe. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> On Fri, Oct 11, 2019 at 3:13 PM leerho < > >>>>>> [email protected]> > >>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Roman, > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> I hope you are doing well. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> I really appreciate the contributions you made to > >>>>> our > >>>>>>>> Memory > >>>>>>>>>>>>>>>> component! > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Given that Druid has made its code base compatible > >>>>>> with > >>>>>>>>>>> OpenJDK > >>>>>>>>>>>>> 11, > >>>>>>>>>>>>>> we > >>>>>>>>>>>>>>>>>> could use your help in what changes do we need to > >>>>> make > >>>>>>> to > >>>>>>>> make > >>>>>>>>>>>>> that > >>>>>>>>>>>>>>>>> happen > >>>>>>>>>>>>>>>>>> for DataSketches. As I recall, our DataSketches > >>>>>> library > >>>>>>>> was > >>>>>>>>>>> not > >>>>>>>>>>>>> the > >>>>>>>>>>>>>>>> only > >>>>>>>>>>>>>>>>>> Druid dependency that took advantage of Unsafe. So > >>>>> I > >>>>>>>> assume > >>>>>>>>>>> you > >>>>>>>>>>>>>> have > >>>>>>>>>>>>>>>>> been > >>>>>>>>>>>>>>>>>> down this path :) > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Currently, I have a check in the static initializer > >>>>> in > >>>>>>>>>>>>>>>> Memory/UnsafeUtil: > >>>>>>>>>>>>>>>>>> parseJavaVersion(...) that checks the string > >>>>> returned > >>>>>>> from > >>>>>>>>>>>>>>>>>> System.getProperty("java.version"). If that string > >>>>>> does > >>>>>>>> not > >>>>>>>>>>>>> contain > >>>>>>>>>>>>>>>>> "1.8" > >>>>>>>>>>>>>>>>>> or "8" it will throw an error. Given that we have > >>>>> not > >>>>>>>> tested > >>>>>>>>>>>>> with > >>>>>>>>>>>>>>>> JDK 9, > >>>>>>>>>>>>>>>>>> 10, 11 or 12, I felt it was safer to explicitly > >>>>> throw > >>>>>>>> rather > >>>>>>>>>>> than > >>>>>>>>>>>>>>>> having > >>>>>>>>>>>>>>>>>> the users experience some weird failure later that > >>>>> may > >>>>>>> be > >>>>>>>>>>>>> difficult > >>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>> debug. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> With only one minor exception (which we could easily > >>>>>>> fix) > >>>>>>>> the > >>>>>>>>>>>>> Memory > >>>>>>>>>>>>>>>>>> component is the only place where Unsafe is used. > >>>>>>>> However, > >>>>>>>>>>> we do > >>>>>>>>>>>>>> use > >>>>>>>>>>>>>>>>>> reflection to gain internal access to a number of > >>>>>> other > >>>>>>>>>>> classes > >>>>>>>>>>>>>>>> including > >>>>>>>>>>>>>>>>>> ByteBuffer, Cleaner, and Bits. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Looking forward to hearing from you. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Cheers, > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Lee. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>>>>>> > >>> --------------------------------------------------------------------- > >>>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>>> For additional commands, e-mail: [email protected] > >>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>> -- > >>>> From my cell phone. > >>> > >>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: [email protected] > >>> For additional commands, e-mail: [email protected] > >>> > >>> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
