I thought that is what I did. Isn't the DATASKETCHES-3 ticket I opened the same thing?
On Fri, Oct 18, 2019 at 5:42 PM Dave Fisher <[email protected]> wrote: > Hi Lee, > > DATASKETCHES Jira issues are for the project. > > If we want Infrastructure to do something of service to the community then > you need to open an INFRA ticket. Got it? > > Regards? > Dave > > Sent from my iPhone > > > On Oct 18, 2019, at 3:53 PM, leerho <[email protected]> wrote: > > > > I opened a ticket (DATASKETCHES-3) with infra to see if we could add this > > capability to our slack channel. > > > >> On Fri, Oct 18, 2019 at 12:29 PM Dave Fisher <[email protected]> wrote: > >> > >> Hi - > >> > >> Apache Pulsar uses this: https://github.com/merlimat/slack-email-digest > >> > >> They don’t have their slack channels at the-asf.slack.com but it does > >> work. You can always ask for infra’s help. > >> > >> Regards, > >> Dave > >> > >>> On Oct 18, 2019, at 12:26 PM, leerho <[email protected]> wrote: > >>> > >>> I wonder if there is a way to have slack auto-post to an email list? > >>> > >>>> On Fri, Oct 18, 2019 at 5:13 AM Evans Ye <[email protected]> wrote: > >>>> > >>>> I don't have the expertise on jdk 11 but would love know more about > >>>> datasketches and specifically the java lib. Would be great if you can > >> bring > >>>> back the result of what you have discussed in the slack dev channel. > >> This > >>>> helps other contributors easily to search and get to know what > happened > >> in > >>>> the community and get involved :) > >>>> > >>>> Best > >>>> Evans > >>>> > >>>>> leerho <[email protected]> 於 2019年10月18日 週五 11:03 寫道： > >>>>> > >>>>> Christopher, > >>>>> > >>>>> This would be fabulous. I have tons of questions, whenever you are > >>>> ready. > >>>>> I might be easier to interact via the ASF slack channel > "datasketches" > >> or > >>>>> "datasketches-dev". > >>>>> > >>>>> Lee. > >>>>> > >>>>> On Thu, Oct 17, 2019 at 5:07 PM Jon Malkin <[email protected]> > >> wrote: > >>>>> > >>>>>> The memory repository is the one that needs the most conversion > help. > >>>>>> Everything using Unsafe in datasketches-java runs through the memory > >>>>>> package. > >>>>>> > >>>>>> I'm not sure off-hand if the characterization repo has low level > tests > >>>> of > >>>>>> Memory (on my phone at the moment), but we'd be looking to minimize > >> the > >>>>>> speed impact in the process since it very directly impacts sketch > >>>> speed. > >>>>>> > >>>>>> jon > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Thu, Oct 17, 2019, 4:43 PM Christopher <[email protected]> > >> wrote: > >>>>>> > >>>>>>> I've converted several other projects to using JDK 11, and can help > >>>>>>> with this if there's still a need for help. I've actually been > >>>> looking > >>>>>>> at ways I can contribute to DataSketches since ApacheCon, and this > >>>>>>> seems to be something I can probably do. > >>>>>>> > >>>>>>>> On Wed, Oct 16, 2019 at 9:47 PM leerho <[email protected]> wrote: > >>>>>>>> > >>>>>>>> Gian, > >>>>>>>> > >>>>>>>> If I were to supply you with a link to a snapshot or a Release > >>>>>> Candidate > >>>>>>>> version of the datasketches jar (and Memory jar) could you test in > >>>>> your > >>>>>>>> environment under JDK 11 and see if it works? > >>>>>>>> > >>>>>>>> Lee > >>>>>>>> > >>>>>>>> > >>>>>>>>> On Mon, Oct 14, 2019 at 9:19 PM leerho <[email protected]> wrote: > >>>>>>>>> > >>>>>>>>> Holy Cow! I have never seen such a complex travis config! > >>>>>>>>> > >>>>>>>>> On Mon, Oct 14, 2019 at 6:54 PM Gian Merlino <[email protected]> > >>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> I should have linked Druid's Travis config file for reference. > >>>>> Here > >>>>>>> it is: > >>>>>>>>>> > >>>> https://github.com/apache/incubator-druid/blob/master/.travis.yml > >>>>>>>>>> > >>>>>>>>>> On Mon, Oct 14, 2019 at 6:53 PM Gian Merlino <[email protected]> > >>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>>>> So what exactly is the requirement? Our DataSketches code > >>>>> does > >>>>>>>>>> compile > >>>>>>>>>>>> under JDK8 and we have lots of customers and users using JDK > >>>>> 8. > >>>>>>>>>>> > >>>>>>>>>>> It depends on who you ask, I guess :) > >>>>>>>>>>> > >>>>>>>>>>> From Druid's perspective, we aren't going to be compiling your > >>>>>> code, > >>>>>>>>>> only > >>>>>>>>>>> running it. So the only requirement we would ask for is that > >>>> the > >>>>>>> code > >>>>>>>>>>> _runs_ under a Java 11 runtime. Being able to compile under > >>>> Java > >>>>>> 11 > >>>>>>> is > >>>>>>>>>> good > >>>>>>>>>>> for future-proofing your code, though. It is starting to > >>>> become > >>>>>> more > >>>>>>>>>>> prevalent. > >>>>>>>>>>> > >>>>>>>>>>>> It sounds like (as far as Unsafe is concerned) converting > >>>> all > >>>>>> the > >>>>>>>>>> direct > >>>>>>>>>>>> unsafe.blah() calls to static MethodHandle.invoke calls is > >>>>> what > >>>>>> is > >>>>>>>>>>> required. > >>>>>>>>>>>> > >>>>>>>>>>>> Is this correct? > >>>>>>>>>>> > >>>>>>>>>>> That is certainly part of it, I'm not 100% sure if that's all > >>>>>> that's > >>>>>>>>>>> needed. Btw, CI can help verify. In Druid's Travis config, we > >>>>> run > >>>>>>> our > >>>>>>>>>> tests > >>>>>>>>>>> on both JDK 8 and JDK 11. In order to make this automated > >>>>> testing > >>>>>>>>>> easier, > >>>>>>>>>>> we chose to do the work needed to compile Druid on JDK 11, > >>>> even > >>>>>>> though > >>>>>>>>>> we > >>>>>>>>>>> still compile release builds using JDK 8 only. (If we hadn't > >>>>> done > >>>>>>> this > >>>>>>>>>>> work, then we'd have to compile on 8 and test on 11, which > >>>>> Travis > >>>>>>>>>> doesn't > >>>>>>>>>>> easily support.) > >>>>>>>>>>> > >>>>>>>>>>> On Mon, Oct 14, 2019 at 5:51 PM leerho <[email protected]> > >>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Thanks, Gian, this is VERY helpful! > >>>>>>>>>>>> > >>>>>>>>>>>> From my cursory glance, it appears that UnsafeUtil acquires > >>>>>> Unsafe > >>>>>>> and > >>>>>>>>>>>> Unsafe-class the usual way using reflection (I didn't think > >>>>> this > >>>>>>> would > >>>>>>>>>>>> work > >>>>>>>>>>>> with 9+ ), and then defines some method-handles that the > >>>> other > >>>>>>> classes > >>>>>>>>>>>> use. > >>>>>>>>>>>> > >>>>>>>>>>>> Will this approach allow code to compile under JDK11 and > >>>> JDK8? > >>>>>>>>>>>> > >>>>>>>>>>>> It also seems that compiling with JDK11 is still a work in > >>>>>>> progress as > >>>>>>>>>> the > >>>>>>>>>>>> pom.xml clearly specifies JDK8. > >>>>>>>>>>>> > >>>>>>>>>>>> So what exactly is the requirement? Our DataSketches code > >>>> does > >>>>>>> compile > >>>>>>>>>>>> under JDK8 and we have lots of customers and users using JDK > >>>> 8. > >>>>>>>>>>>> > >>>>>>>>>>>> It sounds like (as far as Unsafe is concerned) converting all > >>>>> the > >>>>>>>>>> direct > >>>>>>>>>>>> unsafe.blah() calls to static MethodHandle.invoke calls is > >>>> what > >>>>>> is > >>>>>>>>>>>> required. > >>>>>>>>>>>> > >>>>>>>>>>>> Is this correct? > >>>>>>>>>>>> > >>>>>>>>>>>> Lee. > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> On Mon, Oct 14, 2019 at 2:07 PM Gian Merlino < > >>>> [email protected]> > >>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> And nobody should be using the Oracle JDK without a > >>>>> commercial > >>>>>>>>>>>> relationship > >>>>>>>>>>>>> with Oracle, its license is a nightmare. > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Mon, Oct 14, 2019 at 2:04 PM Gian Merlino < > >>>>> [email protected]> > >>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Oracle and OpenJDK builds behave the same as each other > >>>>> now. > >>>>>>> The > >>>>>>>>>>>>>> differences are in terms of licensing, length of support, > >>>>> and > >>>>>>>>>> release > >>>>>>>>>>>>>> cadence. IMO most Java devs should be using one of the > >>>>>>> third-party > >>>>>>>>>>>>> OpenJDK > >>>>>>>>>>>>>> distributions (e.g. Corretto, Zulu, AdoptOpenJDK) rather > >>>>> than > >>>>>>>>>> vanilla > >>>>>>>>>>>>>> OpenJDK these days, because of the new OpenJDK policy > >>>> that > >>>>>>> public > >>>>>>>>>>>> updates > >>>>>>>>>>>>>> will cease for major versions after 6 months. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Mon, Oct 14, 2019 at 11:07 AM leerho < > >>>> [email protected]> > >>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Thanks, Gian, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Your references are helpful and I am studying them. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I didn't see any references to OpenJDK vs Oracle's JDKs. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Are there any differences in the way Unsafe calls (or > >>>>> other > >>>>>>> hidden > >>>>>>>>>>>>>>> classes) > >>>>>>>>>>>>>>> are handled by OpenJDK? Or is this a "don't care" as > >>>> far > >>>>> as > >>>>>>>>>> Druid is > >>>>>>>>>>>>>>> concerned? Are the version numbers between Oracle and > >>>>>> OpenJDK > >>>>>>>>>> always > >>>>>>>>>>>>>>> aligned? > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Lee. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Fri, Oct 11, 2019 at 5:45 PM Gian Merlino < > >>>>>> [email protected] > >>>>>>>> > >>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> You didn't address this message to me, but in Druid > >>>> most > >>>>>> of > >>>>>>> the > >>>>>>>>>>>> work > >>>>>>>>>>>>> for > >>>>>>>>>>>>>>>> Java 9+ compatibility is mentioned in this master > >>>> issue, > >>>>>>> which > >>>>>>>>>> you > >>>>>>>>>>>>> might > >>>>>>>>>>>>>>>> find helpful: > >>>>>>>>>>>> https://github.com/apache/incubator-druid/issues/5589. > >>>>>>>>>>>>>>>> Skimming the list, these might be particularly > >>>> relevant > >>>>> to > >>>>>>>>>>>>> DataSketches: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7487 > >>>>>>> (cleaner > >>>>>>>>>>>>>>> operations) > >>>>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7466 > >>>>>>>>>> (ByteBuffer > >>>>>>>>>>>>> unmap > >>>>>>>>>>>>>>>> operation) > >>>>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7576 > >>>>>>> (Remove > >>>>>>>>>>>> direct > >>>>>>>>>>>>>>>> references to Unsafe) > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Btw, it would be better, I think, to address emails to > >>>>> the > >>>>>>> list > >>>>>>>>>> at > >>>>>>>>>>>>>>> large. > >>>>>>>>>>>>>>>> It encourages more people to participate. If Roman is > >>>> a > >>>>>>>>>> subscriber > >>>>>>>>>>>> he > >>>>>>>>>>>>>>> would > >>>>>>>>>>>>>>>> get a copy anyway. If not, you could encourage him to > >>>>>>> subscribe. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Fri, Oct 11, 2019 at 3:13 PM leerho < > >>>>> [email protected]> > >>>>>>>>>> wrote: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Roman, > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> I hope you are doing well. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> I really appreciate the contributions you made to > >>>> our > >>>>>>> Memory > >>>>>>>>>>>>>>> component! > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Given that Druid has made its code base compatible > >>>>> with > >>>>>>>>>> OpenJDK > >>>>>>>>>>>> 11, > >>>>>>>>>>>>> we > >>>>>>>>>>>>>>>>> could use your help in what changes do we need to > >>>> make > >>>>>> to > >>>>>>> make > >>>>>>>>>>>> that > >>>>>>>>>>>>>>>> happen > >>>>>>>>>>>>>>>>> for DataSketches. As I recall, our DataSketches > >>>>> library > >>>>>>> was > >>>>>>>>>> not > >>>>>>>>>>>> the > >>>>>>>>>>>>>>> only > >>>>>>>>>>>>>>>>> Druid dependency that took advantage of Unsafe. So > >>>> I > >>>>>>> assume > >>>>>>>>>> you > >>>>>>>>>>>>> have > >>>>>>>>>>>>>>>> been > >>>>>>>>>>>>>>>>> down this path :) > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Currently, I have a check in the static initializer > >>>> in > >>>>>>>>>>>>>>> Memory/UnsafeUtil: > >>>>>>>>>>>>>>>>> parseJavaVersion(...) that checks the string > >>>> returned > >>>>>> from > >>>>>>>>>>>>>>>>> System.getProperty("java.version"). If that string > >>>>> does > >>>>>>> not > >>>>>>>>>>>> contain > >>>>>>>>>>>>>>>> "1.8" > >>>>>>>>>>>>>>>>> or "8" it will throw an error. Given that we have > >>>> not > >>>>>>> tested > >>>>>>>>>>>> with > >>>>>>>>>>>>>>> JDK 9, > >>>>>>>>>>>>>>>>> 10, 11 or 12, I felt it was safer to explicitly > >>>> throw > >>>>>>> rather > >>>>>>>>>> than > >>>>>>>>>>>>>>> having > >>>>>>>>>>>>>>>>> the users experience some weird failure later that > >>>> may > >>>>>> be > >>>>>>>>>>>> difficult > >>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>> debug. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> With only one minor exception (which we could easily > >>>>>> fix) > >>>>>>> the > >>>>>>>>>>>> Memory > >>>>>>>>>>>>>>>>> component is the only place where Unsafe is used. > >>>>>>> However, > >>>>>>>>>> we do > >>>>>>>>>>>>> use > >>>>>>>>>>>>>>>>> reflection to gain internal access to a number of > >>>>> other > >>>>>>>>>> classes > >>>>>>>>>>>>>>> including > >>>>>>>>>>>>>>>>> ByteBuffer, Cleaner, and Bits. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Looking forward to hearing from you. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Cheers, > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Lee. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>>> > --------------------------------------------------------------------- > >>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>> For additional commands, e-mail: [email protected] > >>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>> > >>> -- > >>> From my cell phone. > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: [email protected] > >> For additional commands, e-mail: [email protected] > >> > >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
