Hi Lee, DATASKETCHES Jira issues are for the project.
If we want Infrastructure to do something of service to the community then you need to open an INFRA ticket. Got it? Regards? Dave Sent from my iPhone > On Oct 18, 2019, at 3:53 PM, leerho <[email protected]> wrote: > > I opened a ticket (DATASKETCHES-3) with infra to see if we could add this > capability to our slack channel. > >> On Fri, Oct 18, 2019 at 12:29 PM Dave Fisher <[email protected]> wrote: >> >> Hi - >> >> Apache Pulsar uses this: https://github.com/merlimat/slack-email-digest >> >> They don’t have their slack channels at the-asf.slack.com but it does >> work. You can always ask for infra’s help. >> >> Regards, >> Dave >> >>> On Oct 18, 2019, at 12:26 PM, leerho <[email protected]> wrote: >>> >>> I wonder if there is a way to have slack auto-post to an email list? >>> >>>> On Fri, Oct 18, 2019 at 5:13 AM Evans Ye <[email protected]> wrote: >>>> >>>> I don't have the expertise on jdk 11 but would love know more about >>>> datasketches and specifically the java lib. Would be great if you can >> bring >>>> back the result of what you have discussed in the slack dev channel. >> This >>>> helps other contributors easily to search and get to know what happened >> in >>>> the community and get involved :) >>>> >>>> Best >>>> Evans >>>> >>>>> leerho <[email protected]> 於 2019年10月18日 週五 11:03 寫道： >>>>> >>>>> Christopher, >>>>> >>>>> This would be fabulous. I have tons of questions, whenever you are >>>> ready. >>>>> I might be easier to interact via the ASF slack channel "datasketches" >> or >>>>> "datasketches-dev". >>>>> >>>>> Lee. >>>>> >>>>> On Thu, Oct 17, 2019 at 5:07 PM Jon Malkin <[email protected]> >> wrote: >>>>> >>>>>> The memory repository is the one that needs the most conversion help. >>>>>> Everything using Unsafe in datasketches-java runs through the memory >>>>>> package. >>>>>> >>>>>> I'm not sure off-hand if the characterization repo has low level tests >>>> of >>>>>> Memory (on my phone at the moment), but we'd be looking to minimize >> the >>>>>> speed impact in the process since it very directly impacts sketch >>>> speed. >>>>>> >>>>>> jon >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Oct 17, 2019, 4:43 PM Christopher <[email protected]> >> wrote: >>>>>> >>>>>>> I've converted several other projects to using JDK 11, and can help >>>>>>> with this if there's still a need for help. I've actually been >>>> looking >>>>>>> at ways I can contribute to DataSketches since ApacheCon, and this >>>>>>> seems to be something I can probably do. >>>>>>> >>>>>>>> On Wed, Oct 16, 2019 at 9:47 PM leerho <[email protected]> wrote: >>>>>>>> >>>>>>>> Gian, >>>>>>>> >>>>>>>> If I were to supply you with a link to a snapshot or a Release >>>>>> Candidate >>>>>>>> version of the datasketches jar (and Memory jar) could you test in >>>>> your >>>>>>>> environment under JDK 11 and see if it works? >>>>>>>> >>>>>>>> Lee >>>>>>>> >>>>>>>> >>>>>>>>> On Mon, Oct 14, 2019 at 9:19 PM leerho <[email protected]> wrote: >>>>>>>>> >>>>>>>>> Holy Cow! I have never seen such a complex travis config! >>>>>>>>> >>>>>>>>> On Mon, Oct 14, 2019 at 6:54 PM Gian Merlino <[email protected]> >>>>>> wrote: >>>>>>>>> >>>>>>>>>> I should have linked Druid's Travis config file for reference. >>>>> Here >>>>>>> it is: >>>>>>>>>> >>>> https://github.com/apache/incubator-druid/blob/master/.travis.yml >>>>>>>>>> >>>>>>>>>> On Mon, Oct 14, 2019 at 6:53 PM Gian Merlino <[email protected]> >>>>>> wrote: >>>>>>>>>> >>>>>>>>>>>> So what exactly is the requirement? Our DataSketches code >>>>> does >>>>>>>>>> compile >>>>>>>>>>>> under JDK8 and we have lots of customers and users using JDK >>>>> 8. >>>>>>>>>>> >>>>>>>>>>> It depends on who you ask, I guess :) >>>>>>>>>>> >>>>>>>>>>> From Druid's perspective, we aren't going to be compiling your >>>>>> code, >>>>>>>>>> only >>>>>>>>>>> running it. So the only requirement we would ask for is that >>>> the >>>>>>> code >>>>>>>>>>> _runs_ under a Java 11 runtime. Being able to compile under >>>> Java >>>>>> 11 >>>>>>> is >>>>>>>>>> good >>>>>>>>>>> for future-proofing your code, though. It is starting to >>>> become >>>>>> more >>>>>>>>>>> prevalent. >>>>>>>>>>> >>>>>>>>>>>> It sounds like (as far as Unsafe is concerned) converting >>>> all >>>>>> the >>>>>>>>>> direct >>>>>>>>>>>> unsafe.blah() calls to static MethodHandle.invoke calls is >>>>> what >>>>>> is >>>>>>>>>>> required. >>>>>>>>>>>> >>>>>>>>>>>> Is this correct? >>>>>>>>>>> >>>>>>>>>>> That is certainly part of it, I'm not 100% sure if that's all >>>>>> that's >>>>>>>>>>> needed. Btw, CI can help verify. In Druid's Travis config, we >>>>> run >>>>>>> our >>>>>>>>>> tests >>>>>>>>>>> on both JDK 8 and JDK 11. In order to make this automated >>>>> testing >>>>>>>>>> easier, >>>>>>>>>>> we chose to do the work needed to compile Druid on JDK 11, >>>> even >>>>>>> though >>>>>>>>>> we >>>>>>>>>>> still compile release builds using JDK 8 only. (If we hadn't >>>>> done >>>>>>> this >>>>>>>>>>> work, then we'd have to compile on 8 and test on 11, which >>>>> Travis >>>>>>>>>> doesn't >>>>>>>>>>> easily support.) >>>>>>>>>>> >>>>>>>>>>> On Mon, Oct 14, 2019 at 5:51 PM leerho <[email protected]> >>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Thanks, Gian, this is VERY helpful! >>>>>>>>>>>> >>>>>>>>>>>> From my cursory glance, it appears that UnsafeUtil acquires >>>>>> Unsafe >>>>>>> and >>>>>>>>>>>> Unsafe-class the usual way using reflection (I didn't think >>>>> this >>>>>>> would >>>>>>>>>>>> work >>>>>>>>>>>> with 9+ ), and then defines some method-handles that the >>>> other >>>>>>> classes >>>>>>>>>>>> use. >>>>>>>>>>>> >>>>>>>>>>>> Will this approach allow code to compile under JDK11 and >>>> JDK8? >>>>>>>>>>>> >>>>>>>>>>>> It also seems that compiling with JDK11 is still a work in >>>>>>> progress as >>>>>>>>>> the >>>>>>>>>>>> pom.xml clearly specifies JDK8. >>>>>>>>>>>> >>>>>>>>>>>> So what exactly is the requirement? Our DataSketches code >>>> does >>>>>>> compile >>>>>>>>>>>> under JDK8 and we have lots of customers and users using JDK >>>> 8. >>>>>>>>>>>> >>>>>>>>>>>> It sounds like (as far as Unsafe is concerned) converting all >>>>> the >>>>>>>>>> direct >>>>>>>>>>>> unsafe.blah() calls to static MethodHandle.invoke calls is >>>> what >>>>>> is >>>>>>>>>>>> required. >>>>>>>>>>>> >>>>>>>>>>>> Is this correct? >>>>>>>>>>>> >>>>>>>>>>>> Lee. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Oct 14, 2019 at 2:07 PM Gian Merlino < >>>> [email protected]> >>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> And nobody should be using the Oracle JDK without a >>>>> commercial >>>>>>>>>>>> relationship >>>>>>>>>>>>> with Oracle, its license is a nightmare. >>>>>>>>>>>>> >>>>>>>>>>>>> On Mon, Oct 14, 2019 at 2:04 PM Gian Merlino < >>>>> [email protected]> >>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Oracle and OpenJDK builds behave the same as each other >>>>> now. >>>>>>> The >>>>>>>>>>>>>> differences are in terms of licensing, length of support, >>>>> and >>>>>>>>>> release >>>>>>>>>>>>>> cadence. IMO most Java devs should be using one of the >>>>>>> third-party >>>>>>>>>>>>> OpenJDK >>>>>>>>>>>>>> distributions (e.g. Corretto, Zulu, AdoptOpenJDK) rather >>>>> than >>>>>>>>>> vanilla >>>>>>>>>>>>>> OpenJDK these days, because of the new OpenJDK policy >>>> that >>>>>>> public >>>>>>>>>>>> updates >>>>>>>>>>>>>> will cease for major versions after 6 months. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Mon, Oct 14, 2019 at 11:07 AM leerho < >>>> [email protected]> >>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, Gian, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Your references are helpful and I am studying them. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I didn't see any references to OpenJDK vs Oracle's JDKs. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Are there any differences in the way Unsafe calls (or >>>>> other >>>>>>> hidden >>>>>>>>>>>>>>> classes) >>>>>>>>>>>>>>> are handled by OpenJDK? Or is this a "don't care" as >>>> far >>>>> as >>>>>>>>>> Druid is >>>>>>>>>>>>>>> concerned? Are the version numbers between Oracle and >>>>>> OpenJDK >>>>>>>>>> always >>>>>>>>>>>>>>> aligned? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Lee. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Fri, Oct 11, 2019 at 5:45 PM Gian Merlino < >>>>>> [email protected] >>>>>>>> >>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> You didn't address this message to me, but in Druid >>>> most >>>>>> of >>>>>>> the >>>>>>>>>>>> work >>>>>>>>>>>>> for >>>>>>>>>>>>>>>> Java 9+ compatibility is mentioned in this master >>>> issue, >>>>>>> which >>>>>>>>>> you >>>>>>>>>>>>> might >>>>>>>>>>>>>>>> find helpful: >>>>>>>>>>>> https://github.com/apache/incubator-druid/issues/5589. >>>>>>>>>>>>>>>> Skimming the list, these might be particularly >>>> relevant >>>>> to >>>>>>>>>>>>> DataSketches: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7487 >>>>>>> (cleaner >>>>>>>>>>>>>>> operations) >>>>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7466 >>>>>>>>>> (ByteBuffer >>>>>>>>>>>>> unmap >>>>>>>>>>>>>>>> operation) >>>>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7576 >>>>>>> (Remove >>>>>>>>>>>> direct >>>>>>>>>>>>>>>> references to Unsafe) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Btw, it would be better, I think, to address emails to >>>>> the >>>>>>> list >>>>>>>>>> at >>>>>>>>>>>>>>> large. >>>>>>>>>>>>>>>> It encourages more people to participate. If Roman is >>>> a >>>>>>>>>> subscriber >>>>>>>>>>>> he >>>>>>>>>>>>>>> would >>>>>>>>>>>>>>>> get a copy anyway. If not, you could encourage him to >>>>>>> subscribe. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Fri, Oct 11, 2019 at 3:13 PM leerho < >>>>> [email protected]> >>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Roman, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I hope you are doing well. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I really appreciate the contributions you made to >>>> our >>>>>>> Memory >>>>>>>>>>>>>>> component! >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Given that Druid has made its code base compatible >>>>> with >>>>>>>>>> OpenJDK >>>>>>>>>>>> 11, >>>>>>>>>>>>> we >>>>>>>>>>>>>>>>> could use your help in what changes do we need to >>>> make >>>>>> to >>>>>>> make >>>>>>>>>>>> that >>>>>>>>>>>>>>>> happen >>>>>>>>>>>>>>>>> for DataSketches. As I recall, our DataSketches >>>>> library >>>>>>> was >>>>>>>>>> not >>>>>>>>>>>> the >>>>>>>>>>>>>>> only >>>>>>>>>>>>>>>>> Druid dependency that took advantage of Unsafe. So >>>> I >>>>>>> assume >>>>>>>>>> you >>>>>>>>>>>>> have >>>>>>>>>>>>>>>> been >>>>>>>>>>>>>>>>> down this path :) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Currently, I have a check in the static initializer >>>> in >>>>>>>>>>>>>>> Memory/UnsafeUtil: >>>>>>>>>>>>>>>>> parseJavaVersion(...) that checks the string >>>> returned >>>>>> from >>>>>>>>>>>>>>>>> System.getProperty("java.version"). If that string >>>>> does >>>>>>> not >>>>>>>>>>>> contain >>>>>>>>>>>>>>>> "1.8" >>>>>>>>>>>>>>>>> or "8" it will throw an error. Given that we have >>>> not >>>>>>> tested >>>>>>>>>>>> with >>>>>>>>>>>>>>> JDK 9, >>>>>>>>>>>>>>>>> 10, 11 or 12, I felt it was safer to explicitly >>>> throw >>>>>>> rather >>>>>>>>>> than >>>>>>>>>>>>>>> having >>>>>>>>>>>>>>>>> the users experience some weird failure later that >>>> may >>>>>> be >>>>>>>>>>>> difficult >>>>>>>>>>>>> to >>>>>>>>>>>>>>>>> debug. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> With only one minor exception (which we could easily >>>>>> fix) >>>>>>> the >>>>>>>>>>>> Memory >>>>>>>>>>>>>>>>> component is the only place where Unsafe is used. >>>>>>> However, >>>>>>>>>> we do >>>>>>>>>>>>> use >>>>>>>>>>>>>>>>> reflection to gain internal access to a number of >>>>> other >>>>>>>>>> classes >>>>>>>>>>>>>>> including >>>>>>>>>>>>>>>>> ByteBuffer, Cleaner, and Bits. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Looking forward to hearing from you. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Lee. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>> For additional commands, e-mail: [email protected] >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> -- >>> From my cell phone. >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
