I opened a ticket (DATASKETCHES-3) with infra to see if we could add this capability to our slack channel.
On Fri, Oct 18, 2019 at 12:29 PM Dave Fisher <[email protected]> wrote: > Hi - > > Apache Pulsar uses this: https://github.com/merlimat/slack-email-digest > > They don’t have their slack channels at the-asf.slack.com but it does > work. You can always ask for infra’s help. > > Regards, > Dave > > > On Oct 18, 2019, at 12:26 PM, leerho <[email protected]> wrote: > > > > I wonder if there is a way to have slack auto-post to an email list? > > > > On Fri, Oct 18, 2019 at 5:13 AM Evans Ye <[email protected]> wrote: > > > >> I don't have the expertise on jdk 11 but would love know more about > >> datasketches and specifically the java lib. Would be great if you can > bring > >> back the result of what you have discussed in the slack dev channel. > This > >> helps other contributors easily to search and get to know what happened > in > >> the community and get involved :) > >> > >> Best > >> Evans > >> > >> leerho <[email protected]> 於 2019年10月18日 週五 11:03 寫道： > >> > >>> Christopher, > >>> > >>> This would be fabulous. I have tons of questions, whenever you are > >> ready. > >>> I might be easier to interact via the ASF slack channel "datasketches" > or > >>> "datasketches-dev". > >>> > >>> Lee. > >>> > >>> On Thu, Oct 17, 2019 at 5:07 PM Jon Malkin <[email protected]> > wrote: > >>> > >>>> The memory repository is the one that needs the most conversion help. > >>>> Everything using Unsafe in datasketches-java runs through the memory > >>>> package. > >>>> > >>>> I'm not sure off-hand if the characterization repo has low level tests > >> of > >>>> Memory (on my phone at the moment), but we'd be looking to minimize > the > >>>> speed impact in the process since it very directly impacts sketch > >> speed. > >>>> > >>>> jon > >>>> > >>>> > >>>> > >>>> On Thu, Oct 17, 2019, 4:43 PM Christopher <[email protected]> > wrote: > >>>> > >>>>> I've converted several other projects to using JDK 11, and can help > >>>>> with this if there's still a need for help. I've actually been > >> looking > >>>>> at ways I can contribute to DataSketches since ApacheCon, and this > >>>>> seems to be something I can probably do. > >>>>> > >>>>> On Wed, Oct 16, 2019 at 9:47 PM leerho <[email protected]> wrote: > >>>>>> > >>>>>> Gian, > >>>>>> > >>>>>> If I were to supply you with a link to a snapshot or a Release > >>>> Candidate > >>>>>> version of the datasketches jar (and Memory jar) could you test in > >>> your > >>>>>> environment under JDK 11 and see if it works? > >>>>>> > >>>>>> Lee > >>>>>> > >>>>>> > >>>>>> On Mon, Oct 14, 2019 at 9:19 PM leerho <[email protected]> wrote: > >>>>>> > >>>>>>> Holy Cow! I have never seen such a complex travis config! > >>>>>>> > >>>>>>> On Mon, Oct 14, 2019 at 6:54 PM Gian Merlino <[email protected]> > >>>> wrote: > >>>>>>> > >>>>>>>> I should have linked Druid's Travis config file for reference. > >>> Here > >>>>> it is: > >>>>>>>> > >> https://github.com/apache/incubator-druid/blob/master/.travis.yml > >>>>>>>> > >>>>>>>> On Mon, Oct 14, 2019 at 6:53 PM Gian Merlino <[email protected]> > >>>> wrote: > >>>>>>>> > >>>>>>>>>> So what exactly is the requirement? Our DataSketches code > >>> does > >>>>>>>> compile > >>>>>>>>>> under JDK8 and we have lots of customers and users using JDK > >>> 8. > >>>>>>>>> > >>>>>>>>> It depends on who you ask, I guess :) > >>>>>>>>> > >>>>>>>>> From Druid's perspective, we aren't going to be compiling your > >>>> code, > >>>>>>>> only > >>>>>>>>> running it. So the only requirement we would ask for is that > >> the > >>>>> code > >>>>>>>>> _runs_ under a Java 11 runtime. Being able to compile under > >> Java > >>>> 11 > >>>>> is > >>>>>>>> good > >>>>>>>>> for future-proofing your code, though. It is starting to > >> become > >>>> more > >>>>>>>>> prevalent. > >>>>>>>>> > >>>>>>>>>> It sounds like (as far as Unsafe is concerned) converting > >> all > >>>> the > >>>>>>>> direct > >>>>>>>>>> unsafe.blah() calls to static MethodHandle.invoke calls is > >>> what > >>>> is > >>>>>>>>> required. > >>>>>>>>>> > >>>>>>>>>> Is this correct? > >>>>>>>>> > >>>>>>>>> That is certainly part of it, I'm not 100% sure if that's all > >>>> that's > >>>>>>>>> needed. Btw, CI can help verify. In Druid's Travis config, we > >>> run > >>>>> our > >>>>>>>> tests > >>>>>>>>> on both JDK 8 and JDK 11. In order to make this automated > >>> testing > >>>>>>>> easier, > >>>>>>>>> we chose to do the work needed to compile Druid on JDK 11, > >> even > >>>>> though > >>>>>>>> we > >>>>>>>>> still compile release builds using JDK 8 only. (If we hadn't > >>> done > >>>>> this > >>>>>>>>> work, then we'd have to compile on 8 and test on 11, which > >>> Travis > >>>>>>>> doesn't > >>>>>>>>> easily support.) > >>>>>>>>> > >>>>>>>>> On Mon, Oct 14, 2019 at 5:51 PM leerho <[email protected]> > >>> wrote: > >>>>>>>>> > >>>>>>>>>> Thanks, Gian, this is VERY helpful! > >>>>>>>>>> > >>>>>>>>>> From my cursory glance, it appears that UnsafeUtil acquires > >>>> Unsafe > >>>>> and > >>>>>>>>>> Unsafe-class the usual way using reflection (I didn't think > >>> this > >>>>> would > >>>>>>>>>> work > >>>>>>>>>> with 9+ ), and then defines some method-handles that the > >> other > >>>>> classes > >>>>>>>>>> use. > >>>>>>>>>> > >>>>>>>>>> Will this approach allow code to compile under JDK11 and > >> JDK8? > >>>>>>>>>> > >>>>>>>>>> It also seems that compiling with JDK11 is still a work in > >>>>> progress as > >>>>>>>> the > >>>>>>>>>> pom.xml clearly specifies JDK8. > >>>>>>>>>> > >>>>>>>>>> So what exactly is the requirement? Our DataSketches code > >> does > >>>>> compile > >>>>>>>>>> under JDK8 and we have lots of customers and users using JDK > >> 8. > >>>>>>>>>> > >>>>>>>>>> It sounds like (as far as Unsafe is concerned) converting all > >>> the > >>>>>>>> direct > >>>>>>>>>> unsafe.blah() calls to static MethodHandle.invoke calls is > >> what > >>>> is > >>>>>>>>>> required. > >>>>>>>>>> > >>>>>>>>>> Is this correct? > >>>>>>>>>> > >>>>>>>>>> Lee. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> On Mon, Oct 14, 2019 at 2:07 PM Gian Merlino < > >> [email protected]> > >>>>> wrote: > >>>>>>>>>> > >>>>>>>>>>> And nobody should be using the Oracle JDK without a > >>> commercial > >>>>>>>>>> relationship > >>>>>>>>>>> with Oracle, its license is a nightmare. > >>>>>>>>>>> > >>>>>>>>>>> On Mon, Oct 14, 2019 at 2:04 PM Gian Merlino < > >>> [email protected]> > >>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Oracle and OpenJDK builds behave the same as each other > >>> now. > >>>>> The > >>>>>>>>>>>> differences are in terms of licensing, length of support, > >>> and > >>>>>>>> release > >>>>>>>>>>>> cadence. IMO most Java devs should be using one of the > >>>>> third-party > >>>>>>>>>>> OpenJDK > >>>>>>>>>>>> distributions (e.g. Corretto, Zulu, AdoptOpenJDK) rather > >>> than > >>>>>>>> vanilla > >>>>>>>>>>>> OpenJDK these days, because of the new OpenJDK policy > >> that > >>>>> public > >>>>>>>>>> updates > >>>>>>>>>>>> will cease for major versions after 6 months. > >>>>>>>>>>>> > >>>>>>>>>>>> On Mon, Oct 14, 2019 at 11:07 AM leerho < > >> [email protected]> > >>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> Thanks, Gian, > >>>>>>>>>>>>> > >>>>>>>>>>>>> Your references are helpful and I am studying them. > >>>>>>>>>>>>> > >>>>>>>>>>>>> I didn't see any references to OpenJDK vs Oracle's JDKs. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Are there any differences in the way Unsafe calls (or > >>> other > >>>>> hidden > >>>>>>>>>>>>> classes) > >>>>>>>>>>>>> are handled by OpenJDK? Or is this a "don't care" as > >> far > >>> as > >>>>>>>> Druid is > >>>>>>>>>>>>> concerned? Are the version numbers between Oracle and > >>>> OpenJDK > >>>>>>>> always > >>>>>>>>>>>>> aligned? > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> Lee. > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Fri, Oct 11, 2019 at 5:45 PM Gian Merlino < > >>>> [email protected] > >>>>>> > >>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> You didn't address this message to me, but in Druid > >> most > >>>> of > >>>>> the > >>>>>>>>>> work > >>>>>>>>>>> for > >>>>>>>>>>>>>> Java 9+ compatibility is mentioned in this master > >> issue, > >>>>> which > >>>>>>>> you > >>>>>>>>>>> might > >>>>>>>>>>>>>> find helpful: > >>>>>>>>>> https://github.com/apache/incubator-druid/issues/5589. > >>>>>>>>>>>>>> Skimming the list, these might be particularly > >> relevant > >>> to > >>>>>>>>>>> DataSketches: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7487 > >>>>> (cleaner > >>>>>>>>>>>>> operations) > >>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7466 > >>>>>>>> (ByteBuffer > >>>>>>>>>>> unmap > >>>>>>>>>>>>>> operation) > >>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7576 > >>>>> (Remove > >>>>>>>>>> direct > >>>>>>>>>>>>>> references to Unsafe) > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Btw, it would be better, I think, to address emails to > >>> the > >>>>> list > >>>>>>>> at > >>>>>>>>>>>>> large. > >>>>>>>>>>>>>> It encourages more people to participate. If Roman is > >> a > >>>>>>>> subscriber > >>>>>>>>>> he > >>>>>>>>>>>>> would > >>>>>>>>>>>>>> get a copy anyway. If not, you could encourage him to > >>>>> subscribe. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Fri, Oct 11, 2019 at 3:13 PM leerho < > >>> [email protected]> > >>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Roman, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I hope you are doing well. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I really appreciate the contributions you made to > >> our > >>>>> Memory > >>>>>>>>>>>>> component! > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Given that Druid has made its code base compatible > >>> with > >>>>>>>> OpenJDK > >>>>>>>>>> 11, > >>>>>>>>>>> we > >>>>>>>>>>>>>>> could use your help in what changes do we need to > >> make > >>>> to > >>>>> make > >>>>>>>>>> that > >>>>>>>>>>>>>> happen > >>>>>>>>>>>>>>> for DataSketches. As I recall, our DataSketches > >>> library > >>>>> was > >>>>>>>> not > >>>>>>>>>> the > >>>>>>>>>>>>> only > >>>>>>>>>>>>>>> Druid dependency that took advantage of Unsafe. So > >> I > >>>>> assume > >>>>>>>> you > >>>>>>>>>>> have > >>>>>>>>>>>>>> been > >>>>>>>>>>>>>>> down this path :) > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Currently, I have a check in the static initializer > >> in > >>>>>>>>>>>>> Memory/UnsafeUtil: > >>>>>>>>>>>>>>> parseJavaVersion(...) that checks the string > >> returned > >>>> from > >>>>>>>>>>>>>>> System.getProperty("java.version"). If that string > >>> does > >>>>> not > >>>>>>>>>> contain > >>>>>>>>>>>>>> "1.8" > >>>>>>>>>>>>>>> or "8" it will throw an error. Given that we have > >> not > >>>>> tested > >>>>>>>>>> with > >>>>>>>>>>>>> JDK 9, > >>>>>>>>>>>>>>> 10, 11 or 12, I felt it was safer to explicitly > >> throw > >>>>> rather > >>>>>>>> than > >>>>>>>>>>>>> having > >>>>>>>>>>>>>>> the users experience some weird failure later that > >> may > >>>> be > >>>>>>>>>> difficult > >>>>>>>>>>> to > >>>>>>>>>>>>>>> debug. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> With only one minor exception (which we could easily > >>>> fix) > >>>>> the > >>>>>>>>>> Memory > >>>>>>>>>>>>>>> component is the only place where Unsafe is used. > >>>>> However, > >>>>>>>> we do > >>>>>>>>>>> use > >>>>>>>>>>>>>>> reflection to gain internal access to a number of > >>> other > >>>>>>>> classes > >>>>>>>>>>>>> including > >>>>>>>>>>>>>>> ByteBuffer, Cleaner, and Bits. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Looking forward to hearing from you. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Cheers, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Lee. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>> > >>>>> --------------------------------------------------------------------- > >>>>> To unsubscribe, e-mail: [email protected] > >>>>> For additional commands, e-mail: [email protected] > >>>>> > >>>>> > >>>> > >>> > >> > > -- > > From my cell phone. > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
