I guess not. I guess the "DATASKETCHES-XXX" tickets are against our own team?
I moved the "slack" ticket to an INFRA-19303. On Fri, Oct 18, 2019 at 5:48 PM leerho <[email protected]> wrote: > I thought that is what I did. Isn't the DATASKETCHES-3 ticket I opened > the same thing? > > > On Fri, Oct 18, 2019 at 5:42 PM Dave Fisher <[email protected]> wrote: > >> Hi Lee, >> >> DATASKETCHES Jira issues are for the project. >> >> If we want Infrastructure to do something of service to the community >> then you need to open an INFRA ticket. Got it? >> >> Regards? >> Dave >> >> Sent from my iPhone >> >> > On Oct 18, 2019, at 3:53 PM, leerho <[email protected]> wrote: >> > >> > I opened a ticket (DATASKETCHES-3) with infra to see if we could add >> this >> > capability to our slack channel. >> > >> >> On Fri, Oct 18, 2019 at 12:29 PM Dave Fisher <[email protected]> wrote: >> >> >> >> Hi - >> >> >> >> Apache Pulsar uses this: >> https://github.com/merlimat/slack-email-digest >> >> >> >> They don’t have their slack channels at the-asf.slack.com but it does >> >> work. You can always ask for infra’s help. >> >> >> >> Regards, >> >> Dave >> >> >> >>> On Oct 18, 2019, at 12:26 PM, leerho <[email protected]> wrote: >> >>> >> >>> I wonder if there is a way to have slack auto-post to an email list? >> >>> >> >>>> On Fri, Oct 18, 2019 at 5:13 AM Evans Ye <[email protected]> wrote: >> >>>> >> >>>> I don't have the expertise on jdk 11 but would love know more about >> >>>> datasketches and specifically the java lib. Would be great if you can >> >> bring >> >>>> back the result of what you have discussed in the slack dev channel. >> >> This >> >>>> helps other contributors easily to search and get to know what >> happened >> >> in >> >>>> the community and get involved :) >> >>>> >> >>>> Best >> >>>> Evans >> >>>> >> >>>>> leerho <[email protected]> 於 2019年10月18日 週五 11:03 寫道： >> >>>>> >> >>>>> Christopher, >> >>>>> >> >>>>> This would be fabulous. I have tons of questions, whenever you are >> >>>> ready. >> >>>>> I might be easier to interact via the ASF slack channel >> "datasketches" >> >> or >> >>>>> "datasketches-dev". >> >>>>> >> >>>>> Lee. >> >>>>> >> >>>>> On Thu, Oct 17, 2019 at 5:07 PM Jon Malkin <[email protected]> >> >> wrote: >> >>>>> >> >>>>>> The memory repository is the one that needs the most conversion >> help. >> >>>>>> Everything using Unsafe in datasketches-java runs through the >> memory >> >>>>>> package. >> >>>>>> >> >>>>>> I'm not sure off-hand if the characterization repo has low level >> tests >> >>>> of >> >>>>>> Memory (on my phone at the moment), but we'd be looking to minimize >> >> the >> >>>>>> speed impact in the process since it very directly impacts sketch >> >>>> speed. >> >>>>>> >> >>>>>> jon >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> On Thu, Oct 17, 2019, 4:43 PM Christopher <[email protected]> >> >> wrote: >> >>>>>> >> >>>>>>> I've converted several other projects to using JDK 11, and can >> help >> >>>>>>> with this if there's still a need for help. I've actually been >> >>>> looking >> >>>>>>> at ways I can contribute to DataSketches since ApacheCon, and this >> >>>>>>> seems to be something I can probably do. >> >>>>>>> >> >>>>>>>> On Wed, Oct 16, 2019 at 9:47 PM leerho <[email protected]> wrote: >> >>>>>>>> >> >>>>>>>> Gian, >> >>>>>>>> >> >>>>>>>> If I were to supply you with a link to a snapshot or a Release >> >>>>>> Candidate >> >>>>>>>> version of the datasketches jar (and Memory jar) could you test >> in >> >>>>> your >> >>>>>>>> environment under JDK 11 and see if it works? >> >>>>>>>> >> >>>>>>>> Lee >> >>>>>>>> >> >>>>>>>> >> >>>>>>>>> On Mon, Oct 14, 2019 at 9:19 PM leerho <[email protected]> >> wrote: >> >>>>>>>>> >> >>>>>>>>> Holy Cow! I have never seen such a complex travis config! >> >>>>>>>>> >> >>>>>>>>> On Mon, Oct 14, 2019 at 6:54 PM Gian Merlino <[email protected]> >> >>>>>> wrote: >> >>>>>>>>> >> >>>>>>>>>> I should have linked Druid's Travis config file for reference. >> >>>>> Here >> >>>>>>> it is: >> >>>>>>>>>> >> >>>> https://github.com/apache/incubator-druid/blob/master/.travis.yml >> >>>>>>>>>> >> >>>>>>>>>> On Mon, Oct 14, 2019 at 6:53 PM Gian Merlino <[email protected]> >> >>>>>> wrote: >> >>>>>>>>>> >> >>>>>>>>>>>> So what exactly is the requirement? Our DataSketches code >> >>>>> does >> >>>>>>>>>> compile >> >>>>>>>>>>>> under JDK8 and we have lots of customers and users using JDK >> >>>>> 8. >> >>>>>>>>>>> >> >>>>>>>>>>> It depends on who you ask, I guess :) >> >>>>>>>>>>> >> >>>>>>>>>>> From Druid's perspective, we aren't going to be compiling your >> >>>>>> code, >> >>>>>>>>>> only >> >>>>>>>>>>> running it. So the only requirement we would ask for is that >> >>>> the >> >>>>>>> code >> >>>>>>>>>>> _runs_ under a Java 11 runtime. Being able to compile under >> >>>> Java >> >>>>>> 11 >> >>>>>>> is >> >>>>>>>>>> good >> >>>>>>>>>>> for future-proofing your code, though. It is starting to >> >>>> become >> >>>>>> more >> >>>>>>>>>>> prevalent. >> >>>>>>>>>>> >> >>>>>>>>>>>> It sounds like (as far as Unsafe is concerned) converting >> >>>> all >> >>>>>> the >> >>>>>>>>>> direct >> >>>>>>>>>>>> unsafe.blah() calls to static MethodHandle.invoke calls is >> >>>>> what >> >>>>>> is >> >>>>>>>>>>> required. >> >>>>>>>>>>>> >> >>>>>>>>>>>> Is this correct? >> >>>>>>>>>>> >> >>>>>>>>>>> That is certainly part of it, I'm not 100% sure if that's all >> >>>>>> that's >> >>>>>>>>>>> needed. Btw, CI can help verify. In Druid's Travis config, we >> >>>>> run >> >>>>>>> our >> >>>>>>>>>> tests >> >>>>>>>>>>> on both JDK 8 and JDK 11. In order to make this automated >> >>>>> testing >> >>>>>>>>>> easier, >> >>>>>>>>>>> we chose to do the work needed to compile Druid on JDK 11, >> >>>> even >> >>>>>>> though >> >>>>>>>>>> we >> >>>>>>>>>>> still compile release builds using JDK 8 only. (If we hadn't >> >>>>> done >> >>>>>>> this >> >>>>>>>>>>> work, then we'd have to compile on 8 and test on 11, which >> >>>>> Travis >> >>>>>>>>>> doesn't >> >>>>>>>>>>> easily support.) >> >>>>>>>>>>> >> >>>>>>>>>>> On Mon, Oct 14, 2019 at 5:51 PM leerho <[email protected]> >> >>>>> wrote: >> >>>>>>>>>>> >> >>>>>>>>>>>> Thanks, Gian, this is VERY helpful! >> >>>>>>>>>>>> >> >>>>>>>>>>>> From my cursory glance, it appears that UnsafeUtil acquires >> >>>>>> Unsafe >> >>>>>>> and >> >>>>>>>>>>>> Unsafe-class the usual way using reflection (I didn't think >> >>>>> this >> >>>>>>> would >> >>>>>>>>>>>> work >> >>>>>>>>>>>> with 9+ ), and then defines some method-handles that the >> >>>> other >> >>>>>>> classes >> >>>>>>>>>>>> use. >> >>>>>>>>>>>> >> >>>>>>>>>>>> Will this approach allow code to compile under JDK11 and >> >>>> JDK8? >> >>>>>>>>>>>> >> >>>>>>>>>>>> It also seems that compiling with JDK11 is still a work in >> >>>>>>> progress as >> >>>>>>>>>> the >> >>>>>>>>>>>> pom.xml clearly specifies JDK8. >> >>>>>>>>>>>> >> >>>>>>>>>>>> So what exactly is the requirement? Our DataSketches code >> >>>> does >> >>>>>>> compile >> >>>>>>>>>>>> under JDK8 and we have lots of customers and users using JDK >> >>>> 8. >> >>>>>>>>>>>> >> >>>>>>>>>>>> It sounds like (as far as Unsafe is concerned) converting all >> >>>>> the >> >>>>>>>>>> direct >> >>>>>>>>>>>> unsafe.blah() calls to static MethodHandle.invoke calls is >> >>>> what >> >>>>>> is >> >>>>>>>>>>>> required. >> >>>>>>>>>>>> >> >>>>>>>>>>>> Is this correct? >> >>>>>>>>>>>> >> >>>>>>>>>>>> Lee. >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> On Mon, Oct 14, 2019 at 2:07 PM Gian Merlino < >> >>>> [email protected]> >> >>>>>>> wrote: >> >>>>>>>>>>>> >> >>>>>>>>>>>>> And nobody should be using the Oracle JDK without a >> >>>>> commercial >> >>>>>>>>>>>> relationship >> >>>>>>>>>>>>> with Oracle, its license is a nightmare. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> On Mon, Oct 14, 2019 at 2:04 PM Gian Merlino < >> >>>>> [email protected]> >> >>>>>>>>>> wrote: >> >>>>>>>>>>>>> >> >>>>>>>>>>>>>> Oracle and OpenJDK builds behave the same as each other >> >>>>> now. >> >>>>>>> The >> >>>>>>>>>>>>>> differences are in terms of licensing, length of support, >> >>>>> and >> >>>>>>>>>> release >> >>>>>>>>>>>>>> cadence. IMO most Java devs should be using one of the >> >>>>>>> third-party >> >>>>>>>>>>>>> OpenJDK >> >>>>>>>>>>>>>> distributions (e.g. Corretto, Zulu, AdoptOpenJDK) rather >> >>>>> than >> >>>>>>>>>> vanilla >> >>>>>>>>>>>>>> OpenJDK these days, because of the new OpenJDK policy >> >>>> that >> >>>>>>> public >> >>>>>>>>>>>> updates >> >>>>>>>>>>>>>> will cease for major versions after 6 months. >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> On Mon, Oct 14, 2019 at 11:07 AM leerho < >> >>>> [email protected]> >> >>>>>>> wrote: >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Thanks, Gian, >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Your references are helpful and I am studying them. >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> I didn't see any references to OpenJDK vs Oracle's JDKs. >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Are there any differences in the way Unsafe calls (or >> >>>>> other >> >>>>>>> hidden >> >>>>>>>>>>>>>>> classes) >> >>>>>>>>>>>>>>> are handled by OpenJDK? Or is this a "don't care" as >> >>>> far >> >>>>> as >> >>>>>>>>>> Druid is >> >>>>>>>>>>>>>>> concerned? Are the version numbers between Oracle and >> >>>>>> OpenJDK >> >>>>>>>>>> always >> >>>>>>>>>>>>>>> aligned? >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Lee. >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> On Fri, Oct 11, 2019 at 5:45 PM Gian Merlino < >> >>>>>> [email protected] >> >>>>>>>> >> >>>>>>>>>>>> wrote: >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> You didn't address this message to me, but in Druid >> >>>> most >> >>>>>> of >> >>>>>>> the >> >>>>>>>>>>>> work >> >>>>>>>>>>>>> for >> >>>>>>>>>>>>>>>> Java 9+ compatibility is mentioned in this master >> >>>> issue, >> >>>>>>> which >> >>>>>>>>>> you >> >>>>>>>>>>>>> might >> >>>>>>>>>>>>>>>> find helpful: >> >>>>>>>>>>>> https://github.com/apache/incubator-druid/issues/5589. >> >>>>>>>>>>>>>>>> Skimming the list, these might be particularly >> >>>> relevant >> >>>>> to >> >>>>>>>>>>>>> DataSketches: >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7487 >> >>>>>>> (cleaner >> >>>>>>>>>>>>>>> operations) >> >>>>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7466 >> >>>>>>>>>> (ByteBuffer >> >>>>>>>>>>>>> unmap >> >>>>>>>>>>>>>>>> operation) >> >>>>>>>>>>>>>>>> - https://github.com/apache/incubator-druid/pull/7576 >> >>>>>>> (Remove >> >>>>>>>>>>>> direct >> >>>>>>>>>>>>>>>> references to Unsafe) >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> Btw, it would be better, I think, to address emails to >> >>>>> the >> >>>>>>> list >> >>>>>>>>>> at >> >>>>>>>>>>>>>>> large. >> >>>>>>>>>>>>>>>> It encourages more people to participate. If Roman is >> >>>> a >> >>>>>>>>>> subscriber >> >>>>>>>>>>>> he >> >>>>>>>>>>>>>>> would >> >>>>>>>>>>>>>>>> get a copy anyway. If not, you could encourage him to >> >>>>>>> subscribe. >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> On Fri, Oct 11, 2019 at 3:13 PM leerho < >> >>>>> [email protected]> >> >>>>>>>>>> wrote: >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> Roman, >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> I hope you are doing well. >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> I really appreciate the contributions you made to >> >>>> our >> >>>>>>> Memory >> >>>>>>>>>>>>>>> component! >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> Given that Druid has made its code base compatible >> >>>>> with >> >>>>>>>>>> OpenJDK >> >>>>>>>>>>>> 11, >> >>>>>>>>>>>>> we >> >>>>>>>>>>>>>>>>> could use your help in what changes do we need to >> >>>> make >> >>>>>> to >> >>>>>>> make >> >>>>>>>>>>>> that >> >>>>>>>>>>>>>>>> happen >> >>>>>>>>>>>>>>>>> for DataSketches. As I recall, our DataSketches >> >>>>> library >> >>>>>>> was >> >>>>>>>>>> not >> >>>>>>>>>>>> the >> >>>>>>>>>>>>>>> only >> >>>>>>>>>>>>>>>>> Druid dependency that took advantage of Unsafe. So >> >>>> I >> >>>>>>> assume >> >>>>>>>>>> you >> >>>>>>>>>>>>> have >> >>>>>>>>>>>>>>>> been >> >>>>>>>>>>>>>>>>> down this path :) >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> Currently, I have a check in the static initializer >> >>>> in >> >>>>>>>>>>>>>>> Memory/UnsafeUtil: >> >>>>>>>>>>>>>>>>> parseJavaVersion(...) that checks the string >> >>>> returned >> >>>>>> from >> >>>>>>>>>>>>>>>>> System.getProperty("java.version"). If that string >> >>>>> does >> >>>>>>> not >> >>>>>>>>>>>> contain >> >>>>>>>>>>>>>>>> "1.8" >> >>>>>>>>>>>>>>>>> or "8" it will throw an error. Given that we have >> >>>> not >> >>>>>>> tested >> >>>>>>>>>>>> with >> >>>>>>>>>>>>>>> JDK 9, >> >>>>>>>>>>>>>>>>> 10, 11 or 12, I felt it was safer to explicitly >> >>>> throw >> >>>>>>> rather >> >>>>>>>>>> than >> >>>>>>>>>>>>>>> having >> >>>>>>>>>>>>>>>>> the users experience some weird failure later that >> >>>> may >> >>>>>> be >> >>>>>>>>>>>> difficult >> >>>>>>>>>>>>> to >> >>>>>>>>>>>>>>>>> debug. >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> With only one minor exception (which we could easily >> >>>>>> fix) >> >>>>>>> the >> >>>>>>>>>>>> Memory >> >>>>>>>>>>>>>>>>> component is the only place where Unsafe is used. >> >>>>>>> However, >> >>>>>>>>>> we do >> >>>>>>>>>>>>> use >> >>>>>>>>>>>>>>>>> reflection to gain internal access to a number of >> >>>>> other >> >>>>>>>>>> classes >> >>>>>>>>>>>>>>> including >> >>>>>>>>>>>>>>>>> ByteBuffer, Cleaner, and Bits. >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> Looking forward to hearing from you. >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> Cheers, >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> Lee. >> >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>> >> >>>>>>> >> >>>>>>> >> --------------------------------------------------------------------- >> >>>>>>> To unsubscribe, e-mail: [email protected] >> >>>>>>> For additional commands, e-mail: [email protected] >> >>>>>>> >> >>>>>>> >> >>>>>> >> >>>>> >> >>>> >> >>> -- >> >>> From my cell phone. >> >> >> >> >> >> --------------------------------------------------------------------- >> >> To unsubscribe, e-mail: [email protected] >> >> For additional commands, e-mail: [email protected] >> >> >> >> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >>
