Today's dev sync
https://hangouts.google.com/hangouts/_/calendar/Z2lhbm1lcmxpbm9AZ21haWwuY29t.73eguu7d0a9gr27kt1ugitllq3
Dev list migration
Hi all, In the dev sync today there was some general agreement around migrating the dev list to Apache next week. Please +1 or -1 in this thread as desired. I think ideally we'd want an autoresponder but I don't see a way to set one. So the concrete plan I'd propose instead is to do the following next Monday, April 23, 1) Post a message to the list that we have migrated to dev@druid.apache.org 2) Sticky that post on https://groups.google.com/forum/#!forum/druid-development 3) Update the list link on the web site at http://druid.io/community/ 4) Disable posting on druid-developm...@googlegroups.com This message has been cross posted to both lists, but please reply on the Apache list. If you haven't signed up for it yet, you can do that by emailing dev-subscr...@druid.apache.org.
Re: [druid-dev] consider doing a 0.12.1 release
It looks like https://github.com/druid-io/druid/pull/5654 is the last to backport. Shall we do a release candidate after that? This will probably end up being a non-Apache release when all is said and done, since the timelines will probably not line up - we want to get the bug fixes out quickly, but we are not yet ready to do Apache releases from either an IP or technical standpoint. On Mon, Apr 9, 2018 at 9:07 PM, Gian Merlino wrote: > I backported 5504 and 5587. I skipped 5374 since it turns out it isn't > relevant to 0.12.0 -- it fixes something introduced since then -- so I > moved that back to 0.13.0. > > On Mon, Apr 9, 2018 at 1:56 PM, Gian Merlino wrote: > >> I raised backport PRs for everything on my list except for these three, >> since they had some (minor) conflicts and I was using a backport script >> that bails out if it sees any conflicts: >> >> - https://github.com/druid-io/druid/pull/5504 >> - https://github.com/druid-io/druid/pull/5587 >> - https://github.com/druid-io/druid/pull/5374 >> >> I will take a closer look at them later if nobody else gets to them. >> >> Nishant, I added https://github.com/druid-io/druid/pull/5596 to the >> milestone. >> >> Prashant, I added https://github.com/druid-io/druid/pull/5611 to the >> milestone. >> >> On Mon, Apr 9, 2018 at 1:10 PM, Prashant Deva >> wrote: >> >>> also for the list: >>> >>> - compact task throws exception #5611 >>> >>> i dont have permission to add to gian's github milestone >>> >>> Prashant >>> >>> On Mon, Apr 9, 2018 at 12:26 PM, Gian Merlino >>> wrote: >>> >>> > Here we go: https://github.com/druid-io/druid/milestone/26. Please add >>> > stuff if it makes sense (high importance / low risk bug fixes). >>> > >>> > On Mon, Apr 9, 2018 at 12:14 PM, Gian Merlino >>> > wrote: >>> > >>> > > It looks like there's a lot of support for doing a 0.12.1… I'll make >>> a >>> > > milestone in GH. >>> > > >>> > > On Mon, Apr 9, 2018 at 12:12 PM, Slim Bouguerra >>> > wrote: >>> > > >>> > >> +1 especially there is some instability bugs. >>> > >> >>> > >> On Apr 9, 2018, at 11:57 AM, Gian Merlino wrote: >>> > >> >>> > >> My feeling is that #3 and #2 are borderline, but #1 definitely >>> warrants >>> > a >>> > >> new release. Personally I have seen it occur at least a half dozen >>> > times, >>> > >> and I had been thinking about proposing a Druid 0.12.1 release, so >>> I'm >>> > >> glad >>> > >> you brought it up. >>> > >> >>> > >> If we do 0.12.1, it would be another non-ASF release (we haven't >>> got the >>> > >> ASF process set up yet, and are not likely to have it set up in >>> time) so >>> > >> we >>> > >> should notify the incubator folks about it. >>> > >> >>> > >> I would also consider including these fixes in 0.12.1: >>> > >> >>> > >> - DoublesSketchModule: Fix serde for DoublesSketchMergeAggregatorFa >>> > ctory. >>> > >> (#5587) >>> > >> - ArrayAggregation: Use long to avoid overflow (#5544) >>> > >> - Respect forceHashAggregation in queryContext (#5533) >>> > >> - Fix indexTask to respect forceExtendableShardSpecs (#5509) >>> > >> - Add overlord unsecured paths to coordinator when using combined >>> > service >>> > >> (#5579) >>> > >> - Fix SQLMetadataSegmentManager to allow succesive start and stop >>> > (#5554) >>> > >> - Fix supervisor tombstone auth handling (#5504) >>> > >> - Authorize supervisor history instead of current active >>> supervisors for >>> > >> supervisor history API (#5501) >>> > >> - Fix round robining in router. (#5500) >>> > >> - SegmentMetadataQuery: Fix default interval handling. (#5489) >>> > >> - Log exceptions thrown before persist() for indexing tasks (#5374) >>> > >> - More memory limiting for HttpPostEmitter (#5300) >>> > >> - pass configuration from context into JobConf for determining >>> > >> DatasourceInputFormat splits (#5408) >>> > >> - Lookups: Inherit "injective" from registered lookups, improve >>> docs. >>> > >> (#5316) >>> > >> - SQL: Throttle metadata refreshes when they fail. (#5328) >>> > >> >>> > >> On Mon, Apr 9, 2018 at 10:35 AM, Gian Merlino >>> wrote: >>> > >> >>> > >> I think this conversation is worth having. I have cross posted this >>> to >>> > >> dev@druid.apache.org and will reply there. Since we're trying to >>> > migrate >>> > >> the dev list, please cross post any dev messages there, or even only >>> > post >>> > >> to that list. >>> > >> >>> > >> Gian >>> > >> >>> > >> On Sun, Apr 8, 2018 at 5:45 PM, Prashant Deva < >>> prashant.d...@gmail.com> >>> > >> wrote: >>> > >> >>> > >> Current 0.12.0 release has some major issues: >>> > >> >>> > >> >>> > >> 1. Coordinator loses leadership >>> > >> https://github.com/druid-io/druid/issues/5561 >>> > >> >>> > >> 2. Newly introduced Quantiles sketch is broken >>> > >> https://github.com/druid-io/druid/issues/5575 >>> > >> >>> > >> 3. Coordinator+overlord web console broken >>> > >> https://github.com/druid-io/druid/issues/5559 >>> > >> >>> > >> >>> > >> 1. is especially very important. Without a coordinator, druid stops
Re: Dev list migration
+1 On Tue, Apr 17, 2018 at 11:49 AM, Gian Merlino wrote: > Hi all, > > In the dev sync today there was some general agreement around migrating the > dev list to Apache next week. Please +1 or -1 in this thread as desired. > > I think ideally we'd want an autoresponder but I don't see a way to set > one. So the concrete plan I'd propose instead is to do the following next > Monday, April 23, > > 1) Post a message to the list that we have migrated to > dev@druid.apache.org > 2) Sticky that post on > https://groups.google.com/forum/#!forum/druid-development > 3) Update the list link on the web site at http://druid.io/community/ > 4) Disable posting on druid-developm...@googlegroups.com > > This message has been cross posted to both lists, but please reply on the > Apache list. If you haven't signed up for it yet, you can do that by > emailing dev-subscr...@druid.apache.org. >
Re: Dev list migration
+1 On 2018/04/17 18:09:37, David Lim wrote: > +1 > > On Tue, Apr 17, 2018 at 11:49 AM, Gian Merlino wrote: > > > Hi all, > > > > In the dev sync today there was some general agreement around migrating the > > dev list to Apache next week. Please +1 or -1 in this thread as desired. > > > > I think ideally we'd want an autoresponder but I don't see a way to set > > one. So the concrete plan I'd propose instead is to do the following next > > Monday, April 23, > > > > 1) Post a message to the list that we have migrated to > > dev@druid.apache.org > > 2) Sticky that post on > > https://groups.google.com/forum/#!forum/druid-development > > 3) Update the list link on the web site at http://druid.io/community/ > > 4) Disable posting on druid-developm...@googlegroups.com > > > > This message has been cross posted to both lists, but please reply on the > > Apache list. If you haven't signed up for it yet, you can do that by > > emailing dev-subscr...@druid.apache.org. > > > - To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org For additional commands, e-mail: dev-h...@druid.apache.org
Re: Dev list migration
+1 -- Nishant Bangarwa Hortonworks (M): +91-9729200044 On 4/17/18, 10:19 PM, "Jonathan Wei" wrote: >+1 > >On 2018/04/17 18:09:37, David Lim wrote: >> +1 >> >> On Tue, Apr 17, 2018 at 11:49 AM, Gian Merlino wrote: >> >> > Hi all, >> > >> > In the dev sync today there was some general agreement around migrating the >> > dev list to Apache next week. Please +1 or -1 in this thread as desired. >> > >> > I think ideally we'd want an autoresponder but I don't see a way to set >> > one. So the concrete plan I'd propose instead is to do the following next >> > Monday, April 23, >> > >> > 1) Post a message to the list that we have migrated to >> > dev@druid.apache.org >> > 2) Sticky that post on >> > https://groups.google.com/forum/#!forum/druid-development >> > 3) Update the list link on the web site at http://druid.io/community/ >> > 4) Disable posting on druid-developm...@googlegroups.com >> > >> > This message has been cross posted to both lists, but please reply on the >> > Apache list. If you haven't signed up for it yet, you can do that by >> > emailing dev-subscr...@druid.apache.org. >> > >> > >- >To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org >For additional commands, e-mail: dev-h...@druid.apache.org >
Re: Dev list migration
+1 on moving to official incubator dev list as proposed On Tue, Apr 17, 2018 at 1:20 PM Nishant Bangarwa wrote: > +1 > -- > Nishant Bangarwa > Hortonworks > (M): +91-9729200044 <+91%2097292%244> > > > > > > > > On 4/17/18, 10:19 PM, "Jonathan Wei" wrote: > > >+1 > > > >On 2018/04/17 18:09:37, David Lim wrote: > >> +1 > >> > >> On Tue, Apr 17, 2018 at 11:49 AM, Gian Merlino wrote: > >> > >> > Hi all, > >> > > >> > In the dev sync today there was some general agreement around > migrating the > >> > dev list to Apache next week. Please +1 or -1 in this thread as > desired. > >> > > >> > I think ideally we'd want an autoresponder but I don't see a way to > set > >> > one. So the concrete plan I'd propose instead is to do the following > next > >> > Monday, April 23, > >> > > >> > 1) Post a message to the list that we have migrated to > >> > dev@druid.apache.org > >> > 2) Sticky that post on > >> > https://groups.google.com/forum/#!forum/druid-development > >> > 3) Update the list link on the web site at > https://urldefense.proofpoint.com/v2/url?u=http-3A__druid.io_community_&d=DwIGaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=CGUSAtro7RcpRonWXvEi1TpH3WKiK-X4I_grZmhiXz8&s=KaTfGZOG8yjZ29zRcA2uSfqam2J43XaA4RoNrOOldFU&e= > >> > 4) Disable posting on druid-developm...@googlegroups.com > >> > > >> > This message has been cross posted to both lists, but please reply on > the > >> > Apache list. If you haven't signed up for it yet, you can do that by > >> > emailing dev-subscr...@druid.apache.org. > >> > > >> > > > >- > >To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org > >For additional commands, e-mail: dev-h...@druid.apache.org > > >
Re: Dev list migration
+1 > On Apr 17, 2018, at 10:49 AM, Gian Merlino wrote: > > Hi all, > > In the dev sync today there was some general agreement around migrating the > dev list to Apache next week. Please +1 or -1 in this thread as desired. > > I think ideally we'd want an autoresponder but I don't see a way to set > one. So the concrete plan I'd propose instead is to do the following next > Monday, April 23, > > 1) Post a message to the list that we have migrated to dev@druid.apache.org > 2) Sticky that post on > https://groups.google.com/forum/#!forum/druid-development > 3) Update the list link on the web site at http://druid.io/community/ > 4) Disable posting on druid-developm...@googlegroups.com > > This message has been cross posted to both lists, but please reply on the > Apache list. If you haven't signed up for it yet, you can do that by > emailing dev-subscr...@druid.apache.org. - To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org For additional commands, e-mail: dev-h...@druid.apache.org
Re: [druid-dev] Re: Dev list migration
+1 2018년 4월 17일 (화) 오후 4:11, Slim Bouguerra 님이 작성: > +1 > > > On Apr 17, 2018, at 10:49 AM, Gian Merlino wrote: > > > > Hi all, > > > > In the dev sync today there was some general agreement around migrating > the > > dev list to Apache next week. Please +1 or -1 in this thread as desired. > > > > I think ideally we'd want an autoresponder but I don't see a way to set > > one. So the concrete plan I'd propose instead is to do the following next > > Monday, April 23, > > > > 1) Post a message to the list that we have migrated to > dev@druid.apache.org > > 2) Sticky that post on > > https://groups.google.com/forum/#!forum/druid-development > > 3) Update the list link on the web site at http://druid.io/community/ > > 4) Disable posting on druid-developm...@googlegroups.com > > > > This message has been cross posted to both lists, but please reply on the > > Apache list. If you haven't signed up for it yet, you can do that by > > emailing dev-subscr...@druid.apache.org. > > -- > You received this message because you are subscribed to the Google Groups > "Druid Development" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to druid-development+unsubscr...@googlegroups.com. > To post to this group, send email to druid-developm...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/druid-development/0726D556-DE55-433B-AF7E-E47FD4637CA1%40gmail.com > . > For more options, visit https://groups.google.com/d/optout. >
Tranquility Future
Hi Devs, am not sure if i have missed the discussion but am wondering what is the Future of Tranquility after moving out Druid of Druid-io? Will tranquility stay under https://github.com/druid-io/tranquility/ ? in the future is this Repo https://github.com/druid-io/tranquility/ manageable by the Druid community or not anymore? Any thought about the future of Tranquility ? Thanks !
Re: [druid-dev] Re: Dev list migration
BTW, probably we need to use Apache email address instead of personal ones to make individual's vote effective. 2018년 4월 17일 (화) 오후 5:52, Jihoon Son 님이 작성: > +1 > > 2018년 4월 17일 (화) 오후 4:11, Slim Bouguerra 님이 작성: > >> +1 >> >> > On Apr 17, 2018, at 10:49 AM, Gian Merlino wrote: >> > >> > Hi all, >> > >> > In the dev sync today there was some general agreement around migrating >> the >> > dev list to Apache next week. Please +1 or -1 in this thread as desired. >> > >> > I think ideally we'd want an autoresponder but I don't see a way to set >> > one. So the concrete plan I'd propose instead is to do the following >> next >> > Monday, April 23, >> > >> > 1) Post a message to the list that we have migrated to >> dev@druid.apache.org >> > 2) Sticky that post on >> > https://groups.google.com/forum/#!forum/druid-development >> > 3) Update the list link on the web site at http://druid.io/community/ >> > 4) Disable posting on druid-developm...@googlegroups.com >> > >> > This message has been cross posted to both lists, but please reply on >> the >> > Apache list. If you haven't signed up for it yet, you can do that by >> > emailing dev-subscr...@druid.apache.org. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Druid Development" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to druid-development+unsubscr...@googlegroups.com. >> To post to this group, send email to druid-developm...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/druid-development/0726D556-DE55-433B-AF7E-E47FD4637CA1%40gmail.com >> . >> For more options, visit https://groups.google.com/d/optout. >> >
Re: Tranquility Future
Is anyone interested in doing the work to migrate it to Apache? If so, I think we should definitely do that. In general I have been more interested in contributing to the Kafka indexing service lately since it has nicer properties (exactly once, no extra processes, can read late data). But Tranquility is still valuable from a user perspective when best-effort is ok and when you don't have a Kafka server. So I hesitate to get rid of it completely. But of course - to actually migrate it to Apache, someone will have to step up and do the work. On Tue, Apr 17, 2018 at 5:56 PM, Slim Bouguerra wrote: > Hi Devs, > am not sure if i have missed the discussion but am wondering what is the > Future of Tranquility after moving out Druid of Druid-io? > Will tranquility stay under https://github.com/druid-io/tranquility/ ? > in the future is this Repo https://github.com/druid-io/tranquility/ > manageable by the Druid community or not anymore? > Any thought about the future of Tranquility ? > > Thanks ! >
Re: Dev list migration
+1 On Tue, Apr 17, 2018 at 3:20 PM, Nishant Bangarwa wrote: > +1 > -- > Nishant Bangarwa > Hortonworks > (M): +91-9729200044 > > > > > > > > On 4/17/18, 10:19 PM, "Jonathan Wei" wrote: > > >+1 > > > >On 2018/04/17 18:09:37, David Lim wrote: > >> +1 > >> > >> On Tue, Apr 17, 2018 at 11:49 AM, Gian Merlino wrote: > >> > >> > Hi all, > >> > > >> > In the dev sync today there was some general agreement around > migrating the > >> > dev list to Apache next week. Please +1 or -1 in this thread as > desired. > >> > > >> > I think ideally we'd want an autoresponder but I don't see a way to > set > >> > one. So the concrete plan I'd propose instead is to do the following > next > >> > Monday, April 23, > >> > > >> > 1) Post a message to the list that we have migrated to > >> > dev@druid.apache.org > >> > 2) Sticky that post on > >> > https://groups.google.com/forum/#!forum/druid-development > >> > 3) Update the list link on the web site at http://druid.io/community/ > >> > 4) Disable posting on druid-developm...@googlegroups.com > >> > > >> > This message has been cross posted to both lists, but please reply on > the > >> > Apache list. If you haven't signed up for it yet, you can do that by > >> > emailing dev-subscr...@druid.apache.org. > >> > > >> > > > >- > >To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org > >For additional commands, e-mail: dev-h...@druid.apache.org > > >
Dependencies licenses Report
One of the question last dev synch was about the generation of dependency licenses. Some projects (ORC and Hive) use the maven site plugin that can generates reports with all the dependencies and licenses details. I have run it on Druid and this is how it looks for Druid Api Module. cmd mvn project-info-reports:dependencies The site directory can be found under target/site here is an example for one module https://drive.google.com/file/d/1P8R0kZjp8zP4WSOVrKdlJF7Xr8-OI7Oe/view?usp=sharing Also no fancy tools used to detect unwanted licenses, it is done while reviewing PR - To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org For additional commands, e-mail: dev-h...@druid.apache.org