Re: [druid-dev] consider doing a 0.12.1 release

[Gian Merlino]

I backported 5504 and 5587. I skipped 5374 since it turns out it isn't
relevant to 0.12.0 -- it fixes something introduced since then -- so I
moved that back to 0.13.0.

On Mon, Apr 9, 2018 at 1:56 PM, Gian Merlino  wrote:

> I raised backport PRs for everything on my list except for these three,
> since they had some (minor) conflicts and I was using a backport script
> that bails out if it sees any conflicts:
>
> - https://github.com/druid-io/druid/pull/5504
> - https://github.com/druid-io/druid/pull/5587
> - https://github.com/druid-io/druid/pull/5374
>
> I will take a closer look at them later if nobody else gets to them.
>
> Nishant, I added https://github.com/druid-io/druid/pull/5596 to the
> milestone.
>
> Prashant, I added https://github.com/druid-io/druid/pull/5611 to the
> milestone.
>
> On Mon, Apr 9, 2018 at 1:10 PM, Prashant Deva 
> wrote:
>
>> also for the list:
>>
>> - compact task throws exception #5611
>>
>> i dont have permission to add to gian's github milestone
>>
>> Prashant
>>
>> On Mon, Apr 9, 2018 at 12:26 PM, Gian Merlino 
>> wrote:
>>
>> > Here we go: https://github.com/druid-io/druid/milestone/26. Please add
>> > stuff if it makes sense (high importance / low risk bug fixes).
>> >
>> > On Mon, Apr 9, 2018 at 12:14 PM, Gian Merlino 
>> > wrote:
>> >
>> > > It looks like there's a lot of support for doing a 0.12.1… I'll make a
>> > > milestone in GH.
>> > >
>> > > On Mon, Apr 9, 2018 at 12:12 PM, Slim Bouguerra 
>> > wrote:
>> > >
>> > >> +1 especially there is some instability bugs.
>> > >>
>> > >> On Apr 9, 2018, at 11:57 AM, Gian Merlino  wrote:
>> > >>
>> > >> My feeling is that #3 and #2 are borderline, but #1 definitely
>> warrants
>> > a
>> > >> new release. Personally I have seen it occur at least a half dozen
>> > times,
>> > >> and I had been thinking about proposing a Druid 0.12.1 release, so
>> I'm
>> > >> glad
>> > >> you brought it up.
>> > >>
>> > >> If we do 0.12.1, it would be another non-ASF release (we haven't got
>> the
>> > >> ASF process set up yet, and are not likely to have it set up in
>> time) so
>> > >> we
>> > >> should notify the incubator folks about it.
>> > >>
>> > >> I would also consider including these fixes in 0.12.1:
>> > >>
>> > >> - DoublesSketchModule: Fix serde for DoublesSketchMergeAggregatorFa
>> > ctory.
>> > >> (#5587)
>> > >> - ArrayAggregation: Use long to avoid overflow (#5544)
>> > >> - Respect forceHashAggregation in queryContext (#5533)
>> > >> - Fix indexTask to respect forceExtendableShardSpecs (#5509)
>> > >> - Add overlord unsecured paths to coordinator when using combined
>> > service
>> > >> (#5579)
>> > >> - Fix SQLMetadataSegmentManager to allow succesive start and stop
>> > (#5554)
>> > >> - Fix supervisor tombstone auth handling (#5504)
>> > >> - Authorize supervisor history instead of current active supervisors
>> for
>> > >> supervisor history API (#5501)
>> > >> - Fix round robining in router. (#5500)
>> > >> - SegmentMetadataQuery: Fix default interval handling. (#5489)
>> > >> - Log exceptions thrown before persist() for indexing tasks (#5374)
>> > >> - More memory limiting for HttpPostEmitter (#5300)
>> > >> - pass configuration from context into JobConf for determining
>> > >> DatasourceInputFormat splits (#5408)
>> > >> - Lookups: Inherit "injective" from registered lookups, improve docs.
>> > >> (#5316)
>> > >> - SQL: Throttle metadata refreshes when they fail. (#5328)
>> > >>
>> > >> On Mon, Apr 9, 2018 at 10:35 AM, Gian Merlino  wrote:
>> > >>
>> > >> I think this conversation is worth having. I have cross posted this
>> to
>> > >> dev@druid.apache.org and will reply there. Since we're trying to
>> > migrate
>> > >> the dev list, please cross post any dev messages there, or even only
>> > post
>> > >> to that list.
>> > >>
>> > >> Gian
>> > >>
>> > >> On Sun, Apr 8, 2018 at 5:45 PM, Prashant Deva <
>> prashant.d...@gmail.com>
>> > >> wrote:
>> > >>
>> > >> Current 0.12.0 release has some major issues:
>> > >>
>> > >>
>> > >>   1. Coordinator loses leadership
>> > >>   https://github.com/druid-io/druid/issues/5561
>> > >>
>> > >>   2. Newly introduced Quantiles sketch is broken
>> > >>   https://github.com/druid-io/druid/issues/5575
>> > >>
>> > >>   3. Coordinator+overlord web console broken
>> > >>   https://github.com/druid-io/druid/issues/5559
>> > >>
>> > >>
>> > >> 1. is especially very important. Without a coordinator, druid stops
>> > >> functioning.
>> > >> With bug 5561, it is impossible to use druid for long periods since
>> > >> coordinator eventually does lose leadership and the whole process
>> needs
>> > to
>> > >> be restarted for it to come back.
>> > >>
>> > >> *Why not wait till 0.13.0?*
>> > >>
>> > >> A lot of companies like to update one version at a time and may not
>> want
>> > >> to jump directly to 0.13.0.
>> > >> Those companies will hit a bad 

Re: [druid-dev] consider doing a 0.12.1 release

[Prashant Deva]

for https://github.com/druid-io/druid/pull/5587
we deployed the changes in production manually for this and it works fine.

Prashant

On Mon, Apr 9, 2018 at 1:56 PM, Gian Merlino  wrote:

> I raised backport PRs for everything on my list except for these three,
> since they had some (minor) conflicts and I was using a backport script
> that bails out if it sees any conflicts:
>
> - https://github.com/druid-io/druid/pull/5504
> - https://github.com/druid-io/druid/pull/5587
> - https://github.com/druid-io/druid/pull/5374
>
> I will take a closer look at them later if nobody else gets to them.
>
> Nishant, I added https://github.com/druid-io/druid/pull/5596 to the
> milestone.
>
> Prashant, I added https://github.com/druid-io/druid/pull/5611 to the
> milestone.
>
> On Mon, Apr 9, 2018 at 1:10 PM, Prashant Deva 
> wrote:
>
> > also for the list:
> >
> > - compact task throws exception #5611
> >
> > i dont have permission to add to gian's github milestone
> >
> > Prashant
> >
> > On Mon, Apr 9, 2018 at 12:26 PM, Gian Merlino 
> > wrote:
> >
> > > Here we go: https://github.com/druid-io/druid/milestone/26. Please add
> > > stuff if it makes sense (high importance / low risk bug fixes).
> > >
> > > On Mon, Apr 9, 2018 at 12:14 PM, Gian Merlino 
> > > wrote:
> > >
> > > > It looks like there's a lot of support for doing a 0.12.1… I'll make
> a
> > > > milestone in GH.
> > > >
> > > > On Mon, Apr 9, 2018 at 12:12 PM, Slim Bouguerra 
> > > wrote:
> > > >
> > > >> +1 especially there is some instability bugs.
> > > >>
> > > >> On Apr 9, 2018, at 11:57 AM, Gian Merlino  wrote:
> > > >>
> > > >> My feeling is that #3 and #2 are borderline, but #1 definitely
> > warrants
> > > a
> > > >> new release. Personally I have seen it occur at least a half dozen
> > > times,
> > > >> and I had been thinking about proposing a Druid 0.12.1 release, so
> I'm
> > > >> glad
> > > >> you brought it up.
> > > >>
> > > >> If we do 0.12.1, it would be another non-ASF release (we haven't got
> > the
> > > >> ASF process set up yet, and are not likely to have it set up in
> time)
> > so
> > > >> we
> > > >> should notify the incubator folks about it.
> > > >>
> > > >> I would also consider including these fixes in 0.12.1:
> > > >>
> > > >> - DoublesSketchModule: Fix serde for DoublesSketchMergeAggregatorFa
> > > ctory.
> > > >> (#5587)
> > > >> - ArrayAggregation: Use long to avoid overflow (#5544)
> > > >> - Respect forceHashAggregation in queryContext (#5533)
> > > >> - Fix indexTask to respect forceExtendableShardSpecs (#5509)
> > > >> - Add overlord unsecured paths to coordinator when using combined
> > > service
> > > >> (#5579)
> > > >> - Fix SQLMetadataSegmentManager to allow succesive start and stop
> > > (#5554)
> > > >> - Fix supervisor tombstone auth handling (#5504)
> > > >> - Authorize supervisor history instead of current active supervisors
> > for
> > > >> supervisor history API (#5501)
> > > >> - Fix round robining in router. (#5500)
> > > >> - SegmentMetadataQuery: Fix default interval handling. (#5489)
> > > >> - Log exceptions thrown before persist() for indexing tasks (#5374)
> > > >> - More memory limiting for HttpPostEmitter (#5300)
> > > >> - pass configuration from context into JobConf for determining
> > > >> DatasourceInputFormat splits (#5408)
> > > >> - Lookups: Inherit "injective" from registered lookups, improve
> docs.
> > > >> (#5316)
> > > >> - SQL: Throttle metadata refreshes when they fail. (#5328)
> > > >>
> > > >> On Mon, Apr 9, 2018 at 10:35 AM, Gian Merlino 
> wrote:
> > > >>
> > > >> I think this conversation is worth having. I have cross posted this
> to
> > > >> dev@druid.apache.org and will reply there. Since we're trying to
> > > migrate
> > > >> the dev list, please cross post any dev messages there, or even only
> > > post
> > > >> to that list.
> > > >>
> > > >> Gian
> > > >>
> > > >> On Sun, Apr 8, 2018 at 5:45 PM, Prashant Deva <
> > prashant.d...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> Current 0.12.0 release has some major issues:
> > > >>
> > > >>
> > > >>   1. Coordinator loses leadership
> > > >>   https://github.com/druid-io/druid/issues/5561
> > > >>
> > > >>   2. Newly introduced Quantiles sketch is broken
> > > >>   https://github.com/druid-io/druid/issues/5575
> > > >>
> > > >>   3. Coordinator+overlord web console broken
> > > >>   https://github.com/druid-io/druid/issues/5559
> > > >>
> > > >>
> > > >> 1. is especially very important. Without a coordinator, druid stops
> > > >> functioning.
> > > >> With bug 5561, it is impossible to use druid for long periods since
> > > >> coordinator eventually does lose leadership and the whole process
> > needs
> > > to
> > > >> be restarted for it to come back.
> > > >>
> > > >> *Why not wait till 0.13.0?*
> > > >>
> > > >> A lot of companies like to update one version at a time and may not
> > want

Re: [druid-dev] consider doing a 0.12.1 release

[Gian Merlino]

I raised backport PRs for everything on my list except for these three,
since they had some (minor) conflicts and I was using a backport script
that bails out if it sees any conflicts:

- https://github.com/druid-io/druid/pull/5504
- https://github.com/druid-io/druid/pull/5587
- https://github.com/druid-io/druid/pull/5374

I will take a closer look at them later if nobody else gets to them.

Nishant, I added https://github.com/druid-io/druid/pull/5596 to the
milestone.

Prashant, I added https://github.com/druid-io/druid/pull/5611 to the
milestone.

On Mon, Apr 9, 2018 at 1:10 PM, Prashant Deva 
wrote:

> also for the list:
>
> - compact task throws exception #5611
>
> i dont have permission to add to gian's github milestone
>
> Prashant
>
> On Mon, Apr 9, 2018 at 12:26 PM, Gian Merlino 
> wrote:
>
> > Here we go: https://github.com/druid-io/druid/milestone/26. Please add
> > stuff if it makes sense (high importance / low risk bug fixes).
> >
> > On Mon, Apr 9, 2018 at 12:14 PM, Gian Merlino 
> > wrote:
> >
> > > It looks like there's a lot of support for doing a 0.12.1… I'll make a
> > > milestone in GH.
> > >
> > > On Mon, Apr 9, 2018 at 12:12 PM, Slim Bouguerra 
> > wrote:
> > >
> > >> +1 especially there is some instability bugs.
> > >>
> > >> On Apr 9, 2018, at 11:57 AM, Gian Merlino  wrote:
> > >>
> > >> My feeling is that #3 and #2 are borderline, but #1 definitely
> warrants
> > a
> > >> new release. Personally I have seen it occur at least a half dozen
> > times,
> > >> and I had been thinking about proposing a Druid 0.12.1 release, so I'm
> > >> glad
> > >> you brought it up.
> > >>
> > >> If we do 0.12.1, it would be another non-ASF release (we haven't got
> the
> > >> ASF process set up yet, and are not likely to have it set up in time)
> so
> > >> we
> > >> should notify the incubator folks about it.
> > >>
> > >> I would also consider including these fixes in 0.12.1:
> > >>
> > >> - DoublesSketchModule: Fix serde for DoublesSketchMergeAggregatorFa
> > ctory.
> > >> (#5587)
> > >> - ArrayAggregation: Use long to avoid overflow (#5544)
> > >> - Respect forceHashAggregation in queryContext (#5533)
> > >> - Fix indexTask to respect forceExtendableShardSpecs (#5509)
> > >> - Add overlord unsecured paths to coordinator when using combined
> > service
> > >> (#5579)
> > >> - Fix SQLMetadataSegmentManager to allow succesive start and stop
> > (#5554)
> > >> - Fix supervisor tombstone auth handling (#5504)
> > >> - Authorize supervisor history instead of current active supervisors
> for
> > >> supervisor history API (#5501)
> > >> - Fix round robining in router. (#5500)
> > >> - SegmentMetadataQuery: Fix default interval handling. (#5489)
> > >> - Log exceptions thrown before persist() for indexing tasks (#5374)
> > >> - More memory limiting for HttpPostEmitter (#5300)
> > >> - pass configuration from context into JobConf for determining
> > >> DatasourceInputFormat splits (#5408)
> > >> - Lookups: Inherit "injective" from registered lookups, improve docs.
> > >> (#5316)
> > >> - SQL: Throttle metadata refreshes when they fail. (#5328)
> > >>
> > >> On Mon, Apr 9, 2018 at 10:35 AM, Gian Merlino  wrote:
> > >>
> > >> I think this conversation is worth having. I have cross posted this to
> > >> dev@druid.apache.org and will reply there. Since we're trying to
> > migrate
> > >> the dev list, please cross post any dev messages there, or even only
> > post
> > >> to that list.
> > >>
> > >> Gian
> > >>
> > >> On Sun, Apr 8, 2018 at 5:45 PM, Prashant Deva <
> prashant.d...@gmail.com>
> > >> wrote:
> > >>
> > >> Current 0.12.0 release has some major issues:
> > >>
> > >>
> > >>   1. Coordinator loses leadership
> > >>   https://github.com/druid-io/druid/issues/5561
> > >>
> > >>   2. Newly introduced Quantiles sketch is broken
> > >>   https://github.com/druid-io/druid/issues/5575
> > >>
> > >>   3. Coordinator+overlord web console broken
> > >>   https://github.com/druid-io/druid/issues/5559
> > >>
> > >>
> > >> 1. is especially very important. Without a coordinator, druid stops
> > >> functioning.
> > >> With bug 5561, it is impossible to use druid for long periods since
> > >> coordinator eventually does lose leadership and the whole process
> needs
> > to
> > >> be restarted for it to come back.
> > >>
> > >> *Why not wait till 0.13.0?*
> > >>
> > >> A lot of companies like to update one version at a time and may not
> want
> > >> to jump directly to 0.13.0.
> > >> Those companies will hit a bad surprise due to bug 5561 essentially
> > >> rendering the cluster useless in production.
> > >> Also quantiles being the new feature and broken does not look good
> > either.
> > >>
> > >> 0.12.0 is a 'release', not an RC, thus marking it good for production,
> > >> but the bugs listed above prevent it from being used as such.
> > >> I highly recommend 0.12.1 release, thus marking the right 

Re: [druid-dev] consider doing a 0.12.1 release

[Prashant Deva]

also for the list:

- compact task throws exception #5611

i dont have permission to add to gian's github milestone

Prashant

On Mon, Apr 9, 2018 at 12:26 PM, Gian Merlino  wrote:

> Here we go: https://github.com/druid-io/druid/milestone/26. Please add
> stuff if it makes sense (high importance / low risk bug fixes).
>
> On Mon, Apr 9, 2018 at 12:14 PM, Gian Merlino 
> wrote:
>
> > It looks like there's a lot of support for doing a 0.12.1… I'll make a
> > milestone in GH.
> >
> > On Mon, Apr 9, 2018 at 12:12 PM, Slim Bouguerra 
> wrote:
> >
> >> +1 especially there is some instability bugs.
> >>
> >> On Apr 9, 2018, at 11:57 AM, Gian Merlino  wrote:
> >>
> >> My feeling is that #3 and #2 are borderline, but #1 definitely warrants
> a
> >> new release. Personally I have seen it occur at least a half dozen
> times,
> >> and I had been thinking about proposing a Druid 0.12.1 release, so I'm
> >> glad
> >> you brought it up.
> >>
> >> If we do 0.12.1, it would be another non-ASF release (we haven't got the
> >> ASF process set up yet, and are not likely to have it set up in time) so
> >> we
> >> should notify the incubator folks about it.
> >>
> >> I would also consider including these fixes in 0.12.1:
> >>
> >> - DoublesSketchModule: Fix serde for DoublesSketchMergeAggregatorFa
> ctory.
> >> (#5587)
> >> - ArrayAggregation: Use long to avoid overflow (#5544)
> >> - Respect forceHashAggregation in queryContext (#5533)
> >> - Fix indexTask to respect forceExtendableShardSpecs (#5509)
> >> - Add overlord unsecured paths to coordinator when using combined
> service
> >> (#5579)
> >> - Fix SQLMetadataSegmentManager to allow succesive start and stop
> (#5554)
> >> - Fix supervisor tombstone auth handling (#5504)
> >> - Authorize supervisor history instead of current active supervisors for
> >> supervisor history API (#5501)
> >> - Fix round robining in router. (#5500)
> >> - SegmentMetadataQuery: Fix default interval handling. (#5489)
> >> - Log exceptions thrown before persist() for indexing tasks (#5374)
> >> - More memory limiting for HttpPostEmitter (#5300)
> >> - pass configuration from context into JobConf for determining
> >> DatasourceInputFormat splits (#5408)
> >> - Lookups: Inherit "injective" from registered lookups, improve docs.
> >> (#5316)
> >> - SQL: Throttle metadata refreshes when they fail. (#5328)
> >>
> >> On Mon, Apr 9, 2018 at 10:35 AM, Gian Merlino  wrote:
> >>
> >> I think this conversation is worth having. I have cross posted this to
> >> dev@druid.apache.org and will reply there. Since we're trying to
> migrate
> >> the dev list, please cross post any dev messages there, or even only
> post
> >> to that list.
> >>
> >> Gian
> >>
> >> On Sun, Apr 8, 2018 at 5:45 PM, Prashant Deva 
> >> wrote:
> >>
> >> Current 0.12.0 release has some major issues:
> >>
> >>
> >>   1. Coordinator loses leadership
> >>   https://github.com/druid-io/druid/issues/5561
> >>
> >>   2. Newly introduced Quantiles sketch is broken
> >>   https://github.com/druid-io/druid/issues/5575
> >>
> >>   3. Coordinator+overlord web console broken
> >>   https://github.com/druid-io/druid/issues/5559
> >>
> >>
> >> 1. is especially very important. Without a coordinator, druid stops
> >> functioning.
> >> With bug 5561, it is impossible to use druid for long periods since
> >> coordinator eventually does lose leadership and the whole process needs
> to
> >> be restarted for it to come back.
> >>
> >> *Why not wait till 0.13.0?*
> >>
> >> A lot of companies like to update one version at a time and may not want
> >> to jump directly to 0.13.0.
> >> Those companies will hit a bad surprise due to bug 5561 essentially
> >> rendering the cluster useless in production.
> >> Also quantiles being the new feature and broken does not look good
> either.
> >>
> >> 0.12.0 is a 'release', not an RC, thus marking it good for production,
> >> but the bugs listed above prevent it from being used as such.
> >> I highly recommend 0.12.1 release, thus marking the right version to
> >> upgrade to from 0.11.0
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> Groups
> >> "Druid Development" group.
> >> To unsubscribe from this group and stop receiving emails from it, send
> an
> >> email to druid-development+unsubscr...@googlegroups.com.
> >> To post to this group, send email to druid-developm...@googlegroups.com
> .
> >> To view this discussion on the web visit https://groups.google.com/d/ms
> >> gid/druid-development/165b491e-3ec2-4744-a228-d1270c9d283a%4
> >> 0googlegroups.com
> >>  >> 165b491e-3ec2-4744-a228-d1270c9d283a%40googlegroups.
> >> com?utm_medium=email_source=footer>
> >> .
> >> For more options, visit https://groups.google.com/d/optout.
> >>
> >>
> >> --
> >> You received this message because you are subscribed to the 

Re: [druid-dev] consider doing a 0.12.1 release

[Gian Merlino]

Here we go: https://github.com/druid-io/druid/milestone/26. Please add
stuff if it makes sense (high importance / low risk bug fixes).

On Mon, Apr 9, 2018 at 12:14 PM, Gian Merlino  wrote:

> It looks like there's a lot of support for doing a 0.12.1… I'll make a
> milestone in GH.
>
> On Mon, Apr 9, 2018 at 12:12 PM, Slim Bouguerra  wrote:
>
>> +1 especially there is some instability bugs.
>>
>> On Apr 9, 2018, at 11:57 AM, Gian Merlino  wrote:
>>
>> My feeling is that #3 and #2 are borderline, but #1 definitely warrants a
>> new release. Personally I have seen it occur at least a half dozen times,
>> and I had been thinking about proposing a Druid 0.12.1 release, so I'm
>> glad
>> you brought it up.
>>
>> If we do 0.12.1, it would be another non-ASF release (we haven't got the
>> ASF process set up yet, and are not likely to have it set up in time) so
>> we
>> should notify the incubator folks about it.
>>
>> I would also consider including these fixes in 0.12.1:
>>
>> - DoublesSketchModule: Fix serde for DoublesSketchMergeAggregatorFactory.
>> (#5587)
>> - ArrayAggregation: Use long to avoid overflow (#5544)
>> - Respect forceHashAggregation in queryContext (#5533)
>> - Fix indexTask to respect forceExtendableShardSpecs (#5509)
>> - Add overlord unsecured paths to coordinator when using combined service
>> (#5579)
>> - Fix SQLMetadataSegmentManager to allow succesive start and stop (#5554)
>> - Fix supervisor tombstone auth handling (#5504)
>> - Authorize supervisor history instead of current active supervisors for
>> supervisor history API (#5501)
>> - Fix round robining in router. (#5500)
>> - SegmentMetadataQuery: Fix default interval handling. (#5489)
>> - Log exceptions thrown before persist() for indexing tasks (#5374)
>> - More memory limiting for HttpPostEmitter (#5300)
>> - pass configuration from context into JobConf for determining
>> DatasourceInputFormat splits (#5408)
>> - Lookups: Inherit "injective" from registered lookups, improve docs.
>> (#5316)
>> - SQL: Throttle metadata refreshes when they fail. (#5328)
>>
>> On Mon, Apr 9, 2018 at 10:35 AM, Gian Merlino  wrote:
>>
>> I think this conversation is worth having. I have cross posted this to
>> dev@druid.apache.org and will reply there. Since we're trying to migrate
>> the dev list, please cross post any dev messages there, or even only post
>> to that list.
>>
>> Gian
>>
>> On Sun, Apr 8, 2018 at 5:45 PM, Prashant Deva 
>> wrote:
>>
>> Current 0.12.0 release has some major issues:
>>
>>
>>   1. Coordinator loses leadership
>>   https://github.com/druid-io/druid/issues/5561
>>
>>   2. Newly introduced Quantiles sketch is broken
>>   https://github.com/druid-io/druid/issues/5575
>>
>>   3. Coordinator+overlord web console broken
>>   https://github.com/druid-io/druid/issues/5559
>>
>>
>> 1. is especially very important. Without a coordinator, druid stops
>> functioning.
>> With bug 5561, it is impossible to use druid for long periods since
>> coordinator eventually does lose leadership and the whole process needs to
>> be restarted for it to come back.
>>
>> *Why not wait till 0.13.0?*
>>
>> A lot of companies like to update one version at a time and may not want
>> to jump directly to 0.13.0.
>> Those companies will hit a bad surprise due to bug 5561 essentially
>> rendering the cluster useless in production.
>> Also quantiles being the new feature and broken does not look good either.
>>
>> 0.12.0 is a 'release', not an RC, thus marking it good for production,
>> but the bugs listed above prevent it from being used as such.
>> I highly recommend 0.12.1 release, thus marking the right version to
>> upgrade to from 0.11.0
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Druid Development" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to druid-development+unsubscr...@googlegroups.com.
>> To post to this group, send email to druid-developm...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/ms
>> gid/druid-development/165b491e-3ec2-4744-a228-d1270c9d283a%4
>> 0googlegroups.com
>> > 165b491e-3ec2-4744-a228-d1270c9d283a%40googlegroups.
>> com?utm_medium=email_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Druid Development" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to druid-development+unsubscr...@googlegroups.com.
>> To post to this group, send email to druid-developm...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/
>> msgid/druid-development/CACZNdYDbTjn%2BVZfnK-zuugw_
>> 

Re: [druid-dev] consider doing a 0.12.1 release

[Gian Merlino]

It looks like there's a lot of support for doing a 0.12.1… I'll make a
milestone in GH.

On Mon, Apr 9, 2018 at 12:12 PM, Slim Bouguerra  wrote:

> +1 especially there is some instability bugs.
>
> On Apr 9, 2018, at 11:57 AM, Gian Merlino  wrote:
>
> My feeling is that #3 and #2 are borderline, but #1 definitely warrants a
> new release. Personally I have seen it occur at least a half dozen times,
> and I had been thinking about proposing a Druid 0.12.1 release, so I'm glad
> you brought it up.
>
> If we do 0.12.1, it would be another non-ASF release (we haven't got the
> ASF process set up yet, and are not likely to have it set up in time) so we
> should notify the incubator folks about it.
>
> I would also consider including these fixes in 0.12.1:
>
> - DoublesSketchModule: Fix serde for DoublesSketchMergeAggregatorFactory.
> (#5587)
> - ArrayAggregation: Use long to avoid overflow (#5544)
> - Respect forceHashAggregation in queryContext (#5533)
> - Fix indexTask to respect forceExtendableShardSpecs (#5509)
> - Add overlord unsecured paths to coordinator when using combined service
> (#5579)
> - Fix SQLMetadataSegmentManager to allow succesive start and stop (#5554)
> - Fix supervisor tombstone auth handling (#5504)
> - Authorize supervisor history instead of current active supervisors for
> supervisor history API (#5501)
> - Fix round robining in router. (#5500)
> - SegmentMetadataQuery: Fix default interval handling. (#5489)
> - Log exceptions thrown before persist() for indexing tasks (#5374)
> - More memory limiting for HttpPostEmitter (#5300)
> - pass configuration from context into JobConf for determining
> DatasourceInputFormat splits (#5408)
> - Lookups: Inherit "injective" from registered lookups, improve docs.
> (#5316)
> - SQL: Throttle metadata refreshes when they fail. (#5328)
>
> On Mon, Apr 9, 2018 at 10:35 AM, Gian Merlino  wrote:
>
> I think this conversation is worth having. I have cross posted this to
> dev@druid.apache.org and will reply there. Since we're trying to migrate
> the dev list, please cross post any dev messages there, or even only post
> to that list.
>
> Gian
>
> On Sun, Apr 8, 2018 at 5:45 PM, Prashant Deva 
> wrote:
>
> Current 0.12.0 release has some major issues:
>
>
>   1. Coordinator loses leadership
>   https://github.com/druid-io/druid/issues/5561
>
>   2. Newly introduced Quantiles sketch is broken
>   https://github.com/druid-io/druid/issues/5575
>
>   3. Coordinator+overlord web console broken
>   https://github.com/druid-io/druid/issues/5559
>
>
> 1. is especially very important. Without a coordinator, druid stops
> functioning.
> With bug 5561, it is impossible to use druid for long periods since
> coordinator eventually does lose leadership and the whole process needs to
> be restarted for it to come back.
>
> *Why not wait till 0.13.0?*
>
> A lot of companies like to update one version at a time and may not want
> to jump directly to 0.13.0.
> Those companies will hit a bad surprise due to bug 5561 essentially
> rendering the cluster useless in production.
> Also quantiles being the new feature and broken does not look good either.
>
> 0.12.0 is a 'release', not an RC, thus marking it good for production,
> but the bugs listed above prevent it from being used as such.
> I highly recommend 0.12.1 release, thus marking the right version to
> upgrade to from 0.11.0
>
> --
> You received this message because you are subscribed to the Google Groups
> "Druid Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to druid-development+unsubscr...@googlegroups.com.
> To post to this group, send email to druid-developm...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/ms
> gid/druid-development/165b491e-3ec2-4744-a228-d1270c9d283a%4
> 0googlegroups.com
>  165b491e-3ec2-4744-a228-d1270c9d283a%40googlegroups.
> com?utm_medium=email_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Druid Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to druid-development+unsubscr...@googlegroups.com.
> To post to this group, send email to druid-developm...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/druid-development/CACZNdYDbTjn%2BVZfnK-zuugw_
> E1efUNu2F4G%2BJTZJa0h7J-iK3w%40mail.gmail.com
>  CACZNdYDbTjn%2BVZfnK-zuugw_E1efUNu2F4G%2BJTZJa0h7J-iK3w%
> 40mail.gmail.com?utm_medium=email_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

Re: [druid-dev] consider doing a 0.12.1 release

[Slim Bouguerra]

+1 especially there is some instability bugs.

On Apr 9, 2018, at 11:57 AM, Gian Merlino  wrote:

My feeling is that #3 and #2 are borderline, but #1 definitely warrants a
new release. Personally I have seen it occur at least a half dozen times,
and I had been thinking about proposing a Druid 0.12.1 release, so I'm glad
you brought it up.

If we do 0.12.1, it would be another non-ASF release (we haven't got the
ASF process set up yet, and are not likely to have it set up in time) so we
should notify the incubator folks about it.

I would also consider including these fixes in 0.12.1:

- DoublesSketchModule: Fix serde for DoublesSketchMergeAggregatorFactory.
(#5587)
- ArrayAggregation: Use long to avoid overflow (#5544)
- Respect forceHashAggregation in queryContext (#5533)
- Fix indexTask to respect forceExtendableShardSpecs (#5509)
- Add overlord unsecured paths to coordinator when using combined service
(#5579)
- Fix SQLMetadataSegmentManager to allow succesive start and stop (#5554)
- Fix supervisor tombstone auth handling (#5504)
- Authorize supervisor history instead of current active supervisors for
supervisor history API (#5501)
- Fix round robining in router. (#5500)
- SegmentMetadataQuery: Fix default interval handling. (#5489)
- Log exceptions thrown before persist() for indexing tasks (#5374)
- More memory limiting for HttpPostEmitter (#5300)
- pass configuration from context into JobConf for determining
DatasourceInputFormat splits (#5408)
- Lookups: Inherit "injective" from registered lookups, improve docs.
(#5316)
- SQL: Throttle metadata refreshes when they fail. (#5328)

On Mon, Apr 9, 2018 at 10:35 AM, Gian Merlino  wrote:

I think this conversation is worth having. I have cross posted this to
dev@druid.apache.org and will reply there. Since we're trying to migrate
the dev list, please cross post any dev messages there, or even only post
to that list.

Gian

On Sun, Apr 8, 2018 at 5:45 PM, Prashant Deva 
wrote:

Current 0.12.0 release has some major issues:


  1. Coordinator loses leadership
  https://github.com/druid-io/druid/issues/5561

  2. Newly introduced Quantiles sketch is broken
  https://github.com/druid-io/druid/issues/5575

  3. Coordinator+overlord web console broken
  https://github.com/druid-io/druid/issues/5559


1. is especially very important. Without a coordinator, druid stops
functioning.
With bug 5561, it is impossible to use druid for long periods since
coordinator eventually does lose leadership and the whole process needs to
be restarted for it to come back.

*Why not wait till 0.13.0?*

A lot of companies like to update one version at a time and may not want
to jump directly to 0.13.0.
Those companies will hit a bad surprise due to bug 5561 essentially
rendering the cluster useless in production.
Also quantiles being the new feature and broken does not look good either.

0.12.0 is a 'release', not an RC, thus marking it good for production,
but the bugs listed above prevent it from being used as such.
I highly recommend 0.12.1 release, thus marking the right version to
upgrade to from 0.11.0

--
You received this message because you are subscribed to the Google Groups
"Druid Development" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to druid-development+unsubscr...@googlegroups.com.
To post to this group, send email to druid-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/ms
gid/druid-development/165b491e-3ec2-4744-a228-d1270c9d283a%4
0googlegroups.com

.
For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups
"Druid Development" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to druid-development+unsubscr...@googlegroups.com.
To post to this group, send email to druid-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/druid-development/CACZNdYDbTjn%2BVZfnK-zuugw_
E1efUNu2F4G%2BJTZJa0h7J-iK3w%40mail.gmail.com

.

For more options, visit https://groups.google.com/d/optout.

Re: [druid-dev] consider doing a 0.12.1 release

[Nishant Bangarwa]

+1 on doing a 0.12.1 release.
Kerberos security also has some issues -
https://github.com/druid-io/druid/pull/5596
I propose we also get this in 0.12.1.

On Tue, 10 Apr 2018 at 00:32 Prashant Deva  wrote:

> May I also add this to the list Gian proposed:
>
> - Load rules should honor partial overlap #5595
>
> Prashant
>
> On Mon, Apr 9, 2018 at 11:57 AM, Gian Merlino  wrote:
>
>> My feeling is that #3 and #2 are borderline, but #1 definitely warrants a
>> new release. Personally I have seen it occur at least a half dozen times,
>> and I had been thinking about proposing a Druid 0.12.1 release, so I'm
>> glad
>> you brought it up.
>>
>> If we do 0.12.1, it would be another non-ASF release (we haven't got the
>> ASF process set up yet, and are not likely to have it set up in time) so
>> we
>> should notify the incubator folks about it.
>>
>> I would also consider including these fixes in 0.12.1:
>>
>> - DoublesSketchModule: Fix serde for DoublesSketchMergeAggregatorFactory.
>> (#5587)
>> - ArrayAggregation: Use long to avoid overflow (#5544)
>> - Respect forceHashAggregation in queryContext (#5533)
>> - Fix indexTask to respect forceExtendableShardSpecs (#5509)
>> - Add overlord unsecured paths to coordinator when using combined service
>> (#5579)
>> - Fix SQLMetadataSegmentManager to allow succesive start and stop (#5554)
>> - Fix supervisor tombstone auth handling (#5504)
>> - Authorize supervisor history instead of current active supervisors for
>> supervisor history API (#5501)
>> - Fix round robining in router. (#5500)
>> - SegmentMetadataQuery: Fix default interval handling. (#5489)
>> - Log exceptions thrown before persist() for indexing tasks (#5374)
>> - More memory limiting for HttpPostEmitter (#5300)
>> - pass configuration from context into JobConf for determining
>> DatasourceInputFormat splits (#5408)
>> - Lookups: Inherit "injective" from registered lookups, improve docs.
>> (#5316)
>> - SQL: Throttle metadata refreshes when they fail. (#5328)
>>
>> On Mon, Apr 9, 2018 at 10:35 AM, Gian Merlino  wrote:
>>
>> > I think this conversation is worth having. I have cross posted this to
>> > dev@druid.apache.org and will reply there. Since we're trying to
>> migrate
>> > the dev list, please cross post any dev messages there, or even only
>> post
>> > to that list.
>> >
>> > Gian
>> >
>> > On Sun, Apr 8, 2018 at 5:45 PM, Prashant Deva 
>> > wrote:
>> >
>> >> Current 0.12.0 release has some major issues:
>> >>
>> >>
>> >>1. Coordinator loses leadership
>> >>https://github.com/druid-io/druid/issues/5561
>> >>
>> >>2. Newly introduced Quantiles sketch is broken
>> >>https://github.com/druid-io/druid/issues/5575
>> >>
>> >>3. Coordinator+overlord web console broken
>> >>https://github.com/druid-io/druid/issues/5559
>> >>
>> >>
>> >> 1. is especially very important. Without a coordinator, druid stops
>> >> functioning.
>> >> With bug 5561, it is impossible to use druid for long periods since
>> >> coordinator eventually does lose leadership and the whole process
>> needs to
>> >> be restarted for it to come back.
>> >>
>> >> *Why not wait till 0.13.0?*
>> >>
>> >> A lot of companies like to update one version at a time and may not
>> want
>> >> to jump directly to 0.13.0.
>> >> Those companies will hit a bad surprise due to bug 5561 essentially
>> >> rendering the cluster useless in production.
>> >> Also quantiles being the new feature and broken does not look good
>> either.
>> >>
>> >> 0.12.0 is a 'release', not an RC, thus marking it good for production,
>> >> but the bugs listed above prevent it from being used as such.
>> >> I highly recommend 0.12.1 release, thus marking the right version to
>> >> upgrade to from 0.11.0
>> >>
>> >> --
>> >> You received this message because you are subscribed to the Google
>> Groups
>> >> "Druid Development" group.
>> >> To unsubscribe from this group and stop receiving emails from it, send
>> an
>> >> email to druid-development+unsubscr...@googlegroups.com.
>> >> To post to this group, send email to
>> druid-developm...@googlegroups.com.
>> >> To view this discussion on the web visit
>> https://groups.google.com/d/ms
>> >> gid/druid-development/165b491e-3ec2-4744-a228-d1270c9d283a%4
>> >> 0googlegroups.com
>> >> <
>> https://groups.google.com/d/msgid/druid-development/165b491e-3ec2-4744-a228-d1270c9d283a%40googlegroups.com?utm_medium=email_source=footer
>> >
>> >> .
>> >> For more options, visit https://groups.google.com/d/optout.
>> >>
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups
>> > "Druid Development" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an
>> > email to druid-development+unsubscr...@googlegroups.com.
>> > To post to this group, send email to druid-developm...@googlegroups.com
>> .
>> > To view this discussion on the web visit https://groups.google.com/d/
>> 

Re: [druid-dev] consider doing a 0.12.1 release

[Prashant Deva]

May I also add this to the list Gian proposed:

- Load rules should honor partial overlap #5595

Prashant

On Mon, Apr 9, 2018 at 11:57 AM, Gian Merlino  wrote:

> My feeling is that #3 and #2 are borderline, but #1 definitely warrants a
> new release. Personally I have seen it occur at least a half dozen times,
> and I had been thinking about proposing a Druid 0.12.1 release, so I'm glad
> you brought it up.
>
> If we do 0.12.1, it would be another non-ASF release (we haven't got the
> ASF process set up yet, and are not likely to have it set up in time) so we
> should notify the incubator folks about it.
>
> I would also consider including these fixes in 0.12.1:
>
> - DoublesSketchModule: Fix serde for DoublesSketchMergeAggregatorFactory.
> (#5587)
> - ArrayAggregation: Use long to avoid overflow (#5544)
> - Respect forceHashAggregation in queryContext (#5533)
> - Fix indexTask to respect forceExtendableShardSpecs (#5509)
> - Add overlord unsecured paths to coordinator when using combined service
> (#5579)
> - Fix SQLMetadataSegmentManager to allow succesive start and stop (#5554)
> - Fix supervisor tombstone auth handling (#5504)
> - Authorize supervisor history instead of current active supervisors for
> supervisor history API (#5501)
> - Fix round robining in router. (#5500)
> - SegmentMetadataQuery: Fix default interval handling. (#5489)
> - Log exceptions thrown before persist() for indexing tasks (#5374)
> - More memory limiting for HttpPostEmitter (#5300)
> - pass configuration from context into JobConf for determining
> DatasourceInputFormat splits (#5408)
> - Lookups: Inherit "injective" from registered lookups, improve docs.
> (#5316)
> - SQL: Throttle metadata refreshes when they fail. (#5328)
>
> On Mon, Apr 9, 2018 at 10:35 AM, Gian Merlino  wrote:
>
> > I think this conversation is worth having. I have cross posted this to
> > dev@druid.apache.org and will reply there. Since we're trying to migrate
> > the dev list, please cross post any dev messages there, or even only post
> > to that list.
> >
> > Gian
> >
> > On Sun, Apr 8, 2018 at 5:45 PM, Prashant Deva 
> > wrote:
> >
> >> Current 0.12.0 release has some major issues:
> >>
> >>
> >>1. Coordinator loses leadership
> >>https://github.com/druid-io/druid/issues/5561
> >>
> >>2. Newly introduced Quantiles sketch is broken
> >>https://github.com/druid-io/druid/issues/5575
> >>
> >>3. Coordinator+overlord web console broken
> >>https://github.com/druid-io/druid/issues/5559
> >>
> >>
> >> 1. is especially very important. Without a coordinator, druid stops
> >> functioning.
> >> With bug 5561, it is impossible to use druid for long periods since
> >> coordinator eventually does lose leadership and the whole process needs
> to
> >> be restarted for it to come back.
> >>
> >> *Why not wait till 0.13.0?*
> >>
> >> A lot of companies like to update one version at a time and may not want
> >> to jump directly to 0.13.0.
> >> Those companies will hit a bad surprise due to bug 5561 essentially
> >> rendering the cluster useless in production.
> >> Also quantiles being the new feature and broken does not look good
> either.
> >>
> >> 0.12.0 is a 'release', not an RC, thus marking it good for production,
> >> but the bugs listed above prevent it from being used as such.
> >> I highly recommend 0.12.1 release, thus marking the right version to
> >> upgrade to from 0.11.0
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> Groups
> >> "Druid Development" group.
> >> To unsubscribe from this group and stop receiving emails from it, send
> an
> >> email to druid-development+unsubscr...@googlegroups.com.
> >> To post to this group, send email to druid-developm...@googlegroups.com
> .
> >> To view this discussion on the web visit https://groups.google.com/d/ms
> >> gid/druid-development/165b491e-3ec2-4744-a228-d1270c9d283a%4
> >> 0googlegroups.com
> >>  165b491e-3ec2-4744-a228-d1270c9d283a%40googlegroups.
> com?utm_medium=email_source=footer>
> >> .
> >> For more options, visit https://groups.google.com/d/optout.
> >>
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Druid Development" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to druid-development+unsubscr...@googlegroups.com.
> > To post to this group, send email to druid-developm...@googlegroups.com.
> > To view this discussion on the web visit https://groups.google.com/d/
> > msgid/druid-development/CACZNdYDbTjn%2BVZfnK-zuugw_
> > E1efUNu2F4G%2BJTZJa0h7J-iK3w%40mail.gmail.com
> >  CACZNdYDbTjn%2BVZfnK-zuugw_E1efUNu2F4G%2BJTZJa0h7J-iK3w%
> 40mail.gmail.com?utm_medium=email_source=footer>
> > .
> >
> > For more options, visit https://groups.google.com/d/optout.
> >
>

Re: [druid-dev] consider doing a 0.12.1 release

[Gian Merlino]

My feeling is that #3 and #2 are borderline, but #1 definitely warrants a
new release. Personally I have seen it occur at least a half dozen times,
and I had been thinking about proposing a Druid 0.12.1 release, so I'm glad
you brought it up.

If we do 0.12.1, it would be another non-ASF release (we haven't got the
ASF process set up yet, and are not likely to have it set up in time) so we
should notify the incubator folks about it.

I would also consider including these fixes in 0.12.1:

- DoublesSketchModule: Fix serde for DoublesSketchMergeAggregatorFactory.
(#5587)
- ArrayAggregation: Use long to avoid overflow (#5544)
- Respect forceHashAggregation in queryContext (#5533)
- Fix indexTask to respect forceExtendableShardSpecs (#5509)
- Add overlord unsecured paths to coordinator when using combined service
(#5579)
- Fix SQLMetadataSegmentManager to allow succesive start and stop (#5554)
- Fix supervisor tombstone auth handling (#5504)
- Authorize supervisor history instead of current active supervisors for
supervisor history API (#5501)
- Fix round robining in router. (#5500)
- SegmentMetadataQuery: Fix default interval handling. (#5489)
- Log exceptions thrown before persist() for indexing tasks (#5374)
- More memory limiting for HttpPostEmitter (#5300)
- pass configuration from context into JobConf for determining
DatasourceInputFormat splits (#5408)
- Lookups: Inherit "injective" from registered lookups, improve docs.
(#5316)
- SQL: Throttle metadata refreshes when they fail. (#5328)

On Mon, Apr 9, 2018 at 10:35 AM, Gian Merlino  wrote:

> I think this conversation is worth having. I have cross posted this to
> dev@druid.apache.org and will reply there. Since we're trying to migrate
> the dev list, please cross post any dev messages there, or even only post
> to that list.
>
> Gian
>
> On Sun, Apr 8, 2018 at 5:45 PM, Prashant Deva 
> wrote:
>
>> Current 0.12.0 release has some major issues:
>>
>>
>>1. Coordinator loses leadership
>>https://github.com/druid-io/druid/issues/5561
>>
>>2. Newly introduced Quantiles sketch is broken
>>https://github.com/druid-io/druid/issues/5575
>>
>>3. Coordinator+overlord web console broken
>>https://github.com/druid-io/druid/issues/5559
>>
>>
>> 1. is especially very important. Without a coordinator, druid stops
>> functioning.
>> With bug 5561, it is impossible to use druid for long periods since
>> coordinator eventually does lose leadership and the whole process needs to
>> be restarted for it to come back.
>>
>> *Why not wait till 0.13.0?*
>>
>> A lot of companies like to update one version at a time and may not want
>> to jump directly to 0.13.0.
>> Those companies will hit a bad surprise due to bug 5561 essentially
>> rendering the cluster useless in production.
>> Also quantiles being the new feature and broken does not look good either.
>>
>> 0.12.0 is a 'release', not an RC, thus marking it good for production,
>> but the bugs listed above prevent it from being used as such.
>> I highly recommend 0.12.1 release, thus marking the right version to
>> upgrade to from 0.11.0
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Druid Development" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to druid-development+unsubscr...@googlegroups.com.
>> To post to this group, send email to druid-developm...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/ms
>> gid/druid-development/165b491e-3ec2-4744-a228-d1270c9d283a%4
>> 0googlegroups.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Druid Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to druid-development+unsubscr...@googlegroups.com.
> To post to this group, send email to druid-developm...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/druid-development/CACZNdYDbTjn%2BVZfnK-zuugw_
> E1efUNu2F4G%2BJTZJa0h7J-iK3w%40mail.gmail.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

Re: [druid-dev] Load rule doesn't honor intervals properly

[Gian Merlino]

Looks like it is under review right now. Thanks for the patch.

Gian

On Sun, Apr 8, 2018 at 8:04 PM, 'Pala Muthiah' via Druid Development <
druid-developm...@googlegroups.com> wrote:

> Hi Gian,
>
> Thanks for following up. I have submitted a patch: https://github.com/
> druid-io/druid/pull/5595.
>
> Whoever is the right owner please take a look - let me know if i should @
> a specific person and i can do that.
>
>
> Thanks,
> pala
>
>
>
> On Fri, Mar 30, 2018 at 2:12 PM, Gian Merlino  wrote:
>
>> Hi Pala,
>>
>> That sounds like a bug to me - a patch would be welcome!
>>
>> Btw, since we are trying to migrate the dev mailing list to Apache,
>> please cross post this sort of thing with d...@druid.incubator.apache.org,
>> or even only post to that list.
>>
>> Gian
>>
>> On Thu, Mar 29, 2018 at 5:43 PM, 'Pala Muthiah' via Druid Development <
>> druid-developm...@googlegroups.com> wrote:
>>
>>> Hello folks,
>>>
>>> Anybody have insight on the below? Curious to know if there would be
>>> unforeseen side effects if we count even partial overlap as valid.
>>>
>>> On Mon, Mar 19, 2018 at 10:54 AM, pala.muthiah via Druid Development <
>>> druid-developm...@googlegroups.com> wrote:
>>>
 Hi,

 In our deployment, we enabled background segment merging and found that
 some of the data within the load period was actually getting dropped.

 My suspicion was that when a merged segment only partially overlaps
 with a period (e.g: Rule says keep data from Jan 1st onwards, and i have a
 segment that spans Dec 25th - Jan 2nd), for correctness that segment should
 be kept but current implementation seems to drop it.

 I checked the code and found indeed Rules.eligibleForLoad() only keeps
 segments that overlap fully.

 Is this a bug, or is there other reason behind this? In our case, we do
 have data sources that are highly aggregated and therefore a single segment
 could span a month for example.

 I can submit a patch but wanted to get proper context.


 Thanks,
 pala

 --
 You received this message because you are subscribed to a topic in the
 Google Groups "Druid Development" group.
 To unsubscribe from this topic, visit https://groups.google.com/d/to
 pic/druid-development/QYMhjGup2RI/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 druid-development+unsubscr...@googlegroups.com.
 To post to this group, send email to druid-developm...@googlegroups.com
 .
 To view this discussion on the web visit https://groups.google.com/d/ms
 gid/druid-development/d8e3df1d-699e-4747-a681-ebba91327388%4
 0googlegroups.com
 
 .
 For more options, visit https://groups.google.com/d/optout.

>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Druid Development" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to druid-development+unsubscr...@googlegroups.com.
>>> To post to this group, send email to druid-developm...@googlegroups.com.
>>> To view this discussion on the web visit https://groups.google.com/d/ms
>>> gid/druid-development/CALxr%3D2Wo9KKTYmFjbusc%2BsU0dt7K4k%3D
>>> P4JQX1T0sFe%3DLbBHXAA%40mail.gmail.com
>>> 
>>> .
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Druid Development" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/to
>> pic/druid-development/QYMhjGup2RI/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> druid-development+unsubscr...@googlegroups.com.
>> To post to this group, send email to druid-developm...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/ms
>> gid/druid-development/CACZNdYDNQNMmuxZ67Uk5q%3DwzK92EvqtozLW
>> %2BeK1cm8OaaYoR4Q%40mail.gmail.com
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Druid Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to druid-development+unsubscr...@googlegroups.com.
> To post to this group, send email to druid-developm...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> 

Re: [druid-dev] consider doing a 0.12.1 release

[Gian Merlino]

I think this conversation is worth having. I have cross posted this to
dev@druid.apache.org and will reply there. Since we're trying to migrate
the dev list, please cross post any dev messages there, or even only post
to that list.

Gian

On Sun, Apr 8, 2018 at 5:45 PM, Prashant Deva 
wrote:

> Current 0.12.0 release has some major issues:
>
>
>1. Coordinator loses leadership
>https://github.com/druid-io/druid/issues/5561
>
>2. Newly introduced Quantiles sketch is broken
>https://github.com/druid-io/druid/issues/5575
>
>3. Coordinator+overlord web console broken
>https://github.com/druid-io/druid/issues/5559
>
>
> 1. is especially very important. Without a coordinator, druid stops
> functioning.
> With bug 5561, it is impossible to use druid for long periods since
> coordinator eventually does lose leadership and the whole process needs to
> be restarted for it to come back.
>
> *Why not wait till 0.13.0?*
>
> A lot of companies like to update one version at a time and may not want
> to jump directly to 0.13.0.
> Those companies will hit a bad surprise due to bug 5561 essentially
> rendering the cluster useless in production.
> Also quantiles being the new feature and broken does not look good either.
>
> 0.12.0 is a 'release', not an RC, thus marking it good for production, but
> the bugs listed above prevent it from being used as such.
> I highly recommend 0.12.1 release, thus marking the right version to
> upgrade to from 0.11.0
>
> --
> You received this message because you are subscribed to the Google Groups
> "Druid Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to druid-development+unsubscr...@googlegroups.com.
> To post to this group, send email to druid-developm...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/ms
> gid/druid-development/165b491e-3ec2-4744-a228-d1270c9d283a%
> 40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>