Re: Druid 0.12.2 release

2018-08-10 Thread Dongjin Lee 
Congratulations!
Thank you for your great works!

Best,
Dongjin

On Fri, Aug 10, 2018 at 2:17 PM Jihoon Son  wrote:

> We're happy to announce our latest release, Druid 0.12.2!
>
> Druid 0.12.2 contains stability improvements and bug fixes from 13
> contributors. Major stability improvements are for Kafka indexing service,
> data ingestion, segment balancing, and querying & result caching.
>
> Druid 0.12.2 is a non-ASF release.
>
> You can download the release here:
> http://druid.io/downloads.html
>
> Release notes are at:
> https://github.com/apache/incubator-druid/releases/tag/druid-0.12.2
>
> Thanks to everyone who contributed issues, docs, and code!
>
> Jihoon
>
> --
> *Dongjin Lee*
>
> *A hitchhiker in the mathematical world.*
>
> *github:  github.com/dongjinleekr
> linkedin: kr.linkedin.com/in/dongjinleekr
> slideshare: 
> www.slideshare.net/dongjinleekr
> *
>

Re: Druid 0.12.2 release vote

2018-08-10 Thread Prashant Deva 
Charles,
 did you post a screenshot? i cant see it..

Prashant


On Thu, Aug 9, 2018 at 4:23 PM Jihoon Son  wrote:

> Hi all, thank you for the vote!
>
> The vote has passed with 8 positive votes (+1).
>
> I'm going to release 0.12.2.
>
> Best,
> Jihoon
>
> On Thu, Aug 9, 2018 at 10:40 AM Charles Allen  wrote:
>
> > +1
> >
> > Still rolling out but looks good for what I've seen so far
> >
> > On Thu, Aug 9, 2018 at 10:39 AM Charles Allen
> >  wrote:
> >
> > > Oh probably. Long story short: approximately equivalent, new release
> has
> > > slightly tucked in long tail on the bad side of query time (but also
> few
> > > samples in the long tail overall).
> > >
> > >
> > >
> > > On Thu, Aug 9, 2018 at 10:37 AM Gian Merlino  wrote:
> > >
> > > > Nice!!
> > > >
> > > > Although I don't see the graphic attached, maybe the mailing list ate
> > it?
> > > >
> > > > On Wed, Aug 8, 2018 at 4:15 PM Charles Allen  > > > .invalid>
> > > > wrote:
> > > >
> > > > > Blue is 0.12.2 with some minor backports not perf related. Red is
> > from
> > > > the
> > > > > 0.11.x series. This is effectively a bucketed PDF of the query
> times
> > > for
> > > > a
> > > > > live cluster with Timeseries queries as self-reported by historical
> > > > nodes.
> > > > > I mentioned elsewhere I'm not convinced query/time is a good proxy
> > for
> > > > user
> > > > > experience, but it does provide a good baseline for comparisons
> > between
> > > > > versions. Low query times are suspected due to some aggressive
> > caching
> > > or
> > > > > complete node misses (node very little data for that time range for
> > > that
> > > > > datasource). And high query time outliers are often the result of
> bad
> > > GC.
> > > > >
> > > > > On our side there is a new java version going out with the 0.12.2
> > > > > deployment so it is unclear how much is attributed to the new java
> > > > version
> > > > > and how much is attributed to the druid jars or other config
> changes.
> > > > > Overall things seem to consistently display a small % improvement
> in
> > > the
> > > > > mean with our internal 0.12.2 release. This is good!
> > > > >
> > > > > Cheers,
> > > > > Charles Allen
> > > > >
> > > > > [image: Screen Shot 2018-08-08 at 4.01.24 PM.png]
> > > > >
> > > > >
> > > > > On Wed, Aug 8, 2018 at 3:11 PM David Lim 
> > wrote:
> > > > >
> > > > >> +1, thank you!
> > > > >>
> > > > >> On Wed, Aug 8, 2018 at 3:16 PM Jonathan Wei 
> > > wrote:
> > > > >>
> > > > >> > +1, thanks Jihoon!
> > > > >> >
> > > > >> > On Wed, Aug 8, 2018 at 1:18 PM, Jihoon Son <
> jihoon...@apache.org>
> > > > >> wrote:
> > > > >> >
> > > > >> > > Awesome! Thanks Charles!
> > > > >> > >
> > > > >> > > Jihoon
> > > > >> > >
> > > > >> > > On Wed, Aug 8, 2018 at 1:16 PM Gian Merlino 
> > > > wrote:
> > > > >> > >
> > > > >> > > > Thanks, it will be nice to see!
> > > > >> > > >
> > > > >> > > > On Wed, Aug 8, 2018 at 1:15 PM Charles Allen <
> > > > >> charles.al...@snap.com
> > > > >> > > > .invalid>
> > > > >> > > > wrote:
> > > > >> > > >
> > > > >> > > > > I don't think it should be a blocker to release, but I
> have
> > to
> > > > run
> > > > >> > perf
> > > > >> > > > > tests for rollouts anyways so I figured I'd publish what I
> > > find
> > > > >> :-P
> > > > >> > > > >
> > > > >> > > > > Cheers,
> > > > >> > > > > Charles Allen
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > > > On Wed, Aug 8, 2018 at 12:33 PM Gian Merlino <
> > g...@apache.org
> > > >
> > > > >> > wrote:
> > > > >> > > > >
> > > > >> > > > > > That being said, Charles I am definitely looking forward
> > to
> > > > your
> > > > >> > > report
> > > > >> > > > > of
> > > > >> > > > > > what the upgrade from 0.11 -> 0.12.2-rc1 is like in your
> > > > >> cluster!
> > > > >> > > > > >
> > > > >> > > > > > On Wed, Aug 8, 2018 at 12:30 PM Gian Merlino <
> > > g...@apache.org
> > > > >
> > > > >> > > wrote:
> > > > >> > > > > >
> > > > >> > > > > > > My thought is that recently we have started doing
> small
> > > > >> bug-fix
> > > > >> > > > > releases
> > > > >> > > > > > > more often (0.12.1 and 0.12.2 were both small
> releases)
> > > and
> > > > I
> > > > >> > think
> > > > >> > > > it
> > > > >> > > > > > > makes sense to continue this practice. It makes sense
> to
> > > get
> > > > >> them
> > > > >> > > out
> > > > >> > > > > > > quickly, since shipping bug fixes is good. IMO trying
> to
> > > > >> validate
> > > > >> > > bug
> > > > >> > > > > fix
> > > > >> > > > > > > releases within the customary Apache style 72 hour
> > voting
> > > > >> period
> > > > >> > > is a
> > > > >> > > > > > good
> > > > >> > > > > > > goal.
> > > > >> > > > > > >
> > > > >> > > > > > > On the other hand we do strive to put out high quality
> > > > >> releases,
> > > > >> > > and
> > > > >> > > > we
> > > > >> > > > > > > don't want bug fix releases to introduce regressions.
> > > > Testing
> > > > >> > every
> > > > >> > > > > > single
> > > > >> > > > > > > patch in real clusters is an important part of that.
> > Al

Re: Nightly build!

2018-08-10 Thread Jihoon Son 
Good point. Thanks Gian.
I think we can use https://dist.apache.org/repos/dist/dev/druid/nightly/.
It looks that Nexus repository manager is not available for
https://builds.apache.org/.
So, the Jenkins job would create a nightly build and overwrite the artifact
on https://dist.apache.org/repos/dist/dev/druid/nightly/.
Does it make sense?

Jihoon

On Thu, Aug 9, 2018 at 6:39 PM Gian Merlino  wrote:

> I found http://www.apache.org/legal/release-policy.html#host-rc which
> looks
> like the policy we should follow if we start doing nightly builds. I guess
> we shouldn't archive nightly builds, since there would be too many.
>
> On Thu, Aug 9, 2018 at 6:02 PM Jihoon Son  wrote:
>
> > Hi all,
> >
> > Nightly build would be useful for folks who want to stay on the bleeding
> > edge of Druid. I'm thinking to add a Jenkins job to
> > https://builds.apache.org/ which checks every hour that there are
> changes
> > in the master branch and builds a new build. Once the build succeeds, the
> > binary is archived, so that we can add a link to the binary to our web
> > page. If the build fails, the notification would be sent to
> > comm...@druid.apache.org.
> >
> > Welcome any thoughts.
> >
> > Best,
> > Jihoon
> >
>

Docs for 'master'

2018-08-10 Thread Jihoon Son 
Hi all,

We currently have the following system for docs.

- http://druid.io/docs/{version}: docs for a specific version
- http://druid.io/docs/latest: latest docs. These docs are basically based
on the latest release, but it can contain more recent docs which are not
released yet.

This system sometimes makes people confused because 'latest' can be
interpreted as the 'latest release'. So, I'm proposing a new docs for the
'master' branch which always shows the most recent docs. We might call this
'dev' or something better. Then, we would have the following docs:

- http://druid.io/docs/{version}: docs for a specific version
- http://druid.io/docs/latest: docs for the latest release
- http://druid.io/docs/dev: docs for the master branch

IMO, this system would have some benefits of less confusion as well as
quickly publishing recent documents. Quick document publishing is important
for developers as well as users because we also need to refer documents to
test/improve/develop/review some features.

Welcome any thoughts.

Best,
Jihoon

Re: Docs for 'master'

2018-08-10 Thread Gian Merlino 
That sounds nice, I am all for it. It should ideally be automated, maybe
with Jenkins.

On Fri, Aug 10, 2018 at 5:13 PM Jihoon Son  wrote:

> Hi all,
>
> We currently have the following system for docs.
>
> - http://druid.io/docs/{version}: docs for a specific version
> - http://druid.io/docs/latest: latest docs. These docs are basically based
> on the latest release, but it can contain more recent docs which are not
> released yet.
>
> This system sometimes makes people confused because 'latest' can be
> interpreted as the 'latest release'. So, I'm proposing a new docs for the
> 'master' branch which always shows the most recent docs. We might call this
> 'dev' or something better. Then, we would have the following docs:
>
> - http://druid.io/docs/{version}: docs for a specific version
> - http://druid.io/docs/latest: docs for the latest release
> - http://druid.io/docs/dev: docs for the master branch
>
> IMO, this system would have some benefits of less confusion as well as
> quickly publishing recent documents. Quick document publishing is important
> for developers as well as users because we also need to refer documents to
> test/improve/develop/review some features.
>
> Welcome any thoughts.
>
> Best,
> Jihoon
>