Re: [DISCUSS] Dropping support for Java 8

[Clint Wylie]

+1 from me for deprecating first then removing. I'd also like to
officially support java 21 before we totally drop support, at least
experimentally, but preferably fully. We already run unit tests with
21, so maybe we could transition the java 8 integration tests to use
21 instead? I've also been using 21 for all of my debugging and
testing for quite some time now and it seems fine to me.

I know this isn't strictly related, but with 8 still supported maybe
it just seems like we are kind of slow and cautious, but if we drop 8,
it seems like our java support is in a strange place of moderately old
versions if we only officially support 11 and 17, given 21 is also an
LTS (even 11 is starting to seem a bit old to me).

On Tue, Jul 16, 2024 at 9:21 AM Gian Merlino  wrote:
>
> I think this is a good move. Let's give users some warning by deprecating it 
> first prior to removal. IMO, good timing would be to deprecate Java 8 in the 
> next major Druid release (Druid 31). That means a doc update, release note 
> update, and updating the start scripts to log a warning that support for Java 
> 8 will be removed soon (if Java 8 is detected).
>
> When we do remove support for Java 8, we should update the "verify-java" 
> script to require DRUID_SKIP_JAVA_CHECK=1 when Java 8 is detected.
>
> Gian
>
> On 2024/07/16 04:17:06 Abhishek Agarwal wrote:
> > Hello everyone,
> > Starting this thread to discuss, if and when, we can drop Java 8 support.
> > We have been fully supporting Java 11 and Java 17 for a while now. Anyone,
> > who is looking to upgrade Druid, can safely select either of these LTS Java
> > runtimes. There are a few important reasons to drop Java 8 support
> >
> > - It adds extra burden on build/test pipelines to test all these different
> > runtimes. We want to shrink this matrix of Java runtime and test suites.
> > - Being on Java 8 will block us from upgrading dependencies that have
> > dropped Java 8 support. We can get around it by building profiles and shims
> > but it adds more complexity. One example is pac4j which is Java 11 based
> > from 5.x.
> > - As we drop support for older Java releases, developers can use the
> > features offered by the more advanced Java versions.
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> For additional commands, e-mail: dev-h...@druid.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [VOTE] Release Apache Druid 29.0.0 [RC1]

[Clint Wylie]

+1 (binding)

src package:
- verified checksum and signature
- LICENSE and NOTICE present
- rat check passed
- built binary distribution, ran MSQ ingestion, ran some queries

binary package:
- verified checksum and signature
- LICENSE and NOTICE present
- ran native ingestion, ran some queries

docker:
- verified checksum
- ran docker-compose cluster, msq ingestion, some queries

On Wed, Feb 14, 2024 at 7:27 AM Kashif Faraz  wrote:
>
> +1 (binding)
>
> src package:
> - verified checksum and signature
> - checked for LICENSE and NOTICE
> - rat check passed
> - built binary distribution
> - ingested sample wikipedia dataset, ran some queries, reindexed with range
> partitioning
>
> binary package:
> - verified checksum and signature
> - ingested sample wikipedia dataset, ran some queries, reindexed with range
> partitioning
>
> docker:
> - verified checksum
> - ran docker-compose cluster, some queries
>
> On Tue, Feb 13, 2024 at 6:04 PM Laksh Singla  wrote:
>
> > Hi all,
> >
> > I have created a build for Apache Druid 29.0.0, release
> > candidate 1.
> >
> > Thanks to everyone who has helped contribute to the release! You can read
> > the proposed release notes here:
> > https://github.com/apache/druid/issues/15896
> >
> > The release candidate has been tagged in GitHub as
> > druid-29.0.0-rc1 (869bd3978f0c835ef8eb7c1f25c468e23472a81b),
> > available here:
> > https://github.com/apache/druid/tree/druid-29.0.0-rc1
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/druid/29.0.0-rc1/
> >
> > A staged Maven repository is available for review at:
> > https://repository.apache.org/content/repositories/orgapachedruid-1061/
> >
> > Staged druid.apache.org website documentation is available here:
> > https://druid.staged.apache.org/docs/29.0.0/design/
> >
> > A Docker image containing the binary of the release candidate can be
> > retrieved via:
> > docker pull apache/druid:29.0.0-rc1
> >
> > artifact checksums
> > src:
> >
> > 1948fab4500f3571591f887a638631b6a05f040b88b35004406ca852e16884e5d2269a74e8d07c79c477d8b02a2489efec85776f8ef46f4e8defedce4efb9931
> > bin:
> >
> > f2a11ddc71b59a648d01d7c220a6fae527c34d702d4f1e7aed954803a3576543c4cd7149f34e8948eccf89240435f09a8512db83b46991ac6e354eeba8cbada4
> > docker: 2bddcf692f2137dc4094908b63d2043423fca895ba0f479f0db34ec8016c4472
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/lakshsingla.asc
> >
> > This key and the key of other committers can also be found in the project's
> > KEYS file here:
> > https://dist.apache.org/repos/dist/release/druid/KEYS
> >
> > (If you are a committer, please feel free to add your own key to that file
> > by following the instructions in the file's header.)
> >
> >
> > Verify checksums:
> > diff <(shasum -a512 apache-druid-29.0.0-src.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-29.0.0-src.tar.gz.sha512 ; echo)
> >
> > diff <(shasum -a512 apache-druid-29.0.0-bin.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-29.0.0-bin.tar.gz.sha512 ; echo)
> >
> > Verify signatures:
> > gpg --verify apache-druid-29.0.0-src.tar.gz.asc \
> > apache-druid-29.0.0-src.tar.gz
> >
> > gpg --verify apache-druid-29.0.0-bin.tar.gz.asc \
> > apache-druid-29.0.0-bin.tar.gz
> >
> > Please review the proposed artifacts and vote. Note that Apache has
> > specific requirements that must be met before +1 binding votes can be cast
> > by PMC members. Please refer to the policy at
> > http://www.apache.org/legal/release-policy.html#policy for more details.
> >
> > As part of the validation process, the release artifacts can be generated
> > from source by running:
> > mvn clean install -Papache-release,dist -Dgpg.skip
> >
> > The RAT license check can be run from source by:
> > mvn apache-rat:check -Prat
> >
> > This vote will be open for at least 72 hours. The vote will pass if a
> > majority of at least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Druid 29.0.0
> > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Thanks!
> >

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [Discuss] deprecating and eventually removing non-sql compatible configs

[Clint Wylie]

>Are there any performance implications in making those the defaults?
For numeric columns without null values there should be no impact. For
numeric columns with null values, I believe there is currently a very
minor performance impact for scans/aggregations since the null value
bitmap is iterated to check which rows are null. String performance
should not be changed afaik. I will try to find some time to do some
benchmarking to ensure this is true and measure the stuff to confirm
this, and look for opportunities to improve things where possible in
order to make this happen, since performance is important. Longer
term, I have a hunch that if we actually delete all of the default
value mode code there might also be some performance improvements due
to smaller function bytecode sizes and the lack of 'if sql compatible
mode do this, else do this' being littered everywhere across the
codebase.

>Would this break compatibility for any existing native queries?
There are a handful of query shapes which might have some differences,
the most common are probably with strings since in default value mode
the empty string and null are treated interchangeably, so these kinds
of queries will need to be modified.

There are also the possibility for some subtle differences in some
numeric expressions due to the 'pretend nulls do not exist' behavior
of the default value mode config, in default value mode you can write
arguably nonsense expressions that end up evaluating stuff like "3 +
'hello'" and it will evaluate to 3, while in SQL compatible mode the
'hello' will be unable to be implicitly cast to a number and become
zero, resulting in null.

Strict boolean mode also has some subtle differences if selecting the
outputs, since it homogenizes all true/false values to 1/0 instead of
the javascript style behavior when it is off, where the truthy values
are just passed through.
https://druid.apache.org/docs/latest/querying/math-expr#logical-operator-modes

The other main change in behavior is due to the SQL three-value logic,
where a filter like "x != 'hello'" today will in default value mode
match rows with null values, but in SQL compatible mode will not match
nulls. These queries will need to be rewritten into the form "x !=
'hello' OR x is null", or there is also some native filters added
recently which could be used to wrap to make the SQL equivalent of "(x
= 'hello') is not true", https://github.com/apache/druid/pull/15182,
or "not(istrue(eq(x, 'hello'))". I'm actually open to the idea of
keeping the toggle config for 2vl/3vl since it would need checked in
very few places to control the behavior (basically in not filter), my
main goal here is to eventually get rid of
druid.generic.useDefaultValueForNull.

I plan to dig into all of this in greater depth in the migration guide
mentioned in the first email to make the transition as painless as
possible before we remove it.

On Mon, Jan 22, 2024 at 9:54 AM Xavier Léauté
 wrote:
>
> Two questions:
> - Are there any performance implications in making those the defaults?
> - Would this break compatibility for any existing native queries?
>
> On Wed, Jan 17, 2024 at 5:53 PM Clint Wylie  wrote:
>
> > Hi all,
> >
> > I wanted to discuss the deprecation and removal of the Druid configs
> > related to SQL compatibility, with the eventual goal of always running
> > in SQL compatible mode. As of Druid 28, these are the default, but in
> > the interest of dramatically reducing complexity and removing a ton of
> > code, and also cutting a lot of our CI time in half, I would
> > eventually like to remove the related configs and the code that
> > handles the now non-default behaviors completely.
> >
> > The related configs are:
> >
> > runtime.properties:
> > druid.generic.useDefaultValueForNull  - (will become always false)
> > druid.expressions.useStrictBooleans - (will become always true)
> > druid.generic.useThreeValueLogicForNativeFilters - (will become always
> > true)
> > druid.generic.ignoreNullsForStringCardinality - (irrelevant if
> > druid.generic.useDefaultValueForNull=false)
> >
> > query context:
> > sqlUseBoundAndSelectors - (this is moderately related, and defaults to
> > value of druid.generic.useDefaultValueForNull, but enhancements to
> > expressions and sql planning for lookups make this totally unnecessary
> > to keep around)
> >
> > other things to dump while we are at it:
> > druid.expressions.allowNestedArrays - (will become always true)
> >
> > There might be additional configs which can also be removed, we can
> > add to this thread if we can think of them.
> >
> > I would like to get the official deprec

[Discuss] deprecating and eventually removing non-sql compatible configs

[Clint Wylie]

Hi all,

I wanted to discuss the deprecation and removal of the Druid configs
related to SQL compatibility, with the eventual goal of always running
in SQL compatible mode. As of Druid 28, these are the default, but in
the interest of dramatically reducing complexity and removing a ton of
code, and also cutting a lot of our CI time in half, I would
eventually like to remove the related configs and the code that
handles the now non-default behaviors completely.

The related configs are:

runtime.properties:
druid.generic.useDefaultValueForNull  - (will become always false)
druid.expressions.useStrictBooleans - (will become always true)
druid.generic.useThreeValueLogicForNativeFilters - (will become always true)
druid.generic.ignoreNullsForStringCardinality - (irrelevant if
druid.generic.useDefaultValueForNull=false)

query context:
sqlUseBoundAndSelectors - (this is moderately related, and defaults to
value of druid.generic.useDefaultValueForNull, but enhancements to
expressions and sql planning for lookups make this totally unnecessary
to keep around)

other things to dump while we are at it:
druid.expressions.allowNestedArrays - (will become always true)

There might be additional configs which can also be removed, we can
add to this thread if we can think of them.

I would like to get the official deprecation process started now with
Druid 29, and imagine actually removing them sometime towards the end
of the year, so maybe Druid 32 or so?

Before completely removing them I think I would like to get a more in
depth migration guide in place, to help any hold-outs that are
overriding the now default SQL compatible configs so that things still
run in the legacy mode.

Thoughts? Concerns?

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [VOTE] Release Apache Druid 28.0.1 [RC1]

[Clint Wylie]

+1 (binding)

src package:
- verified checksum and signature
- check for LICENSE and NOTICE
- rat check passed
- built binary distribution, ran MSQ quickstart, ran some queries,
tested result cache issue

binary package:
- verified checksum and signature
- check for LICENSE and NOTICE
- ran native quickstart, ran some queries, tested result cache issue,
tested null filter issue

docker:
- verified checksum
- ran docker-compose cluster, some queries, tested result cache issue

On Sun, Dec 10, 2023 at 9:46 PM Laksh Singla  wrote:
>
> Hi all,
>
> I have created a build for Apache Druid 28.0.1, release
> candidate 1.
>
> Thanks to everyone who has helped contribute to the release! You can read
> the proposed release notes here:
> https://github.com/apache/druid/issues/15535
>
> The release candidate has been tagged in GitHub as
> druid-28.0.1-rc1 (266d5a5f1d76f6d33e2d1d2fa52156dfb3cd87a7),
> available here:
> https://github.com/apache/druid/releases/tag/druid-28.0.1-rc1
>
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/druid/28.0.1-rc1/
>
> A staged Maven repository is available for review at:
> https://repository.apache.org/content/repositories/orgapachedruid-1057/
>
> Staged druid.apache.org website documentation is available here:
> https://druid.staged.apache.org/docs/28.0.0/design/index.html
> Release and upgrade notes will be added to the site as well, however the
> patch isn't merged yet, therefore the site isn't updated with the new paths
> and is the same as the 28.0.0 staged site. The staged website along with
> the release website will be updated with the correct version numbers and
> path along with the release.
>
> A Docker image containing the binary of the release candidate can be
> retrieved via:
> docker pull apache/druid:28.0.1-rc1
>
> artifact checksums
> src:
> 557ca5131c72625f5e63b40ad83a6cce27c4209624a95e6be326fe9c5bac01cbafe8bba14bd46e058f237ae8b5d1c2129cd9911ce31476487d36a3b7c988b626
> bin:
> 923336b96fa9e72bb0a2799158e232f0b336841c60561f0da5e2fdcbc22f592f1d847118fa79ddfab1b02f6626a2d5c7b7f54f11eec1f8187c2e66918f3e86f3
> docker:
> aebb7e18883f47df4f272e36601d9daa76e91fc3a89f781ca93d2d556cc8cd1d
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/lakshsingla.asc
>
> This key and the key of other committers can also be found in the project's
> KEYS file here:
> https://dist.apache.org/repos/dist/release/druid/KEYS
>
> (If you are a committer, please feel free to add your own key to that file
> by following the instructions in the file's header.)
>
>
> Verify checksums:
> diff <(shasum -a512 apache-druid-28.0.1-src.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-28.0.1-src.tar.gz.sha512 ; echo)
>
> diff <(shasum -a512 apache-druid-28.0.1-bin.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-28.0.1-bin.tar.gz.sha512 ; echo)
>
> Verify signatures:
> gpg --verify apache-druid-28.0.1-src.tar.gz.asc \
> apache-druid-28.0.1-src.tar.gz
>
> gpg --verify apache-druid-28.0.1-bin.tar.gz.asc \
> apache-druid-28.0.1-bin.tar.gz
>
> Please review the proposed artifacts and vote. Note that Apache has
> specific requirements that must be met before +1 binding votes can be cast
> by PMC members. Please refer to the policy at
> http://www.apache.org/legal/release-policy.html#policy for more details.
>
> As part of the validation process, the release artifacts can be generated
> from source by running:
> mvn clean install -Papache-release,dist -Dgpg.skip
>
> The RAT license check can be run from source by:
> mvn apache-rat:check -Prat
>
> This vote will be open for at least 72 hours. The vote will pass if a
> majority of at least three +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Druid 28.0.1
> [ ] 0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
> Thanks!

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

on removing 'auto' strategy from native search query

[Clint Wylie]

Hi all, just wanted to start a thread to discuss removing the 'auto'
strategy from the native search query, which is the only thing that
uses the 'estimateSelectivity' method of
https://github.com/apache/druid/blob/master/processing/src/main/java/org/apache/druid/segment/index/BitmapColumnIndex.java#L33,
which is what I would actually like to remove to make implementing
index suppliers a bit easier and also tidy some stuff up for some
additional changes I would like to make.

The 'auto' strategy uses these selectivity estimation methods to try
to determine if it should actually use the indexes or not when
performing a search query, however I've been making some improvements
on making index usage itself automatically determine if it should use
indexes or value matchers, which I think is the replacement. The first
part is the changes in https://github.com/apache/druid/pull/13977,
which currently only applies to 'auto' and 'json' columns, but the
strategy could easily be applied to traditional string columns as
well, and the future work I'd like to do would allow cursor creation
to adaptively skip computing the remaining bitmaps of an AND filter
once the intersection is selective enough to make further operations
not worth the cost.

Once these changes are in place, the 'useIndexes' strategy effectively
becomes the 'auto' strategy because all index evaluation is automatic,
and also much cheaper than the 'auto' strategy which currently has to
repeat some work if it actually does decide to use indexes.

I suppose the part worth discussing is if this 'auto' strategy is used
enough for search queries that we need the complete replacement in
place before we can remove it, or if I can just go ahead and remove it
now, or if I should wire up the partial improvements used today by
'auto' and 'json' columns to regular string columns so at least a
partial improvement is available. I think it would make it a bit
easier for me to finish the remaining refactor for the improvements i
have in mind if i can get this method out of my way entirely, but I
can also probably work around it if necessary (e.g. if anyone is
depending on it), so removing it probably isn't a blocker for
finishing this work.

Thoughts?

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [VOTE] Release Apache Druid 28.0.0 [RC1]

[Clint Wylie]

+1 (binding)

src package:
- verified checksum and signature
- check for LICENSE and NOTICE
- rat check passed
- built binary distribution, ran MSQ quickstart, tested some new
features and ran queries

binary package:
- verified checksum and signature
- check for LICENSE and NOTICE
- tested some new features and ran queries

docker:
- verified checksum
- ran docker-compose cluster, got a failure when forking task when
using 'Use Rosetta for x86/amd64 emulation on Apple Silicon', but
queries work fine. I don't recall having trouble with this on previous
releases, maybe related to the Java version update on docker image? I
don't feel like this needs to be a blocker, but we should make people
aware probably
- ran docker-compose cluster without 'rosetta' emulation, stuff really
slow but tasks did complete successfully at least

On Sat, Nov 11, 2023 at 10:09 PM Karan Kumar  wrote:
>
> + 1 Binding
>
> src package:
> - verified signature/checksum
> - LICENSE/NOTICE present
> - built binary distribution,
>  - Loaded all sample MSQ queries
> queries
>  - Tested query from deep storage
>
> binary package:
> - verified signature/checksum
> - LICENSE/NOTICE present
> - ran single server quick start
>  - Loaded all sample MSQ queries
> queries
>  - Tested query from deep storage
>
> docker:
> - verified checksum
>
>
>
>
>
>
>
>
> On Fri, Nov 10, 2023 at 1:30 PM Kashif Faraz  wrote:
>
> > +1 Binding
> >
> > Verified artifacts:
> >
> > src:
> > - Verified signature/checksum
> > - LICENSE/NOTICE present
> > - Built binary distribution
> > - Ingested sample "wikipedia" dataset using native batch
> > - Ran Apache Rat license check successfully
> >
> > binary:
> > - Verified signature/checksum
> > - Built binary distribution
> > - Ingested sample "wikipedia" dataset using native batch, range
> > partitioning
> >
> > docker image:
> > - Verified checksum
> > - Started cluster with docker-compose
> > - Ingested sample "wikipedia" dataset using native batch
> >
> >
> > On Thu, Nov 9, 2023 at 6:36 PM Abhishek Agarwal 
> > wrote:
> >
> > > +1 Binding
> > >
> > > src package:
> > > - verified signature/checksum
> > > - LICENSE/NOTICE present
> > > - built binary distribution,
> > >  - Loaded example koalas-to-the-max nested dataset using MSQ and ran
> > > some
> > > queries
> > >  - Tested Kafka ingestion locally
> > >
> > > binary package:
> > > - verified signature/checksum
> > > - LICENSE/NOTICE present
> > > - built binary distribution,
> > >  - Loaded example koalas-to-the-max nested dataset using MSQ and ran
> > > some
> > > queries
> > >  - Tested Kafka ingestion locally
> > >
> > > docker:
> > > - verified checksum
> > > - started cluster with docker-compose, loaded example koalas-to-the-max
> > > nested dataset using MSQ and ran some
> > > queries
> > > - Added the kafka extension to the environment, started cluster with
> > > docker-compose, and then tested kafka ingestion
> > >
> > > On Tue, Nov 7, 2023 at 11:08 AM Laksh Singla 
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > I have created a build for Apache Druid 28.0.0, release
> > > > candidate 1.
> > > >
> > > > Thanks to everyone who has helped contribute to the release! You can
> > read
> > > > the proposed release notes here:
> > > > https://github.com/apache/druid/issues/15326
> > > >
> > > > The release candidate has been tagged in GitHub as
> > > > druid-28.0.0-rc1 (c91d61dc3f7232242c00a6334ea5d0d9cd98e32c),
> > > > available here:
> > > > https://github.com/apache/druid/releases/tag/druid-28.0.0-rc1
> > > >
> > > > The artifacts to be voted on are located here:
> > > > https://dist.apache.org/repos/dist/dev/druid/28.0.0-rc1/
> > > >
> > > > A staged Maven repository is available for review at:
> > > >
> > https://repository.apache.org/content/repositories/orgapachedruid-1056/
> > > >
> > > > Staged druid.apache.org website documentation is available here:
> > > > https://druid.staged.apache.org/docs/28.0.0/design/index.html
> > > >
> > > > A Docker image containing the binary of the release candidate can be
> > > > retrieved via:
> > > > docker pull apache/druid:28.0.0-rc1
> > > >
> > > > artifact checksums
> > > > src:
> > > >
> > > >
> > >
> > f648143c29610f6874c5946897712bce967a4b9d5304b953c9efc14e514a7ee1269fcac626165544dd485560517fc52ef85ddd2617d3410fb0478e10f3ff2982
> > > > bin:
> > > >
> > > >
> > >
> > c971ccbb4220fa2ae9f8a0a0f82c3e4b87750d27ac15d863d1e9f415863ccb8b8e5c764122dc8ef76044a42d27e1108b980ca2b88de8aff660db5a22356d6ab7
> > > > docker:
> > 708f1bd7221734986d049e1b5bd6ad0c985c65d815037f0650dd5ace8654d38c
> > > >
> > > > Release artifacts are signed with the following key:
> > > > https://people.apache.org/keys/committer/lakshsingla.asc
> > > >
> > > > This key and the key of other committers can also be found in the
> > > project's
> > > > KEYS file here:
> > > > https://dist.apache.org/repos/dist/release/druid/KEYS
> > > >
> > > > (If you are a committer, please feel free to add your own key to that
> > > f

Re: [VOTE] Release Apache Druid 27.0.0 [RC1]

[Clint Wylie]

+1 (binding)

src:
- verified LICENSE/NOTICE present
- verified signature/checksum
- RAT check passed
- built binary distribution, ingested some data, ran some queries

bin:
- verified LICENSE/NOTICE present
- verified signature/checksum
- ingested some data, ran some queries

docker:
- verified checksum
- ingested some data, ran some queries

On Tue, Aug 8, 2023 at 6:22 AM Karan Kumar  wrote:
>
> +1 (binding)
>
> src package:
>
>- verified signature/checksum
>- Build druid on (m1 based chipset)
>- Ran druid cluster and tested
>   - All MSQ demo q's
>   - Tested query from deep storage with results written out to s3.
>   - Ran q's against segment's not loaded on the historicals
>   - Ran q's against segments loaded on the historicals.
>
>
> binary package:
>
>- verified signature/checksum
>- LICENSE/NOTICE present
>- Ran druid cluster and tested
>   - All MSQ demo q's
>   - Tested query from deep storage with results written out to s3.
>   - Ran q's against segment's not loaded on the historicals
>   - Ran q's against segments loaded on the historicals.
>
>
>
> docker :
>
>- verified checksum
>
>
>
> On Mon, Aug 7, 2023 at 11:23 AM Abhishek Agarwal 
> wrote:
>
> > +1 (binding)
> >
> > src package:
> > - verified signature/checksum
> > - LICENSE/NOTICE present
> > - built binary distribution,
> >  - Loaded example Wikipedia dataset using MSQ and ran some
> > queries
> >  - Tested Kafka ingestion locally
> >
> > binary package:
> > - verified signature/checksum
> > - LICENSE/NOTICE present
> > - built binary distribution,
> >  - Loaded example wikipedia dataset using MSQ and ran some
> > queries
> >  - Tested Kafka ingestion locally
> >
> > docker:
> > - verified checksum
> > - started cluster with docker-compose, Loaded example Wikipedia dataset and
> > ran some queries
> > - Added the kafka extension to the environment, started cluster with
> > docker-compose, and then tested kafka ingestion
> >
> > On Sun, Aug 6, 2023 at 1:43 PM Amatya Avadhanula 
> > wrote:
> >
> > > Hi all,
> > >
> > > I have created a build for Apache Druid 27.0.0, release
> > > candidate 1.
> > >
> > > Thanks to everyone who has helped contribute to the release! You can read
> > > the proposed release notes here:
> > > https://github.com/apache/druid/issues/14761
> > >
> > > The release candidate has been tagged in GitHub as
> > > druid-27.0.0-rc1
> > > available here:
> > > https://github.com/apache/druid/tree/druid-27.0.0-rc1
> > >
> > > The artifacts to be voted on are located here:
> > > https://dist.apache.org/repos/dist/dev/druid/27.0.0-rc1/
> > >
> > > A staged Maven repository is available for review at:
> > > https://repository.apache.org/content/repositories/orgapachedruid-1044/
> > >
> > > Staged druid.apache.org website documentation is available here:
> > > https://druid.staged.apache.org/docs/27.0.0/design/index.html
> > >
> > > A Docker image containing the binary of the release candidate can be
> > > retrieved via:
> > > docker pull apache/druid:27.0.0-rc1
> > >
> > > artifact checksums
> > > src:
> > >
> > >
> > a3a755d02e2ed55a125ba562de4b4ce467d27af1132a89ecbc73cbb4a38622f7534813267ed44de1ec0fb85227e93d04c1c1ce24959d6d5c41dedbf2d7c6e4ed
> > > bin:
> > >
> > >
> > b840ed0d77b1e5c11e058b161a56a469b0916febc1d478bf7ad2517cf79d2b724ece5cef21d96581122ba0958a5737f41e76496db3b10db5bfb4ab3123e4091b
> > > docker: ca3df175bc944033c7c56ccf9499c05e2090ae6cefbdcd90095cfce2b7931ead
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/amatya.asc [To be available
> > > within
> > > 48 hours: https://issues.apache.org/jira/browse/INFRA-24865]
> > >
> > > This key and the key of other committers can also be found in the
> > project's
> > > KEYS file here:
> > > https://dist.apache.org/repos/dist/release/druid/KEYS
> > >
> > > (If you are a committer, please feel free to add your own key to that
> > file
> > > by following the instructions in the file's header.)
> > >
> > >
> > > Verify checksums:
> > > diff <(shasum -a512 apache-druid-27.0.0-src.tar.gz | \
> > > cut -d ' ' -f1) \
> > > <(cat apache-druid-27.0.0-src.tar.gz.sha512 ; echo)
> > >
> > > diff <(shasum -a512 apache-druid-27.0.0-bin.tar.gz | \
> > > cut -d ' ' -f1) \
> > > <(cat apache-druid-27.0.0-bin.tar.gz.sha512 ; echo)
> > >
> > > Verify signatures:
> > > gpg --verify apache-druid-27.0.0-src.tar.gz.asc \
> > > apache-druid-27.0.0-src.tar.gz
> > >
> > > gpg --verify apache-druid-27.0.0-bin.tar.gz.asc \
> > > apache-druid-27.0.0-bin.tar.gz
> > >
> > > Please review the proposed artifacts and vote. Note that Apache has
> > > specific requirements that must be met before +1 binding votes can be
> > cast
> > > by PMC members. Please refer to the policy at
> > > http://www.apache.org/legal/release-policy.html#policy for more details.
> > >
> > > As part of the validation process, the release artifacts can be generated
> > > from source by runni

group-by v1

[Clint Wylie]

Is anyone opposed to removing group-by v1? I think it would allow us
to simplify quite a lot of stuff. While it would very nice to
implement 'growable' buffer aggregators so that the v2 algorithm could
be a bit more flexible and finally cover the only potential reason I
can imagine people might still be using v1, I don't think that this
needs to block removal.

So, is anyone out there still using group-by v1 and would be sad by it
going away?

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [DISCUSS] Druid 28 dropping support for Hadoop 2

[Clint Wylie]

obvious +1 from me

On Tue, Jun 27, 2023 at 11:42 PM Gian Merlino  wrote:
>
> I'd like to propose dropping support for Hadoop 2 in Druid 28. Not the very
> next release (which I assume will be Druid 27) but the one after that,
> likely late 2023 timeframe.
>
> In 2021, we had a discussion about moving away from Hadoop 2:
> https://lists.apache.org/thread/zmc389trnkh6x444so8mdb2h0x0noqq4. For
> various reasons, it didn't seem like the right time. However, I believe now
> is the right time:
>
> 1) We didn't support Hadoop 3 in 2021, but we support it now. There is now
> a Hadoop 3 build profile, as well as convenience binaries on
> https://druid.apache.org/downloads.html.
>
> 2) We have SQL-based ingest with MSQ tasks, which provides a built-in /
> scalable / robust alternative to using Hadoop at all.
>
> 3) It has been an additional two years. Hadoop 2 is that much older, that
> much more time has passed since it was superseded by Hadoop 3, and people
> have had that much more time to migrate.
>
> 4) The original main reason for wanting to move away from Hadoop 2 is still
> relevant. It keeps us on various old dependencies, including an ancient
> version of Guava, which in turn has been keeping us on an ancient version
> of Calcite. The Calcite community has graciously decided to support this
> old version of Guava for at least one release, but plans to drop support by
> Calcite 1.36, leaving us back in the same position. Managing this situation
> is time-consuming for both Druid and Calcite maintainers.
>
> 5) Other solutions beyond dropping Hadoop 2 support were proposed in 2021,
> such as reworking Hadoop support to be purely extension based, and
> reworking extensions to be more isolated from each other. However, these
> are both substantially more complex than dropping support, and in the two
> years since the original thread, these more complex solutions have not been
> implemented. So, I think we need to move on with the simpler solution of
> dropping support.
>
> Gian

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: 26.0.1?

[Clint Wylie]

rad, thanks!


On Tue, Jun 20, 2023 at 11:57 PM Amatya Avadhanula  wrote:
>
> Hi
>
> I would like to volunteer to be the release manager for 26.0.1
>
> On 2023/06/21 06:21:07 Clint Wylie wrote:
> > Hi all,
> >
> > We found a couple of bugs related to the Kafka input format which I
> > think are worth making a small 26.0.1 patch release. This is mainly
> > due to the fact that we modified the web-console data loader to always
> > use the Kafka input format in 26.0.0 instead of just the payload
> > format, increasing the likelihood of actually hitting these issues.
> >
> > The fixes are https://github.com/apache/druid/pull/14413 and
> > https://github.com/apache/druid/pull/14421, but I think a couple of
> > other 'auto' column fixes should also be added (since it was new in
> > 26), along with any other minor bugs we think are worth fixing sooner
> > than the 27 release.
> >
> > Anyone have any other bug fixes they would like to get into a 26.0.1,
> > or opinions for or against making this release?
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> > For additional commands, e-mail: dev-h...@druid.apache.org
> >
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> For additional commands, e-mail: dev-h...@druid.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

26.0.1?

[Clint Wylie]

Hi all,

We found a couple of bugs related to the Kafka input format which I
think are worth making a small 26.0.1 patch release. This is mainly
due to the fact that we modified the web-console data loader to always
use the Kafka input format in 26.0.0 instead of just the payload
format, increasing the likelihood of actually hitting these issues.

The fixes are https://github.com/apache/druid/pull/14413 and
https://github.com/apache/druid/pull/14421, but I think a couple of
other 'auto' column fixes should also be added (since it was new in
26), along with any other minor bugs we think are worth fixing sooner
than the 27 release.

Anyone have any other bug fixes they would like to get into a 26.0.1,
or opinions for or against making this release?

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Requirements for relaxing restrictions on github actions usage

[Clint Wylie]

+1, afaict we have been following these guidelines already without knowing it

On Thu, Jun 1, 2023 at 8:48 AM Atul Mohan  wrote:
>
> +1
> The requirements called out in the policy doc seem reasonable to me.
>
> Thanks for driving this effort!
>
> On Wed, May 31, 2023 at 1:10 AM Abhishek Agarwal 
> wrote:
>
> > Hello,
> > I raised an INFRA ticket (
> > https://issues.apache.org/jira/browse/INFRA-24657)
> > for the druid project so the contributors don't need a committer to trigger
> > PR build/test. Infra has agreed to relax the restrictions enough that a
> > contributor will need the approval only for their first contribution.
> >
> > However, as a project, we need to follow certain requirements that are
> > called out here - https://infra.apache.org/github-actions-policy.html
> >
> > They all seem fine to me. We are using `pull_request_target` for the
> > labeler action but that action doesn't export any confidential variables.
> > If others agree as well, I will just link this thread to the INFRA ticket.
> >
> > As a follow-up item, I can add a README.md in .github folder that warns
> > contributors and committers to keep these requirements in mind as they
> > change GitHub workflows in future.
> >

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

[ANNOUNCE] Apache Druid 26.0.0 release

[Clint Wylie]

The Apache Druid team is proud to announce the release of Apache Druid 26.0.0.
Druid is a high performance analytics data store for event-driven data.

Apache Druid 26.0.0 contains over 390 new features, performance
enhancements, bug fixes, and major documentation improvements from 65
contributors. Major new features and improvements include:

- Schema auto-discovery and auto type column schema (experimental)
- UNNEST for arrays (experimental)
- Sort-merge join and hash shuffle join for MSQ
- Storage improvements on dictionary compression

Source and binary distributions can be downloaded from:
https://druid.apache.org/downloads.html

Release notes are at:
https://github.com/apache/druid/releases/tag/druid-26.0.0

A big thank you to all the contributors in this milestone release!

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

[RESULT][VOTE] Release Apache Druid 26.0.0 [RC3]

[Clint Wylie]

Thanks to everyone who participated in the vote! The results are as follows:

Vadim Ogievetsky: +1 (binding)
Suneet Saldanha: +1 (binding)
Clint Wylie: +1 (binding)

The vote has passed with 3 binding +1s.

Thanks
Clint

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [VOTE] Release Apache Druid 26.0.0 [RC3]

[Clint Wylie]

Thanks for the votes! counting me, we have enough to call it passed,
will start the result thread soon.

On Mon, May 22, 2023 at 5:02 PM Suneet Saldanha  wrote:
>
> +1 (binding)
>
> src:
> - verified LICENSE / NOTICE present
> - verified signature / checksum
> - RAT check passed
> - built binary, ingested wikipedia data, ran some queries
>
> bin:
> - verified LICENSE / NOTICE present
> - verified signature / checksum
> - Ran through jupyter notebook tutorial
> - Ingested data in the kafka tutorial
>
> bin (hadoop3):
> - verified LICENSE / NOTICE present
> - verified signature / checksum
>
> docker:
> - Pulled and ran locally
> - Ingested data, ran queries
> - Running the docker image on an M1 mac was extremely slow, but it was
> quick on older hardware.
>
> On 2023/05/19 19:28:05 Clint Wylie wrote:
> > Hi all,
> >
> > I have created a build for Apache Druid 26.0.0, release
> > candidate 3.
> >
> > Thanks to everyone who has helped contribute to the release! You can read
> > the proposed release notes here:
> > https://github.com/apache/druid/issues/14064
> >
> > The release candidate has been tagged in GitHub as
> > druid-26.0.0-rc3 (727c61943381a82e94a0d6c6409f505dcfda11c1),
> > available here:
> > https://github.com/apache/druid/releases/tag/druid-26.0.0-rc3
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/druid/26.0.0-rc3/
> >
> > A staged Maven repository is available for review at:
> > https://repository.apache.org/content/repositories/orgapachedruid-1042/
> >
> > Staged druid.apache.org website documentation is available here:
> > https://druid.staged.apache.org/docs/26.0.0/design/index.html
> >
> > A Docker image containing the binary of the release candidate can be
> > retrieved via:
> > docker pull apache/druid:26.0.0-rc3
> >
> > artifact checksums
> > src:
> > 67f572acf2cdfbf3affdeb665fefb8d91bec0650ad54e1c3fb3d1eedb15932fa33b612bf870676a76a1a2aa0c57f7cee99bec8eeec33de975326d0d6923ac0b4
> > bin:
> > b8d2c982c8fcc0abadecb571727a3e8f69a6f79ee315fbe9628c3cffc7f92df7437d17b80c0e5234222bb55b265591d19081cbdcda0ab37563805fecf3e4
> > hadoop3-bin:
> > 0dc1bfbd90ff4467518c203375278a1cfa0ef1ebceea5a3359615aebd06e70d1af121ddd8590eeae960e5d286e5c317cd4b571bcf94c8c25458e4ae92d6aafab
> > docker:
> > 9a3a07819058693b29bb5fc145e071c3b3a1f551960640529a60ad5198f29b59
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/cwylie.asc
> >
> > This key and the key of other committers can also be found in the project's
> > KEYS file here:
> > https://dist.apache.org/repos/dist/release/druid/KEYS
> >
> > (If you are a committer, please feel free to add your own key to that file
> > by following the instructions in the file's header.)
> >
> >
> > Verify checksums:
> > diff <(shasum -a512 apache-druid-26.0.0-src.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-26.0.0-src.tar.gz.sha512 ; echo)
> >
> > diff <(shasum -a512 apache-druid-26.0.0-bin.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-26.0.0-bin.tar.gz.sha512 ; echo)
> >
> > diff <(shasum -a512 apache-druid-26.0.0-hadoop3-bin.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-26.0.0-hadoop3-bin.tar.gz.sha512 ; echo)
> >
> > Verify signatures:
> > gpg --verify apache-druid-26.0.0-src.tar.gz.asc \
> > apache-druid-26.0.0-src.tar.gz
> >
> > gpg --verify apache-druid-26.0.0-bin.tar.gz.asc \
> > apache-druid-26.0.0-bin.tar.gz
> >
> > gpg --verify apache-druid-26.0.0-hadoop3-bin.tar.gz.asc \
> > apache-druid-26.0.0-hadoop3-bin.tar.gz
> >
> > Please review the proposed artifacts and vote. Note that Apache has
> > specific requirements that must be met before +1 binding votes can be cast
> > by PMC members. Please refer to the policy at
> > http://www.apache.org/legal/release-policy.html#policy for more details.
> >
> > As part of the validation process, the release artifacts can be generated
> > from source by running:
> > mvn clean install -Papache-release,dist -Dgpg.skip
> >
> > The RAT license check can be run from source by:
> > mvn apache-rat:check -Prat
> >
> > This vote will be open for at least 72 hours. The vote will pass if a
> > majority of at least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Druid 26.0.0
> > [ ] 0 I don't feel strongly about it,

[VOTE] Release Apache Druid 26.0.0 [RC3]

[Clint Wylie]

Hi all,

I have created a build for Apache Druid 26.0.0, release
candidate 3.

Thanks to everyone who has helped contribute to the release! You can read
the proposed release notes here:
https://github.com/apache/druid/issues/14064

The release candidate has been tagged in GitHub as
druid-26.0.0-rc3 (727c61943381a82e94a0d6c6409f505dcfda11c1),
available here:
https://github.com/apache/druid/releases/tag/druid-26.0.0-rc3

The artifacts to be voted on are located here:
https://dist.apache.org/repos/dist/dev/druid/26.0.0-rc3/

A staged Maven repository is available for review at:
https://repository.apache.org/content/repositories/orgapachedruid-1042/

Staged druid.apache.org website documentation is available here:
https://druid.staged.apache.org/docs/26.0.0/design/index.html

A Docker image containing the binary of the release candidate can be
retrieved via:
docker pull apache/druid:26.0.0-rc3

artifact checksums
src:
67f572acf2cdfbf3affdeb665fefb8d91bec0650ad54e1c3fb3d1eedb15932fa33b612bf870676a76a1a2aa0c57f7cee99bec8eeec33de975326d0d6923ac0b4
bin:
b8d2c982c8fcc0abadecb571727a3e8f69a6f79ee315fbe9628c3cffc7f92df7437d17b80c0e5234222bb55b265591d19081cbdcda0ab37563805fecf3e4
hadoop3-bin:
0dc1bfbd90ff4467518c203375278a1cfa0ef1ebceea5a3359615aebd06e70d1af121ddd8590eeae960e5d286e5c317cd4b571bcf94c8c25458e4ae92d6aafab
docker:
9a3a07819058693b29bb5fc145e071c3b3a1f551960640529a60ad5198f29b59

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/cwylie.asc

This key and the key of other committers can also be found in the project's
KEYS file here:
https://dist.apache.org/repos/dist/release/druid/KEYS

(If you are a committer, please feel free to add your own key to that file
by following the instructions in the file's header.)


Verify checksums:
diff <(shasum -a512 apache-druid-26.0.0-src.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-26.0.0-src.tar.gz.sha512 ; echo)

diff <(shasum -a512 apache-druid-26.0.0-bin.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-26.0.0-bin.tar.gz.sha512 ; echo)

diff <(shasum -a512 apache-druid-26.0.0-hadoop3-bin.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-26.0.0-hadoop3-bin.tar.gz.sha512 ; echo)

Verify signatures:
gpg --verify apache-druid-26.0.0-src.tar.gz.asc \
apache-druid-26.0.0-src.tar.gz

gpg --verify apache-druid-26.0.0-bin.tar.gz.asc \
apache-druid-26.0.0-bin.tar.gz

gpg --verify apache-druid-26.0.0-hadoop3-bin.tar.gz.asc \
apache-druid-26.0.0-hadoop3-bin.tar.gz

Please review the proposed artifacts and vote. Note that Apache has
specific requirements that must be met before +1 binding votes can be cast
by PMC members. Please refer to the policy at
http://www.apache.org/legal/release-policy.html#policy for more details.

As part of the validation process, the release artifacts can be generated
from source by running:
mvn clean install -Papache-release,dist -Dgpg.skip

The RAT license check can be run from source by:
mvn apache-rat:check -Prat

This vote will be open for at least 72 hours. The vote will pass if a
majority of at least three +1 PMC votes are cast.

[ ] +1 Release this package as Apache Druid 26.0.0
[ ] 0 I don't feel strongly about it, but I'm okay with the release
[ ] -1 Do not release this package because...

Thanks!

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

[VOTE] Release Apache Druid 26.0.0 [RC3]

[Clint Wylie]

Hi all,

I have created a build for Apache Druid 26.0.0, release
candidate 3.

Thanks to everyone who has helped contribute to the release! You can read
the proposed release notes here:
https://github.com/apache/druid/issues/14064

The release candidate has been tagged in GitHub as
druid-26.0.0-rc3 (727c61943381a82e94a0d6c6409f505dcfda11c1),
available here:
https://github.com/apache/druid/releases/tag/druid-26.0.0-rc3

The artifacts to be voted on are located here:
https://dist.apache.org/repos/dist/dev/druid/26.0.0-rc3/

A staged Maven repository is available for review at:
https://repository.apache.org/content/repositories/orgapachedruid-1042/

Staged druid.apache.org website documentation is available here:
https://druid.staged.apache.org/docs/26.0.0/design/index.html

A Docker image containing the binary of the release candidate can be
retrieved via:
docker pull apache/druid:26.0.0-rc3

artifact checksums
src:
67f572acf2cdfbf3affdeb665fefb8d91bec0650ad54e1c3fb3d1eedb15932fa33b612bf870676a76a1a2aa0c57f7cee99bec8eeec33de975326d0d6923ac0b4
bin:
b8d2c982c8fcc0abadecb571727a3e8f69a6f79ee315fbe9628c3cffc7f92df7437d17b80c0e5234222bb55b265591d19081cbdcda0ab37563805fecf3e4
hadoop3-bin:
0dc1bfbd90ff4467518c203375278a1cfa0ef1ebceea5a3359615aebd06e70d1af121ddd8590eeae960e5d286e5c317cd4b571bcf94c8c25458e4ae92d6aafab
docker:
9a3a07819058693b29bb5fc145e071c3b3a1f551960640529a60ad5198f29b59

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/cwylie.asc

This key and the key of other committers can also be found in the project's
KEYS file here:
https://dist.apache.org/repos/dist/release/druid/KEYS

(If you are a committer, please feel free to add your own key to that file
by following the instructions in the file's header.)


Verify checksums:
diff <(shasum -a512 apache-druid-26.0.0-src.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-26.0.0-src.tar.gz.sha512 ; echo)

diff <(shasum -a512 apache-druid-26.0.0-bin.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-26.0.0-bin.tar.gz.sha512 ; echo)

diff <(shasum -a512 apache-druid-26.0.0-hadoop3-bin.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-26.0.0-hadoop3-bin.tar.gz.sha512 ; echo)

Verify signatures:
gpg --verify apache-druid-26.0.0-src.tar.gz.asc \
apache-druid-26.0.0-src.tar.gz

gpg --verify apache-druid-26.0.0-bin.tar.gz.asc \
apache-druid-26.0.0-bin.tar.gz

gpg --verify apache-druid-26.0.0-hadoop3-bin.tar.gz.asc \
apache-druid-26.0.0-hadoop3-bin.tar.gz

Please review the proposed artifacts and vote. Note that Apache has
specific requirements that must be met before +1 binding votes can be cast
by PMC members. Please refer to the policy at
http://www.apache.org/legal/release-policy.html#policy for more details.

As part of the validation process, the release artifacts can be generated
from source by running:
mvn clean install -Papache-release,dist -Dgpg.skip

The RAT license check can be run from source by:
mvn apache-rat:check -Prat

This vote will be open for at least 72 hours. The vote will pass if a
majority of at least three +1 PMC votes are cast.

[ ] +1 Release this package as Apache Druid 26.0.0
[ ] 0 I don't feel strongly about it, but I'm okay with the release
[ ] -1 Do not release this package because...

Thanks!

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

[CANCEL][VOTE] Release Apache Druid 26.0.0 [RC2]

[Clint Wylie]

Found a regression, RC3 vote will start soon.

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

[VOTE] Release Apache Druid 26.0.0 [RC2]

[Clint Wylie]

Hi all,

I have created a build for Apache Druid 26.0.0, release
candidate 2.

Thanks to everyone who has helped contribute to the release! You can read
the proposed release notes here:
https://github.com/apache/druid/issues/14064

The release candidate has been tagged in GitHub as
druid-26.0.0-rc2 (ff5bb88216028c6d4039152cea7e80857c150b16),
available here:
https://github.com/apache/druid/releases/tag/druid-26.0.0-rc2

The artifacts to be voted on are located here:
https://dist.apache.org/repos/dist/dev/druid/26.0.0-rc2/

A staged Maven repository is available for review at:
https://repository.apache.org/content/repositories/orgapachedruid-1041/

Staged druid.apache.org website documentation is available here:
https://druid.staged.apache.org/docs/26.0.0/design/index.html

A Docker image containing the binary of the release candidate can be
retrieved via:
docker pull apache/druid:26.0.0-rc2

artifact checksums
src:
a2943f14d951712f4e946dac5d5b11e1e4ec7bdd934f9e10bdb953287c918a1e751b481e3895a355271fa424fa7b41bd2f5337270cf4cd8b36d7af08d3833d2c
bin:
a2077102e490bcd8c52d5cbf146c0a8a46f69f982fdadea5b39d6e292ac961cb7e3dd9668fbfc153f048788d0a2b12936571737421d74ecde56b914bd56af3d8
hadoop3-bin:
789ff59402f5701aa6828bf893904bdc05a6a563e947c50740f25a9cae8539f0d763370286b04491a9ed0734b0e9b2214f14fb188b26188c24019c86e73a002e
docker:
5408def03670388d9642d447ccf8a1ed32d074335b131193da3df746f53ecfc2

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/cwylie.asc

This key and the key of other committers can also be found in the project's
KEYS file here:
https://dist.apache.org/repos/dist/release/druid/KEYS

(If you are a committer, please feel free to add your own key to that file
by following the instructions in the file's header.)


Verify checksums:
diff <(shasum -a512 apache-druid-26.0.0-src.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-26.0.0-src.tar.gz.sha512 ; echo)

diff <(shasum -a512 apache-druid-26.0.0-bin.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-26.0.0-bin.tar.gz.sha512 ; echo)

diff <(shasum -a512 apache-druid-26.0.0-hadoop3-bin.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-26.0.0-hadoop3-bin.tar.gz.sha512 ; echo)

Verify signatures:
gpg --verify apache-druid-26.0.0-src.tar.gz.asc \
apache-druid-26.0.0-src.tar.gz

gpg --verify apache-druid-26.0.0-bin.tar.gz.asc \
apache-druid-26.0.0-bin.tar.gz

gpg --verify apache-druid-26.0.0-hadoop3-bin.tar.gz.asc \
apache-druid-26.0.0-hadoop3-bin.tar.gz

Please review the proposed artifacts and vote. Note that Apache has
specific requirements that must be met before +1 binding votes can be cast
by PMC members. Please refer to the policy at
http://www.apache.org/legal/release-policy.html#policy for more details.

As part of the validation process, the release artifacts can be generated
from source by running:
mvn clean install -Papache-release,dist -Dgpg.skip

The RAT license check can be run from source by:
mvn apache-rat:check -Prat

This vote will be open for at least 72 hours. The vote will pass if a
majority of at least three +1 PMC votes are cast.

[ ] +1 Release this package as Apache Druid 26.0.0
[ ] 0 I don't feel strongly about it, but I'm okay with the release
[ ] -1 Do not release this package because...

Thanks!

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

[CANCEL][VOTE] Release Apache Druid 26.0.0 [RC1]

[Clint Wylie]

Cancelling the vote because of an ugly issue with the new 'auto'
column schema that can result in breaking SQL schemas because it
causes segment metadata to fail to refresh.

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [VOTE] Release Apache Druid 26.0.0 [RC1]

[Clint Wylie]

Found a pretty ugly bug with the new 'auto' stuff so am going to
cancel this vote and make an RC2

On Sat, May 13, 2023 at 1:56 PM Clint Wylie  wrote:
>
> I forgot to mention that I had to refresh the expiration on my gpg key
> (I think I did everything correctly), but I am unsure if voters might
> need to get the updated keys on their side for stuff to validate
> correctly.
>
> On Fri, May 12, 2023 at 1:07 PM Clint Wylie  wrote:
> >
> > FYI when running build with -Pwebsite i ran into some issues with the
> > vulnerability checker, but didn't think they needed to be blockers for
> > the RC vote (especially since the website can be updated
> > independently). I will be investigating these to ensure they are not a
> > concern for us.
> >
> > Speaking of the website, there are a couple more doc only changes I am
> > aware of that will be backported to the 26 release branch, and I will
> > update the staging website once they are merged.
> >
> > On Fri, May 12, 2023 at 12:39 PM Clint Wylie  wrote:
> > >
> > > Hi all,
> > >
> > > I have created a build for Apache Druid 26.0.0, release
> > > candidate 1.
> > >
> > > Thanks to everyone who has helped contribute to the release! You can read
> > > the proposed release notes here:
> > > https://github.com/apache/druid/issues/14064
> > >
> > > The release candidate has been tagged in GitHub as
> > > druid-26.0.0-rc1 (bb58a23aedccc7f2bf4484d5211c936cfd90038a),
> > > available here:
> > > https://github.com/apache/druid/releases/tag/druid-26.0.0-rc1
> > >
> > > The artifacts to be voted on are located here:
> > > https://dist.apache.org/repos/dist/dev/druid/26.0.0-rc1/
> > >
> > > A staged Maven repository is available for review at:
> > > https://repository.apache.org/content/repositories/orgapachedruid-1040/
> > >
> > > Staged druid.apache.org website documentation is available here:
> > > https://druid.staged.apache.org/docs/26.0.0/design/index.html
> > >
> > > A Docker image containing the binary of the release candidate can be
> > > retrieved via:
> > > docker pull apache/druid:26.0.0-rc1
> > >
> > > artifact checksums
> > > src:
> > > a9483a1f4bcb9a921f98d3a6a0815a2c20c218a8d64b9743f1f14b1b3d7bc2a9220ac7fc5d4fa9572a3973ab40214ed9f527f53d2f6ff401c5bbdee72105c02a
> > > bin:
> > > e40d8644e1546000a27ff0af61f2f3200606af719623ecf8db9a517f8f5c3e54660385a7a99e7cc50eea5a32e4348c58b00b2b89bbab44b97cc59b24b44f
> > > hadoop3-bin:
> > > 12a8fa196a7dcf252d08b7b73e1117a25efc8de49a74a653dd51631a4fc91d53e4fc6d82df3c375142b0c3fa6f2007446f1beb5f680fdbda5510f2ebb7a9bfd3
> > > docker:
> > > 54b0037966884f66aafc426fdd236b237b96885c3a76608f4cf497d8f3369a5c
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/cwylie.asc
> > >
> > > This key and the key of other committers can also be found in the 
> > > project's
> > > KEYS file here:
> > > https://dist.apache.org/repos/dist/release/druid/KEYS
> > >
> > > (If you are a committer, please feel free to add your own key to that file
> > > by following the instructions in the file's header.)
> > >
> > >
> > > Verify checksums:
> > > diff <(shasum -a512 apache-druid-26.0.0-src.tar.gz | \
> > > cut -d ' ' -f1) \
> > > <(cat apache-druid-26.0.0-src.tar.gz.sha512 ; echo)
> > >
> > > diff <(shasum -a512 apache-druid-26.0.0-bin.tar.gz | \
> > > cut -d ' ' -f1) \
> > > <(cat apache-druid-26.0.0-bin.tar.gz.sha512 ; echo)
> > >
> > > diff <(shasum -a512 apache-druid-26.0.0-hadoop3-bin.tar.gz | \
> > > cut -d ' ' -f1) \
> > > <(cat apache-druid-26.0.0-hadoop3-bin.tar.gz.sha512 ; echo)
> > >
> > > Verify signatures:
> > > gpg --verify apache-druid-26.0.0-src.tar.gz.asc \
> > > apache-druid-26.0.0-src.tar.gz
> > >
> > > gpg --verify apache-druid-26.0.0-bin.tar.gz.asc \
> > > apache-druid-26.0.0-bin.tar.gz
> > >
> > > gpg --verify apache-druid-26.0.0-hadoop3-bin.tar.gz.asc \
> > > apache-druid-26.0.0-hadoop3-bin.tar.gz
> > >
> > > Please review the proposed artifacts and vote. Note that Apache has
> > > specific requirements that must be met before +1 binding votes can be cast
> > > by PMC members. Please refer to the policy at
> >

Re: [VOTE] Release Apache Druid 26.0.0 [RC1]

[Clint Wylie]

I forgot to mention that I had to refresh the expiration on my gpg key
(I think I did everything correctly), but I am unsure if voters might
need to get the updated keys on their side for stuff to validate
correctly.

On Fri, May 12, 2023 at 1:07 PM Clint Wylie  wrote:
>
> FYI when running build with -Pwebsite i ran into some issues with the
> vulnerability checker, but didn't think they needed to be blockers for
> the RC vote (especially since the website can be updated
> independently). I will be investigating these to ensure they are not a
> concern for us.
>
> Speaking of the website, there are a couple more doc only changes I am
> aware of that will be backported to the 26 release branch, and I will
> update the staging website once they are merged.
>
> On Fri, May 12, 2023 at 12:39 PM Clint Wylie  wrote:
> >
> > Hi all,
> >
> > I have created a build for Apache Druid 26.0.0, release
> > candidate 1.
> >
> > Thanks to everyone who has helped contribute to the release! You can read
> > the proposed release notes here:
> > https://github.com/apache/druid/issues/14064
> >
> > The release candidate has been tagged in GitHub as
> > druid-26.0.0-rc1 (bb58a23aedccc7f2bf4484d5211c936cfd90038a),
> > available here:
> > https://github.com/apache/druid/releases/tag/druid-26.0.0-rc1
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/druid/26.0.0-rc1/
> >
> > A staged Maven repository is available for review at:
> > https://repository.apache.org/content/repositories/orgapachedruid-1040/
> >
> > Staged druid.apache.org website documentation is available here:
> > https://druid.staged.apache.org/docs/26.0.0/design/index.html
> >
> > A Docker image containing the binary of the release candidate can be
> > retrieved via:
> > docker pull apache/druid:26.0.0-rc1
> >
> > artifact checksums
> > src:
> > a9483a1f4bcb9a921f98d3a6a0815a2c20c218a8d64b9743f1f14b1b3d7bc2a9220ac7fc5d4fa9572a3973ab40214ed9f527f53d2f6ff401c5bbdee72105c02a
> > bin:
> > e40d8644e1546000a27ff0af61f2f3200606af719623ecf8db9a517f8f5c3e54660385a7a99e7cc50eea5a32e4348c58b00b2b89bbab44b97cc59b24b44f
> > hadoop3-bin:
> > 12a8fa196a7dcf252d08b7b73e1117a25efc8de49a74a653dd51631a4fc91d53e4fc6d82df3c375142b0c3fa6f2007446f1beb5f680fdbda5510f2ebb7a9bfd3
> > docker:
> > 54b0037966884f66aafc426fdd236b237b96885c3a76608f4cf497d8f3369a5c
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/cwylie.asc
> >
> > This key and the key of other committers can also be found in the project's
> > KEYS file here:
> > https://dist.apache.org/repos/dist/release/druid/KEYS
> >
> > (If you are a committer, please feel free to add your own key to that file
> > by following the instructions in the file's header.)
> >
> >
> > Verify checksums:
> > diff <(shasum -a512 apache-druid-26.0.0-src.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-26.0.0-src.tar.gz.sha512 ; echo)
> >
> > diff <(shasum -a512 apache-druid-26.0.0-bin.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-26.0.0-bin.tar.gz.sha512 ; echo)
> >
> > diff <(shasum -a512 apache-druid-26.0.0-hadoop3-bin.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-26.0.0-hadoop3-bin.tar.gz.sha512 ; echo)
> >
> > Verify signatures:
> > gpg --verify apache-druid-26.0.0-src.tar.gz.asc \
> > apache-druid-26.0.0-src.tar.gz
> >
> > gpg --verify apache-druid-26.0.0-bin.tar.gz.asc \
> > apache-druid-26.0.0-bin.tar.gz
> >
> > gpg --verify apache-druid-26.0.0-hadoop3-bin.tar.gz.asc \
> > apache-druid-26.0.0-hadoop3-bin.tar.gz
> >
> > Please review the proposed artifacts and vote. Note that Apache has
> > specific requirements that must be met before +1 binding votes can be cast
> > by PMC members. Please refer to the policy at
> > http://www.apache.org/legal/release-policy.html#policy for more details.
> >
> > As part of the validation process, the release artifacts can be generated
> > from source by running:
> > mvn clean install -Papache-release,dist -Dgpg.skip
> >
> > The RAT license check can be run from source by:
> > mvn apache-rat:check -Prat
> >
> > This vote will be open for at least 72 hours. The vote will pass if a
> > majority of at least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Druid 26.0.0
> > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Thanks!

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [VOTE] Release Apache Druid 26.0.0 [RC1]

[Clint Wylie]

FYI when running build with -Pwebsite i ran into some issues with the
vulnerability checker, but didn't think they needed to be blockers for
the RC vote (especially since the website can be updated
independently). I will be investigating these to ensure they are not a
concern for us.

Speaking of the website, there are a couple more doc only changes I am
aware of that will be backported to the 26 release branch, and I will
update the staging website once they are merged.

On Fri, May 12, 2023 at 12:39 PM Clint Wylie  wrote:
>
> Hi all,
>
> I have created a build for Apache Druid 26.0.0, release
> candidate 1.
>
> Thanks to everyone who has helped contribute to the release! You can read
> the proposed release notes here:
> https://github.com/apache/druid/issues/14064
>
> The release candidate has been tagged in GitHub as
> druid-26.0.0-rc1 (bb58a23aedccc7f2bf4484d5211c936cfd90038a),
> available here:
> https://github.com/apache/druid/releases/tag/druid-26.0.0-rc1
>
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/druid/26.0.0-rc1/
>
> A staged Maven repository is available for review at:
> https://repository.apache.org/content/repositories/orgapachedruid-1040/
>
> Staged druid.apache.org website documentation is available here:
> https://druid.staged.apache.org/docs/26.0.0/design/index.html
>
> A Docker image containing the binary of the release candidate can be
> retrieved via:
> docker pull apache/druid:26.0.0-rc1
>
> artifact checksums
> src:
> a9483a1f4bcb9a921f98d3a6a0815a2c20c218a8d64b9743f1f14b1b3d7bc2a9220ac7fc5d4fa9572a3973ab40214ed9f527f53d2f6ff401c5bbdee72105c02a
> bin:
> e40d8644e1546000a27ff0af61f2f3200606af719623ecf8db9a517f8f5c3e54660385a7a99e7cc50eea5a32e4348c58b00b2b89bbab44b97cc59b24b44f
> hadoop3-bin:
> 12a8fa196a7dcf252d08b7b73e1117a25efc8de49a74a653dd51631a4fc91d53e4fc6d82df3c375142b0c3fa6f2007446f1beb5f680fdbda5510f2ebb7a9bfd3
> docker:
> 54b0037966884f66aafc426fdd236b237b96885c3a76608f4cf497d8f3369a5c
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/cwylie.asc
>
> This key and the key of other committers can also be found in the project's
> KEYS file here:
> https://dist.apache.org/repos/dist/release/druid/KEYS
>
> (If you are a committer, please feel free to add your own key to that file
> by following the instructions in the file's header.)
>
>
> Verify checksums:
> diff <(shasum -a512 apache-druid-26.0.0-src.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-26.0.0-src.tar.gz.sha512 ; echo)
>
> diff <(shasum -a512 apache-druid-26.0.0-bin.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-26.0.0-bin.tar.gz.sha512 ; echo)
>
> diff <(shasum -a512 apache-druid-26.0.0-hadoop3-bin.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-26.0.0-hadoop3-bin.tar.gz.sha512 ; echo)
>
> Verify signatures:
> gpg --verify apache-druid-26.0.0-src.tar.gz.asc \
> apache-druid-26.0.0-src.tar.gz
>
> gpg --verify apache-druid-26.0.0-bin.tar.gz.asc \
> apache-druid-26.0.0-bin.tar.gz
>
> gpg --verify apache-druid-26.0.0-hadoop3-bin.tar.gz.asc \
> apache-druid-26.0.0-hadoop3-bin.tar.gz
>
> Please review the proposed artifacts and vote. Note that Apache has
> specific requirements that must be met before +1 binding votes can be cast
> by PMC members. Please refer to the policy at
> http://www.apache.org/legal/release-policy.html#policy for more details.
>
> As part of the validation process, the release artifacts can be generated
> from source by running:
> mvn clean install -Papache-release,dist -Dgpg.skip
>
> The RAT license check can be run from source by:
> mvn apache-rat:check -Prat
>
> This vote will be open for at least 72 hours. The vote will pass if a
> majority of at least three +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Druid 26.0.0
> [ ] 0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
> Thanks!

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

[VOTE] Release Apache Druid 26.0.0 [RC1]

[Clint Wylie]

Hi all,

I have created a build for Apache Druid 26.0.0, release
candidate 1.

Thanks to everyone who has helped contribute to the release! You can read
the proposed release notes here:
https://github.com/apache/druid/issues/14064

The release candidate has been tagged in GitHub as
druid-26.0.0-rc1 (bb58a23aedccc7f2bf4484d5211c936cfd90038a),
available here:
https://github.com/apache/druid/releases/tag/druid-26.0.0-rc1

The artifacts to be voted on are located here:
https://dist.apache.org/repos/dist/dev/druid/26.0.0-rc1/

A staged Maven repository is available for review at:
https://repository.apache.org/content/repositories/orgapachedruid-1040/

Staged druid.apache.org website documentation is available here:
https://druid.staged.apache.org/docs/26.0.0/design/index.html

A Docker image containing the binary of the release candidate can be
retrieved via:
docker pull apache/druid:26.0.0-rc1

artifact checksums
src:
a9483a1f4bcb9a921f98d3a6a0815a2c20c218a8d64b9743f1f14b1b3d7bc2a9220ac7fc5d4fa9572a3973ab40214ed9f527f53d2f6ff401c5bbdee72105c02a
bin:
e40d8644e1546000a27ff0af61f2f3200606af719623ecf8db9a517f8f5c3e54660385a7a99e7cc50eea5a32e4348c58b00b2b89bbab44b97cc59b24b44f
hadoop3-bin:
12a8fa196a7dcf252d08b7b73e1117a25efc8de49a74a653dd51631a4fc91d53e4fc6d82df3c375142b0c3fa6f2007446f1beb5f680fdbda5510f2ebb7a9bfd3
docker:
54b0037966884f66aafc426fdd236b237b96885c3a76608f4cf497d8f3369a5c

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/cwylie.asc

This key and the key of other committers can also be found in the project's
KEYS file here:
https://dist.apache.org/repos/dist/release/druid/KEYS

(If you are a committer, please feel free to add your own key to that file
by following the instructions in the file's header.)


Verify checksums:
diff <(shasum -a512 apache-druid-26.0.0-src.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-26.0.0-src.tar.gz.sha512 ; echo)

diff <(shasum -a512 apache-druid-26.0.0-bin.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-26.0.0-bin.tar.gz.sha512 ; echo)

diff <(shasum -a512 apache-druid-26.0.0-hadoop3-bin.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-26.0.0-hadoop3-bin.tar.gz.sha512 ; echo)

Verify signatures:
gpg --verify apache-druid-26.0.0-src.tar.gz.asc \
apache-druid-26.0.0-src.tar.gz

gpg --verify apache-druid-26.0.0-bin.tar.gz.asc \
apache-druid-26.0.0-bin.tar.gz

gpg --verify apache-druid-26.0.0-hadoop3-bin.tar.gz.asc \
apache-druid-26.0.0-hadoop3-bin.tar.gz

Please review the proposed artifacts and vote. Note that Apache has
specific requirements that must be met before +1 binding votes can be cast
by PMC members. Please refer to the policy at
http://www.apache.org/legal/release-policy.html#policy for more details.

As part of the validation process, the release artifacts can be generated
from source by running:
mvn clean install -Papache-release,dist -Dgpg.skip

The RAT license check can be run from source by:
mvn apache-rat:check -Prat

This vote will be open for at least 72 hours. The vote will pass if a
majority of at least three +1 PMC votes are cast.

[ ] +1 Release this package as Apache Druid 26.0.0
[ ] 0 I don't feel strongly about it, but I'm okay with the release
[ ] -1 Do not release this package because...

Thanks!

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

[DISCUSS] Druid 26.0 release

[Clint Wylie]

Hey all,

It is about that time again to think about doing another release. I
volunteer to be release manager, and if no objections will aim to make
the branch around the first week of April.

Please use this thread to call to attention anything which should be
resolved before we make the release branch.

It would be nice to reach a conclusion on the helm chart IP clearance
question before this release
https://lists.apache.org/thread/ygyzt23m06vc775nq5dsm349rf0j47dg, but
I'm not sure I have the time/energy to spare to drive it to resolution
and personally know very little about helm to commit to maintaining
it. Any volunteers?

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

moving druid-core, extendedset, druid-hll into druid-processing

[Clint Wylie]

Hi all,

I want to discuss moving druid-core, extendedset, and druid-hll into
druid-processing to simplify our code structure and dependencies a
bit. We've been discussing doing something like this off and on for
quite a lot of years now, re
https://github.com/apache/druid/issues/4312, and we've done parts of
it, but .. we just haven't got back to it yet.

I've opened a PR https://github.com/apache/druid/pull/13698 and have
done some testing so I think it should be minimally disruptive (see PR
for details), but wanted to raise this on the list to try to get a
wider audience and see if anyone has any concerns I missed. There are
still a couple of CI issues to work out.

Beyond this, I also think it would be super nice to move druid-sql and
druid-indexing-service into druid-server, though that should be done
in a separate PR.

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [VOTE] Release Apache Druid 25.0.0 [RC2]

[Clint Wylie]

+1 (binding)

src:
- verified LICENSE/NOTICE present
- verified signature/checksum
- RAT check passed
- built binary distribution, ingested some data, ran some queries

bin:
- verified LICENSE/NOTICE present
- verified signature/checksum
- ingested some data, ran some queries

bin (hadoop3):
- verified LICENSE/NOTICE present
- verified signature/checksum
- ingested some data (not with hadoop), ran some queries

docker:
- verified checksum
- ingested some data, ran some queries


On Fri, Dec 23, 2022 at 9:33 AM Jonathan Wei  wrote:
>
> +1 binding
>
> - checked signatures and checksums
>
> src:
> - checked LICENSE/NOTICE present
> - built package from src (intel based mac)
> - ran the new quickstart, and ran test queries (java 8)
>
> bin:
> - checked LICENSE/NOTICE present
> - ran the new quickstart, and ran test queries (java 8)
>
> I did encounter a test failure in druid-it-cases when building from src,
> related to yaml config parsing, but I don't think this is a release blocker.
>
> On Thu, Dec 22, 2022 at 4:37 PM Vadim Ogievetsky 
> wrote:
>
> > I vote +1 (binding)
> >
> > My checks included:
> > 1. Checksum verified for all assets
> > 2. Signature verified for all assets
> > 3. Source code compiled (Intel based Mac)
> > 4. Ran through the quickstart and the 60m NYC taxi query (Java 11)
> > 5. Ran through the demo queries
> > 6. Checked out the web console specific changes, release notes, and new
> > docs
> >
> > Vadim
> >
> > On 2022/12/22 16:41:04 Kashif Faraz wrote:
> > > Hi
> > >
> > > I have created a build for Apache Druid 25.0.0, release candidate 2.
> > >
> > > Thanks to everyone who has helped contribute to the release! You can read
> > > the proposed release notes here:
> > > https://github.com/apache/druid/issues/13592
> > >
> > > The release candidate has been tagged in GitHub as druid-25.0.0-rc2
> > > (9a78059ffbb22f0bf6074cd81b13078a2d4501bc),
> > > available here:
> > > https://github.com/apache/druid/releases/tag/druid-25.0.0-rc2
> > >
> > > The artifacts to be voted on are located here:
> > > https://dist.apache.org/repos/dist/dev/druid/25.0.0-rc2/
> > >
> > > A staged Maven repository is available for review at:
> > > https://repository.apache.org/content/repositories/orgapachedruid-1039/
> > >
> > > A Docker image containing the binary of the release candidate can be
> > > retrieved via:
> > > docker pull apache/druid:25.0.0-rc2
> > >
> > > Artifact checksums:
> > > src:
> > >
> > 0002091f2f76d64315e1242e5f61cf07517add8af5e4e4739d06eeff93944064721843a4e9666c61cb4f1e8bf587c0a4669cea01fe8f894cb3ae1a09849dd565
> > > bin:
> > >
> > 16f917c1c12c8edf85cefd4ad2fb68bf74cdbbb83e791e15fd13e74af18f3fbfddcdfbae8ab3834dba71cd8d45c6fe3920fbe3c75b0e5d050f040332c8ce54b7
> > > hadoop3-bin:
> > >
> > bdbfc5ae261d8bb551e91e0bca366d7a467e2063e174216ce65c574a73dd44bf6b8f9cde0301bddcab548210ab217a2ccfc9d2e077fe010d3959a10fd0e4b23c
> > > docker:
> > > d512f248e70b43fe94f2703b87e28b6f49a5a396f2407f6068cbb95c60153b14
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/kfaraz.asc
> > >
> > > This key and the key of other committers can also be found in the
> > project's
> > > KEYS file here:
> > > https://dist.apache.org/repos/dist/release/druid/KEYS
> > >
> > > (If you are a committer, please feel free to add your own key to that
> > file
> > > by following the instructions in the file's header.)
> > >
> > > Verify checksums:
> > > diff <(shasum -a512 apache-druid-25.0.0-src.tar.gz | \
> > > cut -d ' ' -f1) \
> > > <(cat apache-druid-25.0.0-src.tar.gz.sha512 ; echo)
> > >
> > > diff <(shasum -a512 apache-druid-25.0.0-bin.tar.gz | \
> > > cut -d ' ' -f1) \
> > > <(cat apache-druid-25.0.0-bin.tar.gz.sha512 ; echo)
> > >
> > > diff <(shasum -a512 apache-druid-25.0.0-hadoop3-bin.tar.gz | \
> > > cut -d ' ' -f1) \
> > > <(cat apache-druid-25.0.0-hadoop3-bin.tar.gz.sha512 ; echo)
> > >
> > > Verify signatures:
> > > gpg --verify apache-druid-25.0.0-src.tar.gz.asc \
> > > apache-druid-25.0.0-src.tar.gz
> > >
> > > gpg --verify apache-druid-25.0.0-bin.tar.gz.asc \
> > > apache-druid-25.0.0-bin.tar.gz
> > >
> > > gpg --verify apache-druid-25.0.0-hadoop3-bin.tar.gz.asc \
> > > apache-druid-25.0.0-hadoop3-bin.tar.gz
> > >
> > > Please review the proposed artifacts and vote. Note that Apache has
> > > specific requirements that must be met before +1 binding votes can be
> > cast
> > > by PMC members. Please refer to the policy at
> > > http://www.apache.org/legal/release-policy.html#policy for more details.
> > >
> > > As part of the validation process, the release artifacts can be generated
> > > from source by running:
> > > mvn clean install -Papache-release,dist -Dgpg.skip
> > >
> > > The hadoop-3 based artifacts can be generated by running:
> > > mvn clean install -Phadoop3,apache-release,dist -Dgpg.skip
> > >
> > > The RAT license check can be run from source by:
> > > mvn apache-rat:check -Prat
> > >
> > > This vote will be open for at least 72 hours. The

Re: [VOTE] Release Apache Druid 24.0.2 [RC1]

[Clint Wylie]

+1 (binding)

src package:
- verified signature/checksum
- LICENSE/NOTICE present
- RAT check passed
- built binary distribution, ran into an issue with CVE-2022-1471
failing on druid-kubernetes-extensions, I don't think this needs to be
a blocker
- tested msq ingestion in binary distribution, successfully ran some queries

bin package:
- verified signature/checksum
- LICENSE/NOTICE present
- test native ingest example data and successfully ran some queries

docker:
- verified checksum
- distribution/docker-compose.yml looks like only the coordinator
version was updated to 24.0.2, the others are 24.0.1,
https://github.com/apache/druid/blob/24.0.2/distribution/docker/docker-compose.yml#L68
I don't think this needs to be a blocker
- tested native ingestion (msq extension still not enabled by
default), ran queries

On Tue, Nov 29, 2022 at 2:56 AM Abhishek Agarwal
 wrote:
>
> +1 (binding)
>
> src package:
> - verified signature/checksum
> - LICENSE/NOTICE present
> - built binary distribution, Loaded example wikipedia dataset and ran some
> queries
>
> binary package:
> - verified signature/checksum
> - LICENSE/NOTICE present
> - Loaded example wikipedia dataset and ran some queries
>
> docker:
> - verified checksum
> - started cluster with docker-compose, Loaded example wikipedia dataset and
> ran some queries
>
> On Tue, Nov 29, 2022 at 2:00 PM Laksh Singla 
> wrote:
>
> > +1 (non-binding)
> > Verified that https://github.com/apache/druid/pull/13138 is included and
> > working in the build.
> >
> > Regards
> > Laksh
> >
> > The https://github.com/apache/druid/pull/13138 is
> >
> > On Sat, Nov 26, 2022 at 2:52 PM Kashif Faraz  wrote:
> >
> > > Hi
> > >
> > > I have created a build for Apache Druid 24.0.2, release candidate 1.
> > >
> > > Thanks to everyone who has helped contribute to the release! You can read
> > > the proposed release notes here:
> > > https://github.com/apache/druid/issues/13435
> > >
> > > The release candidate has been tagged in GitHub as druid-24.0.2-rc1
> > > (13c9fdcbafcf61211a9381892615c4b4db6a3535),
> > > available here:
> > > https://github.com/apache/druid/releases/tag/druid-24.0.2-rc1
> > >
> > > The artifacts to be voted on are located here:
> > > https://dist.apache.org/repos/dist/dev/druid/24.0.2-rc1/
> > >
> > > A staged Maven repository is available for review at:
> > > https://repository.apache.org/content/repositories/orgapachedruid-1037/
> > >
> > > A Docker image containing the binary of the release candidate can be
> > > retrieved via:
> > > docker pull apache/druid:24.0.2-rc1
> > >
> > > Artifact checksums:
> > > src:
> > >
> > >
> > 1ca1ea4b9a4418813fc22cdba2626725f802ff886e118b9986b89e8a6df9fe304a0fb0eca1370a1c937ac5b136448c8157d58b1f79712347ca5cb67f7f034cfc
> > > bin:
> > >
> > >
> > ca22923d0c1b60803289ada2be6c7e74b41641e95e3ad36fb625edd71c62d29f5203221062e5bf7af8199075bbc46606ec3ae8b559b455e1ecfc990f0e669f55
> > > docker:
> > > e96ae3293242e90e0a78be128c0d12325b8a6fc5c68ce2f2bc53de3aac568bbf
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/kfaraz.asc
> > >
> > > This key and the key of other committers can also be found in the
> > project's
> > > KEYS file here:
> > > https://dist.apache.org/repos/dist/release/druid/KEYS
> > >
> > > (If you are a committer, please feel free to add your own key to that
> > file
> > > by following the instructions in the file's header.)
> > >
> > > Verify checksums:
> > > diff <(shasum -a512 apache-druid-24.0.2-src.tar.gz | \
> > > cut -d ' ' -f1) \
> > > <(cat apache-druid-24.0.2-src.tar.gz.sha512 ; echo)
> > >
> > > diff <(shasum -a512 apache-druid-24.0.2-bin.tar.gz | \
> > > cut -d ' ' -f1) \
> > > <(cat apache-druid-24.0.2-bin.tar.gz.sha512 ; echo)
> > >
> > > Verify signatures:
> > > gpg --verify apache-druid-24.0.2-src.tar.gz.asc \
> > > apache-druid-24.0.2-src.tar.gz
> > >
> > > gpg --verify apache-druid-24.0.2-bin.tar.gz.asc \
> > > apache-druid-24.0.2-bin.tar.gz
> > >
> > > Please review the proposed artifacts and vote. Note that Apache has
> > > specific requirements that must be met before +1 binding votes can be
> > cast
> > > by PMC members. Please refer to the policy at
> > > http://www.apache.org/legal/release-policy.html#policy for more details.
> > >
> > > As part of the validation process, the release artifacts can be generated
> > > from source by running:
> > > mvn clean install -Papache-release,dist -Dgpg.skip
> > >
> > > The RAT license check can be run from source by:
> > > mvn apache-rat:check -Prat
> > >
> > > This vote will be open for at least 72 hours. The vote will pass if a
> > > majority of at least three +1 PMC votes are cast.
> > >
> > > [ ] +1 Release this package as Apache Druid 24.0.2
> > > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > > [ ] -1 Do not release this package because...
> > >
> > >
> > > Thanks
> > > Kashif
> > >
> >

-
To unsubsc

Re: [VOTE] Release Apache Druid 24.0.1 [RC1]

[Clint Wylie]

+1 (binding)

src package:
- verified signature/checksum
- LICENSE/NOTICE present
- compiled, ran checks, unit tests, ran into some issues with tests
due to being on an m1 mac, but i don't think they are blockers
- built binary distribution, ingested native batch quickstart data and
ran some queries

binary package:
- verified signature/checksum
- LICENSE/NOTICE present
- MSQ ingestion quickstart and ran some queries

docker:
- verified checksum
- started cluster with docker-compose, native batch ingestion
quickstart and ran some queries

On Mon, Nov 14, 2022 at 1:15 PM Vadim Ogievetsky  wrote:
>
> I vote +1
>
> My checks included:
> 1. Checksum verified
> 2. Signature verified
> 3. Source code compiled (Intel based Mac)
> 4. Ran through the quickstart and a few other select queries (Java 11)
> 5. Checked the specific web-console things fixed in this release
>
> On 2022/11/09 05:55:50 Kashif Faraz wrote:
> > Hi
> >
> > I have created a build for Apache Druid 24.0.1, release candidate 1.
> >
> > Thanks to everyone who has helped contribute to the release! You can read
> > the proposed release notes here:
> > https://github.com/apache/druid/issues/13320
> >
> > The release candidate has been tagged in GitHub as druid-24.0.1-rc1
> > (7a04fc1a7b5eddfdf766ae31c1d3ce3c49fc16a8),
> > available here:
> > https://github.com/apache/druid/releases/tag/druid-24.0.1-rc1
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/druid/24.0.1-rc1/
> >
> > A staged Maven repository is available for review at:
> > https://repository.apache.org/content/repositories/orgapachedruid-1035/
> >
> > A Docker image containing the binary of the release candidate can be
> > retrieved via:
> > docker pull apache/druid:24.0.1-rc1
> >
> > Artifact checksums:
> > src:
> > 75b1eaf28b83a014c282e32519edb07402dae35625d5cffcbc7ce3ce3a7fbd5338370a4d8dae7c001b77360c176ca32298b7ff4d71a610ed39423fa078863b03
> > bin:
> > 58430e64ecad15f4a4baf37c3a91b863c20546a012bf488984c4b79b1642f8a398ba9454d2218f57e23a57a70c117a30423eedfb359d3dc3079027c002580f61
> > docker:
> > 40123383a5d6b2a186fbabb7d63382f6903aa4e77aeab272db55a6ff1dd446be
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/kfaraz.asc
> >
> > This key and the key of other committers can also be found in the project's
> > KEYS file here:
> > https://dist.apache.org/repos/dist/release/druid/KEYS
> >
> > (If you are a committer, please feel free to add your own key to that file
> > by following the instructions in the file's header.)
> >
> > Verify checksums:
> > diff <(shasum -a512 apache-druid-24.0.1-src.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-24.0.1-src.tar.gz.sha512 ; echo)
> >
> > diff <(shasum -a512 apache-druid-24.0.1-bin.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-24.0.1-bin.tar.gz.sha512 ; echo)
> >
> > Verify signatures:
> > gpg --verify apache-druid-24.0.1-src.tar.gz.asc \
> > apache-druid-24.0.1-src.tar.gz
> >
> > gpg --verify apache-druid-24.0.1-bin.tar.gz.asc \
> > apache-druid-24.0.1-bin.tar.gz
> >
> > Please review the proposed artifacts and vote. Note that Apache has
> > specific requirements that must be met before +1 binding votes can be cast
> > by PMC members. Please refer to the policy at
> > http://www.apache.org/legal/release-policy.html#policy for more details.
> >
> > As part of the validation process, the release artifacts can be generated
> > from source by running:
> > mvn clean install -Papache-release,dist -Dgpg.skip
> >
> > The RAT license check can be run from source by:
> > mvn apache-rat:check -Prat
> >
> > This vote will be open for at least 72 hours. The vote will pass if a
> > majority of at least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Druid 24.0.1
> > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Thanks
> > Kashif
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> For additional commands, e-mail: dev-h...@druid.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [VOTE] Release Apache Druid 24.0.0 [RC2]

[Clint Wylie]

+1 (binding)

src package:
- verified signature/checksum
- LICENSE/NOTICE present
- compiled, ran checks, unit tests, ran into some issues with
CVE-2022-39135 which was announced a couple of days ago, but I don't
believe we are impacted; also an issue with parquet tests due to being
on an M1 mac with incompatible snappy which should be fixed by
https://github.com/apache/druid/pull/13081
- built binary distribution, ingested native batch quickstart data and
ran some queries

binary package:
- verified signature/checksum
- LICENSE/NOTICE present
- MSQ ingestion quickstart and ran some queries

docker:
- verified checksum
- started cluster with docker-compose, native batch ingestion
quickstart and ran some
queries



On Tue, Sep 13, 2022 at 11:42 PM Vadim Ogievetsky
 wrote:
>
> I vote +1
>
> My checks included:
> 1. Checksum verified
> 2. Signature verified
> 3. Source code compiled (Intel based Mac)
> 4. Ran through the new quickstart and all the example queries from the binary 
> distribution (Java 11)
> 5. Used the binary distribution for a few days just doing all sorts of 
> queries (lots of MSQ usage) on it as part of other projects
>
> Regarding the issues that Frank raised:
> - https://github.com/apache/druid/pull/13068 this does not look like a 
> regression in 24.0
> - https://github.com/apache/druid/pull/13069 I think it is fine to release 
> without it and mention it in the documentation.
>
> On 2022/09/10 08:32:18 Abhishek Agarwal wrote:
> > Hi all,
> > I have created a build for Apache Druid 24.0.0, release candidate 2.
> >
> > Thanks to everyone who has helped contribute to the release. You can read
> > the draft release notes here:
> > https://github.com/apache/druid/issues/12825
> >
> > The release candidate has been tagged in GitHub as druid-24.0.0-rc2
> > (09991dcf17fb9984b65f291f0db47e38058253e4), available here:
> > https://github.com/apache/druid/releases/tag/druid-24.0.0-rc2
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/druid/24.0.0-rc2/
> >
> > A staged Maven repository is available for review at:
> > https://repository.apache.org/content/repositories/orgapachedruid-1032
> >
> > Staged druid.apache.org website documentation is available here:
> > https://druid.staged.apache.org/docs/24.0.0/design/index.html
> >
> > A Docker image containing the binary of the release candidate can be
> > retrieved via:
> > docker pull apache/druid:24.0.0-rc2
> >
> > artifact checksums
> > src:
> > 867e3c4787e4e3f2a35e066cc982698d3e1a77fe90be7de163bd893908c13fec7fd8a84e19020f6f32fb517502a8eb729ae4492d51549f72a57f2bbf42b08e9c
> >
> > bin:
> > 8b15192a7fb5a6f979b0c8f51bfd0dee5d26a5912ec2a3d25f4739ebbf89cadebed353ba30977a0b13b6410f89a2991eb7ee5ebea4217026129ce2262875dc91
> >
> > docker (sha256):
> > c5ab9872c2f6e340f1429fd0bd514f163b0e491a9d808ee85de9a1b8f81ffe0b
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/abhishek
> > This key and the key of other committers can also be found in the project's
> > KEYS file here:
> > https://dist.apache.org/repos/dist/release/druid/KEYS
> >
> > (If you are a committer, please feel free to add your own key to that file
> > by following the instructions in the file's header.)
> >
> > Verify checksums:
> > diff <(shasum -a512 apache-druid-24.0.0-src.tar.gz | cut -d ' ' -f1) <(cat \
> > apache-druid-24.0.0-src.tar.gz.sha512 ; echo)
> >
> > diff <(shasum -a512 apache-druid-24.0.0-bin.tar.gz | cut -d ' ' -f1) <(cat \
> > apache-druid-24.0.0-bin.tar.gz.sha512 ; echo)
> >
> > Verify signatures:
> > gpg --verify apache-druid-24.0.0-src.tar.gz.asc \
> > apache-druid-24.0.0-src.tar.gz
> >
> > gpg --verify apache-druid-24.0.0-bin.tar.gz.asc \
> > apache-druid-24.0.0-bin.tar.gz
> >
> > Please review the proposed artifacts and vote. Note that Apache has
> > specific requirements that must be met before +1 binding votes can be cast
> > by PMC members. Please refer to the policy at
> > http://www.apache.org/legal/release-policy.html#policy for more details.
> >
> > As part of the validation process, the release artifacts can be generated
> > from source by running:
> > mvn clean install -Papache-release,dist -Dgpg.skip
> >
> > The RAT license check can be run from source by:
> > mvn apache-rat:check -Prat
> >
> > This vote will be open for at least 72 hours. The vote will pass if a
> > majority of at least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Druid 24.0.0
> > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Thanks!
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> For additional commands, e-mail: dev-h...@druid.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-ma

Re: [DISCUSS] Removing code related to `FireHose`

[Clint Wylie]

If we remove RealtimeIndexTask and AppenderatorDriverRealtimeIndexTask
then we can remove EventReceiverFirehoseFactory. The former was
primarily used by tranquility which has been sunset, the latter I'm
not sure was ever used for anything. I'm personally in favor of
removing both of them since push based ingestion is very fragile in my
experience, but I think some of the oldest integration tests use
RealtimeIndexTask and so would need to be removed/updated/rewritten to
use something else as appropriate.

I don't think we can completely remove InputRowParser until we drop
Hadoop support (or modify Hadoop ingestion to use
InputSource/InputFormat?), since it still relies on using the older
spec. As far as I know, Thrift is the only data format that has not
been fully migrated to use InputFormat, though there is an old PR that
is mostly done  here https://github.com/apache/druid/pull/11360.

On Thu, Jun 23, 2022 at 5:11 AM Abhishek Agarwal
 wrote:
>
> Hello,
> The `FiniteFirehoseFactory` and `InputRowParser` classes were deprecated in
> 0.17.0 (https://github.com/apache/druid/pull/8823) in favour of
> `InputSource`.  0.17.0 was released more than 2 years ago in Jan 2020.
>
> I think it is about time that we remove this code entirely. Removing
> `InputRowParser` may not be as trivial as `EventReceiverFirehoseFactory`
> depends on it. I didn't find any alternatives for
> `EventReceiverFirehoseFactory` and it is not marked deprecated as well.
>
> But we can still remove `FiniteFirehoseFactory` and the implementations
> safely as there are alternatives available.
>
> Thoughts/Suggestions?

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [VOTE] Release Apache Druid 0.23.0 [RC2]

[Clint Wylie]

+1 (binding)

src package:
- verified signature/checksum
- LICENSE/NOTICE present
- compiled, ran checks, unit tests
- built binary distribution, ingested batch quickstart data and ran some queries

binary package:
- verified signature/checksum
- LICENSE/NOTICE present
- kafka ingestion quickstart and ran some queries

docker:
- verified checksum
- started cluster with docker-compose, kafka ingestion quickstart and ran some
queries

On Fri, Jun 17, 2022 at 5:06 PM Atul Mohan  wrote:
>
> +1 (binding)
>
> Source:
> - NOTICE and LICENSE present
> - Verified checksum and signature
> - Verified git.version file
> - mvn build with tests passed
> - mvn rat check passed
> - Ran druid cluster with the built binaries, ingested wiki data and
> verified using basic queries
>
> Binary:
> - NOTICE and LICENSE present
> - Verified checksum and signature
> - Ran quickstart cluster and ingested wiki data and verified using basic
> queries
>
>
> Docker:
> - Verified checksum
> - Started up cluster using docker compose
> - Ran native ingestion and verified using basic queries
>
> Thanks for all the hard work in managing this release.
>
> On Wed, Jun 15, 2022 at 7:22 AM Abhishek Agarwal 
> wrote:
>
> > Hi all,
> > I have created a build for Apache Druid 0.23.0, release candidate 2.
> >
> > Thanks to everyone who has helped contribute to the release. You can read
> > the draft release notes here:
> > https://github.com/apache/druid/issues/12510
> >
> > The release candidate has been tagged in GitHub as druid-0.23.0-rc2
> > (c085dd1a5e3db02806f1fc8f54fd7f9297ee0f77), available here:
> > https://github.com/apache/druid/tree/druid-0.23.0-rc2
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/druid/0.23.0-rc2/
> >
> > A staged Maven repository is available for review at:
> > https://repository.apache.org/content/repositories/orgapachedruid-1030
> >
> > Staged druid.apache.org website documentation is available here:
> > https://druid.staged.apache.org/docs/0.23.0/design/index.html
> >
> > A Docker image containing the binary of the release candidate can be
> > retrieved via:
> > docker pull apache/druid:0.23.0-rc2
> >
> > artifact checksums
> > src:
> >
> > 50ff149fa274ac10c01040dc30bbf28fc24732da5a27f1b88f89b13bbee5c9ec8a68f5d57e7caabc5813cac5f551e1c4ca94dc8faa9c6d8efc022f3bd3bef7e3
> >
> > bin:
> >
> > 5e7420bdc66247f0309fc0c1eafeff7ba935ed195f3a28c2da4b23a28ff637300b46b1c3d048379ccd570dde164968494ff5d8847e82eaf9b0edb518f8665328
> >
> > docker (sha256):
> > ed9719968b4be3a2f1643907a1a5a5814e2ee1215f3604da8b2da2ac05f65912
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/abhishek
> > This key and the key of other committers can also be found in the project's
> > KEYS file here:
> > https://dist.apache.org/repos/dist/release/druid/KEYS
> >
> > (If you are a committer, please feel free to add your own key to that file
> > by following the instructions in the file's header.)
> >
> > Verify checksums:
> > diff <(shasum -a512 apache-druid-0.23.0-src.tar.gz | cut -d ' ' -f1) <(cat
> > apache-druid-0.23.0-src.tar.gz.sha512 ; echo)
> >
> > diff <(shasum -a512 apache-druid-0.23.0-bin.tar.gz | cut -d ' ' -f1) <(cat
> > apache-druid-0.23.0-bin.tar.gz.sha512 ; echo)
> >
> > Verify signatures:
> > gpg --verify apache-druid-0.23.0-src.tar.gz.asc
> > apache-druid-0.23.0-src.tar.gz
> >
> > gpg --verify apache-druid-0.23.0-bin.tar.gz.asc
> > apache-druid-0.23.0-bin.tar.gz
> >
> > Please review the proposed artifacts and vote. Note that Apache has
> > specific requirements that must be met before +1 binding votes can be cast
> > by PMC members. Please refer to the policy at
> > http://www.apache.org/legal/release-policy.html#policy for more details.
> >
> > As part of the validation process, the release artifacts can be generated
> > from the source by running:
> > mvn clean install -Papache-release,dist -Dgpg.skip
> >
> > The RAT license check can be run from the source by:
> > mvn apache-rat:check -Prat
> >
> > This vote will be open for at least 72 hours. The vote will pass if a
> > majority of at least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Druid 0.23.0
> > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Thanks!
> >

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [VOTE] Release Apache Druid 0.23.0 [RC1]

[Clint Wylie]

-1 (binding)
The RAT check, mvn apache-rat:check -Prat, fails for me. it looks like
there are some stray files in the source distribution:

extensions-core/kubernetes-extensions/.attach_pid19113
extensions-core/druid-ranger-security/.attach_pid15985

possibly due to running tests in the packaging place, which looks like
it might have something to do with these files when running tests
perhaps? At least this is my best guess stumbling upon
https://github.com/raphw/byte-buddy/blob/master/byte-buddy-agent/src/main/java/net/bytebuddy/agent/VirtualMachine.java#L480,
since mockito uses byte-buddy to attach to process for certain types
of tests. This might be an issue that is new to this release, since I
think we migrated some tests recently?

This, coupled with the wrong date in NOTICE, the docker-compose
version, and web-console docs links issues I think make it worth doing
an RC2.

On Wed, Jun 8, 2022 at 9:43 AM Atul Mohan  wrote:
>
> In the process of verifying the release, it looks like the
> docker-compose.yml that we package is pointing to 0.22.0. I've raised
> https://github.com/apache/druid/pull/12618 to fix this.
>
> Thanks,
> Atul
>
> On Wed, Jun 8, 2022 at 7:25 AM Kashif Faraz  wrote:
>
> > +1
> >
> > verified:
> > - gpg signature
> > - shasum
> > - building the source artifacts (tests skipped)
> > - rat license check
> > - running the binary
> > - basic sanity ingestion and query
> >
> > Thanks
> > Kashif Faraz
> > kfa...@apache.org
> >
> >
> > On Mon, Jun 6, 2022 at 1:00 PM Abhishek Agarwal  > >
> > wrote:
> >
> > > Hi all,
> > > I have created a build for Apache Druid 0.23.0, release candidate 1.
> > >
> > > Thanks to everyone who has helped contribute to the release. You can read
> > > the draft release notes here:
> > > https://github.com/apache/druid/issues/12510
> > >
> > > The release candidate has been tagged in GitHub as druid-0.23.0-rc1
> > > (efa67c4eddb31dc6e9ff7d0c070b9d1b1bf10a45), available here:
> > > https://github.com/apache/druid/tree/druid-0.23.0-rc1
> > >
> > > The artifacts to be voted on are located here:
> > > https://dist.apache.org/repos/dist/dev/druid/0.23.0-rc1/
> > >
> > > A staged Maven repository is available for review at:
> > > https://repository.apache.org/content/repositories/orgapachedruid-1029/
> > >
> > > Staged druid.apache.org website documentation is available here:
> > > https://druid.staged.apache.org/docs/0.23.0/design/index.html
> > >
> > > A Docker image containing the binary of the release candidate can be
> > > retrieved via:
> > > docker pull apache/druid:0.23.0-rc1
> > >
> > > artifact checksums
> > > src:
> > >
> > >
> > a8b8124297f4fb954d8f3cb79cf2ea867a689fe8d8b2470718d0d95badcd9e66d3658efcb064c50617b5ea4cbfbc3e650b01d742de7bc7b39050b1ba5da6b068
> > >
> > > bin:
> > >
> > >
> > da1e0703db699fa523b7fa86b57eec5f77aa0b4141bee44324d81d5c8058a414bc896145f3dd7f5e12320593055e20b551ce597628de133a062d091e40c38a54
> > >
> > > docker (sha256):
> > > 510c9be199d6b8568c62c6d4bcf1bb3c98690baf9521eccf8c6b335e90eeef97
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/abhishek
> > > This key and the key of other committers can also be found in the
> > project's
> > > KEYS file here:
> > > https://dist.apache.org/repos/dist/release/druid/KEYS
> > >
> > > (If you are a committer, please feel free to add your own key to that
> > file
> > > by following the instructions in the file's header.)
> > >
> > > Verify checksums:
> > > diff <(shasum -a512 apache-druid-0.23.0-src.tar.gz | cut -d ' ' -f1)
> > <(cat
> > > apache-druid-0.23.0-src.tar.gz.sha512 ; echo)
> > >
> > > diff <(shasum -a512 apache-druid-0.23.0-bin.tar.gz | cut -d ' ' -f1)
> > <(cat
> > > apache-druid-0.23.0-bin.tar.gz.sha512 ; echo)
> > >
> > > Verify signatures:
> > > gpg --verify apache-druid-0.23.0-src.tar.gz.asc \
> > > apache-druid-0.23.0-src.tar.gz
> > >
> > > gpg --verify apache-druid-0.23.0-bin.tar.gz.asc \
> > > apache-druid-0.23.0-bin.tar.gz
> > >
> > > Please review the proposed artifacts and vote. Note that Apache has
> > > specific requirements that must be met before +1 binding votes can be
> > cast
> > > by PMC members. Please refer to the policy at
> > > http://www.apache.org/legal/release-policy.html#policy for more details.
> > >
> > > As part of the validation process, the release artifacts can be generated
> > > from source by running:
> > > mvn clean install -Papache-release,dist -Dgpg.skip
> > >
> > > The RAT license check can be run from source by:
> > > mvn apache-rat:check -Prat
> > >
> > > This vote will be open for at least 72 hours. The vote will pass if a
> > > majority of at least three +1 PMC votes are cast.
> > >
> > > [ ] +1 Release this package as Apache Druid 0.23.0
> > > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > > [ ] -1 Do not release this package because...
> > >
> > > Thanks!
> > >
> >

-
To unsubscribe, e-mai

Re: [VOTE] Release Apache Druid 0.22.1 [RC2]

[Clint Wylie]

+1 (binding)

src package:
- verified signature/checksum
- LICENSE/NOTICE present
- compiled, ran checks, unit tests
- built binary distribution, ingested some data and ran some queries

binary package:
- verified signature/checksum
- LICENSE/NOTICE present
- ingested some data and ran some queries

docker:
- verified checksum
- started cluster with docker-compose, ingested some data and ran some
queries

Also verified that I could no longer repro the vulnerability. Thanks
for the quick work everyone!

On Fri, Dec 10, 2021 at 9:02 PM Jihoon Son  wrote:
>
> Hi all,
>
> I have created a build for Apache Druid 0.22.1, release
> candidate 2.
>
> Thanks to everyone who has helped contribute to the release! You can read
> the proposed release notes here:
> https://github.com/apache/druid/issues/12054
>
> The release candidate has been tagged in GitHub as
> druid-0.22.1-rc2 (81e4da747d4fcfd15fa15bfebb942058152a3bba),
> available here:
> https://github.com/apache/druid/releases/tag/druid-0.22.1-rc2
>
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/druid/0.22.1-rc2/
>
> A staged Maven repository is available for review at:
> https://repository.apache.org/content/repositories/orgapachedruid-1028/
>
> A Docker image containing the binary of the release candidate can be
> retrieved via:
> docker pull apache/druid:0.22.1-rc2
>
> artifact checksums
> src:
> 2fea9417a7c164703d8f8bc19bbe70e743b82e6c5cf3ba9b7bc63c60a545b507757e36412bf06f6f522cf6de1cff1fe5141575c030c5fb779270012110f1427b
> bin:
> 716b83e07a76b5c9e0e26dd49028ca088bde81befb070989b41e71f0e8082d11a26601f4ac1e646bf099a4bc7420bdfeb9f7450d6da53d2a6de301e08c3cab0d
> docker: 93aae94d4509768e455c444ea7d8515e1a1b447179e043f7b39630c2350125a9
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/jihoonson.asc
>
> This key and the key of other committers can also be found in the project's
> KEYS file here:
> https://dist.apache.org/repos/dist/release/druid/KEYS
>
> (If you are a committer, please feel free to add your own key to that file
> by following the instructions in the file's header.)
>
>
> Verify checksums:
> diff <(shasum -a512 apache-druid-0.22.1-src.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-0.22.1-src.tar.gz.sha512 ; echo)
>
> diff <(shasum -a512 apache-druid-0.22.1-bin.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-0.22.1-bin.tar.gz.sha512 ; echo)
>
> Verify signatures:
> gpg --verify apache-druid-0.22.1-src.tar.gz.asc \
> apache-druid-0.22.1-src.tar.gz
>
> gpg --verify apache-druid-0.22.1-bin.tar.gz.asc \
> apache-druid-0.22.1-bin.tar.gz
>
> Please review the proposed artifacts and vote. Note that Apache has
> specific requirements that must be met before +1 binding votes can be cast
> by PMC members. Please refer to the policy at
> http://www.apache.org/legal/release-policy.html#policy for more details.
>
> As part of the validation process, the RAT license check can be run from
> source by:
> mvn apache-rat:check -Prat
>
> The release artifacts can be generated from source by running:
> mvn clean install -Papache-release,dist -Dgpg.skip
>
> The vote will pass if a majority of at least three +1 PMC votes are cast.
> Because we want to release this version as soon as possible, the vote will
> be closed as soon as it passes.
>
> [ ] +1 Release this package as Apache Druid 0.22.1
> [ ] 0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
> Thanks!

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [VOTE] Release Apache Druid 0.22.1 [RC1]

[Clint Wylie]

Also a +0 from me, because the fix for the issue Gian mentioned is
available in https://github.com/apache/druid/pull/12056 and it seems
low risk

Release looked good otherwise, so anyone that doesn't want to wait
would probably be ok to go ahead and start using RC1 if they don't
mind the logging issue.

On Fri, Dec 10, 2021 at 4:23 PM Gian Merlino  wrote:
>
> My vote is 0 on this release.
>
> I verified the usual things, and compared the src and bin packages against
> 0.22.0 to make sure there were no unexpected changes. That all looks OK to
> me. But there is an issue with weird errors at the end of logfiles for
> processes that exit normally. It's especially noticeable for indexing
> tasks. It looks like some issue with the shutter-downer stuff that was
> introduced in https://github.com/apache/druid/pull/1387. The error looks
> like the following. If this is fixable I think it'd be good to fix it in
> 0.22.1-rc2. But if we can't get that done today, I think it's OK to go
> ahead with releasing 0.22.1-rc1 with this listed as a known issue, because
> the security issue is serious and shouldn't wait another day.
>
> 2021-12-10T23:13:38,740 ERROR [main]
> org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler
> - Exception when stopping method[public void
> org.apache.druid.initialization.Log4jShutterDownerModule$Log4jShutterDowner.stop()]
> on
> object[org.apache.druid.initialization.Log4jShutterDownerModule$Log4jShutterDowner@30a5a58d
> ]
> java.lang.reflect.InvocationTargetException: null
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:?]
> at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:?]
> at
> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:?]
> at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
> at
> org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler.stop(Lifecycle.java:465)
> [druid-core-0.22.1.jar:0.22.1]
> at
> org.apache.druid.java.util.common.lifecycle.Lifecycle.stop(Lifecycle.java:368)
> [druid-core-0.22.1.jar:0.22.1]
> at org.apache.druid.cli.CliPeon.run(CliPeon.java:323)
> [druid-services-0.22.1.jar:0.22.1]
> at org.apache.druid.cli.Main.main(Main.java:113)
> [druid-services-0.22.1.jar:0.22.1]
> Caused by: org.apache.druid.java.util.common.ISE: Expected state [STARTED]
> found [INITIALIZED]
> at
> org.apache.druid.common.config.Log4jShutdown.stop(Log4jShutdown.java:105)
> ~[druid-core-0.22.1.jar:0.22.1]
> at
> org.apache.druid.initialization.Log4jShutterDownerModule$Log4jShutterDowner.stop(Log4jShutterDownerModule.java:116)
> ~[druid-server-0.22.1.jar:0.22.1]
> ... 8 more
> Finished peon task
>
> On Fri, Dec 10, 2021 at 12:25 PM Jihoon Son  wrote:
>
> > Hi all,
> >
> > I have created a build for Apache Druid 0.22.1, release
> > candidate 1.
> >
> > Thanks to everyone who has helped contribute to the release! You can read
> > the proposed release notes here:
> > https://github.com/apache/druid/issues/12054
> >
> > The release candidate has been tagged in GitHub as
> > druid-0.22.1-rc1 (c052fa52b2266e25c5c31b9156c530aa29aeb147),
> > available here:
> > https://github.com/apache/druid/releases/tag/druid-0.22.1-rc1
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/druid/0.22.1-rc1/
> >
> > A staged Maven repository is available for review at:
> > https://repository.apache.org/content/repositories/orgapachedruid-1027/
> >
> > A Docker image containing the binary of the release candidate can be
> > retrieved via:
> > docker pull apache/druid:0.22.1-rc1
> >
> > artifact checksums
> > src:
> >
> > 9a6304d2c434e0a8226ef8621c1d6653cf80eca10e2be24936b98ec4f7cc19f80193599aea6cd3992a6998e600134e6e71d001630b7e51ca0ab98310850cc064
> > bin:
> >
> > eceacdb0ffca7da462eddc31aaed735c02f639c6f7bafc826fd050095c00a84bffc5694f0887d43436d704e3686e437d44e1a226b4f0d7701f9c450c63f1e1c8
> > docker: 929d217c5c86d59b69db3f0202387a4b0103c8a35b7011ce8e00e95582ade380
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/jihoonson.asc
> >
> > This key and the key of other committers can also be found in the project's
> > KEYS file here:
> > https://dist.apache.org/repos/dist/release/druid/KEYS
> >
> > (If you are a committer, please feel free to add your own key to that file
> > by following the instructions in the file's header.)
> >
> >
> > Verify checksums:
> > diff <(shasum -a512 apache-druid-0.22.1-src.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-0.22.1-src.tar.gz.sha512 ; echo)
> >
> > diff <(shasum -a512 apache-druid-0.22.1-bin.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-0.22.1-bin.tar.gz.sha512 ; echo)
> >
> > Verify signatures:
> > gpg --verify apache-druid-0.22.1-src.tar.gz.asc \
> > apache-druid-0.22.1-src.tar.gz
> >
> > gpg --verify apache-druid-0.22.1-bin.tar.gz.asc \
> > apache-druid-0.22.1-bin.tar.gz

CVE-2021-36749: Apache Druid: The HTTP inputSource allows authenticated users to read data from other sources than intended (incomplete fix of CVE-2021-26920)

[Clint Wylie]

Severity: low

Description:

In the Druid ingestion system, the InputSource is used for reading
data from a certain data source. However, the HTTP InputSource allows
authenticated users to read data from other sources than intended,
such as the local file system, with the privileges of the Druid server
process. This is not an elevation of privilege when users access Druid
directly, since Druid also provides the Local InputSource, which
allows the same level of access. But it is problematic when users
interact with Druid indirectly through an application that allows
users to specify the HTTP InputSource, but not the Local InputSource.
In this case, users could bypass the application-level restriction by
passing a file URL to the HTTP InputSource.

This issue was previously mentioned as being fixed in 0.21.0 as per
CVE-2021-26920 but was not fixed in 0.21.0 or 0.21.1.

Mitigation:

Users can avoid the issue by upgrading to 0.22.0 or a higher version.

In an earlier version than 0.22.0, when the user application wants to
restrict the access to the local file system, it should disallow all
InputSources that can read local files, that is the Local, HTTP, and
HDFS InputSources.

Credit:

This issue was originally discovered by chybeta from the Security Team
of Alibaba Cloud.
ABKing and g0udan from the Security Team of Xiaomi discovered that it
was still an issue after CVE-2021-26920.

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

[ANNOUNCE] Apache Druid 0.22.0 release

[Clint Wylie]

The Apache Druid team is proud to announce the release of Apache Druid 0.22.0.
Druid is a high performance analytics data store for event-driven data.

Apache Druid 0.22.0 contains over 400 new features, performance
enhancements, bug fixes, and documentation improvements from 73
contributors. Major new features and improvements include:

- Support for number formatting and bitwise operations
- Support for ARRAY_AGG and STRING_AGG SQL aggregators
- Improved batch ingestion memory usage
- Support for using deep storage for native batch shuffle storage
- Experimental task autoscaling for Apache Kafka and Amazon Kinesis
based streaming ingestion
- Apache Avro and Protobuf streaming 'InputFormat' implementations,
and improved support of Confluent Schema Registry
- Improvements to both manual and automatic compaction
- Automatic cleanup of metadata storage

Source and binary distributions can be downloaded from:
https://druid.apache.org/downloads.html

Release notes are at:
https://github.com/apache/druid/releases/tag/druid-0.22.0

A big thank you to all the contributors in this milestone release!

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [VOTE] Release Apache Druid 0.22.0 [RC1]

[Clint Wylie]

This vote has passed, the final results can be seen in this thread:
https://lists.apache.org/thread.html/r6eef770f7786269c97efcbcd856394e7b2496c3ef146a4b633aa1e2a%40%3Cdev.druid.apache.org%3E

Thanks everyone!

On Mon, Sep 20, 2021 at 11:44 PM Maytas Monsereenusorn
 wrote:
>
> +1 (binding)
>
> src
> - verified the signature and checksum
> - LICENSE and NOTICE present
> - compiled and ran the licenses.yaml file check
> - compiled and ran s3 deep storage integration tests
> - ran RAT check
> - built binary, ingested some data using batch and ran some queries
>
> bin
> - verified the signature and checksum
> - LICENSE and NOTICE present
> - ingested some data using batch ingestion and ran some queries
>
> On Tue, Sep 21, 2021 at 10:41 AM Jihoon Son  wrote:
>
> > +1 (binding)
> >
> > src
> > - verified the signature and checksum
> > - LICENSE and NOTICE present
> > - compiled and ran the licenses.yaml file check
> > - ran RAT check
> > - built binary, ingested some data using batch and kafka ingestion, and
> > ran some queries
> >
> > bin
> > - verified the signature and checksum
> > - LICENSE and NOTICE present
> > - ingested some data using batch and kafka ingestion and ran some queries
> >
> > docker
> > - verified checksum
> > - ingested some data using batch ingestion and ran some queries
> >
> > On Mon, Sep 20, 2021 at 8:12 PM Suneet Saldanha
> >  wrote:
> > >
> > > +1 (binding)
> > >
> > > *Source code*
> > >  - NOTICE and LICENSE files present
> > >  - verified signatures and checksums
> > >  - git.version file is present and correct
> > >  - maven build passes locally
> > >  - tests are green on travis for the latest commit in the release branch
> > > https://app.travis-ci.com/github/apache/druid/builds/237760487
> > >  - mvn rat checks passed
> > >  - ran the batch ingestion quickstart with some simple queries
> > >
> > > *Binary*
> > >  - NOTICE and LICENSE files present
> > >  - Verified signatures and checksums
> > >  - ran quickstart and did wikipedia ingest + sample queries
> > >
> > > On Sat, Sep 18, 2021 at 3:34 AM frank chen  wrote:
> > >
> > > > +1
> > > >
> > > > *Source code*
> > > >
> > > >- NOTICE and LICENSE files present
> > > >- verified signatures and checksums
> > > >- git.version file is present and correct
> > > >- mvn build passed with unit tests
> > > >- mvn rat checks passed
> > > >- ran cluster with built binaries and ran native batch ingestion
> > > >followed by basic queries
> > > >
> > > >
> > > > *Binary*
> > > >
> > > >- NOTICE and LICENSE files present
> > > >- Verified signatures and checksums
> > > >- ran cluster with pre-packaged binaries and ran both native and
> > kafka
> > > >ingestion followed by basic queries
> > > >
> > > >
> > > > *Docker*
> > > >
> > > >- verified checksum
> > > >- started docker by docker-compose.xml
> > > >- ran native ingestion and kafka ingestion followed by basic queries
> > > >
> > > >
> > > > Thanks.
> > > >
> > > > On Thu, Sep 16, 2021 at 3:32 PM Clint Wylie  wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > I have created a build for Apache Druid 0.22.0, release
> > > > > candidate 1.
> > > > >
> > > > > Thanks to everyone who has helped contribute to the release! You can
> > read
> > > > > the proposed release notes here:
> > > > > https://github.com/apache/druid/issues/11657
> > > > >
> > > > > The release candidate has been tagged in GitHub as
> > > > > druid-0.22.0-rc1 (cc603d6118cf7f14056744d608b93d4d8fd4e710),
> > > > > available here:
> > > > > https://github.com/apache/druid/releases/tag/druid-0.22.0-rc1
> > > > >
> > > > > The artifacts to be voted on are located here:
> > > > > https://dist.apache.org/repos/dist/dev/druid/0.22.0-rc1/
> > > > >
> > > > > A staged Maven repository is available for review at:
> > > > >
> > https://repository.apache.org/content/repositories/orgapachedruid-1026/
> > > > >
> > > > &

[RESULT] [VOTE] Release Apache Druid 0.22.0 [RC1]

[Clint Wylie]

Thanks to everyone who participated in the vote! The vote has passed
with 4 binding +1s and 1 non-binding +1.

Clint Wylie: +1 (binding)
Frank Chen: +1 (non-binding)
Suneet Saldanha: +1 (binding)
Jihoon Son: +1 (binding)
Maytas Monsereenusorn: +1 (binding)

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

[VOTE] Release Apache Druid 0.22.0 [RC1]

[Clint Wylie]

Hi all,

I have created a build for Apache Druid 0.22.0, release
candidate 1.

Thanks to everyone who has helped contribute to the release! You can read
the proposed release notes here:
https://github.com/apache/druid/issues/11657

The release candidate has been tagged in GitHub as
druid-0.22.0-rc1 (cc603d6118cf7f14056744d608b93d4d8fd4e710),
available here:
https://github.com/apache/druid/releases/tag/druid-0.22.0-rc1

The artifacts to be voted on are located here:
https://dist.apache.org/repos/dist/dev/druid/0.22.0-rc1/

A staged Maven repository is available for review at:
https://repository.apache.org/content/repositories/orgapachedruid-1026/

Staged druid.apache.org website documentation is available here:
https://druid.staged.apache.org/docs/0.22.0/design/index.html

A Docker image containing the binary of the release candidate can be
retrieved via:
docker pull apache/druid:0.22.0-rc1

artifact checksums
src:
9073e4e4f1dedcddc7c644f11480ed5354093d3aff26986780e871b20b1bcfd64f9aafc7cc9e82aeb169c6534274aa3fa3f8710d53f3dd9c704ec32de7b2141d
bin:
6a2d191cda37e712a39e59066259f028fadf407a6b3f7a746908cd3c4ed10e6d43008778a045eefc551dbbc3e47f8c1cdbfaab44845fbc217d22029bc3b3c3de
docker: 626fd96a997361dce8452c68b28e935a2453153f0d743cf208a0b4355a4fc2c3

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/cwylie.asc

This key and the key of other committers can also be found in the project's
KEYS file here:
https://dist.apache.org/repos/dist/release/druid/KEYS

(If you are a committer, please feel free to add your own key to that file
by following the instructions in the file's header.)


Verify checksums:
diff <(shasum -a512 apache-druid-0.22.0-src.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-0.22.0-src.tar.gz.sha512 ; echo)

diff <(shasum -a512 apache-druid-0.22.0-bin.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-0.22.0-bin.tar.gz.sha512 ; echo)

Verify signatures:
gpg --verify apache-druid-0.22.0-src.tar.gz.asc \
apache-druid-0.22.0-src.tar.gz

gpg --verify apache-druid-0.22.0-bin.tar.gz.asc \
apache-druid-0.22.0-bin.tar.gz

Please review the proposed artifacts and vote. Note that Apache has
specific requirements that must be met before +1 binding votes can be cast
by PMC members. Please refer to the policy at
http://www.apache.org/legal/release-policy.html#policy for more details.

As part of the validation process, the release artifacts can be generated
from source by running:
mvn clean install -Papache-release,dist -Dgpg.skip

The RAT license check can be run from source by:
mvn apache-rat:check -Prat

This vote will be open for at least 72 hours. The vote will pass if a
majority of at least three +1 PMC votes are cast.

[ ] +1 Release this package as Apache Druid 0.22.0
[ ] 0 I don't feel strongly about it, but I'm okay with the release
[ ] -1 Do not release this package because...

Thanks!

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: About maintaining the Helm's Chart of Apache Druid

[Clint Wylie]

My understanding of that thread suggests that
https://incubator.apache.org/ip-clearance/ip-clearance-template.html
is the process that our PMC does before the IPMC can continue the
clearance process, meaning someone on our PMC fill out i think this
form 
http://svn.apache.org/repos/asf/incubator/public/trunk/content/ip-clearance/ip-clearance-template.xml,
and check the things in and send the emails and such listed out in the
"process" section.

I'm going to remove the helm chart from the 0.22.0 release for now
since I can't find any record of this having been done yet (I can add
it back if I missed it and we haven't yet released).

On Sun, Jul 4, 2021 at 11:07 PM Benedict Jin  wrote:
>
> Hi Jihoon,
>
> Last week I asked the AFS, and according to the replay, it seems that only 
> our IPMC has the authority to launch the IP Clearance process. FYI, 
> https://lists.apache.org/thread.html/rfbfc5951c4524c0e68223e4fbe05a7d7ee26c185ab557d6f77a4989d%40%3Cgeneral.incubator.apache.org%3E
>
> Regards,
> Benedict Jin
>
> On 2021/07/02 22:56:08, Jihoon Son  wrote:
> > Hey Benedict,
> >
> > Any updates on this issue? I think we are going to start the release
> > process for 0.22.0 soon.
> >
> > On Fri, Jul 2, 2021 at 1:19 AM Benedict Jin  wrote:
> >
> > > Hi Xavier,
> > >
> > > I'm so happy to hear that and look forward to your changes will be
> > > contributed to upstream. In fact, Helm and Operator are not in conflict,
> > > their relationship is kind like RPM and Systemd. You can even convert Helm
> > > into Operator, or build Operator based on Helm. And I agree with you that
> > > it would be better if we can define user scenarios.
> > >
> > > Regards,
> > > Benedict Jin
> > >
> > > On 2021/06/25 22:42:32, Xavier Léauté 
> > > wrote:
> > > > For what it's worth, we have been using a heavily modified version of
> > > this
> > > > helm chart at Confluent.
> > > >
> > > > I would say it is good to get a Druid cluster up and running quickly, 
> > > > but
> > > > we had to make some significant changes to make it easier to operate a
> > > > Druid cluster.
> > > > It's great for initial deployment and getting all the required
> > > dependencies
> > > > in place, but operations are somewhat painful and require a lot of
> > > internal
> > > > Druid knowledge to not shoot yourself in the foot.
> > > > Our original intention was to contribute back those changes upstream, 
> > > > but
> > > > we have not had the time to put it in a shape that would allow others to
> > > > use it.
> > > >
> > > > We should try to define what we want this chart to be used for, since I
> > > > think the Druid k8s operator is probably a better choice for someone to
> > > run
> > > > and upgrade a meaningful cluster.
> > > > Another option would be to focus our effort on the Druid operator and
> > > maybe
> > > > build a helm chart to get that and our external dependencies in place, I
> > > > think we can provide a better experience that way.
> > > > One concern with the pure helm chart is that we'll get a lot of 
> > > > questions
> > > > on how to operate it that will likely take a lot of time to answer.
> > > > Considering we'd have helm, k8s operator, and docker-compose, I think we
> > > > should be conscious of the time it would take to maintain all those ways
> > > of
> > > > running Druid in containers and what purpose each of them serves.
> > > >
> > > > Just my 2¢,
> > > > Xavier
> > > >
> > > > On Tue, Jun 22, 2021 at 8:04 AM Benedict Jin 
> > > wrote:
> > > >
> > > > > Hi Jihoon Son,
> > > > >
> > > > > Cool, thanks a lot 👍👍👍
> > > > >
> > > > > Regards,
> > > > > Benedict Jin
> > > > >
> > > > > On 2021/06/21 17:10:13, Jihoon Son  wrote:
> > > > > > Thanks Benedict.
> > > > > > You can find another example of the IP clearance process here:
> > > > > >
> > > > >
> > > https://mail-archives.apache.org/mod_mbox/incubator-general/202106.mbox/browser
> > > > > .
> > > > > >
> > > > > > On Fri, May 28, 2021 at 1:01 AM Benedict Jin 
> > > > > wrote:
> > > > > > >
> > > > > > > Hi Jihoon Son,
> > > > > > >
> > > > > > > Yes, it has only been tested on the local cluster. Next, I will 
> > > > > > > add
> > > > > the automated test part.
> > > > > > >
> > > > > > > Because Helm Chart usually releases a new version every time it is
> > > > > modified. So Apache Superset has followed this approach. Of course, I
> > > agree
> > > > > with your suggestion, we shouldn't release the version of Chart
> > > separately,
> > > > > and follow the release rhythm of Apache Druid.
> > > > > > >
> > > > > > > I've been busy recently, but I also took time to study this IP
> > > > > licensing process, and I'm filling out related forms. Thank you very
> > > much
> > > > > for the mailing list, it's very useful to me.
> > > > > > >
> > > > > > > Okay, I got it. We will solve this problem as soon as possible
> > > before
> > > > > the release of 0.22.0.
> > > > > > >
> > > > > > > Regards,
> > > > > > > Benedict Jin
> > > > > > >
> > > > > > > On 2021/05/22 18:53:35

0.22

[Clint Wylie]

I know I mentioned I was going to start the 0.22 release like over a
month ago, but haven't really had the chance to actually work on that
until now...

Since it has been a while, I wanted to start a thread to mention I'm
currently looking over the changes between 0.21 and 0.22, as well as
the set of open PRs, to see if there is anything that should be
addressed before making the 0.22 branch. If anyone is aware of
anything that should be included, feel free to add to this thread.
Otherwise, assuming I don't find any blockers, I expect I should make
the branch/code freeze later this week.

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

[ANNOUNCE] Apache Druid 0.21.1 release

[Clint Wylie]

The Apache Druid team is proud to announce the release of Apache Druid 0.21.1.
Druid is a high performance analytics data store for event-driven data.

Apache Druid 0.21.1 is a minor release to fix regressions introduced in 0.21.0.
Source and binary distributions can be downloaded
from:https://druid.apache.org/downloads.html

Release notes are at:https://github.com/apache/druid/releases/tag/druid-0.21.1

A big thank you to all the contributors that helped fixed the issues
for this release!

Re: [E] [DISCUSS] Hadoop 3, dropping support for Hadoop 2.x

[Clint Wylie]

s an extension.
> Since hadoop2 has not been EOF, to achieve balance between compatibility
> and long term evolution, maybe we could provide two extensions, one for
> hadoop2, one for hadoop3.
>
>
>
> Will Lauer  于2021年6月9日周三 上午4:13写道：
>
> > Just to follow up on this, our main problem with hadoop3 right now has
> been
> > instability in HDFS, to the extent that we put on hold any plans to
> deploy
> > it to our production systems. I would claim Hadoop3 isn't mature enough
> yet
> > to consider migrating Druid to it.
> >
> > WIll
> >
> > <http://www.verizonmedia.com>
> >
> > Will Lauer
> >
> > Senior Principal Architect, Audience & Advertising Reporting
> > Data Platforms & Systems Engineering
> >
> > M 508 561 6427
> > 1908 S. First St
> > Champaign, IL 61822
> >
> > <http://www.facebook.com/verizonmedia>   <
> http://twitter.com/verizonmedia>
> > <https://www.linkedin.com/company/verizon-media/>
> > <http://www.instagram.com/verizonmedia>
> >
> >
> >
> > On Tue, Jun 8, 2021 at 2:59 PM Will Lauer 
> wrote:
> >
> > > Unfortunately, the migration off of hadoop3 is a hard one (maybe not
> for
> > > Druid, but certainly for big organizations running large hadoop2
> > > workloads). If druid migrated to hadoop3 after 0.22, that would
> probably
> > > prevent me from taking any new versions of Druid for at least the
> > remainder
> > > of the year and possibly longer.
> > >
> > > Will
> > >
> > >
> > > <http://www.verizonmedia.com>
> > >
> > > Will Lauer
> > >
> > > Senior Principal Architect, Audience & Advertising Reporting
> > > Data Platforms & Systems Engineering
> > >
> > > M 508 561 6427
> > > 1908 S. First St
> > > Champaign, IL 61822
> > >
> > > <http://www.facebook.com/verizonmedia>   <
> > http://twitter.com/verizonmedia>
> > ><https://www.linkedin.com/company/verizon-media/>
> > > <http://www.instagram.com/verizonmedia>
> > >
> > >
> > >
> > > On Tue, Jun 8, 2021 at 3:08 AM Clint Wylie  wrote:
> > >
> > >> Hi all,
> > >>
> > >> I've been assisting with some experiments to see how we might want to
> > >> migrate Druid to support Hadoop 3.x, and more importantly, see if
> maybe
> > we
> > >> can finally be free of some of the dependency issues it has been
> causing
> > >> for as long as I can remember working with Druid.
> > >>
> > >> Hadoop 3 introduced shaded client jars,
> > >>
> > >>
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_HADOOP-2D11804&d=DwIBaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=ULseRJUsY5gTBgFA9-BUxg&m=FRw8adGvb_qAPLtFgQWNJywJiOgU8zgfkkXf_nokPKQ&s=rBnEOMf2IKDMeWUo4TZyqf5CzrnbiYTfZUkjHr8GOHo&e=
> > >> , with the purpose to
> > >> allow applications to talk to the Hadoop cluster without drowning in
> its
> > >> transitive dependencies. The experimental branch that I have been
> > helping
> > >> with, which is using these new shaded client jars, can be seen in this
> > PR
> > >>
> > >>
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_druid_pull_11314&d=DwIBaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=ULseRJUsY5gTBgFA9-BUxg&m=FRw8adGvb_qAPLtFgQWNJywJiOgU8zgfkkXf_nokPKQ&s=424doHggbejAz5XswosgVkJK98VUBcUj0pD5bAcBjT0&e=
> > >> , and is currently working with
> > >> the HDFS integration tests as well as the Hadoop tutorial flow in the
> > >> Druid
> > >> docs (which is pretty much equivalent to the HDFS integration test).
> > >>
> > >> The cloud deep storages still need some further testing and some minor
> > >> cleanup still needs done for the docs and such. Additionally we still
> > need
> > >> to figure out how to handle the Kerberos extension, because it extends
> > >> some
> > >> Hadoop classes so isn't able to use the shaded client jars in a
> > >> straight-forward manner, and so still has heavy dependencies and
> hasn't
> > >> been tested. However, the experiment has started to pan out enough to
> > >> where
> > >> I think it is worth starting this discussion, because it does have
> some
> > >> implications.
> > >>
> > >> Making this change I think will allow us to update our dependencies
> > with a
> > >> lot more freedom (I'm looking at you, Guava), but the catch is that
> once
> > >> we
> > >> make this change and start updating these dependencies, it will become
> > >> hard, nearing impossible to support Hadoop 2.x, since as far as I know
> > >> there isn't an equivalent set of shaded client jars. I am also not
> > certain
> > >> how far back the Hadoop job classpath isolation stuff goes
> > >> (mapreduce.job.classloader = true) which I think is required to be set
> > on
> > >> Druid tasks for this shaded stuff to work alongside updated Druid
> > >> dependencies.
> > >>
> > >> Is anyone opposed to or worried about dropping Hadoop 2.x support
> after
> > >> the
> > >> Druid 0.22 release?
> > >>
> > >
> >
>

Re: [VOTE] Release Apache Druid 0.21.1 [RC2]

[Clint Wylie]

This vote has passed, the final results can be seen in this thread:
https://lists.apache.org/thread.html/r3c7db826cdf9025efa2f3906e4f0d2ff69b66ae4e3513212a0bad2e3%40%3Cdev.druid.apache.org%3E

On Tue, Jun 8, 2021 at 5:38 PM Jonathan Wei  wrote:

> +1 (binding)
> src
> - verified signature/checksum
> - LICENSE/NOTICE present
> - ran RAT check
> - ran unit tests
> - built binary and ran ingestion tutorial and a few queries
>
> bin
> - verified signature/checksum
> - LICENSE/NOTICE present
> - ran ingestion tutorial and a few queries
>
> docker
> - built docker image from source on linux
> - ran docker-compose quickstart cluster on linux, ran ingestion tutorial
> and a few queries
>
> On Sat, Jun 5, 2021 at 2:01 PM Jihoon Son  wrote:
>
> > +1 (binding)
> >
> > src
> > - verified the signature and checksum
> > - LICENSE and NOTICE are present
> > - compiled and ran the license check and unit tests
> > - built binary, ingested some data via batch and kafka ingestion, and
> > ran some queries
> >
> > bin
> > - verified the signature and checksum
> > - LICENSE and NOTICE are present
> > - ingested some data via batch and kafka ingestion and ran some queries
> >
> > docker
> > - verified checksum
> > - ran a cluster after cleaning up existing volumes on linux
> > - ingested some data via batch ingestion and ran some queries
> >
> > On Thu, Jun 3, 2021 at 10:43 PM frank chen  wrote:
> > >
> > > +1
> > >
> > > src
> > > - verified .asc signatures and .sha512 checksums
> > > - LICENSE/NOTICE present
> > > - compiled, ran the license check/unit tests
> > > - started nano-quickstart cluster and ran both native ingestion and
> Kafka
> > > ingestion followed by some basic queries
> > >
> > > bin
> > > - verified .asc signatures and .sha512 checksums verified
> > > - LICENSE/NOTICE present
> > > - started nano-quickstart cluster and ran both native ingestion and
> Kafka
> > > ingestion
> > >
> > > docker
> > > - started docker on Linux(CentOS) and ran both native ingestion and
> Kafka
> > > ingestion followed by some basic queries
> > > - started docker on macOS and ran both native ingestion and Kafka
> > ingestion
> > > followed by some basic queries
> > >
> > >
> > > Clint Wylie  于2021年6月3日周四 下午5:00写道：
> > >
> > > > Hi all,
> > > >
> > > > I have created a build for Apache Druid 0.21.1, release
> > > > candidate 2.
> > > >
> > > > Thanks to everyone who has helped contribute to the release! You can
> > read
> > > > the proposed release notes here:
> > > > https://github.com/apache/druid/issues/11249
> > > >
> > > > The release candidate has been tagged in GitHub as
> > > > druid-0.21.1-rc2 (6ba6b16786eca5a25ebad2f00df4b2a265861b01),
> > > > available here:
> > > > https://github.com/apache/druid/releases/tag/druid-0.21.1-rc2
> > > >
> > > > The artifacts to be voted on are located here:
> > > > https://dist.apache.org/repos/dist/dev/druid/0.21.1-rc2/
> > > >
> > > > A staged Maven repository is available for review at:
> > > >
> > https://repository.apache.org/content/repositories/orgapachedruid-1025/
> > > >
> > > > Staged druid.apache.org website documentation is available here:
> > > > https://druid.staged.apache.org/docs/0.21.1/design/index.html
> > > >
> > > > A Docker image containing the binary of the release candidate can be
> > > > retrieved via:
> > > > docker pull apache/druid:0.21.1-rc2
> > > >
> > > > artifact checksums
> > > > src:
> > > >
> > > >
> >
> 65eff0c302c316afbf4a84c61a9f54a1baf5ff3cd1baf390d9034665f7e4fbc7457e108c8c5a4ac66350ee69d922e5bcdacde14abac4b54398378b99022acd16
> > > > bin:
> > > >
> > > >
> >
> a68c63ddf92a0939315bd8b79fbdd5fae712e888d1a0e0466f0a87a8e8f1270d70fc5dcc0ec64e076391576be29da1cce88f7c5cf725c31418601e4eee8fa354
> > > > docker:
> > a3a301693db0eea5af1278535586e7f9768bd497231d6748f35a983962780371
> > > >
> > > > Release artifacts are signed with the following key:
> > > > https://people.apache.org/keys/committer/cwylie.asc
> > > >
> > > > This key and the key of other committers can also be found in the
> > project's
> > > > KEYS fil

[RESULT] [VOTE] Release Apache Druid 0.21.1 [RC2]

[Clint Wylie]

Thanks to everyone who participated in the vote! The vote has passed with 3
binding +1s and 1 non-binding +1.

Clint Wylie: +1 (binding)
Frank Chen: +1 (non-binding)
Jihoon Son: +1 (binding)
Jon Wei: +1 (binding)

[DISCUSS] Hadoop 3, dropping support for Hadoop 2.x

[Clint Wylie]

Hi all,

I've been assisting with some experiments to see how we might want to
migrate Druid to support Hadoop 3.x, and more importantly, see if maybe we
can finally be free of some of the dependency issues it has been causing
for as long as I can remember working with Druid.

Hadoop 3 introduced shaded client jars,
https://issues.apache.org/jira/browse/HADOOP-11804, with the purpose to
allow applications to talk to the Hadoop cluster without drowning in its
transitive dependencies. The experimental branch that I have been helping
with, which is using these new shaded client jars, can be seen in this PR
https://github.com/apache/druid/pull/11314, and is currently working with
the HDFS integration tests as well as the Hadoop tutorial flow in the Druid
docs (which is pretty much equivalent to the HDFS integration test).

The cloud deep storages still need some further testing and some minor
cleanup still needs done for the docs and such. Additionally we still need
to figure out how to handle the Kerberos extension, because it extends some
Hadoop classes so isn't able to use the shaded client jars in a
straight-forward manner, and so still has heavy dependencies and hasn't
been tested. However, the experiment has started to pan out enough to where
I think it is worth starting this discussion, because it does have some
implications.

Making this change I think will allow us to update our dependencies with a
lot more freedom (I'm looking at you, Guava), but the catch is that once we
make this change and start updating these dependencies, it will become
hard, nearing impossible to support Hadoop 2.x, since as far as I know
there isn't an equivalent set of shaded client jars. I am also not certain
how far back the Hadoop job classpath isolation stuff goes
(mapreduce.job.classloader = true) which I think is required to be set on
Druid tasks for this shaded stuff to work alongside updated Druid
dependencies.

Is anyone opposed to or worried about dropping Hadoop 2.x support after the
Druid 0.22 release?

[VOTE] Release Apache Druid 0.21.1 [RC2]

[Clint Wylie]

Hi all,

I have created a build for Apache Druid 0.21.1, release
candidate 2.

Thanks to everyone who has helped contribute to the release! You can read
the proposed release notes here:
https://github.com/apache/druid/issues/11249

The release candidate has been tagged in GitHub as
druid-0.21.1-rc2 (6ba6b16786eca5a25ebad2f00df4b2a265861b01),
available here:
https://github.com/apache/druid/releases/tag/druid-0.21.1-rc2

The artifacts to be voted on are located here:
https://dist.apache.org/repos/dist/dev/druid/0.21.1-rc2/

A staged Maven repository is available for review at:
https://repository.apache.org/content/repositories/orgapachedruid-1025/

Staged druid.apache.org website documentation is available here:
https://druid.staged.apache.org/docs/0.21.1/design/index.html

A Docker image containing the binary of the release candidate can be
retrieved via:
docker pull apache/druid:0.21.1-rc2

artifact checksums
src:
65eff0c302c316afbf4a84c61a9f54a1baf5ff3cd1baf390d9034665f7e4fbc7457e108c8c5a4ac66350ee69d922e5bcdacde14abac4b54398378b99022acd16
bin:
a68c63ddf92a0939315bd8b79fbdd5fae712e888d1a0e0466f0a87a8e8f1270d70fc5dcc0ec64e076391576be29da1cce88f7c5cf725c31418601e4eee8fa354
docker: a3a301693db0eea5af1278535586e7f9768bd497231d6748f35a983962780371

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/cwylie.asc

This key and the key of other committers can also be found in the project's
KEYS file here:
https://dist.apache.org/repos/dist/release/druid/KEYS

(If you are a committer, please feel free to add your own key to that file
by following the instructions in the file's header.)


Verify checksums:
diff <(shasum -a512 apache-druid-0.21.1-src.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-0.21.1-src.tar.gz.sha512 ; echo)

diff <(shasum -a512 apache-druid-0.21.1-bin.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-0.21.1-bin.tar.gz.sha512 ; echo)

Verify signatures:
gpg --verify apache-druid-0.21.1-src.tar.gz.asc \
apache-druid-0.21.1-src.tar.gz

gpg --verify apache-druid-0.21.1-bin.tar.gz.asc \
apache-druid-0.21.1-bin.tar.gz

Please review the proposed artifacts and vote. Note that Apache has
specific requirements that must be met before +1 binding votes can be cast
by PMC members. Please refer to the policy at
http://www.apache.org/legal/release-policy.html#policy for more details.

As part of the validation process, the release artifacts can be generated
from source by running:
mvn clean install -Papache-release,dist -Dgpg.skip

The RAT license check can be run from source by:
mvn apache-rat:check -Prat

This vote will be open for at least 72 hours. The vote will pass if a
majority of at least three +1 PMC votes are cast.

[ ] +1 Release this package as Apache Druid 0.21.1
[ ] 0 I don't feel strongly about it, but I'm okay with the release
[ ] -1 Do not release this package because...

Thanks!

[CANCEL] [VOTE] Release Apache Druid 0.21.1 [RC1]

[Clint Wylie]

cancelling to add fix for docker volume permissions,
https://github.com/apache/druid/pull/11299

Re: [VOTE] Release Apache Druid 0.21.1 [RC1]

[Clint Wylie]

Thanks for the votes, though I think since
https://github.com/apache/druid/pull/11299 appears to fix the issue
described in this thread that I should go ahead and cancel this vote and do
an RC2. I'll try to get new artifacts prepared and a vote up soon.

On Fri, May 28, 2021 at 7:49 AM Lucas Capistrant 
wrote:

> +1 (binding)
>
> src
> - Proper NOTICE and LICENSE files present
> - Verified .asc signatures and .sha512 checksums
> - git.version file is present and correct
> - mvn build passed with unit tests
> - mvn rat checks passed
> - ran cluster with built binaries and executed native batch ingestion
> followed by basic queries and api issued kill task on data. Tested with
> HDFS deep store.
>
> bin
> - Proper NOTICE and LICENSE files present
> - Verified .asc signatures and .sha512 checksums
> - ran cluster with pre-packaged binaries and executed native batch
> ingestion followed by basic queries and api issued kill task on data.
> Tested with HDFS deep store.
>
> On Fri, May 14, 2021 at 2:42 PM Clint Wylie  wrote:
>
> > Hi all,
> >
> > I have created a build for Apache Druid 0.21.1, release
> > candidate 1.
> >
> > Thanks to everyone who has helped contribute to the release! You can read
> > the proposed release notes here:
> > https://github.com/apache/druid/issues/11249
> >
> > The release candidate has been tagged in GitHub as
> > druid-0.21.1-rc3 (9d142a2f19cceef38173e8d463a8cc1dfe1cb7ec),
> > available here:
> > https://github.com/apache/druid/releases/tag/druid-0.21.1-rc1
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/druid/0.21.1-rc1/
> >
> > A staged Maven repository is available for review at:
> > https://repository.apache.org/content/repositories/orgapachedruid-1024/
> >
> > Staged druid.apache.org website documentation is available here:
> > https://druid.staged.apache.org/docs/0.21.1/design/index.html
> >
> > A Docker image containing the binary of the release candidate can be
> > retrieved via:
> > docker pull apache/druid:0.21.1-rc1
> >
> > artifact checksums
> > src:
> >
> >
> 8ac8267e9fa8aebbd7e2aa7bf30cfa598083eea4e862856f2e28521443a74c8b4f3d7ede5ba9837ee802028913995567d8f10e991f2aac069fea0155b30083f9
> > bin:
> >
> >
> 170a8861c2bd00078689316013b32d16e110c021be87bf183898b4eb3ffc1c06ca56e3c7b92ca38f9732ba3cf4ff4c1de22a68d7a4b93e6dc271b3c994416053
> > docker: 16b47d4d03b41aa6c5784b6f356d51bb7d3c50198cb5f8a289d0e9c4d425bb26
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/cwylie.asc
> >
> > This key and the key of other committers can also be found in the
> project's
> > KEYS file here:
> > https://dist.apache.org/repos/dist/release/druid/KEYS
> >
> > (If you are a committer, please feel free to add your own key to that
> file
> > by following the instructions in the file's header.)
> >
> >
> > Verify checksums:
> > diff <(shasum -a512 apache-druid-0.21.1-src.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-0.21.1-src.tar.gz.sha512 ; echo)
> >
> > diff <(shasum -a512 apache-druid-0.21.1-bin.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-0.21.1-bin.tar.gz.sha512 ; echo)
> >
> > Verify signatures:
> > gpg --verify apache-druid-0.21.1-src.tar.gz.asc \
> > apache-druid-0.21.1-src.tar.gz
> >
> > gpg --verify apache-druid-0.21.1-bin.tar.gz.asc \
> > apache-druid-0.21.1-bin.tar.gz
> >
> > Please review the proposed artifacts and vote. Note that Apache has
> > specific requirements that must be met before +1 binding votes can be
> cast
> > by PMC members. Please refer to the policy at
> > http://www.apache.org/legal/release-policy.html#policy for more details.
> >
> > As part of the validation process, the release artifacts can be generated
> > from source by running:
> > mvn clean install -Papache-release,dist -Dgpg.skip
> >
> > The RAT license check can be run from source by:
> > mvn apache-rat:check -Prat
> >
> > This vote will be open for at least 72 hours. The vote will pass if a
> > majority of at least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Druid 0.21.1
> > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Thanks!
> >
>

Re: [VOTE] Release Apache Druid 0.21.1 [RC1]

[Clint Wylie]

I agree that this probably isn't a blocker, but also agree based on the
number of reports that we should consider either fixing so that the
docker-compose quickstart instructions work as is, or adjust the docs so
that the instructions at least include something about setting the
permissions so that things can work correctly. Given that I still don't
have enough PMC votes yet to release 0.21.1, and I don't really have a
strong preference of which thing we do (nor am quite sure on what the
correct docker fix is), anyone have opinions on the best path forward?

On Sat, May 22, 2021 at 3:56 PM Jihoon Son  wrote:

> Xavier, that's good to know! Glad we already have some.
>
> On Fri, May 21, 2021 at 8:41 AM Xavier Léauté
>  wrote:
> >
> > > I agree with you that the root cause is lack of some automated tests
> > > against released docker images.
> > >
> >
> > Since https://github.com/apache/druid/pull/11187 we are now running k8s
> > integration tests against the same image we release.
> > So this should at least catch the kind of issues people would encounter
> > with a typical k8s deployment.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> For additional commands, e-mail: dev-h...@druid.apache.org
>
>

Re: [VOTE] Release Apache Druid 0.21.1 [RC1]

[Clint Wylie]

Hmm, I'm actually still seeing a permissions related issue with the rc1
Docker image and Linux (Docker on Mac OS seems to work ok, which is where I
initially tested).

middlemanager| 2021-05-18T21:20:39,239 INFO [forking-task-runner-0]
org.apache.druid.indexing.overlord.ForkingTaskRunner - Exception caught
during execution
middlemanager| org.apache.druid.java.util.common.IOE: Unable to create
task log dir[/opt/data/indexing-logs]
middlemanager|  at
org.apache.druid.indexing.common.tasklogs.FileTaskLogs.pushTaskLog(FileTaskLogs.java:59)
~[druid-indexing-service-0.21.1.jar:0.21.1]
middlemanager|  at
org.apache.druid.indexing.overlord.ForkingTaskRunner$1.call(ForkingTaskRunner.java:386)
[druid-indexing-service-0.21.1.jar:0.21.1]
middlemanager|  at
org.apache.druid.indexing.overlord.ForkingTaskRunner$1.call(ForkingTaskRunner.java:137)
[druid-indexing-service-0.21.1.jar:0.21.1]
middlemanager|  at
java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_275]
middlemanager|  at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_275]
middlemanager|  at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_275]
middlemanager|  at java.lang.Thread.run(Thread.java:748)
[?:1.8.0_275]
middlemanager| 2021-05-18T21:20:39,247 INFO [forking-task-runner-0]
org.apache.druid.indexing.overlord.ForkingTaskRunner - Removing task
directory:
var/druid/task/index_parallel_wikipedia_gmkopeph_2021-05-18T21:20:10.899Z

the 'storage' directory that is used for tasklogs and deep storage in the
docker-compose file seems to belong to root. If I change ownership to the
user it works.

I am unsure if this is necessarily a blocker because the containers
themselves do seem to actually work ok, and this seems to be an issue with
the locally attached volume deep storage used by the example compose file
(which I also forgot to update the version to 0.21.1 in this release).
Presumably everything would work ok if using something like s3 for deep
storage, though I haven't had the chance to confirm this yet.


On Fri, May 14, 2021 at 12:42 PM Clint Wylie  wrote:

> Hi all,
>
> I have created a build for Apache Druid 0.21.1, release
> candidate 1.
>
> Thanks to everyone who has helped contribute to the release! You can read
> the proposed release notes here:
> https://github.com/apache/druid/issues/11249
>
> The release candidate has been tagged in GitHub as
> druid-0.21.1-rc3 (9d142a2f19cceef38173e8d463a8cc1dfe1cb7ec),
> available here:
> https://github.com/apache/druid/releases/tag/druid-0.21.1-rc1
>
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/druid/0.21.1-rc1/
>
> A staged Maven repository is available for review at:
> https://repository.apache.org/content/repositories/orgapachedruid-1024/
>
> Staged druid.apache.org website documentation is available here:
> https://druid.staged.apache.org/docs/0.21.1/design/index.html
>
> A Docker image containing the binary of the release candidate can be
> retrieved via:
> docker pull apache/druid:0.21.1-rc1
>
> artifact checksums
> src:
>
> 8ac8267e9fa8aebbd7e2aa7bf30cfa598083eea4e862856f2e28521443a74c8b4f3d7ede5ba9837ee802028913995567d8f10e991f2aac069fea0155b30083f9
> bin:
>
> 170a8861c2bd00078689316013b32d16e110c021be87bf183898b4eb3ffc1c06ca56e3c7b92ca38f9732ba3cf4ff4c1de22a68d7a4b93e6dc271b3c994416053
> docker: 16b47d4d03b41aa6c5784b6f356d51bb7d3c50198cb5f8a289d0e9c4d425bb26
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/cwylie.asc
>
> This key and the key of other committers can also be found in the project's
> KEYS file here:
> https://dist.apache.org/repos/dist/release/druid/KEYS
>
> (If you are a committer, please feel free to add your own key to that file
> by following the instructions in the file's header.)
>
>
> Verify checksums:
> diff <(shasum -a512 apache-druid-0.21.1-src.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-0.21.1-src.tar.gz.sha512 ; echo)
>
> diff <(shasum -a512 apache-druid-0.21.1-bin.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-0.21.1-bin.tar.gz.sha512 ; echo)
>
> Verify signatures:
> gpg --verify apache-druid-0.21.1-src.tar.gz.asc \
> apache-druid-0.21.1-src.tar.gz
>
> gpg --verify apache-druid-0.21.1-bin.tar.gz.asc \
> apache-druid-0.21.1-bin.tar.gz
>
> Please review the proposed artifacts and vote. Note that Apache has
> specific requirements that must be met before +1 binding votes can be cast
> by PMC members. Please refer to the policy at
> http://www.apache.org/legal/release-policy.html#policy for more details.
>
> As part of the validation process, the 

[VOTE] Release Apache Druid 0.21.1 [RC1]

[Clint Wylie]

Hi all,

I have created a build for Apache Druid 0.21.1, release
candidate 1.

Thanks to everyone who has helped contribute to the release! You can read
the proposed release notes here:
https://github.com/apache/druid/issues/11249

The release candidate has been tagged in GitHub as
druid-0.21.1-rc3 (9d142a2f19cceef38173e8d463a8cc1dfe1cb7ec),
available here:
https://github.com/apache/druid/releases/tag/druid-0.21.1-rc1

The artifacts to be voted on are located here:
https://dist.apache.org/repos/dist/dev/druid/0.21.1-rc1/

A staged Maven repository is available for review at:
https://repository.apache.org/content/repositories/orgapachedruid-1024/

Staged druid.apache.org website documentation is available here:
https://druid.staged.apache.org/docs/0.21.1/design/index.html

A Docker image containing the binary of the release candidate can be
retrieved via:
docker pull apache/druid:0.21.1-rc1

artifact checksums
src:
8ac8267e9fa8aebbd7e2aa7bf30cfa598083eea4e862856f2e28521443a74c8b4f3d7ede5ba9837ee802028913995567d8f10e991f2aac069fea0155b30083f9
bin:
170a8861c2bd00078689316013b32d16e110c021be87bf183898b4eb3ffc1c06ca56e3c7b92ca38f9732ba3cf4ff4c1de22a68d7a4b93e6dc271b3c994416053
docker: 16b47d4d03b41aa6c5784b6f356d51bb7d3c50198cb5f8a289d0e9c4d425bb26

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/cwylie.asc

This key and the key of other committers can also be found in the project's
KEYS file here:
https://dist.apache.org/repos/dist/release/druid/KEYS

(If you are a committer, please feel free to add your own key to that file
by following the instructions in the file's header.)


Verify checksums:
diff <(shasum -a512 apache-druid-0.21.1-src.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-0.21.1-src.tar.gz.sha512 ; echo)

diff <(shasum -a512 apache-druid-0.21.1-bin.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-0.21.1-bin.tar.gz.sha512 ; echo)

Verify signatures:
gpg --verify apache-druid-0.21.1-src.tar.gz.asc \
apache-druid-0.21.1-src.tar.gz

gpg --verify apache-druid-0.21.1-bin.tar.gz.asc \
apache-druid-0.21.1-bin.tar.gz

Please review the proposed artifacts and vote. Note that Apache has
specific requirements that must be met before +1 binding votes can be cast
by PMC members. Please refer to the policy at
http://www.apache.org/legal/release-policy.html#policy for more details.

As part of the validation process, the release artifacts can be generated
from source by running:
mvn clean install -Papache-release,dist -Dgpg.skip

The RAT license check can be run from source by:
mvn apache-rat:check -Prat

This vote will be open for at least 72 hours. The vote will pass if a
majority of at least three +1 PMC votes are cast.

[ ] +1 Release this package as Apache Druid 0.21.1
[ ] 0 I don't feel strongly about it, but I'm okay with the release
[ ] -1 Do not release this package because...

Thanks!

Re: Push-down of operations for SystemSchema tables

[Clint Wylie]

Hi Jason,

Thanks for thinking about this, I would agree that the system tables are a
pain point, especially the segments table in larger clusters.

In the mid term, I think that some of us have been thinking that moving
system tables into the Druid native query engine is the way to go, and have
been working on resolving a number of hurdles that are required to make
this happen. One of the main motivators to do this is so that we have just
the Druid query path in the planner in the Calcite layer, and deprecating
and eventually dropping the "bindable" path completely, described in
https://github.com/apache/druid/issues/9896. System tables would be pushed
into Druid Datasource implementations, and queries would be handled in the
native engine. Gian has even made a prototype of what this might look like,
https://github.com/apache/druid/compare/master...gianm:sql-sys-table-native
since much of the ground work is now in place, though it takes a hard-line
approach of completely removing bindable instead of hiding it behind a
flag, and doesn't implement all of the system tables yet, at least last
time I looked at it.

I think there are a couple of remaining parts to resolve that would make
this feasible. The first is native scan queries need support for ordering
by arbitrary columns, instead of just time, so that we can retain
capabilities of the existing system tables. This isn't actually a blocker
for adding native system table queries, but rather a blocker for replacing
the bindable convention by default so that there isn't a loss (or rather
trade) of functionality. Additionally, I think there is maybe some matters
regarding authorization of system tables when handled by the native engine
that will need resolved, but this can be done while adding the native
implementations.

I think there are some various ideas and experiments underway of how to do
sorting on scan queries at normal Druid datasource scale, which is sort of
a big project, but in the short term we might be able to do something less
ambitious that works well enough at system tables scale to allow this plan
to fully proceed.

Does this approach make sense? I don't believe Gian is actively working on
this at the moment, so I think if you're interested in moving along this
approach and want to start laying the groundwork I'm happy to provide
guidance and help out.

Cheers,
Clint





On Thu, May 13, 2021 at 4:40 PM Julian Hyde  wrote:

> Jason,
>
> > I'm new to Calcite (and Druid) so if I have some terminology
> > incorrect, please point it out.
>
> From a Calcite perspective, I can tell you that your terminology (and
> ideas) seem spot on.
>
> I can’t say whether they make sense in Druid (or are easy to achieve).
>
> Julian
>
>
> > On May 13, 2021, at 4:21 PM, Jason Koch 
> wrote:
> >
> > Hi all,
> >
> > I'm looking to implement push-down for some operations in the
> > SystemSchema class, and looking for your input on the best way to
> > tackle this.
> >
> > With profiling, we have found some UI slowness related to large
> > segment counts and task counts. Inspecting the code, it seems that
> > much of the data is fully materialized before often being discarded
> > [1], which makes it a good opportunity for a pushdown optimization.
> > This would make for a more snappy UI experience for segments, tasks
> > and so on. It would also I believe address #6827 [2].
> >
> > In looking at the code I can see a couple of approaches that might be
> sensible:
> >
> > - Modify the tables to support the linq4j Queryable interface, and
> > have all inputs provided in a single pass. This would be a
> > (relatively) straightforward way of fixing this specific problem,
> > however I am not sure how extensible/reusable this is, and whether I
> > have a correct understanding of Queryable.
> >
> > - Build up a custom RelNode structure for the sys. tables, along with
> > some rules, that could perform the required Logical operations in a
> > single pass on the underlying structures. This seems that perhaps some
> > of the rules would be more reusable and more in line with existing
> > Druid query architecture, however, seems like a more complex solution.
> > I think a starting point would be to convert these tables to
> > `ProjectableFilterableTable` and then develop a RelOptRule to match a
> > LogicalSort+tables and pushing down the required additional sort
> > comparators in. On scan(), then, the sort, projection, and filter
> > details would all be available to perform a single pass.
> >
> > - Any other suggestions or pointers?
> >
> > Other thoughts:
> > - Are there any common query use cases in these schemas that you think
> > we should target as a goal for opt rules? If I know in advance then I
> > can use that to guide the work.
> > - If complexity goes up a lot, especially for the second option, it
> > might be beneficial to move the rules and configuration to a new
> > package (org.apache.druid.sql.calcite.schema.sys?).
> > - It seems that these queries are curr

Re: Apache Druid Release voting requirements

[Clint Wylie]

I agree with Jihoon that deciding what should and shouldn't be in a release
on a feature by feature basis is way too expensive to do at release time.
I think the PR that adds a thing is the most appropriate venue to discuss
whether or not a change should be accepted to a release, unless a major
issue is discovered after being merged. Even then reverting instead of
fixing in place should be a relatively exceptional circumstance when there
is a major issue and a fix would take a long time for a release that is
otherwise ready. It is healthy to review changes while voting on a release
as a sort of spot check (I tend to do it whenever I am release manager
while searching the set of changes to determine if anything should be in
the release notes which was not tagged), since it can help to uncover any
of these potential issues that might have been missed, but anything nearing
formally voting on every feature sounds tedious.

With regards to integration tests, I don't think it is very realistic to
expect voters to be running the integration tests across the entire set of
platforms, access to cloud infrastructure or a hadoop cluster should not be
a barrier to approving a release or not. Not to mention it takes hours and
hours of compute time to run the complete suite. I don't really have a good
solution to this.

On Mon, May 3, 2021 at 10:38 PM Jihoon Son  wrote:

> Thanks Suneet. All look good to me except the ones below.
>
> >- Verify release notes contains a note about all issues marked with
>release notes label
>
> I personally treat the "release notes" label as a mark for the
> candidate PRs that should be noted in the release notes. I sometimes
> intentionally drop some notes if the release notes grow too large when
> I'm the release manager. I'm not sure what others do when they are the
> release manager but think this is OK.
>
> >- [Druid requirement] Vote on whether any features are being released
> that are incomplete - missing integration tests, un-expected change in
> behavior, etc.
>
> Can you elaborate more on this suggestion? Are you suggesting making
> it a part of the release vote process? Or is it voting for each
> feature before starting the release vote?
> To me, making it a part of the release vote process seems too
> expensive because the release vote can start only when the tag is
> ready. I would like to have such discussions before the release vote
> is started.
>
> Today, we have been having such discussions in each PR when someone
> thinks a particular PR should be or should not be in the release.
> These discussions usually do not involve a vote because we usually
> reach a consensus after discussion.
>
> > Does a person voting need to run
> these integration tests today, since they are not part of the automated
> pipeline?
>
> This is a good point. I think we should run them manually for now. In
> the long term, we should find a way to automate those integration
> tests.
>
> On Wed, Apr 28, 2021 at 10:18 PM Suneet Saldanha
>  wrote:
> >
> > Hi Jihoon,
> >
> > Thanks for pointing that out. Then, I'd update my proposal to include
> this
> > step as something that always needs to be done manually.
> >
> > > What I think should always be done manually:
> > >
> > >- [ASF requirement] download all signed source code packages,
> validate
> > checksums + signatures, compile without tests. start a Druid cluster and
> > ingest wikipedia data onto a micro-quickstart cluster and run a
> > simple query against it.
> > >- [Druid requirement] Vote on whether any features are being
> released
> > that are incomplete - missing integration tests, un-expected change in
> > behavior, etc.
> > >- [Druid requirement] Verify that all the automated tests that run
> for
> > the tag have actually passed (maybe this can be automated too?). I
> imagine
> > this step is to look at travis for the tag to ensure that all the tests
> > actually passed.
> https://travis-ci.com/github/apache/druid/builds/224182054
> > - this is the travis build for the 0.21.0 tag (which appears to be
> failing)
> >
> > If we get agreement on this, I'll be happy to add / update a doc that
> talks
> > about what specifically is expected from someone voting on an Apache
> Druid
> > release.
> >
> > While writing out this proposal below, I've realized we have a set of
> > integration tests that do not run as part of the CI pipeline because they
> > require credentials to some third party systems (eg. Kinesis ITs) or we
> > can't get the tests to consistently run in the Travis pipeline without
> > intermittent failures (eg. Hadoop ITs). Does a person voting need to run
> > these integration tests today, since they are not part of the automated
> > pipeline?
> >
> >
> >
> >
> >
> >
> > On Mon, Apr 26, 2021 at 12:30 PM Jihoon Son 
> wrote:
> >
> > > Hi Suneet, thanks for your suggestion.
> > >
> > > However, the Apache release policy [1] explicitly says the below.
> > >
> > > > Before casting +1 binding votes, individua

0.21.1, maybe 0.22?

[Clint Wylie]

Hi all,
>From the 0.21 release vote  thread,
https://lists.apache.org/thread.html/r72b70181bd61585703fdc2a4699b28701bc437da8410444106f2c57a%40%3Cdev.druid.apache.org%3E,
discussing bug https://github.com/apache/druid/issues/11166, I think we've
decided it is best to do a 0.21.1 patch release that includes
https://github.com/apache/druid/pull/11167.

Since the master branch is quite a bit ahead of 0.21, I also think we
should avoid pulling in too many additional bug fixes, and instead I will
volunteer to begin the release process on 0.22 in maybe 2-3 weeks? I am
open to pulling in other fixes if there is anything else that is needed
asap and can't wait until 0.22 though.

I'll get a branch started soon and if nothing else gets added, should have
a vote up later this week.

Re: [VOTE] Release Apache Druid 0.21.0 [RC1]

[Clint Wylie]

I'll take care of 0.21.1 Jihoon since you've handled the last few, and I
was planning on volunteering to get the ball rolling on 0.22 pretty soon
anyway.


On Sat, May 1, 2021 at 1:26 PM Jihoon Son  wrote:

> Thanks Xavier, it makes sense to me.
> I will start a new thread to discuss what should be in 0.21.1 besides
> the docker fix.
>
> On Thu, Apr 29, 2021 at 4:49 PM Xavier Léauté
>  wrote:
> >
> > It might be worthwhile to do a patch release. While there is a
> workaround,
> > it is not very convenient for someone running the docker image in
> > kubernetes.
> > Especially for query nodes that rarely get deployed with persistent
> > storage, you now have to define a volume or mount point of some sort.
> > This makes it very cumbersome to upgrade, and someone may not hit this
> > issue until after they have upgraded all their data nodes, once they
> > upgrade brokers.
> >
> > Once https://github.com/apache/druid/pull/11167 is merged, we can
> backport
> > it.
> >
> >
> > On Wed, Apr 28, 2021 at 11:39 PM frank chen 
> wrote:
> >
> > > Yes, this problem is not bound to macOS, It also exists on my linux
> server
> > > running CentOS 7.2.1511 with Docker 19.03.4, build 9013bf583a.
> > >
> > > I think the reason why the images work on some environments is pointed
> by
> > > Jihoon: the docker volume that was created a long time ago with proper
> > > ownership.
> > >
> > > Since there's an easy workaround, I think it's acceptable for me to fix
> > > this problem in the next release 0.22.0
> > >
> > > Jihoon Son  于2021年4月29日周四 上午7:46写道：
> > >
> > > > I added this issue and the workaround suggested here in the release
> > > > notes:
> > > >
> > >
> https://github.com/apache/druid/releases/tag/druid-0.21.0#21-docker-volume-ownership
> > > > .
> > > >
> > > > On Wed, Apr 28, 2021 at 4:23 PM Jihoon Son 
> wrote:
> > > > >
> > > > > The workaround Xavier suggested seems working for me.
> > > > > I think this workaround can work for the users who don't use the
> > > > docker-compose.
> > > > > For the users who use the docker-compose, I think the easiest
> > > > > workaround is that they first create docker volumes using an old
> image
> > > > > and then run the 0.21.0 image.
> > > > > You can do this by running these commands.
> > > > >
> > > > > $ cd ${PREV_SRC_DIR}
> > > > > $ docker-compose -f distribution/docker/docker-compose.yml create
> > > > > $ cd ${0.21.0_SRC_DIR}
> > > > > $ docker-compose -f distribution/docker/docker-compose.yml up
> > > > >
> > > > > This workaround seems easy enough to me. Also, given that code
> freeze
> > > > > for the 0.22.0 release is coming, I think this workaround could be
> > > > > enough for now.
> > > > > Does anyone strongly think that we should do a patch release to fix
> > > this?
> > > > >
> > > > > On Wed, Apr 28, 2021 at 2:00 PM Jihoon Son 
> > > wrote:
> > > > > >
> > > > > > I could reproduce it. It seems that my previous testing passed
> > > because
> > > > > > of the docker volume that was created a long time ago with a
> proper
> > > > > > ownership.
> > > > > > It also seems not particular to the specific OS version because I
> > > > > > could reproduce the same issue in linux as well. I used the
> ubuntu
> > > > > > linux of the kernel version of 5.4.0-70 and docker 20.10.5.
> > > > > > I'm now looking for an easy workaround. Please let me know if you
> > > know
> > > > > > such a workaround.
> > > > > > Otherwise, maybe we should do a patch release.
> > > > > >
> > > > > >
> > > > > > On Wed, Apr 28, 2021 at 10:35 AM Xavier Léauté
> > > > > >  wrote:
> > > > > > >
> > > > > > > I'm running 3.3.1 on Big Sur, so this might be particular to
> the
> > > > specific
> > > > > > > OS version?
> > > > > > >
> > > > > > > On Wed, Apr 28, 2021 at 10:33 AM Jihoon Son <
> jihoon...@apache.org>
> > > > wrote:
> > > > > > >
> > > > > > > > Interesting. I also used Docker Desktop 3.3.1 on the Catalina
> > > macOS
> > > > > > > > for my testing. The current docker image worked fine for me.
> > > > > > > > It sounds good to me to add this as a known issue in the
> release
> > > > > > > > notes, but I hope it to be more specific.
> > > > > > > > Do you have more details about the issue such as in what
> > > conditions
> > > > > > > > you could see it?
> > > > > > > >
> > > > > > > > On Wed, Apr 28, 2021 at 9:48 AM Xavier Léauté
> > > > > > > >  wrote:
> > > > > > > > >
> > > > > > > > > Hi Frank,
> > > > > > > > >
> > > > > > > > > I am noticing a similar problem when running on Docker for
> Mac.
> > > > > > > > >
> > > > > > > > > This appears to be specific to macOS, and how docker
> > > initializes
> > > > the
> > > > > > > > volume
> > > > > > > > > mount point permissions.
> > > > > > > > > As a workaround you can pass a volume at the command-line,
> > > which
> > > > somehow
> > > > > > > > > sets the right ownership.
> > > > > > > > >
> > > > > > > > > We might want to add a known issue related to that in the
> > > > release notes.
> > > > > > > > >
> > > > > > > > > On Mon

Re: [VOTE] Release Apache Druid 0.21.0 [RC1]

[Clint Wylie]

+1 (binding)

src package:
- verified signature/checksum
- LICENSE/NOTICE present
- compiled, ran checks, unit tests
- built binary distribution, ingested some data and ran some queries

binary package:
- verified signature/checksum
- LICENSE/NOTICE present
- ingested some data and ran some queries

docker:
- verified checksum
- started cluster with docker-compose, ingested some data and ran some
queries

Thanks for putting the release together Jihoon!

On Fri, Apr 16, 2021 at 5:59 PM Jihoon Son  wrote:

> Hi all,
>
> I have created a build for Apache Druid 0.21.0, release
> candidate 1.
>
> Thanks for everyone who has helped contribute to the release! You can read
> the proposed release notes here:
> https://github.com/apache/druid/issues/10752
>
> The release candidate has been tagged in GitHub as
> druid-0.21.0-rc1 (733697c25ff22045f14016d83b123fa18556dec8),
> available here:
> https://github.com/apache/druid/releases/tag/druid-0.21.0-rc1
>
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/druid/0.21.0-rc1/
>
> A staged Maven repository is available for review at:
> https://repository.apache.org/content/repositories/orgapachedruid-1023/
>
> Staged druid.apache.org website documentation is available here:
> https://druid.staged.apache.org/docs/0.21.0/design/index.html
>
> A Docker image containing the binary of the release candidate can be
> retrieved via:
> docker pull apache/druid:0.21.0-rc1
>
> artifact checksums
> src:
>
> 8ff3c5ce96b6eff67a68945284e9d2280ea6fbca4ee4a3a023e74685f05dfbed84d1e9071ed5331cb0b1416cb87895d146ce733ae228070a9437375e1baca022
> bin:
>
> 4c1b9ff4c8d89e1c78f0bc9e414ea4e855a637925959b5e4e4edd79bdbd0311f0b09cc332c6f48f982f10d9d46d2658cee802bac4e60116598d1aaf3deebf9b1
> docker: 33ff4044017f5974f2e250512a1dd2449078dbf1fa18dd2bd4fa511a4c9f2f78
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/jihoonson.asc
>
> This key and the key of other committers can also be found in the project's
> KEYS file here:
> https://dist.apache.org/repos/dist/release/druid/KEYS
>
> (If you are a committer, please feel free to add your own key to that file
> by following the instructions in the file's header.)
>
>
> Verify checksums:
> diff <(shasum -a512 apache-druid-0.21.0-src.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-0.21.0-src.tar.gz.sha512 ; echo)
>
> diff <(shasum -a512 apache-druid-0.21.0-bin.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-0.21.0-bin.tar.gz.sha512 ; echo)
>
> Verify signatures:
> gpg --verify apache-druid-0.21.0-src.tar.gz.asc \
> apache-druid-0.21.0-src.tar.gz
>
> gpg --verify apache-druid-0.21.0-bin.tar.gz.asc \
> apache-druid-0.21.0-bin.tar.gz
>
> Please review the proposed artifacts and vote. Note that Apache has
> specific requirements that must be met before +1 binding votes can be cast
> by PMC members. Please refer to the policy at
> http://www.apache.org/legal/release-policy.html#policy for more details.
>
> As part of the validation process, the release artifacts can be generated
> from source by running:
> mvn clean install -Papache-release,dist -Dgpg.skip
>
> The RAT license check can be run from source by:
> mvn apache-rat:check -Prat
>
> This vote will be open for at least 72 hours. The vote will pass if a
> majority of at least three +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Druid 0.21.0
> [ ] 0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
> Thanks!
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> For additional commands, e-mail: dev-h...@druid.apache.org
>
>

Re: Regarding the 0.21.0 release

[Clint Wylie]

+1 for splitting releases. I don't think we need to delay 0.22 too much,
just enough so they are not on top of each other.

On Wed, Mar 31, 2021 at 11:20 AM Suneet Saldanha 
wrote:

> +1 for delaying the 0.22 release as well
>
> 0.21 is already 5+ months behind the 0.20 release, so it's not like it's a
> hard requirement to stick to a 3 month release schedule.
>
> + the longer cycle time allows users to test features in 0.21 and harden
> them for 0.22
>
> On Wed, Mar 31, 2021 at 10:52 AM Atul Mohan  wrote:
>
> > +1 to the second option of releasing 0.21.0 asap and postpone 0.22.0.
> > If we do end up finding bugs in 0.21.0, they can be fixed into 0.22.0
> > instead of creating a smaller release via 0.21.1.
> >
> >
> > On Wed, Mar 31, 2021 at 12:43 PM Jihoon Son 
> wrote:
> >
> > > Hi all,
> > >
> > > The 0.21.0 release has been blocked by several issues for a long time
> > > including 2 security vulnerabilities (CVE-2021-25646 and
> > > CVE-2021-26919 which are addressed in 0.20.1 and 0.20.2,
> > > respectively). Since we created the release branch for 0.21.0 on
> > > January 11th, it is almost the time to prepare the next release (the
> > > next branch cut is scheduled in April). If we stick to the schedule
> > > and prepare another release right after 0.21.0, it will not be great
> > > for our users because they won't have enough time to test out the
> > > 0.21.0 release. I think we have 2 options to avoid this issue:
> > >
> > > - Consolidating the 0.21.0 and 0.22.0 releases. We can create one
> > > release that will contain all commits that are originally scheduled to
> > > be released separately via 0.21.0 and 0.22.0.
> > > - Finalizing the 0.21.0 release quickly and postponing the 0.22.0
> > > release e.g., a month.
> > >
> > > To me, the second option seems more reasonable because of the below
> > > reasons.
> > >
> > > - The 0.21.0 release is almost ready. We can finish it quickly without
> > > much further preparation.
> > > - There are users waiting for the features introduced in 0.21.0.
> > > Consolidating 2 releases will make them wait longer.
> > > - There could be unknown bugs in the release branch. Consolidating
> > > releases will make testing harder since it will increase the scope of
> > > testing.
> > > - Consolidating releases will remove one option for users what release
> > > they can use. This can be problematic especially when some bugs are
> > > found in the new release.
> > >
> > > I'd like to hear what other people think.
> > >
> > > Thanks,
> > > Jihoon
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> > > For additional commands, e-mail: dev-h...@druid.apache.org
> > >
> > >
> >
>

failing integration tests using segments in s3 bucket static.druid.io

[Clint Wylie]

Hi all,

I've noticed that the integration tests in Travis that use the segments
stored in the static.druid.io s3 bucket are all failing, starting sometime
between yesterday and today. Looking closer, they are failing to load on
historicals with a 403 Forbidden error.

Is anyone familiar with the details on this bucket? The access key is
https://github.com/apache/druid/blob/master/integration-tests/docker/druid.sh#L93,
perhaps it has been disabled?

I have a copy of the segments from these tests, so I guess we could find an
alternative way to load them and run the tests, if necessary, since this I
think is going to be blocking CI until we get it fixed.

It was nice to have them in s3 since it was providing some basic tests on
s3 deep storage, but I guess we could do these tests in an alternative way
as well, and make the s3 integration tests be like the other cloud service
tests and only run manually when bringing your own resources and
credentials.

Re: Forbidding forced git push

[Clint Wylie]

It seems like this will basically only affect release managers.

I am maybe -1 since I have personally had to force push to a release branch
while making an RC, when I optimistically pushed the tags and then found a
mistake doing preflight checks before sending the artifacts out to vote. I
did this so that I didn't have to do something like jump from RC1 to RC3
with a dead RC2 before it was even voted on.

I find since github added
https://github.blog/changelog/2018-11-15-force-push-timeline-event/ that
force-pushes aren't that terrible to deal with during a review even, so
would probably personally be in favor of relaxing our soft policy on them,
but it seems like everyone else is opposed to them, so i think it is also
fine to keep the soft policy as is and adding the link to it to the PR
template.

On Fri, Jan 15, 2021 at 1:26 PM Gian Merlino  wrote:

> Will this help for the (common) case where PR branches are in people's
> forks?
>
> On Fri, Jan 15, 2021 at 1:00 PM Jihoon Son  wrote:
>
> > Hi all,
> >
> > The forced git push is usually used to make the commit history clean,
> which
> > I understand its importance. However, one of its downsides is, because it
> > overwrites the commit history, we cannot tell the exact change between
> > commits while reviewing a PR. This increases the burden for reviewers
> > because they have to go through the entire PR again after a forced push.
> > For the same reason, we are suggesting to not use it in our
> documentation (
> >
> >
> https://github.com/apache/druid/blob/master/CONTRIBUTING.md#if-your-pull-request-shows-conflicts-with-master
> > ),
> > but I don't believe this documentation is well read by many people (It
> is a
> > good doc, BTW. Maybe we should promote it more effectively).
> >
> > Since branch sharing doesn't usually happen for us (AFAIK, there has been
> > no branch sharing so far), I think this is the biggest downside of using
> > forced push. To me, clean commit history is not a big gain compared to
> how
> > much it can make the review process worse, especially when the PR is big.
> >
> > So, I would like to suggest forbidding git forced push for the Druid
> > repository. It seems possible to disable it by creating an infra ticket (
> >
> >
> https://issues.apache.org/jira/browse/INFRA-13613?jql=text%20~%20%22force%20push%22
> > ).
> > I can do it if everyone agrees.
> >
> > Would like to hear what people think.
> > Jihoon
> >
>

Re: [VOTE] Release Apache Druid 0.20.0 [RC2]

[Clint Wylie]

+1 (binding)

src package:
- verified signature/checksum
- LICENSE/NOTICE present
- compiled, ran unit tests (I actually had issue with a Kafka supervisor
test also, testFailedInitializationAndRecovery, but I think it might be
environmental, because I am seeing this same issue with 0.19 src package)
- built binary distribution, ingested some data and ran some queries

binary package:
- verified signature/checksum
- LICENSE/NOTICE present
- ingested some data and ran some queries

docker:
- verified checksum
- started cluster with docker-compose, ingested some data and ran some
queries

On Tue, Oct 13, 2020 at 9:53 PM Jonathan Wei  wrote:

> Thanks for checking, Furkan!
>
> I tried building from source locally again and it succeeded, I think it
> could be flakiness in that test.
>
> Best,
> Jon
>
> On Tue, Oct 13, 2020 at 1:39 AM Furkan KAMACI 
> wrote:
>
> > Hi,
> >
> > Tests couldn't passed on my environment (Darwin Kernel Version 19.5.0):
> >
> > [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> > 33.255 s <<< FAILURE! - in
> > org.apache.druid.indexing.kafka.supervisor.KafkaSupervisorTest
> > [ERROR] org.apache.druid.indexing.kafka.supervisor.KafkaSupervisorTest
> >  Time elapsed: 33.253 s  <<< ERROR!
> > kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for
> > connection while in state: CONNECTING
> > at
> >
> >
> kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:262)
> > at kafka.zookeeper.ZooKeeperClient.(ZooKeeperClient.scala:119)
> > at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1865)
> > at kafka.server.KafkaServer.createZkClient$1(KafkaServer.scala:419)
> > at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:444)
> > at kafka.server.KafkaServer.startup(KafkaServer.scala:222)
> > at
> > org.apache.druid.indexing.kafka.test.TestBroker.start(TestBroker.java:94)
> > at
> >
> >
> org.apache.druid.indexing.kafka.supervisor.KafkaSupervisorTest.setupClass(KafkaSupervisorTest.java:195)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at
> >
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> > at
> >
> >
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> > at
> >
> >
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> > at
> >
> >
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> > at
> >
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> > at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> > at
> >
> >
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> > at
> >
> >
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> > at
> >
> >
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> > at
> >
> >
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> > at
> >
> >
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> > at
> >
> >
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> > at
> >
> >
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> > at
> > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> >
> >
> > [INFO] BUILD FAILURE
> > [INFO]
> > 
> > [INFO] Total time:  01:02 h
> >
> > and rat checks have failed due to that reason:
> >
> > [INFO] BUILD FAILURE
> > [INFO]
> > 
> > [INFO] Total time:  48.835 s
> > [INFO] Finished at: 2020-10-13T11:34:48+03:00
> > [INFO]
> > 
> > [ERROR] Failed to execute goal on project druid-integration-tests: Could
> > not resolve dependencies for project
> > org.apache.druid:druid-integration-tests:jar:0.20.0: The following
> > artifacts could not be resolved:
> > org.apache.druid.extensions:druid-kafka-indexing-service:jar:0.20.0,
> > org.apache.druid.extensions:druid-basic-security:jar:0.20.0,
> > org.apache.druid.extensions:druid-lookups-cached-global:jar:0.20.0,
> > org.apache.druid.extensions:simple-client-sslcontext:jar:0.20.0: Could
> not
> > find artifact
> > org.apache.druid.extensions:druid-kafka-indexing-service:jar:0.20.0 in
> > apache.snapshots (https://repository.apache.org/snapshots) -> [Help 1]
> >
> > Kind Regards,
> > Furkan KAMACI
> >
> > On Tue, Oct 13, 2020 at 3:49 AM Jihoon Son  wrote:
> >
> > > +1 (binding)
> > >
> > > src

[ANNOUNCE] Apache Druid 0.19.0 release

[Clint Wylie]

The Apache Druid team is proud to announce the release of Apache Druid
0.19.0.
Druid is a high performance analytics data store for event-driven data.

Apache Druid 0.19.0 contains around 200 new features, bug fixes,
performance enhancements, documentation improvements, and additional test
coverage from 51 contributors. Major new features and improvements include:

- GroupBy and Timeseries vectorized query engines enabled by default
- Druid native batch support for Apache Avro Object Container Files
- Updated Druid native batch support for SQL databases
- Apache Ranger based authorization
- Alibaba Object Storage Service support
- Ingestion worker autoscaling for Google Compute Engine

Source and binary distributions can be downloaded from:
https://druid.apache.org/downloads.html

Release notes are at:
https://github.com/apache/druid/releases/tag/druid-0.19.0

A big thank you to all the contributors in this release!

[RESULT][VOTE] Release Apache Druid 0.19.0 [RC1]

[Clint Wylie]

Thanks to everyone who participated in the vote! The results are as follows:

Clint Wylie: +1 (binding)
Jihoon Son: +1 (binding)
Furkan KAMACI: +1 (binding)

Re: [VOTE] Release Apache Druid 0.19.0 [RC1]

[Clint Wylie]

Thanks for the votes, counting myself puts us at three +1 binding. I'll
start the thread to announce the results and continue the release process
soon.

[ERROR] Failed to execute goal on project druid-ranger-security: Could not
> resolve dependencies for project
> org.apache.druid.extensions:druid-ranger-security:jar:0.19.0: Could not
> transfer artifact org.eclipse.persistence:eclipselink:jar:2.5.2 from/to
> central (https://repo.maven.apache.org/maven2): GET request of:
> org/eclipse/persistence/eclipselink/2.5.2/eclipselink-2.5.2.jar from
> central failed: Read timed out
>

This seems like perhaps a transient issue with failing to fetch
dependencies?

[VOTE] Release Apache Druid 0.19.0 [RC1]

[Clint Wylie]

Hi all,

I have created a build for Apache Druid 0.19.0, release
candidate 1.

Thanks to everyone who has helped contribute to the release! You can read
the proposed release notes here:
https://github.com/apache/druid/issues/10139

The release candidate has been tagged in GitHub as
druid-0.19.0-rc1 (c557a1448d872e3aab03aec3bafb035e74583656),
available here:
https://github.com/apache/druid/releases/tag/druid-0.19.0-rc1

The artifacts to be voted on are located here:
https://dist.apache.org/repos/dist/dev/druid/0.19.0-rc1/

A staged Maven repository is available for review at:
https://repository.apache.org/content/repositories/orgapachedruid-1018/

Staged druid.apache.org website documentation is available here:
https://druid.staged.apache.org/docs/0.19.0/design/index.html

A Docker image containing the binary of the release candidate can be
retrieved via:
docker pull apache/druid:0.19.0-rc1

artifact checksums
src:
c6e9d4e38f371989ec89f745ae5c6b8c8bfe78a8abd89ee6013f53bf19a39f0c34d7e3e077d15b083cfad472d7f12b9466bcc2a05e786dbecbcd428ce3be85f0
bin:
24dce0d5fad8be7608c0cd3cee183e43ac1bc63b5139fc7ef8e37207937825ca999f488224834d4265213051b8e3eb84186fba4b665cf9c98d1f97a02cf1764b
docker: 233db3b3397c6a28e69afb0d29ed698ea39f10d360c834526a0d76a5b510cc21

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/cwylie.asc

This key and the key of other committers can also be found in the project's
KEYS file here:
https://dist.apache.org/repos/dist/release/druid/KEYS

(If you are a committer, please feel free to add your own key to that file
by following the instructions in the file's header.)


Verify checksums:
diff <(shasum -a512 apache-druid-0.19.0-src.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-0.19.0-src.tar.gz.sha512 ; echo)

diff <(shasum -a512 apache-druid-0.19.0-bin.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-0.19.0-bin.tar.gz.sha512 ; echo)

Verify signatures:
gpg --verify apache-druid-0.19.0-src.tar.gz.asc \
apache-druid-0.19.0-src.tar.gz

gpg --verify apache-druid-0.19.0-bin.tar.gz.asc \
apache-druid-0.19.0-bin.tar.gz

Please review the proposed artifacts and vote. Note that Apache has
specific requirements that must be met before +1 binding votes can be cast
by PMC members. Please refer to the policy at
http://www.apache.org/legal/release-policy.html#policy for more details.

As part of the validation process, the release artifacts can be generated
from source by running:
mvn clean install -Papache-release,dist -Dgpg.skip

The RAT license check can be run from source by:
mvn apache-rat:check -Prat

This vote will be open for at least 72 hours. The vote will pass if a
majority of at least three +1 PMC votes are cast.

[ ] +1 Release this package as Apache Druid 0.19.0
[ ] 0 I don't feel strongly about it, but I'm okay with the release
[ ] -1 Do not release this package because...

Thanks!

Druid 0.19.0

[Clint Wylie]

Hi all,

It's about that time again to start preparing to make the next release (the
last major branch was created in March) - I volunteer to be release
manager. I will probably try to make the branch sometime in the next week
or so, to allow us time to start seriously testing to get everything in the
branch stable and ready to go.

If anyone is aware of any bugs which should be addressed before release,
please mention them in this thread so we can discuss and potentially tag
them to the milestone and try to get fixes in place.

Cheers,
Clint

Re: 0.18.1?

[Clint Wylie]

+1

Would it also be worth including
https://github.com/apache/druid/pull/9723? Additionally
if we can confirm and get a fix for
https://github.com/apache/druid/issues/9736 in time it would be worth
including, but if not it might also be good enough to warrant a 0.18.2.

On Tue, Apr 21, 2020 at 10:38 AM Himanshu  wrote:

> +1
>
> https://github.com/apache/druid/issues/9728 appears to be a regression and
> impacting users, so it would be good to do a release fixing that or else
> those users can't upgrade to 0.18
>
> -- Himanshu
>
>
> On Tue, Apr 21, 2020 at 10:07 AM Jihoon Son  wrote:
>
> > Hi all,
> >
> > We have just released 0.18.0 and it seems like there are a couple of
> > critical bugs in it such as https://github.com/apache/druid/issues/9728
> or
> > https://github.com/apache/druid/issues/9697.
> > https://github.com/apache/druid/issues/9728 seems pretty critical since
> > there are many use cases that use Kafka indexing service with Avro, which
> > prevent users from upgrading to 0.18.0. It looks worth making a 0.18.1
> > release. I'll volunteer to manage the release if we agree on it.
> >
>

Re: [VOTE] Release Apache Druid 0.18.0 [RC3]

[Clint Wylie]

+1 (binding)

src package:
- verified signature/hash
- checked LICENSE and NOTICE
- rat check and unit tests passed
- built distribution, tested native batch quickstart and queries

bin package:
- verified signature/hash
- checked LICENSE and NOTICE
- tested kafka quickstart and queries

docker:
- verified checksum
- tested native batch quickstart and queries

On Thu, Apr 16, 2020 at 3:52 PM Jihoon Son  wrote:

> Hi all,
>
> I have created a build for Apache Druid 0.18.0, release candidate 3.
>
> Thanks for everyone who has helped contribute to the release! You can read
> the proposed release notes here:
> https://github.com/apache/druid/issues/9652
>
> The release candidate has been tagged in GitHub as
> druid-0.18.0-rc3 (6a4407345a9b2e862c1dd87092086d05b30f2ef2),
> available here:
> https://github.com/apache/druid/releases/tag/druid-0.18.0-rc3
>
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/druid/0.18.0-rc3/
>
> Staged druid.apache.org website documentation is available here:
> https://druid.staged.apache.org/docs/0.18.0/design/index.html
>
> A Docker image containing the binary of the release candidate can be
> retrieved via:
> docker pull apache/druid:0.18.0-rc3
>
> artifact checksums
> src:
>
> 48e24af92603d5a7529d803ac2ddba45548516866e69650beae129400b3bdf561710fe415dfe79005cbe1e57f3e20f7096537bc2e7c03b7c171e398d371dc5ce
> bin:
>
> d9a22581bfa5c6d5a57c8bc703683928a04f5938f5da77013747cec46b0acdfbbc457f7aa50e505bc868fcf3f7abf4909eb75c56067bbdd84be7587ac7040bad
> docker: b4ccd2fe9e534afce13253272c0e6be8f89274d4b5ef38fb9751611da4027da9
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/jihoonson.asc
>
> This key and the key of other committers can also be found in the project's
> KEYS file here:
> https://dist.apache.org/repos/dist/release/druid/KEYS
>
> (If you are a committer, please feel free to add your own key to that file
> by following the instructions in the file's header.)
>
>
> Verify checksums:
> diff <(shasum -a512 apache-druid-0.18.0-src.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-0.18.0-src.tar.gz.sha512 ; echo)
>
> diff <(shasum -a512 apache-druid-0.18.0-bin.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-0.18.0-bin.tar.gz.sha512 ; echo)
>
> Verify signatures:
> gpg --verify apache-druid-0.18.0-src.tar.gz.asc \
> apache-druid-0.18.0-src.tar.gz
>
> gpg --verify apache-druid-0.18.0-bin.tar.gz.asc \
> apache-druid-0.18.0-bin.tar.gz
>
> Please review the proposed artifacts and vote. Note that Apache has
> specific requirements that must be met before +1 binding votes can be cast
> by PMC members. Please refer to the policy at
> http://www.apache.org/legal/release-policy.html#policy for more details.
>
> As part of the validation process, the release artifacts can be generated
> from source by running:
> mvn clean install -Papache-release,dist -Dgpg.skip
>
> The RAT license check can be run from source by:
> mvn apache-rat:check -Prat
>
> This vote will be open for at least 72 hours. The vote will pass if a
> majority of at least three +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Druid 0.18.0
> [ ] 0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
> Thanks!
>

Re: [VOTE] Release Apache Druid 0.18.0 [RC2]

[Clint Wylie]

-1 (binding)

It looks like there are some extra code coverage analysis results in the
source package in apache-druid-0.18.0-src/web-console/coverage/ which is
causing this command to fail:

mvn apache-rat:check -Prat

since these files have no license information.

On Wed, Apr 15, 2020 at 2:52 PM Jihoon Son  wrote:

> Hi all,
>
> I have created a build for Apache Druid 0.18.0, release candidate 2.
>
> Thanks for everyone who has helped contribute to the release! You can read
> the proposed release notes here:
> https://github.com/apache/druid/issues/9652
>
> The release candidate has been tagged in GitHub as
> druid-0.18.0-rc2 (3280d14ba523d0b6d623e73945201980e48c8238),
> available here:
> https://github.com/apache/druid/releases/tag/druid-0.18.0-rc2
>
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/druid/0.18.0-rc2/
>
> Staged druid.apache.org website documentation is available here:
> https://druid.staged.apache.org/docs/0.18.0/design/index.html
>
> A Docker image containing the binary of the release candidate can be
> retrieved via:
> docker pull apache/druid:0.18.0-rc2
>
> artifact checksums
> src:
>
> 4b20047f715b03aa8d5fd0d7eac9e7cc4caa3883c538ae0cde5de0777136e687bb7ce05f7b07aaf9815e1b93cd95fd6e04ab2c64c5d7a11beff72528611018b5
> bin:
>
> a95faa297dab5501a98c4e1d34593eddb052652f1c2b77245b44f155d9ea355c5a4a856afb65f53daa6660ef67a8372dd802f728d3ca0869095301ab822e3545
> docker: 812ae45f1226c146300d04a7b1096e4a9470c78c90548b4645bf28275f840a6c
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/jihoonson.asc
>
> This key and the key of other committers can also be found in the project's
> KEYS file here:
> https://dist.apache.org/repos/dist/release/druid/KEYS
>
> (If you are a committer, please feel free to add your own key to that file
> by following the instructions in the file's header.)
>
>
> Verify checksums:
> diff <(shasum -a512 apache-druid-0.18.0-src.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-0.18.0-src.tar.gz.sha512 ; echo)
>
> diff <(shasum -a512 apache-druid-0.18.0-bin.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-0.18.0-bin.tar.gz.sha512 ; echo)
>
> Verify signatures:
> gpg --verify apache-druid-0.18.0-src.tar.gz.asc \
> apache-druid-0.18.0-src.tar.gz
>
> gpg --verify apache-druid-0.18.0-bin.tar.gz.asc \
> apache-druid-0.18.0-bin.tar.gz
>
> Please review the proposed artifacts and vote. Note that Apache has
> specific requirements that must be met before +1 binding votes can be cast
> by PMC members. Please refer to the policy at
> http://www.apache.org/legal/release-policy.html#policy for more details.
>
> As part of the validation process, the release artifacts can be generated
> from source by running:
> mvn clean install -Papache-release,dist -Dgpg.skip
>
> The RAT license check can be run from source by:
> mvn apache-rat:check -Prat
>
> This vote will be open for at least 72 hours. The vote will pass if a
> majority of at least three +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Druid 0.18.0
> [ ] 0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
> Thanks!
>

Re: [VOTE] Release Apache Druid 0.18.0 [RC1]

[Clint Wylie]

+1 (binding)

src package:
- verified signature/hash
- LICENSE and NOTICE lgtm
- tests passed
- built binary distribution and ran through native batch quickstart, issued
queries successfully

bin package:
- verified signature/hash
- LICENSE and NOTICE lgtm
- ran native batch quickstart, issued queries successfully

docker:
- verified checksum
- ran native batch quickstart using docker-compose, issued queries
successfully


On Mon, Apr 13, 2020 at 9:50 PM Himanshu  wrote:

> +1 (binding)
>
> src package:
> - verified signature/hash
> - LICENSE and NOTICE are present
> - unit tests & integration tests passed
> - built binary distribution
>
> bin package:
> - verified signature/hash
> - LICENSE and NOTICE are present
>
> Jihoon: Thanks for coordinating the release.
>
>
> On Sun, Apr 12, 2020 at 1:13 AM Jihoon Son  wrote:
>
> > Hi all,
> >
> > I have created a build for Apache Druid 0.18.0, release candidate 1.
> >
> > Thanks for everyone who has helped contribute to the release! You can
> read
> > the proposed release notes here:
> > https://github.com/apache/druid/issues/9652
> >
> > The release candidate has been tagged in GitHub as
> > druid-0.18.0-rc1 (b579bff520ade833fc4b648f28633ebf85841e0f),
> > available here:
> > https://github.com/apache/druid/releases/tag/druid-0.18.0-rc1
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/druid/0.18.0-rc1/
> >
> > Staged druid.apache.org website documentation is available here:
> > https://druid.staged.apache.org/docs/0.18.0/design/index.html
> >
> > A Docker image containing the binary of the release candidate can be
> > retrieved via:
> > docker pull apache/druid:0.18.0-rc1
> >
> > artifact checksums
> > src:
> >
> >
> 035d2409c17d2d7fbe4931b7d9678db9d4047d71fada48d1a69b058634381eb747109d442bd2b3cc85793e5bd671f3cc5934325c3b7d697fbcb5081df83fc4c6
> > bin:
> >
> >
> d622465d043915c9ae4fe38700855374368fde17cd8091748b2d3272c87feeff0a843fdfa6bb1b719a2a8eb1dc1f0e8f238d43e87a376d6a79e8b7d142c7bc1b
> > docker: 2074498b92ed6baf293a2f20e45791295c614becc0d4fac03b76c44d885e1c18
> >
> > Release artifacts are signed with the following key:
> > B5E27DA9B51A47300DBE1B32875BF09B17E98806
> > (This is a new key that I recently created. It will be available at
> > https://people.apache.org/keys/committer/jihoonson.asc soon.)
> >
> > This key and the key of other committers can also be found in the
> project's
> > KEYS file here:
> > https://dist.apache.org/repos/dist/release/druid/KEYS
> >
> > (If you are a committer, please feel free to add your own key to that
> file
> > by following the instructions in the file's header.)
> >
> >
> > Verify checksums:
> > diff <(shasum -a512 apache-druid-0.18.0-src.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-0.18.0-src.tar.gz.sha512 ; echo)
> >
> > diff <(shasum -a512 apache-druid-0.18.0-bin.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-0.18.0-bin.tar.gz.sha512 ; echo)
> >
> > Verify signatures:
> > gpg --verify apache-druid-0.18.0-src.tar.gz.asc \
> > apache-druid-0.18.0-src.tar.gz
> >
> > gpg --verify apache-druid-0.18.0-bin.tar.gz.asc \
> > apache-druid-0.18.0-bin.tar.gz
> >
> > Please review the proposed artifacts and vote. Note that Apache has
> > specific requirements that must be met before +1 binding votes can be
> cast
> > by PMC members. Please refer to the policy at
> > http://www.apache.org/legal/release-policy.html#policy for more details.
> >
> > As part of the validation process, the release artifacts can be generated
> > from source by running:
> > mvn clean install -Papache-release,dist -Dgpg.skip
> >
> > The RAT license check can be run from source by:
> > mvn apache-rat:check -Prat
> >
> > This vote will be open for at least 72 hours. The vote will pass if a
> > majority of at least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Druid 0.18.0
> > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Thanks!
> >
>

Re: Draft April ASF Board Report

[Clint Wylie]

Hmm, I did the numbers as if it were a quarterly report this time, but I
didn't include the stuff from the previous monthly reports and wasn't sure
if it is necessary to go over it again. I think it is probably fine without
it since the information was included in previous reports?

On Wed, Apr 8, 2020 at 1:44 PM Gian Merlino  wrote:

> It does matter! But, we mentioned those in a previous report (our last one
> was just a month ago — so this one covers the last month). After this
> report they'll start being quarterly and covering 3 months.
>
> On Wed, Apr 8, 2020 at 1:18 PM itai yaffe  wrote:
>
> > Hey,
> > Not sure it matters, but we actually had at least 3 in-person meetups
> > during Q1 organized by the Druid community:
> >
> >1. January 15th in London -
> >https://www.meetup.com/Apache-Druid-London/events/267380924/
> >2. January 28th in Athens -
> >https://www.meetup.com/Athens-Big-Data/events/266900242/
> >3. January 29th in Tel-Aviv (hosted by Nielsen) -
> >
> https://www.meetup.com/Big-things-are-happening-here/events/267578817/
> >
> > Sorry if I left out other events I'm not aware of...
> >
> > On Wed, Apr 8, 2020 at 4:05 AM Clint Wylie  wrote:
> >
> > > Hey all,
> > >
> > > I put together a draft for the quarterly ASF board report due tomorrow,
> > > sorry for the short notice. Let me know if I missed anything or should
> > make
> > > any changes. Thanks!
> > >
> > > -
> > >
> > > ## Description
> > >
> > > Apache Druid is a high performance real-time analytics database. It is
> > > designed for workflows where low-latency query and ingest are the main
> > > requirements. It implements ingestion, storage, and querying
> subsystems.
> > > Users interface with Druid through built-in SQL and JSON APIs, as well
> > > as third-party applications.
> > >
> > > Druid has an extensive web of connections with other Apache projects:
> > > Calcite for SQL planning, Curator and ZooKeeper for coordination, Kafka
> > > and Hadoop as data sources, Avro, ORC, or Parquet as supported data
> input
> > > formats, and DataSketches for scalable approximate algorithms. Druid
> > > can also be used as a data source by Superset.
> > >
> > > ## Issues
> > >
> > > There are no issues requiring board attention at this time.
> > >
> > > ## Activity
> > >
> > > We are currently finishing up our 2nd post-graduation release, 0.18.0,
> > > which we hope to have wrapped up and ready for release in the week of
> > > April 13th. Additionally, we released 0.17.1 on April 1st, in response
> > > to a vulnerabilty report recieved by the Apache Security Team. The
> > > corresponding CVE is CVE-2020-1958 and details are available at
> > > https://nvd.nist.gov/vuln/detail/CVE-2020-1958.
> > >
> > > To update on community happenings, since our last board report we
> > > have had 1 virtual meetup which was a success, with an additional
> > > virtual meetup scheduled for April 8th. Due to COVID-19, all in-person
> > > meetups have been put on hold, in favor of virtual meetups. Likewise,
> > > the Druid Summit event has been rescheduled for November 2-4, with a
> > > smaller virtual event scheduled for April 15th.
> > >
> > > Mailing list activity is healthy with 156 emails on the dev list
> > > (dev@druid.apache.org) over the last quarter. Our ASF slack channel,
> > > #druid, has nearly 750 members, with daily activity of users asking for
> > > and offering support to each other.
> > >
> > > ## Recent PMC changes
> > >
> > >  - Currently 27 PMC members.
> > >  - No changes to PMC since graduation.
> > >
> > > ## Recent committer changes
> > >
> > >  - Currently 35 committers.
> > >  - No recent changes to committers, the most recent addition was
> > >Chi Cao Minh on 2020-01-15
> > >
> > > ## Recent releases
> > >
> > >  - 0.17.1, a security release, was released on April 1 2020
> > >
> > > ## Development activity by the numbers
> > >
> > > In the last quarter:
> > >
> > >  - 317 pull requests opened
> > >  - 323 pull requests merged/closed
> > >  - 178 issues opened
> > >  - 112 issues closed
> > >  - 878 comments on pull requests
> > >  - 541 comments on issues
> > >
> >
>

Draft April ASF Board Report

[Clint Wylie]

Hey all,

I put together a draft for the quarterly ASF board report due tomorrow,
sorry for the short notice. Let me know if I missed anything or should make
any changes. Thanks!

-

## Description

Apache Druid is a high performance real-time analytics database. It is
designed for workflows where low-latency query and ingest are the main
requirements. It implements ingestion, storage, and querying subsystems.
Users interface with Druid through built-in SQL and JSON APIs, as well
as third-party applications.

Druid has an extensive web of connections with other Apache projects:
Calcite for SQL planning, Curator and ZooKeeper for coordination, Kafka
and Hadoop as data sources, Avro, ORC, or Parquet as supported data input
formats, and DataSketches for scalable approximate algorithms. Druid
can also be used as a data source by Superset.

## Issues

There are no issues requiring board attention at this time.

## Activity

We are currently finishing up our 2nd post-graduation release, 0.18.0,
which we hope to have wrapped up and ready for release in the week of
April 13th. Additionally, we released 0.17.1 on April 1st, in response
to a vulnerabilty report recieved by the Apache Security Team. The
corresponding CVE is CVE-2020-1958 and details are available at
https://nvd.nist.gov/vuln/detail/CVE-2020-1958.

To update on community happenings, since our last board report we
have had 1 virtual meetup which was a success, with an additional
virtual meetup scheduled for April 8th. Due to COVID-19, all in-person
meetups have been put on hold, in favor of virtual meetups. Likewise,
the Druid Summit event has been rescheduled for November 2-4, with a
smaller virtual event scheduled for April 15th.

Mailing list activity is healthy with 156 emails on the dev list
(dev@druid.apache.org) over the last quarter. Our ASF slack channel,
#druid, has nearly 750 members, with daily activity of users asking for
and offering support to each other.

## Recent PMC changes

 - Currently 27 PMC members.
 - No changes to PMC since graduation.

## Recent committer changes

 - Currently 35 committers.
 - No recent changes to committers, the most recent addition was
   Chi Cao Minh on 2020-01-15

## Recent releases

 - 0.17.1, a security release, was released on April 1 2020

## Development activity by the numbers

In the last quarter:

 - 317 pull requests opened
 - 323 pull requests merged/closed
 - 178 issues opened
 - 112 issues closed
 - 878 comments on pull requests
 - 541 comments on issues

draft ASF Board Report Feb 2020

[Clint Wylie]

Hey all,

I've put together our ASF board report for Feb 2020, and while I haven't
yet determined how to actually submit it, I wanted to go ahead and send out
a copy to the dev list in case anyone has any feedback.

---


## Description

Apache Druid is a high performance real-time analytics database. It is
designed for workflows where low-latency query and ingest are the main
requirements. It implements ingestion, storage, and querying subsystems.
Users interface with Druid through built-in SQL and JSON APIs, as well
as third-party applications.

Druid has an extensive web of connections with other Apache projects:
Calcite for SQL planning, Curator and ZooKeeper for coordination, Kafka
and Hadoop as data sources, Avro, ORC, or Parquet as supported data input
formats, and DataSketches for scalable approximate algorithms. Druid
can also be used as a data source by Superset.

## Issues

There are no issues requiring board attention at this time.

## Activity

We have done our first post-graduation release, Druid 0.17.0, containing
over 250 new features, performance enhancements, bug fixes, and major
documentation improvements from 52 contributors. Major improvements
include improved native batch indexing, performance improvements, LDAP
support, improved complaince with SQL standards, and much more. Work
on the upcoming 0.18.0 release is underway and on track for a likely
end of March release date.

Community activity continues to be strong, with a healthy rate of commits,
issues filed on github, user mailing list posts, and ever increasing
activity in our ASF Slack channel #druid, which now has over 600 members.

Keeping boots on the ground to improve adoption, over the last month we
have had Druid meetups in New Dehli, London, Athens, and Tel Aviv, with
an addtional meetup in Sydney scheduled in March. Planning for the
Druid Summit event is progressing with over 80 proposed talks currently
being evaluated, and half of the 40 scheduled speaking slots filled with
a diverse set of both PMC members and passionate users. The event will
take place April 13 - 15 in the San Francisco Bay Area.

## Recent PMC changes

 - Currently 27 PMC members.
 - No recent changes to PMC.

## Recent committer changes

 - Currently 35 committers.
 - New commmitters since last board report (Jan 15 2019):
- Chi Cao Minh (Jan 21 2020)

## Recent releases

 - 0.17.0, our first release post graduation was released on Jan 26 2020

## Development activity by the numbers

In the last month:

 - 79 pull requests opened
 - 78 pull requests merged/closed
 - 55 issues opened
 - 25 issues closed
 - 476 comments on pull requests
 - 231 comments on issues

Re: [VOTE] Release Apache Druid 0.17.0 [RC1]

[Clint Wylie]

+1 (binding)

src package:
- verified signature/hash
- verified LICENSE and NOTICE present
- verified packaging appears successfully de-incubated (no DISCLAIMER,
references to incubating/incubator, versions look good)
- RAT checks passed
- compiled, ran unit and integration tests
- built binary artifacts, tested artifacts with a subset of quickstart
exercises

bin package:
- verified signature/hash
- verified binary LICENSE and NOTICE present
- verified packaging appears successfully de-incubated (no DISCLAIMER,
references to incubating/incubator, versions look good)
- tested with a subset of quickstart exercises

docker image:
- verified hash
- tested with docker-compose and running through a subset quickstart
exercises


On Thu, Jan 23, 2020 at 11:42 AM Jihoon Son  wrote:

> +1 (binding)
>
> src package:
> - verified signature/hash
> - LICENSE and NOTICE are present
> - unit tests & integration tests passed
> - built binary distribution
> - ran the Kafka ingestion tutorial
> - ran a parallel indexing task
>
> bin package:
> - verified signature/hash
> - LICENSE and NOTICE are present
> - ran the Kafka ingestion tutorial
> - ran a parallel indexing task
>
> Jihoon
>
> On Wed, Jan 22, 2020 at 2:40 AM Jonathan Wei  wrote:
>
> > Hi all,
> >
> > I have created a build for Apache Druid 0.17.0, release
> > candidate 1.
> >
> > Thanks for everyone who has helped contribute to the release! You can
> read
> > the proposed release notes here:
> > https://github.com/apache/druid/issues/9066
> >
> > The release candidate has been tagged in GitHub as
> > druid-0.17.0-rc1 (f37b9842f22c0a243a5811ac7e887cc188291ef1),
> > available here:
> > https://github.com/apache/druid/releases/tag/druid-0.17.0-rc1
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/druid/0.17.0-rc1/
> >
> > Staged druid.apache.org website documentation is available here:
> > https://druid.staged.apache.org/docs/0.17.0/design/index.html
> >
> > A Docker image containing the binary of the release candidate can be
> > retrieved via:
> > docker pull apache/druid:0.17.0-rc1
> >
> > artifact checksums
> > src:
> >
> >
> 5b0ca592759d5b59674429a2e1f9ec2c4c41ec3075bb9f8d9c4d6e57625c3f703c90433a8be4863488dc7008b6266c0e0bcb4524d9d8cfb9c6547cc5f711c0fa
> > bin:
> >
> >
> 0a4054c73a965d1c7bac390e147f2733d7aa2de6632400e205c879410d5c1588d9eaa4af742c762295b7b0c462d8e67a58c52c0e16c4a81243eea02959a03732
> > docker: 48e561f0640a725f58e1e0363e568f83854f4c577042e63ab21457dfa00a8acf
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/jonwei.asc
> >
> > This key and the key of other committers can also be found in the
> project's
> > KEYS file here:
> > https://dist.apache.org/repos/dist/release/druid/KEYS
> >
> > (If you are a committer, please feel free to add your own key to that
> file
> > by following the instructions in the file's header.)
> >
> > Verify checksums:
> > diff <(shasum -a512 apache-druid-0.17.0-src.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-0.17.0-src.tar.gz.sha512 ; echo)
> >
> > diff <(shasum -a512 apache-druid-0.17.0-bin.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-0.17.0-bin.tar.gz.sha512 ; echo)
> >
> > Verify signatures:
> > gpg --verify apache-druid-0.17.0-src.tar.gz.asc \
> > apache-druid-0.17.0-src.tar.gz
> >
> > gpg --verify apache-druid-0.17.0-bin.tar.gz.asc \
> > apache-druid-0.17.0-bin.tar.gz
> >
> > Please review the proposed artifacts and vote. Note that Apache has
> > specific requirements that must be met before +1 binding votes can be
> cast
> > by PMC members. Please refer to the policy at
> > http://www.apache.org/legal/release-policy.html#policy for more details.
> >
> > As part of the validation process, the release artifacts can be generated
> > from source by running:
> > mvn clean install -Papache-release,dist -Dgpg.skip
> >
> > The RAT license check can be run from source by:
> > mvn apache-rat:check -Prat
> >
> > This vote will be open for at least 72 hours. The vote will pass if a
> > majority of at least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Druid 0.17.0
> > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Thanks!
> >
>

Re: Review request (autoscaling for GCE)

[Clint Wylie]

Apologies for the delay, I haven't had much extra time to review PRs lately
because I've been focusing on testing and fixing up bugs for 0.17 release,
but I will try to have a look at this PR sometime in the next couple of
weeks because once upon a time I also implemented an autoscaler for google
cloud, though it was more like the aws autoscaler and didn't use managed
instance groups.

On Fri, Jan 17, 2020 at 4:15 AM Roman Leventov 
wrote:

> Committers not having enough motivation to review doesn't necessarily mean
> there is a lack of interest. You may try to review someone else's sizable
> PR and kindly ask them to reciprocate by reviewing your PR. This strategy
> worked for me in the past.
>
> On Fri, 17 Jan 2020 at 11:15, Francesco Nidito <
> francesco.nid...@viralize.com> wrote:
>
> > Hello again!
> >
> > Is there any interest from the community on this change?
> >
> > Cheers,
> > Francesco
> >
> > On Tue, Jan 7, 2020 at 9:05 AM Francesco Nidito <
> > francesco.nid...@viralize.com> wrote:
> >
> > > Happy new year!
> > >
> > > At the beginning of December we opened a PR to add support for
> > autoscaling
> > > for GCE but it has not been given any attention so far.
> > >
> > > This is a change that we made internally at Viralize for our own use
> and
> > > it has been running for a while now.  We contributed it back into the
> > > project as we believe it may be useful to a wider audience of users on
> > GCE.
> > >
> > > The PR can be found at the following link →
> > > https://github.com/apache/druid/pull/8987
> > >
> > > Thanks in advance for any feedback you could provide to us :)
> > >
> > > Happy new year again!
> > >
> > > Francesco
> > >
> > > --
> > > Francesco Nidito
> > > *Lead Software Development Engineer*
> > > +39 339 2446640
> > > Skype: francesco.nidito
> > > francesco.nid...@viralize.com <%7Bthis.state.email%7D>Viralize
> > > Milan | London | Madrid | Rome | Florence (HQ)viralize.com
> > > 
> > >
> >
> >
> > --
> > Francesco Nidito
> > *Lead Software Development Engineer*
> > +39 339 2446640
> > Skype: francesco.nidito
> > francesco.nid...@viralize.com <%7Bthis.state.email%7D>Viralize
> > Milan | London | Madrid | Rome | Florence (HQ)viralize.com
> > 
> >
>

Re: Test naming in Druid

[Clint Wylie]

I strongly agree with Gian about underscores being ugly, though I think
your suggested formatting would be just as readable (to me at least) using
just camel-case and would also remain consistent with the styling of the
majority of Druids existing tests. I share similar concerns with the others
on how well it holds up as a convention beyond simple unit tests. I also
don't really feel that the utility this naming adds is worth treating it as
a hard rule, at least at this point. In my experience, the most useful
thing in determining why a test has failed and what my code did to do it is
generally either the stack trace of the error or the mismatched values in
the test expectation. The test name doesn't really matter to me at least,
and is more of just an anchor point to navigate to a position in a file to
let me know which test(s) to re-run, where as the other information is
usually all it takes to clue me into what I part of the changes in my PR
likely caused the test to fail.

Because of this, and more importantly that there will be no possible way
that tooling can enforce such a naming rule, I think it instead should just
be captured as best practices in CONTRIBUTING.md or something like that, as
just a suggestion, so that we don't have to nag at people to enforce this.

On Mon, Dec 23, 2019 at 7:15 PM Jonathan Wei  wrote:

> I think the suggested format could be useful when applicable, I would
> consider expanding "function under test" to "area under test" to
> accommodate tests that have a wider scope than a single method.
>
> For cases where that format may not be applicable, perhaps that means it's
> a test where additional javadocs/comments explaining what the test
> does/expects is a good idea (that would be enough info in my view).
>
> Since I don't think the proposed format applies universally, I would prefer
> starting it off as a suggestion/best practice instead of as a hard
> requirement and seeing how that goes.
>
> I'm indifferent to the use of underscores.
>
>
>
> On Mon, Dec 23, 2019 at 3:19 PM Gian Merlino  wrote:
>
> > Suneet,
> >
> > Sometimes it's hard to understand how things would improve without an
> > example. Could you point to a test file that you think would be improved
> by
> > this change? Also, there are some test files that I would struggle to fit
> > into this framework. It seems best suited to simple single-method unit
> > tests. What would you suggest doing for these examples?
> >
> > - CalciteQueryTest: Tests the SQL layer. There's one function per SQL
> query
> > being tested, but there is no specific function under test, and the
> > expected result is too complex to include in a method name.
> > - GroupByQueryRunnerTest: Tests the groupBy query engine. Similar
> structure
> > and similar issue to CalciteQueryTest.
> > - SelectorFilterTest: Tests the selector filter. But no specific function
> > is being tested. The "assertFilterMatches" helper is used to create test
> > cases that say 'this filter should match these rows'.
> >
> > Other notes:
> >
> > - IMO, the underscores are ugly, but acceptable if they buy us something.
> > - I'm not sure that the "expectedResult" part of the name is going to
> work
> > well. It's often something complex (like an array, or object, or long
> > string) that would be awkward to include in a method name. And with JUnit
> > assertions, the expected result is right there in the method call anyway.
> >
> > Looking forward to hearing your thoughts. Thanks for thinking about how
> to
> > make our testing better.
> >
> > Gian
> >
> > On Mon, Dec 23, 2019 at 11:06 AM Suneet Saldanha <
> suneet.salda...@imply.io
> > >
> > wrote:
> >
> > > Hello,
> > >
> > > I've started writing tests in the druid repo and would like to propose
> a
> > > new test naming structure for the project. Inspired by this thread -
> > > https://www.mail-archive.com/dev@druid.apache.org/msg02426.html but
> > > focused
> > > only on the naming of tests. I'd like to propose we start using the
> > format
> > > below
> > >
> > > test___
> > >
> > > This makes it a lot easier for devs to name tests in a way that's
> easily
> > > understood by someone else without having to read the test to know
> what's
> > > going on. Here's some of my rationale:
> > >
> > >- Explicit - the name tells you everything you need to know about
> the
> > >test
> > >- Forces you to write one test per condition/ expected result
> > >- Underscores make it easy to delineate the different components of
> > the
> > >test
> > >- Minimal effort to think of a short name that correctly captures
> > >everything about the test while still being different from all the
> > other
> > >tests
> > >
> > > Happy to hear feedback / concerns about this approach.
> > >
> > > Suneet
> > >
> >
>

Re: [VOTE] Release Apache Druid (incubating) 0.16.1 [RC2]

[Clint Wylie]

+1 (non-binding)

src package:
- verified signature/hash
- ran RAT check
- checked for LICENSE, NOTICE, and DISCLAIMER
- compiled and ran unit tests
- built binary distribution, ran quickstart on built artifacts

bin package:
- verified signature/hash
- checked for LICENSE, NOTICE, and DISCLAIMER
- ran quickstart

On Wed, Dec 4, 2019 at 12:47 PM David Lim  wrote:

> +1 (binding)
>
> src package:
> - verified signature/hash
> - compared source distribution contents against git tag (144bd78)
> - LICENSE, NOTICE, and DISCLAIMER are present
> - unit tests passed
> - licenses checked
> - built binary distribution
> - ran quickstart
>
> bin package:
> - verified signature/hash
> - verified META-INF/MANIFEST.MF:Build-Revision tag in JAR files matches
> source distribution git.version:Build-Revision (144bd78)
> - LICENSE, NOTICE, and DISCLAIMER are present
> - ran quickstart
>
> On Tue, Dec 3, 2019 at 6:20 PM Jonathan Wei  wrote:
>
> > Hi all,
> >
> > I have created a build for Apache Druid (incubating) 0.16.1, release
> > candidate 2.
> >
> > Thanks for everyone who has helped contribute to the release! You can
> read
> > the proposed release notes here:
> > https://github.com/apache/incubator-druid/issues/8972
> >
> > The release candidate has been tagged in GitHub as
> > druid-0.16.1-incubating-rc2 (144bd782964cac3d3d31f21b9e03b3c215f7f42d),
> > available here:
> >
> >
> https://github.com/apache/incubator-druid/releases/tag/druid-0.16.1-incubating-rc2
> >
> > The artifacts to be voted on are located here:
> >
> >
> https://dist.apache.org/repos/dist/dev/incubator/druid/0.16.1-incubating-rc2/
> >
> > Staged druid.apache.org website documentation is available here:
> > https://druid.staged.apache.org/docs/0.16.1-incubating/design/index.html
> >
> > A Docker image containing the binary of the release candidate can be
> > retrieved via:
> > docker pull apache/incubator-druid:0.16.1-incubating-rc2
> >
> > artifact checksums
> > src:
> >
> >
> 5644217e475885782bca18928cdfcbc75f3e95ace55ea11238f4350ea25b39a7febbca3e58d7de7d5145633a1e549f9022596ec65486b963c1978537c9a138dc
> > bin:
> >
> >
> f637453dfbdec4b22384e9bf8ebe93ccf17898e8f0ebc45312ca71aa7e3722dc79f558e6792fd67619d3c3abc21948e08e2fd8fe63d4c108a01330961fca
> > docker: 169ca79ea8f8623c08cef32d0570260b96c5c68fbb839e308afa41a75e2cec60
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/jonwei.asc
> >
> > This key and the key of other committers can also be found in the
> project's
> > KEYS file here:
> > https://dist.apache.org/repos/dist/release/incubator/druid/KEYS
> >
> > (If you are a committer, please feel free to add your own key to that
> file
> > by following the instructions in the file's header.)
> >
> > Verify checksums:
> > diff <(shasum -a512 apache-druid-0.16.1-incubating-src.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-0.16.1-incubating-src.tar.gz.sha512 ; echo)
> >
> > diff <(shasum -a512 apache-druid-0.16.1-incubating-bin.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-0.16.1-incubating-bin.tar.gz.sha512 ; echo)
> >
> > Verify signatures:
> > gpg --verify apache-druid-0.16.1-incubating-src.tar.gz.asc \
> > apache-druid-0.16.1-incubating-src.tar.gz
> >
> > gpg --verify apache-druid-0.16.1-incubating-bin.tar.gz.asc \
> > apache-druid-0.16.1-incubating-bin.tar.gz
> >
> > Please review the proposed artifacts and vote. Note that Apache has
> > specific requirements that must be met before +1 binding votes can be
> cast
> > by PMC members. Please refer to the policy at
> > http://www.apache.org/legal/release-policy.html#policy for more details.
> >
> > As part of the validation process, the release artifacts can be generated
> > from source by running:
> > mvn clean install -Papache-release,dist -Dgpg.skip
> >
> > The RAT license check can be run from source by:
> > mvn apache-rat:check -Prat
> >
> > This vote will be open for at least 72 hours. The vote will pass if a
> > majority of at least three +1 PMC votes are cast.
> >
> > Once the vote has passed, the second stage vote will be called on the
> > Apache Incubator mailing list to get approval from the Incubator PMC.
> >
> > [ ] +1 Release this package as Apache Druid (incubating) 0.16.0
> > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Thanks!
> >
> > Apache Druid (incubating) is an effort undergoing incubation at The
> Apache
> > Software Foundation (ASF), sponsored by the Apache Incubator. Incubation
> is
> > required of all newly accepted projects until a further review indicates
> > that the infrastructure, communications, and decision making process have
> > stabilized in a manner consistent with other successful ASF projects.
> While
> > incubation status is not necessarily a reflection of the completeness or
> > stability of the code, it does indicate that the project has yet to be
> > fully endorsed by the ASF.
> >
>

Re: [VOTE] Apache Druid graduation to top level project

[Clint Wylie]

+1

On Wed, Dec 4, 2019 at 1:21 PM Jihoon Son  wrote:

> +1
>
> On Wed, Dec 4, 2019 at 1:20 PM Furkan KAMACI 
> wrote:
>
> > Hi,
> >
> > +1!
> >
> > Kind Regards,
> > Furkan KAMACI
> >
> > 4 Ara 2019 Çar, saat 23:58 tarihinde David Lim 
> şunu
> > yazdı:
> >
> > > +1
> > >
> > > On Wed, Dec 4, 2019 at 1:47 PM Fangjin Yang  wrote:
> > >
> > > > +1
> > > >
> > > > On Wed, Dec 4, 2019 at 12:36 PM Julian Hyde 
> wrote:
> > > >
> > > > > +1
> > > > >
> > > > >
> > > > > > On Dec 4, 2019, at 12:19 PM, Gian Merlino 
> wrote:
> > > > > >
> > > > > > Earlier this year, Druid voted to graduate to a top level
> project.
> > > The
> > > > > vote
> > > > > > passed and a resolution was submitted to the Board, but needed to
> > be
> > > > > > shelved at the last minute. We are now ready to proceed to
> > graduation
> > > > > once
> > > > > > again, and so I would like to call another vote.
> > > > > >
> > > > > > For reference, our last vote thread:
> > > > > >
> > > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/7517dd38fbeb28978b1188bed1d69e91445927d13d520a4c71ae9fcf%40%3Cdev.druid.apache.org%3E
> > > > > >
> > > > > > I would like to propose the resolution below. It is the same as
> the
> > > one
> > > > > we
> > > > > > voted on last time, except it adds new committers as initial PMC
> > > > members
> > > > > > (based on our previous decision to include any interested podling
> > > > > > committers on the newly formed PMC).
> > > > > >
> > > > > > Vote:
> > > > > > [ ] +1 - Recommend graduation of Apache Druid as a TLP
> > > > > > [ ] -1 - Do not recommend graduation of Apache Druid because...
> > > > > >
> > > > > > The VOTE is open for a minimum of 72 hours.
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Establish Apache Druid Project
> > > > > >
> > > > > > WHEREAS, the Board of Directors deems it to be in the best
> > > > > > interests of the Foundation and consistent with the Foundation's
> > > > > > purpose to establish a Project Management Committee charged with
> > > > > > the creation and maintenance of open-source analytical database
> > > > > > software, for distribution at no charge to the public.
> > > > > >
> > > > > > NOW, THEREFORE, BE IT RESOLVED, that a Project Management
> > > > > > Committee (PMC), to be known as the "The Apache Druid Project",
> > > > > > be and hereby is established pursuant to Bylaws of the
> > > > > > Foundation; and be it further
> > > > > >
> > > > > > RESOLVED, that The Apache Druid Project be and hereby is
> > > > > > responsible for the creation and maintenance of an analytical
> > > > > > database software project; and be it further
> > > > > >
> > > > > > RESOLVED, that the office of "Vice President, Druid" be and
> > > > > > hereby is created, the person holding such office to serve at the
> > > > > > direction of the Board of Directors as the chair of The Apache
> > > > > > Druid Project, and to have primary responsibility for
> > > > > > management of the projects within the scope of responsibility of
> > > > > > The Apache Druid Project; and be it further
> > > > > >
> > > > > > RESOLVED, that the persons listed immediately below be and
> > > > > > hereby are appointed to serve as the initial members of The
> > > > > > Apache Druid Project:
> > > > > >
> > > > > >  * Benedict Jin (asdf2...@apache.org)
> > > > > >  * Charles Allen(cral...@apache.org)
> > > > > >  * Clint Wylie  (cwy...@apache.org)
> > > > > >  * David Lim(david...@apache.org)
> > > > > >  * Dylan Wylie  (dylanwy...@apache.org)
> > > > > >  * Eric T

replacing the old web consoles

[Clint Wylie]

Hi all,

Vadim and I have been working on a PR to replace and remove all of the old
coordinator and overlord web consoles with the new web console that is
currently only served from the router. The new console will operate with
the subset of functionality which is appropriate to each service type. This
new ui will be functionally equivalent to what was available, with the
exception of the segment timeline that is available on the newer old
coordinator console, which I believe Vad plans to add in a follow-up (there
is already a version of this on the new router web console). We think this
will give users a more consistent and coherent experience when operating
Druid, and it removes a rather large chunk of code from the code-base which
should make maintenance a lot easier on our end.

You can find the PR here:
https://github.com/apache/incubator-druid/pull/8838. We think this PR is
ready to merge, but wanted to ask the list if anyone has any concerns
before we move forward with this.

Cheers,
Clint

[ANNOUNCE] Apache Druid (incubating) 0.16.0 release

[Clint Wylie]

The Apache Druid team is proud to announce the release of Apache Druid
(incubating) 0.16.0. Druid is a high performance analytics data store for
event-driven data.

Apache Druid 0.16.0-incubating contains over 350 new features, performance
enhancements, bug fixes, and major documentation improvements from 50
contributors. Major new features and improvements include:

- 'Vectorized' query processing
- 'Minor' compaction
- Native parallel indexing with shuffle
- New 'indexer' process
- Huge improvements to the web console
- New documentation website
- Official Docker image

Source and binary distributions can be downloaded from:
https://druid.apache.org/downloads.html

Release notes are at:
https://github.com/apache/incubator-druid/releases/tag/druid-0.16.0-incubating

A big thank you to all the contributors in this milestone release!



Disclaimer: Apache Druid is an effort undergoing incubation at The Apache
Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is
required of all newly accepted projects until a further review indicates
that the infrastructure, communications, and decision making process have
stabilized in a manner consistent with other successful ASF projects. While
incubation status is not necessarily a reflection of the completeness or
stability of the code, it does indicate that the project has yet to be
fully endorsed by the ASF.

[RESULT][VOTE] Release Apache Druid (incubating) 0.16.0 [RC3]

[Clint Wylie]

Thanks to everyone who participated in the vote! The results are as follows:

Clint Wylie: +1 (non-binding)
Jihoon Son: +1 (binding)
Furkan KAMACI: +1 (non-binding)
Julian Hyde: +1 (binding)
David Lim: +1 (binding)
Surekha Saharan: +1 (non-binding)

Additionally, I think we should be able to carry over votes from Julian and
Furkan as +1 (binding) for the IPMC vote. However, before I start that
thread I wanted to point out that we currently have an open issue regarding
distribution of example tutorial data which might cause issue with our IPMC
vote, described at https://issues.apache.org/jira/browse/LEGAL-480. I do
have a backup plan if we think we should fix this now as part of this
release, which transitions the tutorials to use a different public domain
dataset, but there is still a fair bit of work to do before it would be
complete, and I would rather not try to rush this through risk mistakes and
then go through an RC4. You can find the work in progress branch here:
https://github.com/apache/incubator-druid/compare/master...implydata:koalas-to-the-min

My opinion is that since we are working to make progress to resolve the
issue that we should proceed with the vote anyway, but I wanted to give the
chance for anyone else to chime in with their thoughts. I plan to open the
IPMC vote thread for RC3 sometime tomorrow if no one thinks that this
unresolved issue should be a blocker.

Re: [VOTE] Release Apache Druid (incubating) 0.16.0 [RC3]

[Clint Wylie]

Thanks for the votes everyone! We've got enough binding +1 so I'm going to
close this vote soon and prepare for the IPMC vote thread.

On Mon, Sep 16, 2019 at 9:38 PM Surekha Saharan 
wrote:

> +1 (non-binding)
>
> src package:
> - verified signature and hash
> - compiled source and ran unit tests
> - ran integration tests
> - ran RAT check
> - checked LICENSE, NOTICE, DISCLAIMER
>
>
> bin package:
> - verified signature and hash
> - ran quickstart batch and kafka ingestion tutorial
> - checked LICENSE, NOTICE, DISCLAIMER
>
>
> On Mon, Sep 16, 2019 at 9:07 PM David Lim  wrote:
>
> >  +1 (binding)
> >
> > src package:
> > - verified signature/hash
> > - compared source distribution contents against git tag (54d29e4)
> > - LICENSE, NOTICE, and DISCLAIMER are present
> > - unit tests passed
> > - licenses checked
> > - built binary distribution
> > - ran quickstart
> >
> > bin package:
> > - verified signature/hash
> > - verified META-INF/MANIFEST.MF:Build-Revision tag in JAR files matches
> > source distribution git.version:Build-Revision (54d29e4)
> > - LICENSE, NOTICE, and DISCLAIMER are present
> > - ran quickstart
> >
> > On Mon, Sep 16, 2019 at 1:29 PM Julian Hyde  wrote:
> >
> > > Full checksum. An attacker can easily generate a binary that matches a
> > > given 32 bit bit (8 digit) hash. That’s why we use SHA-256 or SHA-512.
> > >
> > > If it helps, here is a typical Calcite vote email:
> > >
> > >
> > >
> >
> http://mail-archives.apache.org/mod_mbox/calcite-dev/201906.mbox/%3cCA+EpF8vwOceAeUjv+DJU=zqrkoqu3dwckwsypqhrj6crw9e...@mail.gmail.com%3e
> > > <
> > >
> >
> http://mail-archives.apache.org/mod_mbox/calcite-dev/201906.mbox/%3CCA+EpF8vwOceAeUjv+DJU=zqrkoqu3dwckwsypqhrj6crw9e...@mail.gmail.com%3E
> > >
> > >
> > >
> > >
> > >
> > >
> > > > On Sep 16, 2019, at 1:43 AM, Clint Wylie  wrote:
> > > >
> > > > Ah, oops, yes indeed they are reversed, my bad! I certainly agree
> with
> > > all
> > > > your points on why it is a good idea, and will update our template
> > after
> > > > the release to make sure we do it in the future. Is it better
> practice
> > to
> > > > include the full checksum, or would truncated to the first 8 or so
> > > > characters be preferable to play nice with email?
> > > >
> > > > On Sun, Sep 15, 2019 at 8:34 PM Julian Hyde 
> wrote:
> > > >
> > > >> Sorry for my rather terse -1 vote. I had assumed that we had been
> > > >> following the policy for a while, so when I noticed that we were
> not I
> > > >> assumed it was a mistake by the release manager.
> > > >>
> > > >> Actually I am not sure whether it is policy, but there's definitely
> a
> > > >> strong case for including hashes. The point is this: we are voting
> on
> > > >> artifacts, principally apache-druid-0.16.0-incubating-src.tar.gz.
> > > >>
> > > >> Suppose we all vote on the current
> > > >> apache-druid-0.16.0-incubating-src.tar.gz, the vote passes, and then
> > > >> someone replaces it with a similar file that contains some bad
> stuff.
> > > >> How are we to know whether that is the file we voted on?
> > > >>
> > > >> Putting the file hash in the vote email guarantees that we are all
> > > >> voting on the same set of artifacts, and that set of artifacts is
> > > >> recorded.
> > > >>
> > > >> I think you reversed the hashes (I got 0c4b71f0 for bin, 1f25c55e
> for
> > > >> src), but that's close enough, so let's proceed.
> > > >>
> > > >>
> > > >> +1 (binding)
> > > >>
> > > >> Checked hashes, LICENSE, NOTICE, DISCLAIMER; ran RAT; compiled
> > > >> (skipping tests) using JDK 8 on Ubuntu. Checked that src.tar.gz
> > > >> matches git commit.
> > > >>
> > > >> Julian
> > > >>
> > > >>
> > > >> Julian
> > > >>
> > > >> On Sun, Sep 15, 2019 at 7:24 PM Clint Wylie 
> > wrote:
> > > >>>
> > > >>>> The vote email must contain the checksums of the artifacts we are
> > > >

Re: [VOTE] Release Apache Druid (incubating) 0.16.0 [RC3]

[Clint Wylie]

Ah, oops, yes indeed they are reversed, my bad! I certainly agree with all
your points on why it is a good idea, and will update our template after
the release to make sure we do it in the future. Is it better practice to
include the full checksum, or would truncated to the first 8 or so
characters be preferable to play nice with email?

On Sun, Sep 15, 2019 at 8:34 PM Julian Hyde  wrote:

> Sorry for my rather terse -1 vote. I had assumed that we had been
> following the policy for a while, so when I noticed that we were not I
> assumed it was a mistake by the release manager.
>
> Actually I am not sure whether it is policy, but there's definitely a
> strong case for including hashes. The point is this: we are voting on
> artifacts, principally apache-druid-0.16.0-incubating-src.tar.gz.
>
> Suppose we all vote on the current
> apache-druid-0.16.0-incubating-src.tar.gz, the vote passes, and then
> someone replaces it with a similar file that contains some bad stuff.
> How are we to know whether that is the file we voted on?
>
> Putting the file hash in the vote email guarantees that we are all
> voting on the same set of artifacts, and that set of artifacts is
> recorded.
>
> I think you reversed the hashes (I got 0c4b71f0 for bin, 1f25c55e for
> src), but that's close enough, so let's proceed.
>
>
> +1 (binding)
>
> Checked hashes, LICENSE, NOTICE, DISCLAIMER; ran RAT; compiled
> (skipping tests) using JDK 8 on Ubuntu. Checked that src.tar.gz
> matches git commit.
>
> Julian
>
>
> Julian
>
> On Sun, Sep 15, 2019 at 7:24 PM Clint Wylie  wrote:
> >
> > > The vote email must contain the checksums of the artifacts we are
> voting
> > on.
> >
> > Apologies, I wasn't aware of this requirement since we haven't put them
> in
> > our prior incubating release vote threads and I was just copying the same
> > basic template I and others have previously used. Out of curiosity is
> this
> > a new-ish requirement that I missed, or one we just didn't notice or have
> > just been turning a blind eye to? Regardless, since we are now
> maintaining
> > a 'how to ASF release' guide in the github repo that includes templates
> for
> > voting threads,
> >
> https://github.com/apache/incubator-druid/blob/master/distribution/asf-release-process-guide.md#body
> ,
> > I'll
> > be sure to update it, thanks!
> >
> > > No need for a new RC; I change my vote if the release manager sends an
> > > email with the checksums.
> >
> > If this thread is ok, here they are:
> >
> > artifact checksums
> > src:
> >
> 0c4b71f077e28d2f4d3bc3f072543374570b98ec6a1918a5e1828e1da7e3871b5efb04070a8bcdbc172a817e43254640ce28a99757984be7d8dd3d607f1d870e
> > bin:
> >
> 1f25c55e83069cf7071a97c1e0d56732437dbac4ef373ed1ed72b5b618021b74c107269642226e80081354c8da2e92dc26f1541b01072a4720fd6cfe8dc161a8
> > docker: df9b900d3726ce123a5c054768da1ea08eba6efe635ced5abc3ad72d6c835e2c
> >
> > Thanks!
> > Clint
> >
> > On Sun, Sep 15, 2019 at 6:22 PM Julian Hyde  wrote:
> >
> > > -1
> > >
> > > The vote email must contain the checksums of the artifacts we are
> voting
> > > on.
> > >
> > > No need for a new RC; I change my vote if the release manager sends an
> > > email with the checksums.
> > >
> > > Julian
> > >
> > > On Fri, Sep 13, 2019 at 11:57 PM Clint Wylie 
> wrote:
> > > >
> > > > Hi all,
> > > >
> > > > I have created a build for Apache Druid (incubating) 0.16.0, release
> > > > candidate 3.
> > > >
> > > > Thanks for everyone who has helped contribute to the release! You can
> > > read
> > > > the proposed release notes here:
> > > > https://github.com/apache/incubator-druid/issues/8369
> > > >
> > > > The release candidate has been tagged in GitHub as
> > > > druid-0.16.0-incubating-rc3
> (54d29e438a4df34d75e2385af6cefd1092c4ebb3),
> > > > available here:
> > > >
> > >
> https://github.com/apache/incubator-druid/releases/tag/druid-0.16.0-incubating-rc3
> > > >
> > > > The artifacts to be voted on are located here:
> > > >
> > >
> https://dist.apache.org/repos/dist/dev/incubator/druid/0.16.0-incubating-rc3/
> > > >
> > > > Staged druid.apache.org website documentation is available here:
> > > >
> https://druid.staged.apache.org/docs/0.16.0-incubating/design/index.html
> > > >
> > > &

Re: [VOTE] Release Apache Druid (incubating) 0.16.0 [RC3]

[Clint Wylie]

> The vote email must contain the checksums of the artifacts we are voting
on.

Apologies, I wasn't aware of this requirement since we haven't put them in
our prior incubating release vote threads and I was just copying the same
basic template I and others have previously used. Out of curiosity is this
a new-ish requirement that I missed, or one we just didn't notice or have
just been turning a blind eye to? Regardless, since we are now maintaining
a 'how to ASF release' guide in the github repo that includes templates for
voting threads,
https://github.com/apache/incubator-druid/blob/master/distribution/asf-release-process-guide.md#body,
I'll
be sure to update it, thanks!

> No need for a new RC; I change my vote if the release manager sends an
> email with the checksums.

If this thread is ok, here they are:

artifact checksums
src:
0c4b71f077e28d2f4d3bc3f072543374570b98ec6a1918a5e1828e1da7e3871b5efb04070a8bcdbc172a817e43254640ce28a99757984be7d8dd3d607f1d870e
bin:
1f25c55e83069cf7071a97c1e0d56732437dbac4ef373ed1ed72b5b618021b74c107269642226e80081354c8da2e92dc26f1541b01072a4720fd6cfe8dc161a8
docker: df9b900d3726ce123a5c054768da1ea08eba6efe635ced5abc3ad72d6c835e2c

Thanks!
Clint

On Sun, Sep 15, 2019 at 6:22 PM Julian Hyde  wrote:

> -1
>
> The vote email must contain the checksums of the artifacts we are voting
> on.
>
> No need for a new RC; I change my vote if the release manager sends an
> email with the checksums.
>
> Julian
>
> On Fri, Sep 13, 2019 at 11:57 PM Clint Wylie  wrote:
> >
> > Hi all,
> >
> > I have created a build for Apache Druid (incubating) 0.16.0, release
> > candidate 3.
> >
> > Thanks for everyone who has helped contribute to the release! You can
> read
> > the proposed release notes here:
> > https://github.com/apache/incubator-druid/issues/8369
> >
> > The release candidate has been tagged in GitHub as
> > druid-0.16.0-incubating-rc3 (54d29e438a4df34d75e2385af6cefd1092c4ebb3),
> > available here:
> >
> https://github.com/apache/incubator-druid/releases/tag/druid-0.16.0-incubating-rc3
> >
> > The artifacts to be voted on are located here:
> >
> https://dist.apache.org/repos/dist/dev/incubator/druid/0.16.0-incubating-rc3/
> >
> > Staged druid.apache.org website documentation is available here:
> > https://druid.staged.apache.org/docs/0.16.0-incubating/design/index.html
> >
> > A Docker image containing the binary of the release candidate can be
> > retrieved via:
> > docker pull apache/incubator-druid:0.16.0-incubating-rc3
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/cwylie.asc
> >
> > This key and the key of other committers can also be found in the
> project's
> > KEYS file here:
> > https://dist.apache.org/repos/dist/release/incubator/druid/KEYS
> >
> > (If you are a committer, please feel free to add your own key to that
> file
> > by following the instructions in the file's header.)
> >
> >
> > Verify checksums:
> > diff <(shasum -a512 apache-druid-0.16.0-incubating-src.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-0.16.0-incubating-src.tar.gz.sha512 ; echo)
> >
> > diff <(shasum -a512 apache-druid-0.16.0-incubating-bin.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-0.16.0-incubating-bin.tar.gz.sha512 ; echo)
> >
> > Verify signatures:
> > gpg --verify apache-druid-0.16.0-incubating-src.tar.gz.asc \
> > apache-druid-0.16.0-incubating-src.tar.gz
> >
> > gpg --verify apache-druid-0.16.0-incubating-bin.tar.gz.asc \
> > apache-druid-0.16.0-incubating-bin.tar.gz
> >
> > Please review the proposed artifacts and vote. Note that Apache has
> > specific requirements that must be met before +1 binding votes can be
> cast
> > by PMC members. Please refer to the policy at
> > http://www.apache.org/legal/release-policy.html#policy for more details.
> >
> > As part of the validation process, the release artifacts can be generated
> > from source by running:
> > mvn clean install -Papache-release,dist -Dgpg.skip
> >
> > The RAT license check can be run from source by:
> > mvn apache-rat:check -Prat
> >
> > This vote will be open for at least 72 hours. The vote will pass if a
> > majority of at least three +1 PMC votes are cast.
> >
> > Once the vote has passed, the second stage vote will be called on the
> > Apache Incubator mailing list to get approval from the Incubator PMC.
> >
> > [ ] +1 Release this package as Apache Druid (incubating) 0.16.0
> > [ 

[VOTE] Release Apache Druid (incubating) 0.16.0 [RC3]

[Clint Wylie]

Hi all,

I have created a build for Apache Druid (incubating) 0.16.0, release
candidate 3.

Thanks for everyone who has helped contribute to the release! You can read
the proposed release notes here:
https://github.com/apache/incubator-druid/issues/8369

The release candidate has been tagged in GitHub as
druid-0.16.0-incubating-rc3 (54d29e438a4df34d75e2385af6cefd1092c4ebb3),
available here:
https://github.com/apache/incubator-druid/releases/tag/druid-0.16.0-incubating-rc3

The artifacts to be voted on are located here:
https://dist.apache.org/repos/dist/dev/incubator/druid/0.16.0-incubating-rc3/

Staged druid.apache.org website documentation is available here:
https://druid.staged.apache.org/docs/0.16.0-incubating/design/index.html

A Docker image containing the binary of the release candidate can be
retrieved via:
docker pull apache/incubator-druid:0.16.0-incubating-rc3

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/cwylie.asc

This key and the key of other committers can also be found in the project's
KEYS file here:
https://dist.apache.org/repos/dist/release/incubator/druid/KEYS

(If you are a committer, please feel free to add your own key to that file
by following the instructions in the file's header.)


Verify checksums:
diff <(shasum -a512 apache-druid-0.16.0-incubating-src.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-0.16.0-incubating-src.tar.gz.sha512 ; echo)

diff <(shasum -a512 apache-druid-0.16.0-incubating-bin.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-0.16.0-incubating-bin.tar.gz.sha512 ; echo)

Verify signatures:
gpg --verify apache-druid-0.16.0-incubating-src.tar.gz.asc \
apache-druid-0.16.0-incubating-src.tar.gz

gpg --verify apache-druid-0.16.0-incubating-bin.tar.gz.asc \
apache-druid-0.16.0-incubating-bin.tar.gz

Please review the proposed artifacts and vote. Note that Apache has
specific requirements that must be met before +1 binding votes can be cast
by PMC members. Please refer to the policy at
http://www.apache.org/legal/release-policy.html#policy for more details.

As part of the validation process, the release artifacts can be generated
from source by running:
mvn clean install -Papache-release,dist -Dgpg.skip

The RAT license check can be run from source by:
mvn apache-rat:check -Prat

This vote will be open for at least 72 hours. The vote will pass if a
majority of at least three +1 PMC votes are cast.

Once the vote has passed, the second stage vote will be called on the
Apache Incubator mailing list to get approval from the Incubator PMC.

[ ] +1 Release this package as Apache Druid (incubating) 0.16.0
[ ] 0 I don't feel strongly about it, but I'm okay with the release
[ ] -1 Do not release this package because...

Thanks!

Apache Druid (incubating) is an effort undergoing incubation at The Apache
Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is
required of all newly accepted projects until a further review indicates
that the infrastructure, communications, and decision making process have
stabilized in a manner consistent with other successful ASF projects. While
incubation status is not necessarily a reflection of the completeness or
stability of the code, it does indicate that the project has yet to be
fully endorsed by the ASF.

[CANCEL] [VOTE] Release Apache Druid (incubating) 0.16.0 [RC2]

[Clint Wylie]

Cancelling vote for RC2 due to
https://github.com/apache/incubator-druid/pull/8535. I will create an RC3
as soon as possible.

Re: [VOTE] Release Apache Druid (incubating) 0.16.0 [RC2]

[Clint Wylie]

Unfortunately we have found a rather critical bug with group by query
cache, so I am going to send out a cancel of this vote soon. A PR for the
fix has been opened, https://github.com/apache/incubator-druid/pull/8535,
but apologies for the false start.

On Thu, Sep 12, 2019 at 6:00 AM Clint Wylie  wrote:

> Hey all,
>
> I have created a build for Apache Druid (incubating) 0.16.0, release
> candidate 2.
>
> Thanks for everyone who has helped contribute to the release! You can read
> the proposed release notes here:
> https://github.com/apache/incubator-druid/issues/8369
>
> The release candidate has been tagged in GitHub as
> druid-0.16.0-incubating-rc2 (e406d6d2136cf2306c27907bdc0c71c8a2b9d957),
> available here:
>
> https://github.com/apache/incubator-druid/releases/tag/druid-0.16.0-incubating-rc2
>
> The artifacts to be voted on are located here:
>
> https://dist.apache.org/repos/dist/dev/incubator/druid/0.16.0-incubating-rc2/
>
> Staged druid.apache.org website documentation is available here:
> https://druid.staged.apache.org/docs/0.16.0-incubating/design/index.html
>
> A Docker image containing the binary of the release candidate can be
> retrieved via:
> docker pull apache/incubator-druid:0.16.0-incubating-rc2
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/cwylie.asc
>
> This key and the key of other committers can also be found in the
> project's KEYS file here:
> https://dist.apache.org/repos/dist/release/incubator/druid/KEYS
>
> (If you are a committer, please feel free to add your own key to that file
> by following the instructions in the file's header.)
>
>
> Verify checksums:
> diff <(shasum -a512 apache-druid-0.16.0-incubating-src.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-0.16.0-incubating-src.tar.gz.sha512 ; echo)
>
> diff <(shasum -a512 apache-druid-0.16.0-incubating-bin.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-0.16.0-incubating-bin.tar.gz.sha512 ; echo)
>
> Verify signatures:
> gpg --verify apache-druid-0.16.0-incubating-src.tar.gz.asc \
> apache-druid-0.16.0-incubating-src.tar.gz
>
> gpg --verify apache-druid-0.16.0-incubating-bin.tar.gz.asc \
> apache-druid-0.16.0-incubating-bin.tar.gz
>
> Please review the proposed artifacts and vote. Note that Apache has
> specific requirements that must be met before +1 binding votes can be cast
> by PMC members. Please refer to the policy at
> http://www.apache.org/legal/release-policy.html#policy for more details.
>
> As part of the validation process, the release artifacts can be generated
> from source by running:
> mvn clean install -Papache-release,dist -Dgpg.skip
>
> The RAT license check can be run from source by:
> mvn apache-rat:check -Prat
>
> This vote will be open for at least 72 hours. The vote will pass if a
> majority of at least three +1 PMC votes are cast.
>
> Once the vote has passed, the second stage vote will be called on the
> Apache Incubator mailing list to get approval from the Incubator PMC.
>
> [ ] +1 Release this package as Apache Druid (incubating) 0.16.0
> [ ] 0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
> Thanks!
>
> Apache Druid (incubating) is an effort undergoing incubation at The Apache
> Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is
> required of all newly accepted projects until a further review indicates
> that the infrastructure, communications, and decision making process have
> stabilized in a manner consistent with other successful ASF projects. While
> incubation status is not necessarily a reflection of the completeness or
> stability of the code, it does indicate that the project has yet to be
> fully endorsed by the ASF.
>

[VOTE] Release Apache Druid (incubating) 0.16.0 [RC2]

[Clint Wylie]

Hey all,

I have created a build for Apache Druid (incubating) 0.16.0, release
candidate 2.

Thanks for everyone who has helped contribute to the release! You can read
the proposed release notes here:
https://github.com/apache/incubator-druid/issues/8369

The release candidate has been tagged in GitHub as
druid-0.16.0-incubating-rc2 (e406d6d2136cf2306c27907bdc0c71c8a2b9d957),
available here:
https://github.com/apache/incubator-druid/releases/tag/druid-0.16.0-incubating-rc2

The artifacts to be voted on are located here:
https://dist.apache.org/repos/dist/dev/incubator/druid/0.16.0-incubating-rc2/

Staged druid.apache.org website documentation is available here:
https://druid.staged.apache.org/docs/0.16.0-incubating/design/index.html

A Docker image containing the binary of the release candidate can be
retrieved via:
docker pull apache/incubator-druid:0.16.0-incubating-rc2

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/cwylie.asc

This key and the key of other committers can also be found in the project's
KEYS file here:
https://dist.apache.org/repos/dist/release/incubator/druid/KEYS

(If you are a committer, please feel free to add your own key to that file
by following the instructions in the file's header.)


Verify checksums:
diff <(shasum -a512 apache-druid-0.16.0-incubating-src.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-0.16.0-incubating-src.tar.gz.sha512 ; echo)

diff <(shasum -a512 apache-druid-0.16.0-incubating-bin.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-0.16.0-incubating-bin.tar.gz.sha512 ; echo)

Verify signatures:
gpg --verify apache-druid-0.16.0-incubating-src.tar.gz.asc \
apache-druid-0.16.0-incubating-src.tar.gz

gpg --verify apache-druid-0.16.0-incubating-bin.tar.gz.asc \
apache-druid-0.16.0-incubating-bin.tar.gz

Please review the proposed artifacts and vote. Note that Apache has
specific requirements that must be met before +1 binding votes can be cast
by PMC members. Please refer to the policy at
http://www.apache.org/legal/release-policy.html#policy for more details.

As part of the validation process, the release artifacts can be generated
from source by running:
mvn clean install -Papache-release,dist -Dgpg.skip

The RAT license check can be run from source by:
mvn apache-rat:check -Prat

This vote will be open for at least 72 hours. The vote will pass if a
majority of at least three +1 PMC votes are cast.

Once the vote has passed, the second stage vote will be called on the
Apache Incubator mailing list to get approval from the Incubator PMC.

[ ] +1 Release this package as Apache Druid (incubating) 0.16.0
[ ] 0 I don't feel strongly about it, but I'm okay with the release
[ ] -1 Do not release this package because...

Thanks!

Apache Druid (incubating) is an effort undergoing incubation at The Apache
Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is
required of all newly accepted projects until a further review indicates
that the infrastructure, communications, and decision making process have
stabilized in a manner consistent with other successful ASF projects. While
incubation status is not necessarily a reflection of the completeness or
stability of the code, it does indicate that the project has yet to be
fully endorsed by the ASF.

[CANCEL] [VOTE] Release Apache Druid (incubating) 0.16.0 [RC1]

[Clint Wylie]

Cancelling vote for RC1 due to
https://github.com/apache/incubator-druid/pull/8506. I will try to make an
RC2 and get a new vote started later today if possible.

Re: [VOTE] Release Apache Druid (incubating) 0.16.0 [RC1]

[Clint Wylie]

Unfortunately a small issue was found, going to cancel this vote and make
an RC2 later today.

On Wed, Sep 11, 2019 at 12:44 AM Clint Wylie  wrote:

> Hi Druids,
>
> I have created a build for Apache Druid (incubating) 0.16.0, release
> candidate 1.
>
> Thanks for everyone who has helped contribute to the release! You can read
> the proposed release notes here:
> https://github.com/apache/incubator-druid/issues/8369
>
> The release candidate has been tagged in GitHub as
> druid-0.16.0-incubating-rc1 (8b7f0de9d2bcb95b529c24f8ab2e8e2ccc4f7a0a),
> available here:
>
> https://github.com/apache/incubator-druid/releases/tag/druid-0.16.0-incubating-rc1
>
> The artifacts to be voted on are located here:
>
> https://dist.apache.org/repos/dist/dev/incubator/druid/0.16.0-incubating-rc1/
>
> Staged documentation for the release candidate is available here:
> https://druid.staged.apache.org/docs/0.16.0-incubating/design/index.html
>
> A Docker image containing the binary of the release candidate can be
> retrieved via:
> docker pull apache/incubator-druid:0.16.0-incubating-rc1
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/cwylie.asc
>
> This key and the key of other committers can also be found in the
> project's KEYS file here:
> https://dist.apache.org/repos/dist/release/incubator/druid/KEYS
>
> (If you are a committer, please feel free to add your own key to that file
> by following the instructions in the file's header.)
>
> Verify checksums:
> diff <(shasum -a512 apache-druid-0.16.0-incubating-bin.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-0.16.0-incubating-bin.tar.gz.sha512 ; echo)
>
> diff <(shasum -a512 apache-druid-0.16.0-incubating-src.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-0.16.0-incubating-src.tar.gz.sha512 ; echo)
>
> Verify signatures:
> gpg --verify apache-druid-0.16.0-incubating-bin.tar.gz.asc \
> apache-druid-0.16.0-incubating-bin.tar.gz
>
> gpg --verify apache-druid-0.16.0-incubating-src.tar.gz.asc \
> apache-druid-0.16.0-incubating-src.tar.gz
>
> Please review the proposed artifacts and vote. Note that Apache has
> specific requirements that must be met before +1 binding votes can be cast
> by PMC members. Please refer to the policy at
> http://www.apache.org/legal/release-policy.html#policy for more details.
>
> As part of the validation process, the release artifacts can be generated
> from source by running:
> mvn clean install -Papache-release,dist -Dgpg.skip
>
> The RAT license check can be run from source by:
> mvn apache-rat:check -Prat
>
> This vote will be open for at least 72 hours. The vote will pass if a
> majority of at least three +1 PMC votes are cast.
>
> Once the vote has passed, the second stage vote will be called on the
> Apache Incubator mailing list to get approval from the Incubator PMC.
>
> [ ] +1  Release this package as Apache Druid (incubating) 0.16.0
> [ ]   0  I don't feel strongly about it, but I'm okay with the release
> [ ]  -1  Do not release this package because...
>
> Thanks!
>
> Apache Druid (incubating) is an effort undergoing incubation at The Apache
> Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is
> required of all newly accepted projects until a further review indicates
> that the infrastructure, communications, and decision making process have
> stabilized in a manner consistent with other successful ASF projects. While
> incubation status is not necessarily a reflection of the completeness or
> stability of the code, it does indicate that the project has yet to be
> fully endorsed by the ASF.
>

[VOTE] Release Apache Druid (incubating) 0.16.0 [RC1]

[Clint Wylie]

Hi Druids,

I have created a build for Apache Druid (incubating) 0.16.0, release
candidate 1.

Thanks for everyone who has helped contribute to the release! You can read
the proposed release notes here:
https://github.com/apache/incubator-druid/issues/8369

The release candidate has been tagged in GitHub as
druid-0.16.0-incubating-rc1 (8b7f0de9d2bcb95b529c24f8ab2e8e2ccc4f7a0a),
available here:
https://github.com/apache/incubator-druid/releases/tag/druid-0.16.0-incubating-rc1

The artifacts to be voted on are located here:
https://dist.apache.org/repos/dist/dev/incubator/druid/0.16.0-incubating-rc1/

Staged documentation for the release candidate is available here:
https://druid.staged.apache.org/docs/0.16.0-incubating/design/index.html

A Docker image containing the binary of the release candidate can be
retrieved via:
docker pull apache/incubator-druid:0.16.0-incubating-rc1

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/cwylie.asc

This key and the key of other committers can also be found in the project's
KEYS file here:
https://dist.apache.org/repos/dist/release/incubator/druid/KEYS

(If you are a committer, please feel free to add your own key to that file
by following the instructions in the file's header.)

Verify checksums:
diff <(shasum -a512 apache-druid-0.16.0-incubating-bin.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-0.16.0-incubating-bin.tar.gz.sha512 ; echo)

diff <(shasum -a512 apache-druid-0.16.0-incubating-src.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-0.16.0-incubating-src.tar.gz.sha512 ; echo)

Verify signatures:
gpg --verify apache-druid-0.16.0-incubating-bin.tar.gz.asc \
apache-druid-0.16.0-incubating-bin.tar.gz

gpg --verify apache-druid-0.16.0-incubating-src.tar.gz.asc \
apache-druid-0.16.0-incubating-src.tar.gz

Please review the proposed artifacts and vote. Note that Apache has
specific requirements that must be met before +1 binding votes can be cast
by PMC members. Please refer to the policy at
http://www.apache.org/legal/release-policy.html#policy for more details.

As part of the validation process, the release artifacts can be generated
from source by running:
mvn clean install -Papache-release,dist -Dgpg.skip

The RAT license check can be run from source by:
mvn apache-rat:check -Prat

This vote will be open for at least 72 hours. The vote will pass if a
majority of at least three +1 PMC votes are cast.

Once the vote has passed, the second stage vote will be called on the
Apache Incubator mailing list to get approval from the Incubator PMC.

[ ] +1  Release this package as Apache Druid (incubating) 0.16.0
[ ]   0  I don't feel strongly about it, but I'm okay with the release
[ ]  -1  Do not release this package because...

Thanks!

Apache Druid (incubating) is an effort undergoing incubation at The Apache
Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is
required of all newly accepted projects until a further review indicates
that the infrastructure, communications, and decision making process have
stabilized in a manner consistent with other successful ASF projects. While
incubation status is not necessarily a reflection of the completeness or
stability of the code, it does indicate that the project has yet to be
fully endorsed by the ASF.

Re: Druid website stuff

[Clint Wylie]

Staging site is live at https://druid.staged.apache.org/ (had to rebase
asf-staging on asf-site after merging that PR)

On Tue, Sep 10, 2019 at 4:32 PM Clint Wylie  wrote:

> I have druid-0.16.0-incubating-rc1 docs built to the asf-staging branch
> and opened PR https://github.com/apache/incubator-druid-website/pull/34 to
> add the .asf.yaml, so if I understand correctly it should be available at
> druid.staged.apache.org after we merge that.
>
> On Tue, Sep 10, 2019 at 4:20 PM Jonathan Wei  wrote:
>
>> +1, that sounds useful to me.
>>
>> On Thu, Sep 5, 2019 at 2:23 PM Himanshu  wrote:
>>
>> > +1 , can't think of any reason to not do it, thanks.
>> >
>> > On Thu, Sep 5, 2019 at 1:15 PM Clint Wylie  wrote:
>> >
>> > > Hey everyone,
>> > >
>> > > I saw the email last night to committers that announced the
>> '.asf.yaml'
>> > > feature,
>> > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/INFRA/.asf.yaml+features+for+git+repositories
>> > > ,
>> > > and was thinking it would be nice to get set up for
>> > > https://github.com/apache/incubator-druid-website, so we can have
>> > preview
>> > > documentation for the 0.16 available during the release candidate
>> voting
>> > > phase. Thoughts? If everyone is on board I will look into making it
>> > happen.
>> > >
>> > > Clint
>> > >
>> >
>>
>

Re: Druid website stuff

[Clint Wylie]

I have druid-0.16.0-incubating-rc1 docs built to the asf-staging branch and
opened PR https://github.com/apache/incubator-druid-website/pull/34 to add
the .asf.yaml, so if I understand correctly it should be available at
druid.staged.apache.org after we merge that.

On Tue, Sep 10, 2019 at 4:20 PM Jonathan Wei  wrote:

> +1, that sounds useful to me.
>
> On Thu, Sep 5, 2019 at 2:23 PM Himanshu  wrote:
>
> > +1 , can't think of any reason to not do it, thanks.
> >
> > On Thu, Sep 5, 2019 at 1:15 PM Clint Wylie  wrote:
> >
> > > Hey everyone,
> > >
> > > I saw the email last night to committers that announced the '.asf.yaml'
> > > feature,
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/INFRA/.asf.yaml+features+for+git+repositories
> > > ,
> > > and was thinking it would be nice to get set up for
> > > https://github.com/apache/incubator-druid-website, so we can have
> > preview
> > > documentation for the 0.16 available during the release candidate
> voting
> > > phase. Thoughts? If everyone is on board I will look into making it
> > happen.
> > >
> > > Clint
> > >
> >
>

Druid website stuff

[Clint Wylie]

Hey everyone,

I saw the email last night to committers that announced the '.asf.yaml'
feature,
https://cwiki.apache.org/confluence/display/INFRA/.asf.yaml+features+for+git+repositories,
and was thinking it would be nice to get set up for
https://github.com/apache/incubator-druid-website, so we can have preview
documentation for the 0.16 available during the release candidate voting
phase. Thoughts? If everyone is on board I will look into making it happen.

Clint

Re: Discuss Java 11 support

[Clint Wylie]

I tried testing hadoop with java 11 using the quickstart tutorial, but ran
into the following issue on peon before it even got to hadoop


2019-08-28T22:14:56,959 ERROR [task-runner-0-priority-0]
org.apache.druid.indexing.common.task.HadoopIndexTask - Got invocation
target exception in run(), cause:
java.lang.NoClassDefFoundError: javax/script/ScriptEngineManager
at
org.apache.logging.log4j.core.script.ScriptManager.(ScriptManager.java:49)
~[log4j-core-2.5.jar:2.5]
at
org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:179)
~[log4j-core-2.5.jar:2.5]
at
org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:209)
~[log4j-core-2.5.jar:2.5]
at
org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:492)
~[log4j-core-2.5.jar:2.5]
at
org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:562)
~[log4j-core-2.5.jar:2.5]
at
org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:578)
~[log4j-core-2.5.jar:2.5]
at
org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:214)
~[log4j-core-2.5.jar:2.5]
at
org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:145)
~[log4j-core-2.5.jar:2.5]
at
org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:41)
~[log4j-core-2.5.jar:2.5]
at org.apache.logging.log4j.LogManager.getContext(LogManager.java:182)
~[log4j-api-2.5.jar:2.5]
at
org.apache.logging.log4j.spi.AbstractLoggerAdapter.getContext(AbstractLoggerAdapter.java:103)
~[log4j-api-2.5.jar:2.5]
at
org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:43)
~[log4j-slf4j-impl-2.5.jar:2.5]
at
org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:42)
~[log4j-api-2.5.jar:2.5]
at
org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:29)
~[log4j-slf4j-impl-2.5.jar:2.5]
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:358)
~[slf4j-api-1.7.25.jar:1.7.25]
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
~[slf4j-api-1.7.25.jar:1.7.25]
at org.apache.druid.java.util.common.logger.Logger.(Logger.java:38)
[druid-core-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT]
at
org.apache.druid.guice.PropertiesModule.(PropertiesModule.java:42)
~[druid-processing-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT]
at
org.apache.druid.guice.GuiceInjectors.makeDefaultStartupModules(GuiceInjectors.java:39)
~[druid-processing-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT]
at
org.apache.druid.guice.GuiceInjectors.makeStartupInjector(GuiceInjectors.java:56)
~[druid-processing-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT]
at
org.apache.druid.indexer.HadoopDruidIndexerConfig.(HadoopDruidIndexerConfig.java:103)
~[druid-indexing-hadoop-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT]
at
org.apache.druid.indexing.common.task.HadoopIndexTask$HadoopDetermineConfigInnerProcessingRunner.runTask(HadoopIndexTask.java:644)
~[druid-indexing-service-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) ~[?:?]
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:?]
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:?]
at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at
org.apache.druid.indexing.common.task.HadoopIndexTask.runInternal(HadoopIndexTask.java:353)
~[druid-indexing-service-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT]
at
org.apache.druid.indexing.common.task.HadoopIndexTask.runTask(HadoopIndexTask.java:287)
[druid-indexing-service-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT]
at
org.apache.druid.indexing.common.task.AbstractBatchIndexTask.run(AbstractBatchIndexTask.java:137)
[druid-indexing-service-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT]
at
org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:419)
[druid-indexing-service-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT]
at
org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:391)
[druid-indexing-service-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[?:?]
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[?:?]
at java.base/java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.lang.ClassNotFoundException:
javax.script.ScriptEngineManager
at java.base/java.net.URLClassLoader.findClass

Re: 0.16.0-incubating

[Clint Wylie]

I've created the 0.16.0-incubating branch and will try to get a release
candidate built and up for testing/voting as soon as possible.


On Fri, Aug 23, 2019 at 4:09 AM Dylan Wylie  wrote:

> Hey all,
>
> This causes (what seems to be) a key JAR to be missing from the build -
> https://github.com/apache/incubator-druid/issues/8338
>
> It could well be something weird in our setup & since we're not seeing
> anyone else reporting it, that seems likely. Anyone able to triage the
> issue & my suggested fix?
>
> Cheers,
> Dylan
>
>
> On Thu, 22 Aug 2019 at 21:05, Clint Wylie  wrote:
>
> > Hi everyone,
> >
> > Just a heads up that I'm planning on branching 0.16.0-incubating off of
> > master at the end of the week, is there anything that *needs* to be
> > included that is not already part of the milestone so we can get it added
> > and prioritize review?
> >
> > Thanks,
> > Clint
> >
>

Re: Discuss Java 11 support

[Clint Wylie]

> I ran through the quickstart and now everything is starting up, but I ran
> into some issues with JAXB even after [
> https://github.com/apache/incubator-druid/pull/7604] went in (see stack
> trace at end - Google search seems to indicate we also need to include
> org.glassfish.jaxb:jaxb-runtime).

Confirmed that adding jaxb-runtime fixes this issue, raised
https://github.com/apache/incubator-druid/pull/8409. I don't see any
unexpected exceptions at least with quick-start style native indexing after
the changes in that PR.

> I just tried to run Druid 0.15.1 against Java11, but got an exception:
> https://gist.github.com/Fokko/72b57edd47e2d2da08426bf6caca6d55

This should be fixed in latest master, was fixed very recently.

On Tue, Aug 27, 2019 at 1:00 AM Driesprong, Fokko 
wrote:

> I just tried to run Druid 0.15.1 against Java11, but got an exception:
> https://gist.github.com/Fokko/72b57edd47e2d2da08426bf6caca6d55
>
> This also killed the processes. Not sure if this is already fixed in
> master. I'll dive into it later.
>
> Cheers, Fokko
>
> Op di 27 aug. 2019 om 09:41 schreef Clint Wylie :
>
> > I just tested latest master with openjdk 12 and saw the same error in my
> > logs, however overall things appeared to be functioning. I was able to
> > bring up a cluster, index some data with native batch, and query the
> > resulting segments. Maybe we call it "experimental" and let people help
> us
> > shake out the issues? ;)
> >
> > On Mon, Aug 26, 2019 at 10:47 PM David Lim  wrote:
> >
> > > Hey Druids,
> > >
> > > Recently I've been seeing more activity around Java 11 support, both
> from
> > > the side of user inquiries and from contributors submitting patches to
> > > resolve compatibility issues. I thought it would be worthwhile to have
> a
> > > thread to identify:
> > >
> > > - the level of testing that has been done so far by community members
> > > (including of various extensions and ingestion methods); related, it
> > would
> > > be helpful to know if anyone has been running Druid in production on
> Java
> > > 11
> > > - if there are any known remaining issues that need to be addressed
> > > - in which release does the community feel comfortable declaring
> official
> > > Java 11 support
> > >
> > > I ran through the quickstart and now everything is starting up, but I
> ran
> > > into some issues with JAXB even after [
> > > https://github.com/apache/incubator-druid/pull/7604] went in (see
> stack
> > > trace at end - Google search seems to indicate we also need to include
> > > org.glassfish.jaxb:jaxb-runtime).
> > >
> > > Big thanks to @xvrl for spearheading much of the work on Java 11
> support,
> > > most of which is linked from the issue:
> > > https://github.com/apache/incubator-druid/issues/5589. Xavier, do you
> > know
> > > of any significant areas of the codebase that need to be looked at
> before
> > > we can declare Java 11 support? What about issues/restrictions around
> > > Hadoop integration?
> > >
> > > ---
> > > 2019-08-27T05:05:54,776 ERROR [main]
> > > com.sun.jersey.server.impl.wadl.WadlApplicationContextImpl -
> > Implementation
> > > of JAXB-API has not been found on module path or classpath.
> > > javax.xml.bind.JAXBException: Implementation of JAXB-API has not been
> > found
> > > on module path or classpath.
> > > at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:177)
> > > ~[jaxb-api-2.3.1.jar:2.3.0]
> > > at javax.xml.bind.ContextFinder.find(ContextFinder.java:364)
> > > ~[jaxb-api-2.3.1.jar:2.3.0]
> > > at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:508)
> > > ~[jaxb-api-2.3.1.jar:2.3.0]
> > > at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:465)
> > > ~[jaxb-api-2.3.1.jar:2.3.0]
> > > at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:366)
> > > ~[jaxb-api-2.3.1.jar:2.3.0]
> > > at
> > >
> > >
> >
> com.sun.jersey.server.impl.wadl.WadlApplicationContextImpl.(WadlApplicationContextImpl.java:107)
> > > [jersey-server-1.19.3.jar:1.19.3]
> > > ...
> > > Caused by: java.lang.ClassNotFoundException:
> > > com.sun.xml.internal.bind.v2.ContextFactory
> > > at java.base/
> > >
> > >
> >
> jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
> > > ~[?:?]
> > > at
> > >
> > >
> >
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
> > > ~[?:?]
> > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
> ~[?:?]
> > > at
> > >
> > >
> >
> javax.xml.bind.ServiceLoaderUtil.nullSafeLoadClass(ServiceLoaderUtil.java:122)
> > > ~[jaxb-api-2.3.1.jar:2.3.0]
> > > at
> > >
> >
> javax.xml.bind.ServiceLoaderUtil.safeLoadClass(ServiceLoaderUtil.java:155)
> > > ~[jaxb-api-2.3.1.jar:2.3.0]
> > > at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:174)
> > > ~[jaxb-api-2.3.1.jar:2.3.0]
> > >
> >
>

Re: Discuss Java 11 support

[Clint Wylie]

I just tested latest master with openjdk 12 and saw the same error in my
logs, however overall things appeared to be functioning. I was able to
bring up a cluster, index some data with native batch, and query the
resulting segments. Maybe we call it "experimental" and let people help us
shake out the issues? ;)

On Mon, Aug 26, 2019 at 10:47 PM David Lim  wrote:

> Hey Druids,
>
> Recently I've been seeing more activity around Java 11 support, both from
> the side of user inquiries and from contributors submitting patches to
> resolve compatibility issues. I thought it would be worthwhile to have a
> thread to identify:
>
> - the level of testing that has been done so far by community members
> (including of various extensions and ingestion methods); related, it would
> be helpful to know if anyone has been running Druid in production on Java
> 11
> - if there are any known remaining issues that need to be addressed
> - in which release does the community feel comfortable declaring official
> Java 11 support
>
> I ran through the quickstart and now everything is starting up, but I ran
> into some issues with JAXB even after [
> https://github.com/apache/incubator-druid/pull/7604] went in (see stack
> trace at end - Google search seems to indicate we also need to include
> org.glassfish.jaxb:jaxb-runtime).
>
> Big thanks to @xvrl for spearheading much of the work on Java 11 support,
> most of which is linked from the issue:
> https://github.com/apache/incubator-druid/issues/5589. Xavier, do you know
> of any significant areas of the codebase that need to be looked at before
> we can declare Java 11 support? What about issues/restrictions around
> Hadoop integration?
>
> ---
> 2019-08-27T05:05:54,776 ERROR [main]
> com.sun.jersey.server.impl.wadl.WadlApplicationContextImpl - Implementation
> of JAXB-API has not been found on module path or classpath.
> javax.xml.bind.JAXBException: Implementation of JAXB-API has not been found
> on module path or classpath.
> at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:177)
> ~[jaxb-api-2.3.1.jar:2.3.0]
> at javax.xml.bind.ContextFinder.find(ContextFinder.java:364)
> ~[jaxb-api-2.3.1.jar:2.3.0]
> at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:508)
> ~[jaxb-api-2.3.1.jar:2.3.0]
> at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:465)
> ~[jaxb-api-2.3.1.jar:2.3.0]
> at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:366)
> ~[jaxb-api-2.3.1.jar:2.3.0]
> at
>
> com.sun.jersey.server.impl.wadl.WadlApplicationContextImpl.(WadlApplicationContextImpl.java:107)
> [jersey-server-1.19.3.jar:1.19.3]
> ...
> Caused by: java.lang.ClassNotFoundException:
> com.sun.xml.internal.bind.v2.ContextFactory
> at java.base/
>
> jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
> ~[?:?]
> at
>
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
> ~[?:?]
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) ~[?:?]
> at
>
> javax.xml.bind.ServiceLoaderUtil.nullSafeLoadClass(ServiceLoaderUtil.java:122)
> ~[jaxb-api-2.3.1.jar:2.3.0]
> at
> javax.xml.bind.ServiceLoaderUtil.safeLoadClass(ServiceLoaderUtil.java:155)
> ~[jaxb-api-2.3.1.jar:2.3.0]
> at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:174)
> ~[jaxb-api-2.3.1.jar:2.3.0]
>