CVE-2024-45537: Apache Druid: Users can provide MySQL JDBC properties not on allow list

[Karan Kumar]

Severity: low

Affected versions:

- Apache Druid through 30.0.0

Description:

Apache Druid allows users with certain permissions to read data from other 
database systems using JDBC. This functionality allows trusted users to set up 
Druid lookups or run ingestion tasks. Druid also allows administrators to 
configure a list of allowed properties that users are able to provide for their 
JDBC connections. By default, this allowed properties list restricts users to 
TLS-related properties only. However, when configuration a MySQL JDBC 
connection, users can use a particularly-crafted JDBC connection string to 
provide properties that are not on this allow list.

Users without the permission to configure JDBC connections are not able to 
exploit this vulnerability.
CVE-2021-26919 describes a similar vulnerability which was partially addressed 
in Apache Druid 0.20.2.

This issue is fixed in Apache Druid 30.0.1.

References:

https://druid.apache.org
https://www.cve.org/CVERecord?id=CVE-2024-45537


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

CVE-2024-45384: Apache Druid: Padding oracle in druid-pac4j extension that allows an attacker to manipulate a pac4j session cookie via Padding Oracle Attack

[Karan Kumar]

Severity: low

Affected versions:

- Apache Druid 0.18.0 through 30.0.0

Description:

Padding Oracle vulnerability in Apache Druid extension, druid-pac4j.
This could allow an attacker to manipulate a pac4j session cookie.

This issue affects Apache Druid versions 0.18.0 through 30.0.0.
Since the druid-pac4j extension is optional and disabled by default, Druid 
installations not using the druid-pac4j extension are not affected by this 
vulnerability.

While we are not aware of a way to meaningfully exploit this flaw, we 
nevertheless recommend upgrading to version 30.0.1 or higher which fixes the 
issue
and ensuring you have a strong 
druid.auth.pac4j.cookiePassphrase as a precaution.

Credit:

mr-n30 (reporter)

References:

https://druid.apache.org
https://www.cve.org/CVERecord?id=CVE-2024-45384


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

[ANNOUNCE] Apache Druid 30.0.1 release

[Karan Kumar]

The Apache Druid team is proud to announce the release of Apache Druid
30.0.1.
Druid is a high performance analytics data store for event-driven data.

Apache Druid 30.0.1 contains security, doc and task monitor fixes.

Source and binary distributions can be downloaded from:
https://druid.apache.org/downloads.html

Release notes are at:https://github.com/apache/druid/releases/tag/druid-30.0.1

Re: [VOTE] Release Apache Druid 30.0.0 [RC3]

[Karan Kumar]

+1 (binding)

src package:
* verified checksum and signature
* built binary distribution
* ran MSQ ingestion and demo MSQ queries on the dataset (using the built
binary)

binary package:
* verified checksum and signature
* checked for LICENSE and NOTICE
* ran MSQ ingestion and demo MSQ queries on the dataset

docker:
* verified checksum




On Wed, Jun 12, 2024 at 4:06 PM Amatya Avadhanula  wrote:

> +1 (binding)
>
> src package:
> * verified checksum and signature
> * checked for LICENSE and NOTICE
> * rat check successful
> * built binary distribution
> * Ran Kafka ingestion
> * Ran Compaction and MSQ replace ingestion concurrently with the Kafka
> ingestion using the experimental concurrent locks
> * Verified results using simple queries
>
> binary package:
> * verified checksum and signature
> * checked for LICENSE and NOTICE
> * Ran Kafka ingestion
> * Ran Compaction and MSQ replace ingestion concurrently with the Kafka
> ingestion using the experimental concurrent locks
> * Verified results using simple queries
>
> docker:
> * verified checksum
> * ran druid cluster from the docker image
> * Ran Kafka ingestion
>
> On 2024/06/08 10:27:47 Adarsh Sanjeev wrote:
> > Hi all,
> >
> > I have created a build for Apache Druid 30.0.0, release candidate 3.
> >
> > Thanks to everyone who has helped contribute to the release! You can read
> > the proposed release notes here:
> > https://github.com/apache/druid/issues/16505
> >
> > The release candidate has been tagged in GitHub as druid-30.0.0-rc3
> > (09d36ee324747f1407705c27618b6d415c3fa8a9), available here:
> > https://github.com/apache/druid/releases/tag/druid-30.0.0-rc3
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/druid/30.0.0-rc3/
> >
> > A staged Maven repository is available for review at:
> > https://repository.apache.org/content/repositories/orgapachedruid-1066/
> >
> > Staged druid.apache.org website documentation is available here:
> > https://druid.staged.apache.org/docs/30.0.0/design/index.html
> >
> > A Docker image containing the binary of the release candidate can be
> > retrieved via:
> > docker pull apache/druid:30.0.0-rc3
> >
> > artifact checksums
> > src:
> >
> 87e145684a646667811ab7db9f812b13bf78a4bafb23d407853f9dd3fbbbd0f523a1a0d12aeed67543f9e1e462c41c7cde6f7420319bffe5f564119464d83e09
> > bin:
> >
> fe50f134e4c2bba2bf04e0f5ddba1886af1c7ed8a6b4e361347efc853ef50aba8e35b664af0531cdabce69c6063fff0913d2165ae160614730912698f35aee83
> > docker: 978d9380fb6fcab79cafc6c53f11edbb6e80a664481e843d6fe069b1fd05f129
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/adarshsanjeev.asc
> >
> > This key and the key of other committers can also be found in the
> project's
> > KEYS file here:
> > https://dist.apache.org/repos/dist/release/druid/KEYS
> >
> > (If you are a committer, please feel free to add your own key to that
> file
> > by following the instructions in the file's header.)
> >
> >
> > Verify checksums:
> > diff <(shasum -a512 apache-druid-30.0.0-src.tar.gz | cut -d ' ' -f1)
> <(cat
> > apache-druid-30.0.0-src.tar.gz.sha512 ; echo)
> > diff <(shasum -a512 apache-druid-30.0.0-bin.tar.gz | cut -d ' ' -f1)
> <(cat
> > apache-druid-30.0.0-bin.tar.gz.sha512 ; echo)
> >
> > Verify signatures:
> > gpg --verify apache-druid-30.0.0-src.tar.gz.asc
> > apache-druid-30.0.0-src.tar.gz
> > gpg --verify apache-druid-30.0.0-bin.tar.gz.asc
> > apache-druid-30.0.0-bin.tar.gz
> >
> > Please review the proposed artifacts and vote. Note that Apache has
> > specific requirements that must be met before +1 binding votes can be
> cast
> > by PMC members. Please refer to the policy at
> > http://www.apache.org/legal/release-policy.html#policy for more details.
> >
> > As part of the validation process, the release artifacts can be generated
> > from source by running:
> > mvn clean install -Papache-release,dist -Dgpg.skip
> >
> > The RAT license check can be run from source by:
> > mvn apache-rat:check -Prat
> >
> > This vote will be open for at least 72 hours. The vote will pass if a
> > majority of at least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Druid 30.0.0
> > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Thanks!
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> For additional commands, e-mail: dev-h...@druid.apache.org
>
>

Re: [VOTE] Release Apache Druid 30.0.0 [RC1]

[Karan Kumar]

While testing for aure support I ran into :
https://github.com/apache/druid/pull/16525.
I feel it is a RC blocker since Azure support is broken without the above
patch..
Therefore -1 for my side.

On Wed, May 29, 2024 at 10:01 AM Adarsh Sanjeev 
wrote:

> Hi all,
>
> I have created a build for Apache Druid 30.0.0, release candidate 1.
>
> Thanks to everyone who has helped contribute to the release! You can read
> the proposed release notes here:
> https://github.com/apache/druid/issues/16505
>
> The release candidate has been tagged in GitHub as druid-30.0.0-rc1
> (2a3a0052067d217ff0a7d3358f1ced33a24e935a), available here:
> https://github.com/apache/druid/releases/tag/druid-30.0.0-rc1
>
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/druid/30.0.0-rc1/
>
> A staged Maven repository is available for review at:
> https://repository.apache.org/content/repositories/orgapachedruid-1063
>
> Staged druid.apache.org website documentation is available here:
> https://druid.staged.apache.org/docs/30.0.0/design/index.html
>
> A Docker image containing the binary of the release candidate can be
> retrieved via:
> docker pull apache/druid:30.0.0-rc1
>
> artifact checksums
> src:
>
> 8fe404dabf6d9f50a5d2144e0850e0fd5fe3a74eddc30032dbc29a751fab5dba3cb65bdd3a472afaaa5e04ce4f625a8ae7f173663feb5e11c65268b5e7171d1a
> bin:
>
> bae6e9fe3359829227f249b86b62e8fe79dcfb29cdb9e88692190ab7972dbfc1bd206a010dbc4ba3dbd517f3e24517ffb19c90aabd632c010e62458d05cdb683
> docker: 288979c8de866fefef404d2ac03403f11d4272e78291cf8133ec1fb004d3eb2b
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/adarshsanjeev.asc
>
> This key and the key of other committers can also be found in the project's
> KEYS file here:
> https://dist.apache.org/repos/dist/release/druid/KEYS
>
> (If you are a committer, please feel free to add your own key to that file
> by following the instructions in the file's header.)
>
>
> Verify checksums:
> diff <(shasum -a512 apache-druid-30.0.0-src.tar.gz | cut -d ' ' -f1) <(cat
> apache-druid-30.0.0-src.tar.gz.sha512 ; echo)
> diff <(shasum -a512 apache-druid-30.0.0-bin.tar.gz | cut -d ' ' -f1) <(cat
> apache-druid-30.0.0-bin.tar.gz.sha512 ; echo)
>
> Verify signatures:
> gpg --verify apache-druid-30.0.0-src.tar.gz.asc
> apache-druid-30.0.0-src.tar.gz
> gpg --verify apache-druid-30.0.0-bin.tar.gz.asc
> apache-druid-30.0.0-bin.tar.gz
>
> Please review the proposed artifacts and vote. Note that Apache has
> specific requirements that must be met before +1 binding votes can be cast
> by PMC members. Please refer to the policy at
> http://www.apache.org/legal/release-policy.html#policy for more details.
>
> As part of the validation process, the release artifacts can be generated
> from source by running:
> mvn clean install -Papache-release,dist -Dgpg.skip
>
> The RAT license check can be run from source by:
> mvn apache-rat:check -Prat
>
> This vote will be open for at least 72 hours. The vote will pass if a
> majority of at least three +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Druid 0.17.0
> [ ] 0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
> Thanks!
> Adarsh Sanjeev
>


-- 
Thanks
Karan

[ANNOUNCE] Apache Druid 29.0.1 release

[Karan Kumar]

Hi All

The Apache Druid team is proud to announce the release of Apache Druid
29.0.1 .
Druid is a high performance analytics data store for event-driven data.

Apache Druid 29.0.1 is a patch release on top of druid 29.0.0, that
contains the following patches:
Bug fixes

   - Added type verification for INSERT and REPLACE to validate that
   strings and string arrays aren't mixed #15920
   
   - Concurrent replace
   
   now allows pending Peon segments to be upgraded using the Supervisor
   #15995 
   - Changed the targetDataSource attribute to return a string containing
   the name of the datasource. This reverts the breaking change introduced in
   Druid 29.0.0 for INSERT and REPLACE MSQ queries #16004
    #16031
   
   - Decreased the size of the distribution Docker image #15968
   
   - Fixed an issue with SQL-based ingestion where string inputs, such as
   from CSV, TSV, or string-value fields in JSON, are ingested as null values
   when they are typed as LONG or BIGINT #15999
   
   - Fixed an issue where a web console-generated Kafka supervisor spec has
   flattenSpec in the wrong location #15946
   
   - Fixed an issue with filters on expression virtual column indexes
   incorrectly considering values null in some cases for expressions which
   translate null values into not null values #15959
   
   - Fixed an issue where the data loader crashes if the incoming data
   can't be parsed #15983 
   - Improved DOUBLE type detection in the web console #15998
   
   - Web console-generated queries now only set the context parameter
   arrayIngestMode to array when you explicitly opt in to use arrays #15927
   
   - The web console now displays the results of an MSQ query that writes
   to an external destination through the EXTERN
   

   function #15969 

Incompatible changes Changes to targetDataSource in EXPLAIN queries

Druid 29.0.1 includes a breaking change that restores the behavior for
targetDataSource to its 28.0.0 and earlier state, different from Druid
29.0.0 and only 29.0.0. In 29.0.0, targetDataSource returns a JSON object
that includes the datasource name. In all other versions, targetDataSource
returns a string containing the name of the datasource.

If you're upgrading from any version other than 29.0.0, there is no change
in behavior.

If you are upgrading from 29.0.0, this is an incompatible change. #16004

Dependency updates

   - Updated PostgreSQL JDBC Driver version to 42.7.2 #15931
   


Source and binary distributions can be downloaded from:
https://druid.apache.org/downloads.html

All the release notes are present at:
https://github.com/apache/druid/releases/tag/druid-29.0.1
A big thank you to all the contributors in this milestone release!

[RESULT][VOTE] Release Apache Druid 29.0.1 [RC1]

[Karan Kumar]

Thanks to everyone who participated in the vote! The results are as follows:

Karan Kumarl: +1 (binding)
Laksh Singla: +1 (binding)
Amatya Avadhanula: +1 (binding)

The vote has passed with 3 binding +1's.

Re: [VOTE] Release Apache Druid 29.0.1 [RC1]

[Karan Kumar]

+1(binding)
|src package:
* verified checksum and signature
* checked for LICENSE and NOTICE
* rat check passing
* built binary distribution
* Ran MSQ demo Q's

binary package:
* verified checksum and signature
* checked for LICENSE and NOTICE
* ran MSQ demo q's

On Fri, Mar 29, 2024 at 12:51 PM Amatya Avadhanula 
wrote:

> +1 (binding)
>
> src package:
> * verified checksum and signature
> * checked for LICENSE and NOTICE
> * rat check passing
> * built binary distribution
> * ran Kafka streaming ingestion with concurrent MSQ replace, and verified
> queries and data
>
> binary package:
> * verified checksum and signature
> * checked for LICENSE and NOTICE
> * ran Kafka streaming ingestion with concurrent MSQ replace, and verified
> queries and data
>
> docker:
> * verified checksum
>
> On 2024/03/21 12:14:50 Karan Kumar wrote:
> > Hi all,
> >
> > I have created a build for Apache Druid 29.0.1, release
> > candidate 1.
> >
> > Thanks to everyone who has helped contribute to the release! You can read
> > the proposed release notes here:
> > https://github.com/apache/druid/issues/16183
> >
> > The release candidate has been tagged in GitHub as
> > druid-29.0.1-rc1 (15b3efd085d3e69417a00d06b04e325fd26e729a),
> > available here:
> > https://github.com/apache/druid/releases/tag/druid-29.0.1-rc1
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/druid/29.0.1-rc1/
> >
> > A staged Maven repository is available for review at:
> > https://repository.apache.org/content/repositories/orgapachedruid-1062/
> >
> > Staged druid.apache.org website documentation is available here:
> > https://druid.staged.apache.org/docs/29.0.1/design/index.html
> >
> > A Docker image containing the binary of the release candidate can be
> > retrieved via:
> > *docker pull apache/druid:29.0.1-rc1*
> >
> > artifact checksums
> > src:
> >
> 7afa7d767ecb141b4151c0b2682e6bb77071d30365fb1091b1ea800807b5a00acb5605768e5d77aebced30e09e4f50f6a9237ff8706945d9218af27694d7288b
> > bin:
> >
> 5e8dfea56e33bf4c1f2e82fa4a8478a3fdfb9205cd88e3f2fc58beaf90f2493fe6a0b42b15bacc11e97809da2e439008fcf84d8998c0dea689055c00e98b
> > docker: bbffcf53e8d999c880fbbfde64b35785e99cb094e4d5e3e69d423f04a49abb91
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/karan.asc
> >
> > This key and the key of other committers can also be found in the
> project's
> > KEYS file here:
> > https://dist.apache.org/repos/dist/release/druid/KEYS
> >
> > (If you are a committer, please feel free to add your own key to that
> file
> > by following the instructions in the file's header.)
> >
> >
> > Verify checksums:
> >
> >
> >
> >
> >
> >
> > *diff <(shasum -a512 apache-druid-29.0.1-src.tar.gz | \cut -d ' ' -f1)
> > \<(cat apache-druid-29.0.1-src.tar.gz.sha512 ; echo)diff <(shasum -a512
> > apache-druid-29.0.1-bin.tar.gz | \cut -d ' ' -f1) \<(cat
> > apache-druid-29.0.1-bin.tar.gz.sha512 ; echo)*
> >
> > Verify signatures:
> >
> > *gpg --verify apache-druid-29.0.1-src.tar.gz.asc
> > \apache-druid-29.0.1-src.tar.gz*
> >
> >
> > *gpg --verify apache-druid-29.0.1-bin.tar.gz.asc
> > \apache-druid-29.0.1-bin.tar.gz*
> >
> > Please review the proposed artifacts and vote. Note that Apache has
> > specific requirements that must be met before +1 binding votes can be
> cast
> > by PMC members. Please refer to the policy at
> > http://www.apache.org/legal/release-policy.html#policy for more details.
> >
> > As part of the validation process, the release artifacts can be generated
> > from source by running:
> > *mvn clean install -Papache-release,dist -Dgpg.skip*
> >
> > The RAT license check can be run from source by:
> > *mvn apache-rat:check -Prat*
> >
> > This vote will be open for at least *72 hours*. The vote will pass if a
> > majority of at least *three +1 PMC* votes are cast.
> >
> > [ ] +1 Release this package as Apache Druid 29.0.1
> > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> >
> > Thanks
> > Karan
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> For additional commands, e-mail: dev-h...@druid.apache.org
>
>

Re: [VOTE] Release Apache Druid 29.0.0 [RC1]

[Karan Kumar]

Hi
Vote thread for druid 29.0.1 is up.
https://lists.apache.org/thread/6syof9fmnb7vbyjrpowrt7s73rh2bqy4

Thanks
Karan

On Wed, Feb 21, 2024 at 1:10 AM Vadim Ogievetsky 
wrote:

> That sounds great to me. Thank you +1
>
> On 2024/02/20 18:58:35 Laksh Singla wrote:
> > Since the artifacts were already out at the time of the Vad's vote, I
> will
> > be going ahead with the release of Druid 29.0.0. However, I do echo with
> > the comments mentioned in the mail, therefore we can take the following
> > steps to prevent misfortune for the unexpecting users:
> >
> > For the current Druid 29.0.0 release:
> > 1. The docs (update, release, and general docs) will be updated with the
> > possible pitfalls arising out of console injecting the "arrayIngestMode"
> > automatically, and persisting it for the given tab.
> >
> > Further, we can have a patch release, 29.0.1, with the following patches
> > mentioned in the Vad's mail, that would have been a part of RC2, had the
> > vote not passed through at the time:
> > 1. https://github.com/apache/druid/pull/15927
> > 2. https://github.com/apache/druid/pull/15920
> >
> > This will address the concerns regarding the 29.0.0 release, and also
> give
> > enough time for the PRs to get reviewed without being hastened by the
> > release timeline. I'll volunteer to be the release manager for the same.
> > Thoughts?
> >
> > Regards
> > Laksh
> >
> >
> >
> > On Tue, 20 Feb 2024 at 10:50, Abhishek Agarwal 
> wrote:
> >
> > > Vad, The vote was closed by Laksh, given that 72 hours had passed and
> > > artifacts had been released. Given we had the three binding +1s, we can
> > > proceed as usual. I suggest that we call out the caveat clearly in the
> > > release notes section.
> > >
> > > On Mon, Feb 19, 2024 at 11:53 PM Vadim Ogievetsky <
> vogievet...@apache.org>
> > > wrote:
> > >
> > > > Thank you for all the work getting this release to this point. As the
> > > > author of https://github.com/apache/druid/pull/15588 I regretfully
> vote
> > > > -1 on this release. To echo Gian's point I think the chance of some
> user
> > > > injuring a production Datasource with an innocent query pasted into
> the
> > > > wrong web console tab is just too large. The fact that the
> > > > "arrayIngestMode" setting itself is not new in this release is
> immaterial
> > > > to my consideration as I am more worried about "what might happen by
> > > > accident" than "what is technically possible to mess up".
> > > >
> > > > I have made a PR (https://github.com/apache/druid/pull/15927) that
> > > > implements Gian's suggestion of: Making it obvious in the web console
> > > > whether a tab is in "arrayIngestMode: array" or "arrayIngestMode:
> mvd" or
> > > > "server default". Furthermore this PR also minimizes the use of
> > > > "arrayIngestMode: array" to only when needed while still generating
> the
> > > > "correct" form of the SQL queries at all times. If this PR was
> merged and
> > > > back-ported I would change my vote to +1.
> > > >
> > > > I think that Gian's validation PR (
> > > > https://github.com/apache/druid/pull/15920) is an even better
> solution,
> > > > as it would protect API usage and usage via other frontends also.
> > > Obviously
> > > > we could merge both.
> > > >
> > > > Another idea that would solve this issue would be have the ability
> to set
> > > > context flags via SQL perhaps with a PARAMETER keyword. If this was
> > > > possible the console could exclusively use that for setting query
> context
> > > > parameters on generated queries.
> > > >
> > > > Vadim
> > > >
> > > > On 2024/02/17 02:52:38 Gian Merlino wrote:
> > > > > Here's a patch with the validation idea:
> > > > https://github.com/apache/druid/pull/15920
> > > > >
> > > > > It adds validation for the most problematic case (mixing strings
> and
> > > > arrays), provides a way to override the validation, and makes the
> warning
> > > > log on the controller task when arrayIngestMode is 'mvd' more
> friendly
> > > and
> > > > explanatory.
> > > > >
> > > > > Depending on which direction you're going in, the errors look like
> > > > either:
> > > > >
> > > > >   Cannot write into field[flags] using type[VARCHAR ARRAY] and
> > > > arrayIngestMode[mvd], since the existing type is[VARCHAR ARRAY]. Try
> > > > setting arrayIngestMode to[array] to retain the SQL type[VARCHAR
> ARRAY]
> > > > >
> > > > > Or:
> > > > >
> > > > >   Cannot write into field[flags] using type[VARCHAR ARRAY] and
> > > > arrayIngestMode[array], since the existing type is[VARCHAR]. Try
> wrapping
> > > > this field using ARRAY_TO_MV(...) AS "flags"
> > > > >
> > > > > The "try" language pushes people towards the behavior we'd like
> for the
> > > > future: using arrayIngestMode[array] and wrapping MVDs in
> ARRAY_TO_MV.
> > > > >
> > > > > I'm changing my vote to a plain 0, given that _most_ of the changes
> > > > related to arrayIngestMode went out in Druid 28. However I do think
> it
> > > > would be nice to get a patch like this in, given that th

Re: [DISCUSS] Apache Druid Release 29.0.1

[Karan Kumar]

Hi All
Just started a vote thread :
https://lists.apache.org/thread/6syof9fmnb7vbyjrpowrt7s73rh2bqy4

Thanks
Karan

On Mon, Feb 26, 2024 at 6:10 PM Karan Kumar  wrote:

>
> Hi folks,
>
> As discussed in Druid 29.0 RC thread :
> https://lists.apache.org/thread/t3q3288lr0ypky8bpnmrkxowl36305vo a new
> druid 29.0.1 release is neded. I volunteer to shepherd the process.
> I have marked relevant PR with the druid 29.0.1 milestone here :
> https://github.com/apache/druid/pulls?q=is%3Aopen+is%3Apr+milestone%3A29.0.1
> Please mention if there are other high priority fixes that should go into
> this patch release.
>
> Thanks
> Karan
>


-- 
Thanks
Karan

[VOTE] Release Apache Druid 29.0.1 [RC1]

[Karan Kumar]

Hi all,

I have created a build for Apache Druid 29.0.1, release
candidate 1.

Thanks to everyone who has helped contribute to the release! You can read
the proposed release notes here:
https://github.com/apache/druid/issues/16183

The release candidate has been tagged in GitHub as
druid-29.0.1-rc1 (15b3efd085d3e69417a00d06b04e325fd26e729a),
available here:
https://github.com/apache/druid/releases/tag/druid-29.0.1-rc1

The artifacts to be voted on are located here:
https://dist.apache.org/repos/dist/dev/druid/29.0.1-rc1/

A staged Maven repository is available for review at:
https://repository.apache.org/content/repositories/orgapachedruid-1062/

Staged druid.apache.org website documentation is available here:
https://druid.staged.apache.org/docs/29.0.1/design/index.html

A Docker image containing the binary of the release candidate can be
retrieved via:
*docker pull apache/druid:29.0.1-rc1*

artifact checksums
src:
7afa7d767ecb141b4151c0b2682e6bb77071d30365fb1091b1ea800807b5a00acb5605768e5d77aebced30e09e4f50f6a9237ff8706945d9218af27694d7288b
bin:
5e8dfea56e33bf4c1f2e82fa4a8478a3fdfb9205cd88e3f2fc58beaf90f2493fe6a0b42b15bacc11e97809da2e439008fcf84d8998c0dea689055c00e98b
docker: bbffcf53e8d999c880fbbfde64b35785e99cb094e4d5e3e69d423f04a49abb91

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/karan.asc

This key and the key of other committers can also be found in the project's
KEYS file here:
https://dist.apache.org/repos/dist/release/druid/KEYS

(If you are a committer, please feel free to add your own key to that file
by following the instructions in the file's header.)


Verify checksums:






*diff <(shasum -a512 apache-druid-29.0.1-src.tar.gz | \cut -d ' ' -f1)
\<(cat apache-druid-29.0.1-src.tar.gz.sha512 ; echo)diff <(shasum -a512
apache-druid-29.0.1-bin.tar.gz | \cut -d ' ' -f1) \<(cat
apache-druid-29.0.1-bin.tar.gz.sha512 ; echo)*

Verify signatures:

*gpg --verify apache-druid-29.0.1-src.tar.gz.asc
\apache-druid-29.0.1-src.tar.gz*


*gpg --verify apache-druid-29.0.1-bin.tar.gz.asc
\apache-druid-29.0.1-bin.tar.gz*

Please review the proposed artifacts and vote. Note that Apache has
specific requirements that must be met before +1 binding votes can be cast
by PMC members. Please refer to the policy at
http://www.apache.org/legal/release-policy.html#policy for more details.

As part of the validation process, the release artifacts can be generated
from source by running:
*mvn clean install -Papache-release,dist -Dgpg.skip*

The RAT license check can be run from source by:
*mvn apache-rat:check -Prat*

This vote will be open for at least *72 hours*. The vote will pass if a
majority of at least *three +1 PMC* votes are cast.

[ ] +1 Release this package as Apache Druid 29.0.1
[ ] 0 I don't feel strongly about it, but I'm okay with the release
[ ] -1 Do not release this package because...


Thanks
Karan

[DISCUSS] Apache Druid Release 29.0.1

[Karan Kumar]

Hi folks,

As discussed in Druid 29.0 RC thread :
https://lists.apache.org/thread/t3q3288lr0ypky8bpnmrkxowl36305vo a new
druid 29.0.1 release is neded. I volunteer to shepherd the process.
I have marked relevant PR with the druid 29.0.1 milestone here :
https://github.com/apache/druid/pulls?q=is%3Aopen+is%3Apr+milestone%3A29.0.1
Please mention if there are other high priority fixes that should go into
this patch release.

Thanks
Karan

Re: [VOTE] Release Apache Druid 29.0.0 [RC1]

[Karan Kumar]

+1 (binding)

src package:
- verified checksum and signature
- LICENSE and NOTICE present
- built binary distribution, ran MSQ quickstart

binary package:
- verified checksum and signature
- LICENSE and NOTICE present
- ran MSQ quickstart

docker:
- verified checksum

On Thu, Feb 15, 2024 at 5:41 AM Clint Wylie  wrote:

> +1 (binding)
>
> src package:
> - verified checksum and signature
> - LICENSE and NOTICE present
> - rat check passed
> - built binary distribution, ran MSQ ingestion, ran some queries
>
> binary package:
> - verified checksum and signature
> - LICENSE and NOTICE present
> - ran native ingestion, ran some queries
>
> docker:
> - verified checksum
> - ran docker-compose cluster, msq ingestion, some queries
>
> On Wed, Feb 14, 2024 at 7:27 AM Kashif Faraz  wrote:
> >
> > +1 (binding)
> >
> > src package:
> > - verified checksum and signature
> > - checked for LICENSE and NOTICE
> > - rat check passed
> > - built binary distribution
> > - ingested sample wikipedia dataset, ran some queries, reindexed with
> range
> > partitioning
> >
> > binary package:
> > - verified checksum and signature
> > - ingested sample wikipedia dataset, ran some queries, reindexed with
> range
> > partitioning
> >
> > docker:
> > - verified checksum
> > - ran docker-compose cluster, some queries
> >
> > On Tue, Feb 13, 2024 at 6:04 PM Laksh Singla 
> wrote:
> >
> > > Hi all,
> > >
> > > I have created a build for Apache Druid 29.0.0, release
> > > candidate 1.
> > >
> > > Thanks to everyone who has helped contribute to the release! You can
> read
> > > the proposed release notes here:
> > > https://github.com/apache/druid/issues/15896
> > >
> > > The release candidate has been tagged in GitHub as
> > > druid-29.0.0-rc1 (869bd3978f0c835ef8eb7c1f25c468e23472a81b),
> > > available here:
> > > https://github.com/apache/druid/tree/druid-29.0.0-rc1
> > >
> > > The artifacts to be voted on are located here:
> > > https://dist.apache.org/repos/dist/dev/druid/29.0.0-rc1/
> > >
> > > A staged Maven repository is available for review at:
> > >
> https://repository.apache.org/content/repositories/orgapachedruid-1061/
> > >
> > > Staged druid.apache.org website documentation is available here:
> > > https://druid.staged.apache.org/docs/29.0.0/design/
> > >
> > > A Docker image containing the binary of the release candidate can be
> > > retrieved via:
> > > docker pull apache/druid:29.0.0-rc1
> > >
> > > artifact checksums
> > > src:
> > >
> > >
> 1948fab4500f3571591f887a638631b6a05f040b88b35004406ca852e16884e5d2269a74e8d07c79c477d8b02a2489efec85776f8ef46f4e8defedce4efb9931
> > > bin:
> > >
> > >
> f2a11ddc71b59a648d01d7c220a6fae527c34d702d4f1e7aed954803a3576543c4cd7149f34e8948eccf89240435f09a8512db83b46991ac6e354eeba8cbada4
> > > docker:
> 2bddcf692f2137dc4094908b63d2043423fca895ba0f479f0db34ec8016c4472
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/lakshsingla.asc
> > >
> > > This key and the key of other committers can also be found in the
> project's
> > > KEYS file here:
> > > https://dist.apache.org/repos/dist/release/druid/KEYS
> > >
> > > (If you are a committer, please feel free to add your own key to that
> file
> > > by following the instructions in the file's header.)
> > >
> > >
> > > Verify checksums:
> > > diff <(shasum -a512 apache-druid-29.0.0-src.tar.gz | \
> > > cut -d ' ' -f1) \
> > > <(cat apache-druid-29.0.0-src.tar.gz.sha512 ; echo)
> > >
> > > diff <(shasum -a512 apache-druid-29.0.0-bin.tar.gz | \
> > > cut -d ' ' -f1) \
> > > <(cat apache-druid-29.0.0-bin.tar.gz.sha512 ; echo)
> > >
> > > Verify signatures:
> > > gpg --verify apache-druid-29.0.0-src.tar.gz.asc \
> > > apache-druid-29.0.0-src.tar.gz
> > >
> > > gpg --verify apache-druid-29.0.0-bin.tar.gz.asc \
> > > apache-druid-29.0.0-bin.tar.gz
> > >
> > > Please review the proposed artifacts and vote. Note that Apache has
> > > specific requirements that must be met before +1 binding votes can be
> cast
> > > by PMC members. Please refer to the policy at
> > > http://www.apache.org/legal/release-policy.html#policy for more
> details.
> > >
> > > As part of the validation process, the release artifacts can be
> generated
> > > from source by running:
> > > mvn clean install -Papache-release,dist -Dgpg.skip
> > >
> > > The RAT license check can be run from source by:
> > > mvn apache-rat:check -Prat
> > >
> > > This vote will be open for at least 72 hours. The vote will pass if a
> > > majority of at least three +1 PMC votes are cast.
> > >
> > > [ ] +1 Release this package as Apache Druid 29.0.0
> > > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > > [ ] -1 Do not release this package because...
> > >
> > > Thanks!
> > >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> For additional commands, e-mail: dev-h...@druid.apache.org
>
>

-- 
Thanks
Karan

[Discuss] Druid 29.0.0 release

[Karan Kumar]

Hello all,

I am starting a discussion for the Druid 29.0.0 release. I am volunteering
to be the release manager for the same. The branch for Druid 28.0.0 was cut
on 9th October 2023, and since we do quarterly releases, we should cut the
branch for Druid 29.0.0 on 15th January 2024 (Monday). Please use this
thread to discuss the important issues or features that are in flight, that
should be a part of the Druid 29.0.0 release.

Thanks
Karan

Re: [VOTE] Release Apache Druid 28.0.1 [RC1]

[Karan Kumar]

+1 (Binding)
src package:
- verified checksum and signature
- check for LICENSE and NOTICE
- built binary distribution, ran MSQ quickstart, ran some queries,
tested rolling overlord patch for new TaskActions.

binary package:
- verified checksum and signature
- check for LICENSE and NOTICE
- ran native quickstart, ran MSQ quickstart




On Fri, Dec 15, 2023 at 11:06 AM Kashif Faraz  wrote:

> +1 (binding)
>
> src package:
> - verified checksum and signature
> - checked for LICENSE and NOTICE
> - rat check passed
> - built binary distribution
> - ingested sample wikipedia dataset, ran some queries,
> reindexed into a different datasource
>
> binary package:
> - verified checksum and signature
> - ingested sample wikipedia dataset, ran some queries,
> reindexed into a different datasource
>
> docker:
> - verified checksum
> - ran docker-compose cluster, some queries
>
>
> On Thu, Dec 14, 2023 at 11:29 PM Clint Wylie  wrote:
>
> > +1 (binding)
> >
> > src package:
> > - verified checksum and signature
> > - check for LICENSE and NOTICE
> > - rat check passed
> > - built binary distribution, ran MSQ quickstart, ran some queries,
> > tested result cache issue
> >
> > binary package:
> > - verified checksum and signature
> > - check for LICENSE and NOTICE
> > - ran native quickstart, ran some queries, tested result cache issue,
> > tested null filter issue
> >
> > docker:
> > - verified checksum
> > - ran docker-compose cluster, some queries, tested result cache issue
> >
> > On Sun, Dec 10, 2023 at 9:46 PM Laksh Singla 
> > wrote:
> > >
> > > Hi all,
> > >
> > > I have created a build for Apache Druid 28.0.1, release
> > > candidate 1.
> > >
> > > Thanks to everyone who has helped contribute to the release! You can
> read
> > > the proposed release notes here:
> > > https://github.com/apache/druid/issues/15535
> > >
> > > The release candidate has been tagged in GitHub as
> > > druid-28.0.1-rc1 (266d5a5f1d76f6d33e2d1d2fa52156dfb3cd87a7),
> > > available here:
> > > https://github.com/apache/druid/releases/tag/druid-28.0.1-rc1
> > >
> > > The artifacts to be voted on are located here:
> > > https://dist.apache.org/repos/dist/dev/druid/28.0.1-rc1/
> > >
> > > A staged Maven repository is available for review at:
> > >
> https://repository.apache.org/content/repositories/orgapachedruid-1057/
> > >
> > > Staged druid.apache.org website documentation is available here:
> > > https://druid.staged.apache.org/docs/28.0.0/design/index.html
> > > Release and upgrade notes will be added to the site as well, however
> the
> > > patch isn't merged yet, therefore the site isn't updated with the new
> > paths
> > > and is the same as the 28.0.0 staged site. The staged website along
> with
> > > the release website will be updated with the correct version numbers
> and
> > > path along with the release.
> > >
> > > A Docker image containing the binary of the release candidate can be
> > > retrieved via:
> > > docker pull apache/druid:28.0.1-rc1
> > >
> > > artifact checksums
> > > src:
> > >
> >
> 557ca5131c72625f5e63b40ad83a6cce27c4209624a95e6be326fe9c5bac01cbafe8bba14bd46e058f237ae8b5d1c2129cd9911ce31476487d36a3b7c988b626
> > > bin:
> > >
> >
> 923336b96fa9e72bb0a2799158e232f0b336841c60561f0da5e2fdcbc22f592f1d847118fa79ddfab1b02f6626a2d5c7b7f54f11eec1f8187c2e66918f3e86f3
> > > docker:
> > > aebb7e18883f47df4f272e36601d9daa76e91fc3a89f781ca93d2d556cc8cd1d
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/lakshsingla.asc
> > >
> > > This key and the key of other committers can also be found in the
> > project's
> > > KEYS file here:
> > > https://dist.apache.org/repos/dist/release/druid/KEYS
> > >
> > > (If you are a committer, please feel free to add your own key to that
> > file
> > > by following the instructions in the file's header.)
> > >
> > >
> > > Verify checksums:
> > > diff <(shasum -a512 apache-druid-28.0.1-src.tar.gz | \
> > > cut -d ' ' -f1) \
> > > <(cat apache-druid-28.0.1-src.tar.gz.sha512 ; echo)
> > >
> > > diff <(shasum -a512 apache-druid-28.0.1-bin.tar.gz | \
> > > cut -d ' ' -f1) \
> > > <(cat apache-druid-28.0.1-bin.tar.gz.sha512 ; echo)
> > >
> > > Verify signatures:
> > > gpg --verify apache-druid-28.0.1-src.tar.gz.asc \
> > > apache-druid-28.0.1-src.tar.gz
> > >
> > > gpg --verify apache-druid-28.0.1-bin.tar.gz.asc \
> > > apache-druid-28.0.1-bin.tar.gz
> > >
> > > Please review the proposed artifacts and vote. Note that Apache has
> > > specific requirements that must be met before +1 binding votes can be
> > cast
> > > by PMC members. Please refer to the policy at
> > > http://www.apache.org/legal/release-policy.html#policy for more
> details.
> > >
> > > As part of the validation process, the release artifacts can be
> generated
> > > from source by running:
> > > mvn clean install -Papache-release,dist -Dgpg.skip
> > >
> > > The RAT license check can be run from source by:
> > > mvn apache-rat:check -Prat
> > >
> > > This vote will be o

Re: [VOTE] Release Apache Druid 28.0.0 [RC1]

[Karan Kumar]

+ 1 Binding

src package:
- verified signature/checksum
- LICENSE/NOTICE present
- built binary distribution,
 - Loaded all sample MSQ queries
queries
 - Tested query from deep storage

binary package:
- verified signature/checksum
- LICENSE/NOTICE present
- ran single server quick start
 - Loaded all sample MSQ queries
queries
 - Tested query from deep storage

docker:
- verified checksum








On Fri, Nov 10, 2023 at 1:30 PM Kashif Faraz  wrote:

> +1 Binding
>
> Verified artifacts:
>
> src:
> - Verified signature/checksum
> - LICENSE/NOTICE present
> - Built binary distribution
> - Ingested sample "wikipedia" dataset using native batch
> - Ran Apache Rat license check successfully
>
> binary:
> - Verified signature/checksum
> - Built binary distribution
> - Ingested sample "wikipedia" dataset using native batch, range
> partitioning
>
> docker image:
> - Verified checksum
> - Started cluster with docker-compose
> - Ingested sample "wikipedia" dataset using native batch
>
>
> On Thu, Nov 9, 2023 at 6:36 PM Abhishek Agarwal 
> wrote:
>
> > +1 Binding
> >
> > src package:
> > - verified signature/checksum
> > - LICENSE/NOTICE present
> > - built binary distribution,
> >  - Loaded example koalas-to-the-max nested dataset using MSQ and ran
> > some
> > queries
> >  - Tested Kafka ingestion locally
> >
> > binary package:
> > - verified signature/checksum
> > - LICENSE/NOTICE present
> > - built binary distribution,
> >  - Loaded example koalas-to-the-max nested dataset using MSQ and ran
> > some
> > queries
> >  - Tested Kafka ingestion locally
> >
> > docker:
> > - verified checksum
> > - started cluster with docker-compose, loaded example koalas-to-the-max
> > nested dataset using MSQ and ran some
> > queries
> > - Added the kafka extension to the environment, started cluster with
> > docker-compose, and then tested kafka ingestion
> >
> > On Tue, Nov 7, 2023 at 11:08 AM Laksh Singla 
> > wrote:
> >
> > > Hi all,
> > >
> > > I have created a build for Apache Druid 28.0.0, release
> > > candidate 1.
> > >
> > > Thanks to everyone who has helped contribute to the release! You can
> read
> > > the proposed release notes here:
> > > https://github.com/apache/druid/issues/15326
> > >
> > > The release candidate has been tagged in GitHub as
> > > druid-28.0.0-rc1 (c91d61dc3f7232242c00a6334ea5d0d9cd98e32c),
> > > available here:
> > > https://github.com/apache/druid/releases/tag/druid-28.0.0-rc1
> > >
> > > The artifacts to be voted on are located here:
> > > https://dist.apache.org/repos/dist/dev/druid/28.0.0-rc1/
> > >
> > > A staged Maven repository is available for review at:
> > >
> https://repository.apache.org/content/repositories/orgapachedruid-1056/
> > >
> > > Staged druid.apache.org website documentation is available here:
> > > https://druid.staged.apache.org/docs/28.0.0/design/index.html
> > >
> > > A Docker image containing the binary of the release candidate can be
> > > retrieved via:
> > > docker pull apache/druid:28.0.0-rc1
> > >
> > > artifact checksums
> > > src:
> > >
> > >
> >
> f648143c29610f6874c5946897712bce967a4b9d5304b953c9efc14e514a7ee1269fcac626165544dd485560517fc52ef85ddd2617d3410fb0478e10f3ff2982
> > > bin:
> > >
> > >
> >
> c971ccbb4220fa2ae9f8a0a0f82c3e4b87750d27ac15d863d1e9f415863ccb8b8e5c764122dc8ef76044a42d27e1108b980ca2b88de8aff660db5a22356d6ab7
> > > docker:
> 708f1bd7221734986d049e1b5bd6ad0c985c65d815037f0650dd5ace8654d38c
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/lakshsingla.asc
> > >
> > > This key and the key of other committers can also be found in the
> > project's
> > > KEYS file here:
> > > https://dist.apache.org/repos/dist/release/druid/KEYS
> > >
> > > (If you are a committer, please feel free to add your own key to that
> > file
> > > by following the instructions in the file's header.)
> > >
> > >
> > > Verify checksums:
> > > diff <(shasum -a512 apache-druid-28.0.0-src.tar.gz | \
> > > cut -d ' ' -f1) \
> > > <(cat apache-druid-28.0.0-src.tar.gz.sha512 ; echo)
> > >
> > > diff <(shasum -a512 apache-druid-28.0.0-bin.tar.gz | \
> > > cut -d ' ' -f1) \
> > > <(cat apache-druid-28.0.0-bin.tar.gz.sha512 ; echo)
> > >
> > > Verify signatures:
> > > gpg --verify apache-druid-28.0.0-src.tar.gz.asc \
> > > apache-druid-28.0.0-src.tar.gz
> > >
> > > gpg --verify apache-druid-28.0.0-bin.tar.gz.asc \
> > > apache-druid-28.0.0-bin.tar.gz
> > >
> > > Please review the proposed artifacts and vote. Note that Apache has
> > > specific requirements that must be met before +1 binding votes can be
> > cast
> > > by PMC members. Please refer to the policy at
> > > http://www.apache.org/legal/release-policy.html#policy for more
> details.
> > >
> > > As part of the validation process, the release artifacts can be
> generated
> > > from source by running:
> > > mvn clean install -Papache-release,dist -Dgpg.skip
> > >
> > > The RAT license check can be run from source by:
> > > mvn a

Re: New committer: Rahul Gidwani

[Karan Kumar]

Congratulations!!

On Tue, Aug 29, 2023 at 9:59 PM Atul Mohan  wrote:

> Congratulations Rahul!
>
> On Mon, Aug 28, 2023 at 10:35 PM suneet Saldanha 
> wrote:
>
> > The Project Management Committee (PMC) for Apache Druid
> > has invited Rahul Gidwani and we are pleased to announce that
> > he has accepted.
> >
> > Rahul's major contribution has been introducing the ability to launch
> > ingestion tasks as Kubernetes jobs aka middle managerless ingestion.
> >
> > Congrats Rahul!
> >
>


-- 
Thanks
Karan

New Committer : Soumyava Das

[Karan Kumar]

Hello everyone,

The Project Management Committee (PMC) for Apache Druid has invited
Soumyava to become a committer and we are pleased to announce that
Soumyava has accepted.

Soumyava has been a consistent contributor for over a year now. He has over
29 commits in druid. His majority commits are in the calcite layer and the
query processing layer. His major contributions are :
1. Developing the unnest feature for array typed columns.
2. Fixing various bugs in the druid query planning and query processing
area.
3. Vectorizing druid aggregators for faster processing.
4. Calcite 1.35 upgrade.


Congratulations Soumyava.

New Committer : Adarsh Sanjeev

[Karan Kumar]

Hello everyone,

The Project Management Committee (PMC) for Apache Druid has invited
Adarsh to become a committer and we are pleased to announce that
Adarsh has accepted.

Adarsh has been a consistent contributor for over a year now. He has over
49 commits in druid. His majority commits are in the calcite layer and MSQ.
Some of his most notable contributions are:
1. Adding the replace SQL syntax for MSQ.
2. Adding the sequential merge feature in MSQ which helps in generating
better segment sizes.
3. Changes in load rules and brokers to enable the query from deep storage
feature.


Congratulations Adarsh.

Re: [VOTE] Release Apache Druid 27.0.0 [RC1]

[Karan Kumar]

+1 (binding)

src package:

   - verified signature/checksum
   - Build druid on (m1 based chipset)
   - Ran druid cluster and tested
  - All MSQ demo q's
  - Tested query from deep storage with results written out to s3.
  - Ran q's against segment's not loaded on the historicals
  - Ran q's against segments loaded on the historicals.


binary package:

   - verified signature/checksum
   - LICENSE/NOTICE present
   - Ran druid cluster and tested
  - All MSQ demo q's
  - Tested query from deep storage with results written out to s3.
  - Ran q's against segment's not loaded on the historicals
  - Ran q's against segments loaded on the historicals.



docker :

   - verified checksum



On Mon, Aug 7, 2023 at 11:23 AM Abhishek Agarwal 
wrote:

> +1 (binding)
>
> src package:
> - verified signature/checksum
> - LICENSE/NOTICE present
> - built binary distribution,
>  - Loaded example Wikipedia dataset using MSQ and ran some
> queries
>  - Tested Kafka ingestion locally
>
> binary package:
> - verified signature/checksum
> - LICENSE/NOTICE present
> - built binary distribution,
>  - Loaded example wikipedia dataset using MSQ and ran some
> queries
>  - Tested Kafka ingestion locally
>
> docker:
> - verified checksum
> - started cluster with docker-compose, Loaded example Wikipedia dataset and
> ran some queries
> - Added the kafka extension to the environment, started cluster with
> docker-compose, and then tested kafka ingestion
>
> On Sun, Aug 6, 2023 at 1:43 PM Amatya Avadhanula 
> wrote:
>
> > Hi all,
> >
> > I have created a build for Apache Druid 27.0.0, release
> > candidate 1.
> >
> > Thanks to everyone who has helped contribute to the release! You can read
> > the proposed release notes here:
> > https://github.com/apache/druid/issues/14761
> >
> > The release candidate has been tagged in GitHub as
> > druid-27.0.0-rc1
> > available here:
> > https://github.com/apache/druid/tree/druid-27.0.0-rc1
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/druid/27.0.0-rc1/
> >
> > A staged Maven repository is available for review at:
> > https://repository.apache.org/content/repositories/orgapachedruid-1044/
> >
> > Staged druid.apache.org website documentation is available here:
> > https://druid.staged.apache.org/docs/27.0.0/design/index.html
> >
> > A Docker image containing the binary of the release candidate can be
> > retrieved via:
> > docker pull apache/druid:27.0.0-rc1
> >
> > artifact checksums
> > src:
> >
> >
> a3a755d02e2ed55a125ba562de4b4ce467d27af1132a89ecbc73cbb4a38622f7534813267ed44de1ec0fb85227e93d04c1c1ce24959d6d5c41dedbf2d7c6e4ed
> > bin:
> >
> >
> b840ed0d77b1e5c11e058b161a56a469b0916febc1d478bf7ad2517cf79d2b724ece5cef21d96581122ba0958a5737f41e76496db3b10db5bfb4ab3123e4091b
> > docker: ca3df175bc944033c7c56ccf9499c05e2090ae6cefbdcd90095cfce2b7931ead
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/amatya.asc [To be available
> > within
> > 48 hours: https://issues.apache.org/jira/browse/INFRA-24865]
> >
> > This key and the key of other committers can also be found in the
> project's
> > KEYS file here:
> > https://dist.apache.org/repos/dist/release/druid/KEYS
> >
> > (If you are a committer, please feel free to add your own key to that
> file
> > by following the instructions in the file's header.)
> >
> >
> > Verify checksums:
> > diff <(shasum -a512 apache-druid-27.0.0-src.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-27.0.0-src.tar.gz.sha512 ; echo)
> >
> > diff <(shasum -a512 apache-druid-27.0.0-bin.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-27.0.0-bin.tar.gz.sha512 ; echo)
> >
> > Verify signatures:
> > gpg --verify apache-druid-27.0.0-src.tar.gz.asc \
> > apache-druid-27.0.0-src.tar.gz
> >
> > gpg --verify apache-druid-27.0.0-bin.tar.gz.asc \
> > apache-druid-27.0.0-bin.tar.gz
> >
> > Please review the proposed artifacts and vote. Note that Apache has
> > specific requirements that must be met before +1 binding votes can be
> cast
> > by PMC members. Please refer to the policy at
> > http://www.apache.org/legal/release-policy.html#policy for more details.
> >
> > As part of the validation process, the release artifacts can be generated
> > from source by running:
> > mvn clean install -Papache-release,dist -Dgpg.skip
> >
> > The RAT license check can be run from source by:
> > mvn apache-rat:check -Prat
> >
> > This vote will be open for at least 72 hours. The vote will pass if a
> > majority of at least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Druid 0.17.0
> > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Thank you!
> >
>

Re: [DISCUSS] Druid 28 dropping support for Hadoop 2

[Karan Kumar]

In favour of dropping hadoop 2 support . Another point is the lack of
security and vulnerability fixes in hadoop2.



On Wed, Jun 28, 2023 at 12:17 PM Clint Wylie  wrote:

> obvious +1 from me
>
> On Tue, Jun 27, 2023 at 11:42 PM Gian Merlino  wrote:
> >
> > I'd like to propose dropping support for Hadoop 2 in Druid 28. Not the
> very
> > next release (which I assume will be Druid 27) but the one after that,
> > likely late 2023 timeframe.
> >
> > In 2021, we had a discussion about moving away from Hadoop 2:
> > https://lists.apache.org/thread/zmc389trnkh6x444so8mdb2h0x0noqq4. For
> > various reasons, it didn't seem like the right time. However, I believe
> now
> > is the right time:
> >
> > 1) We didn't support Hadoop 3 in 2021, but we support it now. There is
> now
> > a Hadoop 3 build profile, as well as convenience binaries on
> > https://druid.apache.org/downloads.html.
> >
> > 2) We have SQL-based ingest with MSQ tasks, which provides a built-in /
> > scalable / robust alternative to using Hadoop at all.
> >
> > 3) It has been an additional two years. Hadoop 2 is that much older, that
> > much more time has passed since it was superseded by Hadoop 3, and people
> > have had that much more time to migrate.
> >
> > 4) The original main reason for wanting to move away from Hadoop 2 is
> still
> > relevant. It keeps us on various old dependencies, including an ancient
> > version of Guava, which in turn has been keeping us on an ancient version
> > of Calcite. The Calcite community has graciously decided to support this
> > old version of Guava for at least one release, but plans to drop support
> by
> > Calcite 1.36, leaving us back in the same position. Managing this
> situation
> > is time-consuming for both Druid and Calcite maintainers.
> >
> > 5) Other solutions beyond dropping Hadoop 2 support were proposed in
> 2021,
> > such as reworking Hadoop support to be purely extension based, and
> > reworking extensions to be more isolated from each other. However, these
> > are both substantially more complex than dropping support, and in the two
> > years since the original thread, these more complex solutions have not
> been
> > implemented. So, I think we need to move on with the simpler solution of
> > dropping support.
> >
> > Gian
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> For additional commands, e-mail: dev-h...@druid.apache.org
>
>

-- 
Thanks
Karan

Re: New Committer : Didip Kerabat

[Karan Kumar]

Congratulations!!

On Wed, May 3, 2023 at 9:33 PM Rohan Garg 
wrote:

> Congratulations Didip!
>
> On Wed, May 3, 2023 at 7:48 AM Kashif Faraz  wrote:
>
> > Congratulations, Didip!!
> >
> > On Tue, May 2, 2023 at 3:43 AM Sergio Ferragut
> >  wrote:
> >
> > > Congratulations on becoming a committer Didip!
> > > The whole Apache Druid community appreciates your in-depth
> understanding
> > > and willingness to share it on a regular basis.
> > > Thank you.
> > >
> > > On Mon, May 1, 2023 at 5:02 AM Abhishek Agarwal 
> > > wrote:
> > >
> > > > Hello everyone,
> > > >
> > > > The Project Management Committee (PMC) for Apache Druid has invited
> > Didip
> > > > to become a committer and we are pleased to announce that Didip has
> > > > accepted.
> > > >
> > > > Didip is very active on the Druid slack channel and helps Druid users
> > > > troubleshoot their problems regularly. He has also fixed bugs and
> added
> > > > features that have benefited our users.
> > > >
> > > > Congrats Didip.
> > > >
> > >
> >
>


-- 
Thanks
Karan

Re: New Committer : Jason Koch

[Karan Kumar]

Congratulations Jason !!!

On Tue, 18 Apr, 2023, 5:11 am Jason Koch,  wrote:

> Thank you!!
>
> On Mon, Apr 17, 2023 at 10:28 AM Maytas Monsereenusorn  >
> wrote:
>
> > Hello everyone,
> >
> > The Project Management Committee (PMC) for Apache Druid has invited
> > Jason to become a committer and we are pleased to announce that
> > Jason has accepted.
> >
> > Jason has been working on Apache Druid for about two years now. Jason's
> > focus has been in the area of performance and cost saving. His most
> notable
> > contribution is performance improvement of Druid ingestion, both batch
> > ingestion and streaming ingestion. Jason also introduced a new "tree"
> type
> > to the flattenSpec allowing faster JSON parsing for certain use cases.
> >
> > Congratulations Jason.
> >
>

Re: New Committer : Laksh Singla

[Karan Kumar]

Congratulations Laksh!!

On Mon, 10 Apr, 2023, 7:37 pm Kashif Faraz,  wrote:

> Congrats, Laksh!!
>
> On Mon, Apr 10, 2023 at 2:20 PM Rohan Garg 
> wrote:
>
> > Congratulations Laksh!
> >
> > On Mon, Apr 10, 2023 at 11:38 AM Frank Chen 
> wrote:
> >
> > > Congratulations to Laksh
> > >
> > > On Mon, Apr 10, 2023 at 2:01 PM Abhishek Agarwal 
> > > wrote:
> > >
> > > > Hello everyone,
> > > >
> > > > The Project Management Committee (PMC) for Apache Druid has invited
> > Laksh
> > > > to become a committer and we are pleased to announce that Laksh has
> > > > accepted.
> > > >
> > > > Laksh has made a lot of contributions to MSQ over one year. He has
> been
> > > > helping the druid community on Slack. His most notable contribution
> is
> > > > extending the calcite parser that enabled us to add custom SQL syntax
> > for
> > > > MSQ. He also worked on improving the test coverage of MSQ by porting
> > > > CalciteQueryTests to the MSQ engine. He has made the parse exceptions
> > > more
> > > > informative to aid in troubleshooting.
> > > >
> > > > Congratulations Laksh.
> > > >
> > >
> >
>

Re: New Committer : Tejaswini Bandlamudi

[Karan Kumar]

Congratulations Tejaswini!!

On Mon, 10 Apr, 2023, 7:37 pm Kashif Faraz,  wrote:

> Congrats, Tejaswini!
>
> On Mon, Apr 10, 2023 at 2:20 PM Rohan Garg 
> wrote:
>
> > Congratulations Tejaswini!
> >
> > On Mon, Apr 10, 2023 at 11:37 AM Frank Chen 
> wrote:
> >
> > > Congratulation to Tejaswini
> > >
> > > On Mon, Apr 10, 2023 at 1:59 PM Abhishek Agarwal 
> > > wrote:
> > >
> > > > Hello everyone,
> > > >
> > > > The Project Management Committee (PMC) for Apache Druid has invited
> > > > Tejaswini to become a committer and we are pleased to announce that
> > > > Tejaswini has accepted.
> > > >
> > > > Tejaswini has been working on Apache Druid for over a year now. Her
> > major
> > > > contribution has been porting our test infra from Travis to GHA. She
> > has
> > > > fixed various bugs in the ingestion area. She added the feature for a
> > > > supervisor to become idle when there is no data on the topic. This
> can
> > > help
> > > > cluster operators save money when data sources receive data sparsely.
> > > >
> > > > Congratulation Tejaswini.
> > > >
> > >
> >
>

Re: CI requiring approval for external contributors

[Karan Kumar]

+1
Pasting some of the examples that I shared on slack:

   - PR : https://github.com/apache/druid/pull/13934 which is raised by
   Soumyava Das who contributes regularly to druid should not block on a
   committer to approve CI runs.
   - PR : https://github.com/apache/druid/pull/13909 by Adarsh Sanjeev who
   has added a lot of features in MSQE.
   - PR : https://github.com/apache/druid/pull/13991 by Jason Witkowski who
   contributes regularly to druid helm charts.


On Wed, Mar 29, 2023 at 10:55 AM Frank Chen  wrote:

> +1
>
> I don't see there's a need for committers to click the 'Approve' button to
> run CI for every PR.
>
>
>
> On Tue, Mar 28, 2023 at 2:27 PM Austin Bennett  wrote:
>
> > Beam did the same.
> >
> > +1 on allowing contributors to keep things moving [where appropriate,
> > which this can be if CI is setup well], allowing committers to focus on
> > more/other activities
> >
> > On Mon, Mar 27, 2023, 11:24 PM Gian Merlino  wrote:
> >
> > > Recently, ASF GitHub repos had their defaults for GitHub Actions
> changed
> > to
> > > "always require approval for external contributors". In Slack, Karan
> > > pointed out that Airflow has recently submitted a ticket to have that
> > > changed back: https://issues.apache.org/jira/browse/INFRA-24200. IMO,
> we
> > > should do the same. I don't think we have a problem with fake PRs, but
> we
> > > can always improve our responsiveness to contributors from outside the
> > > project! Every little bit helps, including running CI automatically.
> > >
> > > If others have opinions on this, let me know. I'd like to raise our own
> > > ticket to change our default.
> > >
> > > Gian
> > >
> >
>


-- 
Thanks
Karan

Re: [Discuss] S3 buckets or IT tests

[Karan Kumar]

Thanks everyone. I have closed the ticket
https://issues.apache.org/jira/browse/INFRA-23952 since min.io seems like
the path ahead.

On Tue, Mar 7, 2023 at 12:24 AM Austin Bennett  wrote:

> +1 for Minio to keep things sufficiently simple.  Full functional testing [
> ex: using s3 ] is great.  But, it doesn't seem critical for this use case?
>
>
> On Sun, Mar 5, 2023, 8:27 PM Abhishek Agarwal  wrote:
>
> > I would prefer using something like min.io than running tests against
> S3.
> > Primarily because tests no longer remain portable if we use S3. Which
> also
> > makes them harder to debug. Setting up minio using GHA seems
> > straightforward (https://github.com/mozilla/sccache/pull/1513/files).
> >
> > On Mon, Mar 6, 2023 at 7:01 AM Karan Kumar 
> > wrote:
> >
> > > Oh yes, https://issues.apache.org/jira/browse/INFRA-23952 Is the
> ticket
> > I
> > > am talking about. As the INFRA folks require approval to proceed, can
> one
> > > of the PMC's drop a comment regarding approval/rejection on the same
> > jira.
> > > If we are not able to procure a bucket, then we can start exploring
> mini
> > > IO.
> > >
> > > On Wed, Feb 22, 2023 at 6:07 PM Gian Merlino  wrote:
> > >
> > > > I think the ticket you're referring to is
> > > > https://issues.apache.org/jira/browse/INFRA-23952.
> > > >
> > > > It would definitely be valuable to run S3 integration tests as part
> of
> > > the
> > > > automated test suite in GitHub Actions. If Infra is willing to
> provide
> > a
> > > > bucket for this purpose then we would certainly be able to use that.
> I
> > > bet
> > > > we could also use Minio (https://min.io/) or something similar.
> > > >
> > > > Gian
> > > >
> > > > On 2023/02/15 21:38:47 Karan Kumar wrote:
> > > > > Hey Folks
> > > > > S3 read write tests currently are not executed in github actions
> > since
> > > we
> > > > > do not have public creds to read/write from s3.
> > > > > Have raised an ASF infra ticket
> > > > >
> > > >
> > >
> >
> https://apachedruidworkspace.slack.com/archives/C030CMF6B70/p1675916945323839
> > > > > so that they can give us a bucket.
> > > > > They require a PMC approval.
> > > > > As discussed on this slack thread
> > > > >
> > > >
> > >
> >
> https://apachedruidworkspace.slack.com/archives/C030CMF6B70/p1675916945323839
> > > > ,
> > > > > starting a formal discussion around it.
> > > > >
> > > > > The objective would be to integrate our s3 IT's tests to run on
> each
> > > PR.
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Thanks
> > > > > Karan
> > > > >
> > > >
> > > > -
> > > > To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> > > > For additional commands, e-mail: dev-h...@druid.apache.org
> > > >
> > > >
> > >
> > > --
> > > Thanks
> > > Karan
> > >
> >
>


-- 
Thanks
Karan

Re: [Discuss] S3 buckets or IT tests

[Karan Kumar]

Oh yes, https://issues.apache.org/jira/browse/INFRA-23952 Is the ticket I
am talking about. As the INFRA folks require approval to proceed, can one
of the PMC's drop a comment regarding approval/rejection on the same jira.
If we are not able to procure a bucket, then we can start exploring mini
IO.

On Wed, Feb 22, 2023 at 6:07 PM Gian Merlino  wrote:

> I think the ticket you're referring to is
> https://issues.apache.org/jira/browse/INFRA-23952.
>
> It would definitely be valuable to run S3 integration tests as part of the
> automated test suite in GitHub Actions. If Infra is willing to provide a
> bucket for this purpose then we would certainly be able to use that. I bet
> we could also use Minio (https://min.io/) or something similar.
>
> Gian
>
> On 2023/02/15 21:38:47 Karan Kumar wrote:
> > Hey Folks
> > S3 read write tests currently are not executed in github actions since we
> > do not have public creds to read/write from s3.
> > Have raised an ASF infra ticket
> >
> https://apachedruidworkspace.slack.com/archives/C030CMF6B70/p1675916945323839
> > so that they can give us a bucket.
> > They require a PMC approval.
> > As discussed on this slack thread
> >
> https://apachedruidworkspace.slack.com/archives/C030CMF6B70/p1675916945323839
> ,
> > starting a formal discussion around it.
> >
> > The objective would be to integrate our s3 IT's tests to run on each PR.
> >
> >
> >
> > --
> > Thanks
> > Karan
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> For additional commands, e-mail: dev-h...@druid.apache.org
>
>

-- 
Thanks
Karan

[Discuss] S3 buckets or IT tests

[Karan Kumar]

Hey Folks
S3 read write tests currently are not executed in github actions since we
do not have public creds to read/write from s3.
Have raised an ASF infra ticket
https://apachedruidworkspace.slack.com/archives/C030CMF6B70/p1675916945323839
so that they can give us a bucket.
They require a PMC approval.
As discussed on this slack thread
https://apachedruidworkspace.slack.com/archives/C030CMF6B70/p1675916945323839,
starting a formal discussion around it.

The objective would be to integrate our s3 IT's tests to run on each PR.



-- 
Thanks
Karan

Re: [E] Re: Apache Druid Slack

[Karan Kumar]

Hey
I fully support having our slack history indexed on google and I think we
should do it sooner rather than later as there are a lot of good threads on
slack that are getting purged out.


On Wed, Oct 19, 2022 at 9:47 AM Abhishek Agarwal 
wrote:

> I recently learned about Apache Flink making slack conversations google
> indexable through linen.dev. That might be worth exploring. Does anyone
> know how it works?
>
> On Wed, Oct 19, 2022 at 4:18 AM Eyal Yurman  .invalid>
> wrote:
>
> > What should we do about Slack canceling message history for free tier?
> >
> > Perhaps we should export history elsewhere?
> >
> > On Thu, Jan 27, 2022 at 1:10 PM Vadim Ogievetsky  >
> > wrote:
> >
> > > Alright, I have setup the new Slack workspace and updated the link on
> > >
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__druid.apache.org_community_&d=DwIFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=XsVtrQ1n66sQ9WUOQT8K162x-6-hXEAYJwF2JdxJbMU&m=adK2ZRhIznXu2_uvV3MLkV-rmdB2gTTJbxUdYqZHQ_DabAl8uQCbiEDLwBUVdYxF&s=hV9FPFkuEXDmMbTGo3m_iqUc-baVKjOHznj9kig1I1Q&e=
> > > to point to it.
> > > Please join with the
> > >
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__druid.apache.org_community_join-2Dslack&d=DwIFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=XsVtrQ1n66sQ9WUOQT8K162x-6-hXEAYJwF2JdxJbMU&m=adK2ZRhIznXu2_uvV3MLkV-rmdB2gTTJbxUdYqZHQ_DabAl8uQCbiEDLwBUVdYxF&s=07e2sPTwojKxi997mSN4a5Ktcr1lyHcekXuTwEZj9JY&e=
> > > link (and share it with others).
> > > Sadly there is no way to migrate content from one Slack workspace to
> > > workspace, I am sure we will fill the new channel with great content
> very
> > > quickly!
> > >
> > > On 2022/01/20 18:15:23 Vadim Ogievetsky wrote:
> > > > I think that the PMC should create a new Slack channel for Apache
> Druid
> > > and
> > > > shift the community towards using it away from the ASF Slack. I
> > volunteer
> > > > to do this on the PMCs behalf and do all the setup/admin work. I
> would
> > > > share admin access with any PMC member.
> > > >
> > > > What is the motivation for this?
> > > >
> > > > As you may have heard, it’s become increasingly difficult for new
> users
> > > > without an @apache.org email address to join the ASF #druid Slack
> > > channel.
> > > > ASF Infra disabled the option to publicly provide a link to the
> > workspace
> > > > to anyone who wanted it, after encountering issues with spammers.
> > > >
> > > > Per Infra’s guidance (
> > >
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__infra.apache.org_slack.html&d=DwIFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=XsVtrQ1n66sQ9WUOQT8K162x-6-hXEAYJwF2JdxJbMU&m=adK2ZRhIznXu2_uvV3MLkV-rmdB2gTTJbxUdYqZHQ_DabAl8uQCbiEDLwBUVdYxF&s=qZfDyv1d4QnrCrEjvTgSbm8qeEj-B-dFWKil2k4XOyQ&e=
> > > ), new community
> > > > members should only be invited as single-channel guests.
> Unfortunately,
> > > > single-channel guests are unable to extend invitations to new
> members,
> > > > including their colleagues who are using Druid. Only someone with
> full
> > > > member privileges is able to extend an invitation to new members.
> This
> > > lack
> > > > of consistency doesn’t make the community feel inclusive.
> > > >
> > > > There is a workaround in place (
> > > >
> > >
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_druid-2Dwebsite-2Dsrc_pull_278&d=DwIFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=XsVtrQ1n66sQ9WUOQT8K162x-6-hXEAYJwF2JdxJbMU&m=adK2ZRhIznXu2_uvV3MLkV-rmdB2gTTJbxUdYqZHQ_DabAl8uQCbiEDLwBUVdYxF&s=LZAv4yoEkFJcPmJckcAR0xXyuo7V81ZfR2X_jxQjH58&e=
> > > ) – users can send an
> > > > email to druid-u...@googlegroups.com to request an invite to the
> Slack
> > > > channel from an existing member – but this still poses a barrier to
> > > entry,
> > > > and isn’t a viable permanent solution. It also creates potential
> > privacy
> > > > issues as not everyone is at liberty to announce they’re using Druid
> > nor
> > > > wishes to display their email address in a public forum.
> > > >
> > > > I propose we make our own free Slack channel for Apache Druid and
> > > encourage
> > > > people to migrate to it. Then we can have our own policy on Slack
> > > > invitations - I would like to restore the ability for anyone on the
> web
> > > to
> > > > join our Slack.
> > > >
> > > > This is not a 100% original idea, in fact this is what other Apache
> > > > projects have done, notably Apache Pinot (see "join our slack" on
> > > >
> > >
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__pinot.apache.org_&d=DwIFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=XsVtrQ1n66sQ9WUOQT8K162x-6-hXEAYJwF2JdxJbMU&m=adK2ZRhIznXu2_uvV3MLkV-rmdB2gTTJbxUdYqZHQ_DabAl8uQCbiEDLwBUVdYxF&s=GYTpyHD-1U_Er2eKNXk41b9aiy4zW9EtCt-2YXxSh9o&e=
> > > ). I propose we do the same.
> > > >
> > > > Vadim
> > > >
> > > > -
> > > > To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> > > > For additional commands

Re: New Committer : Karan Kumar

[Karan Kumar]

Thank you all for your wishes.


On Thu, Nov 3, 2022 at 2:13 PM Furkan KAMACI  wrote:

> Welcome on board!
>
> On Tue, Nov 1, 2022 at 5:37 AM Paul Rogers  wrote:
>
> > Well deserved. Congrats Karan!
> >
> > - Paul
> >
> > > On Oct 31, 2022, at 7:36 PM, Kashif Faraz  wrote:
> > >
> > > Congrats, Karan!
> > >
> > > On Mon, Oct 31, 2022 at 8:46 PM Tijo Thomas 
> > wrote:
> > >
> > >> Awesome... Congratulations Karan..
> > >>
> > >> On Mon, Oct 31, 2022 at 5:22 PM Rohan Garg 
> wrote:
> > >>
> > >>> Congratulations Karan!
> > >>>
> > >>> On Mon, Oct 31, 2022 at 3:33 PM Abhishek Agarwal <
> abhis...@apache.org>
> > >>> wrote:
> > >>>
> > >>>> Hello everyone,
> > >>>>
> > >>>> The Project Management Committee (PMC) for Apache Druid has invited
> > >> Karan
> > >>>> Kumar to become a committer and we are pleased to announce that
> Karan
> > >> has
> > >>>> accepted.
> > >>>>
> > >>>> Karan contributed group-by on arrays without unnesting (
> > >>>> https://github.com/apache/druid/pull/12078), an essential
> requirement
> > >>> for
> > >>>> MSQ. Karan has been making a lot of contributions to MSQ. He has
> been
> > >>> doing
> > >>>> many code reviews and helps out users regularly on apache druid
> slack.
> > >>>>
> > >>>> Congratulations Karan.
> > >>>>
> > >>>
> > >>
> > >>
> > >> --
> > >> Thanks & Regards
> > >> Tijo Thomas
> > >>
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> > For additional commands, e-mail: dev-h...@druid.apache.org
> >
> >
>


-- 
Thanks
Karan

Re: New Committer : Paul Rogers

[Karan Kumar]

Congratulations Paul!!

On Wed, Sep 28, 2022 at 10:17 PM Abhishek Agarwal 
wrote:

> Hello everyone,
>
> The Project Management Committee (PMC) for Apache Druid has invited Paul
> Rogers to become a committer and we are pleased to announce that Paul has
> accepted.
>
> Paul has been very active in the community. He has done a lot of thorough
> code reviews. He single-handedly built the new IT framework. He has been
> cleaning up the SQL wiring for some time now. He has put many
> forward-looking design proposals that are detailed and in-depth e.g. druid
> catalog, calcite planner test framework, etc. Paul also brings in a wealth
> of experience and ideas from the Drill community.
>
> Congratulations Paul.
>


-- 
Thanks
Karan

Re: New Committer : Amatya Avadhanula

[Karan Kumar]

Congratulations Amatya!!

On Wed, Sep 28, 2022 at 10:17 PM Abhishek Agarwal 
wrote:

> Hello everyone,
>
> The Project Management Committee (PMC) for Apache Druid has invited Amatya
> Avadhanula to become a committer and we are pleased to announce that Amatya
> has accepted.
>
> Amatya has been making a lot of fixes on the kinesis ingestion and systems
> (overlord, task locking) side. He optimized the task fetching in overlord
> from the console which is going to benefit many heavy clusters. He fixed
> the kinesis ingestion to work with empty shards which had been a major
> limitation in working with kinesis ingestion. His work has been
> high-quality with great test coverage. He has also been helping the
> community on slack and Github.
>
> Congratulations Amatya.
>


-- 
Thanks
Karan

Re: New Committer : Rohan Garg

[Karan Kumar]

Congrats Rohan !!

On Tue, Jul 26, 2022 at 12:20 PM Abhishek Agarwal 
wrote:

> Hello everyone,
>
> The Project Management Committee (PMC) for Apache Druid has invited Rohan
> Garg to become a committer and we are pleased to announce that Rohan has
> accepted.
>
> Rohan has been an active contributor for about a year now. He has made over
> 20 commits. A majority of his work has been around SQL planning for better
> query performance. He has also helped the community via PR reviews.
>
> Congratulations Rohan.
>


-- 
Thanks
Karan

Re: New PMC member: Abhishek Agarwal

[Karan Kumar]

Congratulations Abhishek!

On Wed, Jun 8, 2022 at 8:42 AM Tijo Thomas  wrote:

> Congratulations...
>
>
> On Wed, Jun 8, 2022 at 6:48 AM Frank Chen  wrote:
>
> > Congratulations.
> >
> >
> > On Wed, Jun 8, 2022 at 7:44 AM Furkan KAMACI 
> > wrote:
> >
> > > Hi,
> > >
> > > Congrats and welcome on board Abhishek!
> > >
> > > Kind Regards,
> > > Furkan KAMACI
> > >
> > > On Wed, Jun 8, 2022 at 2:43 AM Suneet Saldanha <
> suneet.salda...@imply.io
> > >
> > > wrote:
> > >
> > > > Congratulations Abhishek! And thanks for all your work shepherding
> the
> > > > 0.23.0 release!!
> > > >
> > > > On Tue, Jun 7, 2022 at 12:04 PM Gian Merlino 
> wrote:
> > > >
> > > > > Hey Druids,
> > > > >
> > > > > The Druid PMC has invited Abhishek Agarwal (asf id abhishek, github
> > id
> > > > > abhishekagarwal87) to become a PMC member, and we are pleased to
> > > announce
> > > > > that he has accepted. Abhishek has authored dozens of commits,
> > > > participated
> > > > > in nearly 200 code reviews, and is release manager for the upcoming
> > > > 0.23.0
> > > > > release.
> > > > >
> > > > > Congratulations, Abhishek!
> > > > >
> > > >
> > >
> >
>
>
> --
> Tijo Thomas
> Solutions Architect  | => Imply , Bangalore , India
>


-- 
Thanks
Karan

Re: new committer: Kashif Faraz

[Karan Kumar]

Congratulations Kashif!!

On Mon, Jan 17, 2022 at 10:43 AM Tijo Thomas  wrote:

> Congratulations Kashif !!
>
> On Fri, Jan 14, 2022 at 8:40 AM Kashif Faraz 
> wrote:
>
> > Thanks a lot, Suneet, Abhishek!
> >
> > I am really grateful for the recognition and for the community's active
> > feedback on my PRs 😀.
> > I look forward to learning more from the Druid community and seeing Druid
> > grow by leaps and bounds.
> >
> > Regards
> > Kashif Faraz
> >
> > On Thu, Jan 13, 2022 at 11:28 PM Abhishek Agarwal <
> > abhishek.agar...@imply.io>
> > wrote:
> >
> > > Congratulations Kashif. You have done really great work, particularly
> on
> > > the ingestion side. Looking forward to many more contributions in
> future.
> > >
> > > On Thu, Jan 13, 2022 at 11:04 PM suneet Saldanha 
> > > wrote:
> > >
> > > > Hey Druids,
> > > >
> > > > The Druid PMC has invited Kashif Faraz (@kfaraz on GitHub)
> > > > to become a committer and we are pleased to announce that
> > > > Kashif has accepted.
> > > >
> > > > Kashif has contributed several impactful features to Druid like
> support
> > > > for range partitioning on multiple dimensions and the ability to
> route
> > > sql
> > > > queries at the router.
> > > >
> > > > Congratulations Kashif!
> > > >
> > >
> >
>
>
> --
> Tijo Thomas
> Solutions Architect  | => Imply , Bangalore , India
>


-- 
Thanks
Karan

Re: New PMC member: Atul Mohan

[Karan Kumar]

Congrats Atul!!

On Tue, Nov 2, 2021 at 11:35 AM Abhishek Agarwal 
wrote:

> Congratulations Atul.
>
> On Tue, Nov 2, 2021 at 2:43 AM Jihoon Son  wrote:
>
> > Hey Druids,
> >
> > The Druid PMC has invited Atul Mohan (@a2l007 on github) to become a
> > PMC member and we are pleased to announce that he has accepted. Atul
> > has been making substantial contributions in the Druid community, not
> > only code contributions but also helping people in the user community.
> >
> > Congratulations Atul!
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> > For additional commands, e-mail: dev-h...@druid.apache.org
> >
> >
>


-- 
Thanks
Karan

Re: New PMC member: Frank Chen

[Karan Kumar]

Congrats Frank!!

On Tue, Nov 2, 2021 at 11:36 AM Abhishek Agarwal 
wrote:

> Congratulations Frank.
>
> On Tue, Nov 2, 2021 at 2:44 AM Jihoon Son  wrote:
>
> > Hey Druids,
> >
> > The Druid PMC has invited Frank Chen (@FrankChen021 on github) to
> > become a PMC member and we are pleased to announce that he has
> > accepted. Frank has been active in various parts of the Druid
> > community including contributing features, fixing bugs, reviewing PRs,
> > responding to user issues, releases, and so on.
> >
> > Congratulations Frank!
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> > For additional commands, e-mail: dev-h...@druid.apache.org
> >
> >
>


-- 
Thanks
Karan

Re: New committer: Agustin Gonzalez Tuchmann

[Karan Kumar]

Congrats Agustin !!

On Tue, Nov 2, 2021 at 11:35 AM Abhishek Agarwal 
wrote:

> Congratulations Agustin. Keep up the good work.
>
> On Tue, Nov 2, 2021 at 2:42 AM Jihoon Son  wrote:
>
> > Hey Druids,
> >
> > The Druid PMC has invited Agustin Gonzalez Tuchmann (@loquisgon on
> > github) to become a committer and we are pleased to announce that he
> > has
> > accepted. Agustin has made several significant contributions in the
> > ingestion, including reducing memory pressure for batch ingestion.
> >
> > Congratulations Agustin!
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> > For additional commands, e-mail: dev-h...@druid.apache.org
> >
> >
>


-- 
Thanks
Karan

Re: [E] [DISCUSS] Hadoop 3, dropping support for Hadoop 2.x

[Karan Kumar]

Hello 
We can also use maven profiles. We keep hadoop2 support by default and add a 
new maven profile with hadoop3. This will allow the user to choose the profile 
which is best suited for the use case. 
Agreed, it will not help in the Hadoop dependency problems but does enable our 
users to use druid with multiple flavors. 
Also with hadoop3, as clint mentioned, the dependencies come pre-shaded so we 
significantly reduce our effort in solving the dependency problems. 
I have the PR in the last phases where I am able to run the entire test suit 
unit + integration tests on both the default ie hadoop2 and the new hadoop3 
profile. 



On 2021/06/09 11:55:31, Will Lauer  wrote: 
> Clint,
> 
> I fully understand what type of headache dealing with these dependency
> issues is. We deal with this all the time, and based on conversations I've
> had with our internal hadoop development team, they are quite aware of them
> and just as frustrated by them as you are. I'm certainly in favor of doing
> something to improve this situation, as long as it doesn't abandon a large
> section of the user base, which I think DROPPING hadoop2 would do.
> 
> I think there are solutions there that can help solve the conflicting
> dependency problem. Refactoring Hadoop support into an independent
> extension is certainly a start. But I think the dependency problem is
> bigger than that. There are always going to be conflicts between
> dependencies in the core system and in extensions as the system gets
> bigger. We have one right now internally that prevents us from enabling SQL
> in our instance of Druid due to conflicts between versions of protobuf used
> by Calcite vs one of our critical extensions. Long term, I think you are
> going to need to carefully think through a ClassLoader based strategy to
> truly separate the impact of various dependencies.
> 
> While I'm not seriously suggesting it for Druid, OSGi WOULD solve this
> problem. It's a system that allows you to explicitly declare what each
> bundle exposes to the system, and what each bundle consumes from the
> system, allowing multiple conflicting dependencies to co-exist without
> impacting each other. OSGi is the big hammer approach, but I bet a more
> appropriate solution would be a simpler custom-ClassLoader based solution
> that hid all dependencies in extensions, keeping them from impacting the
> core, and that only exposed "public" pieces of the core to extensions. If
> Druid's core could be extended without impacting the various extensions,
> and the extensions' dependencies could be modified without impacting the
> core, this would go a long way towards solving the problem that you have
> described.
> 
> Will
> 
> 
> 
> Will Lauer
> 
> Senior Principal Architect, Audience & Advertising Reporting
> Data Platforms & Systems Engineering
> 
> M 508 561 6427
> 1908 S. First St
> Champaign, IL 61822
> 
>    
> 
> 
> 
> 
> 
> On Wed, Jun 9, 2021 at 12:47 AM Clint Wylie  wrote:
> 
> > @itai, I think pending the outcome of this discussion that it makes sense
> > to have a wider community thread to announce any decisions we make here,
> > thanks for bringing that up.
> >
> > @rajiv, Minio support seems unrelated to this discussion. It seems like a
> > reasonable request, but I recommend starting another thread to see if
> > someone is interested in taking up this effort.
> >
> > @jihoon I definitely agree that Hadoop should be refactored to be an
> > extension longer term. I don't think this upgrade would necessarily
> > make doing such a refactor any easier, but not harder either. Just moving
> > Hadoop to an extension also unfortunately doesn't really do anything to
> > help our dependency problem though, which is the thing that has agitated me
> > enough to start this thread and start looking into solutions.
> >
> > @will/@frank I feel like the stranglehold Hadoop has on our dependencies
> > has started to become especially more painful in the last couple of
> > years. Most painful to me is that we are stuck using a version of Apache
> > Calcite from 2019 (six versions behind the latest), because newer versions
> > require a newer version of Guava. This means we cannot get any bug fixes
> > and improvements in our SQL parsing layer without doing something like
> > packaging a shaded version of it ourselves or solving our Hadoop dependency
> > problem.
> >
> > Many other dependencies have also proved problematic with Hadoop as well in
> > the past, and since we aren't able to run the Hadoop integration tests in
> > Travis, there is always the chance that sometimes we don't catch these when
> > they go in. I imagine now that we have turned on dependabot this week,
> >
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_druid_pull_11079&d=DwIFaQ&c=sWW_bEwW_mLyN