Re: Next Druid release version scheme

[Julian Hyde]

That's good to hear. I still think it would be good if there was a
stated policy about API changes (e.g. "Druid will give notice of not
less than two x.0 releases and 1 year before removing a public API")
because "We follow semantic versioning" no longer has the same heft
under the new version scheme.

On Wed, Jul 6, 2022 at 9:05 PM Gian Merlino  wrote:
>
> I'd say yes, in a way that's similar to today. Today we treat increments of
> the version after the 0 as potentially allowing breaking changes. We also
> try to avoid them whenever feasible, because we know they're painful for
> users. I'm not suggesting we immediately get any more, or less, eager about
> making breaking changes as part of dropping the "0.". Over time, though,
> I'd like to see us get less eager about making breaking changes.
>
> On Wed, Jul 6, 2022 at 9:47 AM Julian Hyde  wrote:
>
> > Would 24.0 and 25.0 each be regarded as major versions for the purposes of
> > semantic versioning?
> >
> > If so, under the rules of semantic versioning, we *can* make breaking API
> > changes but that doesn’t mean that we *should*. (For an example of a
> > project that followed the letter of semantic versioning but still
> > undermined the trust of their users by making too many API changes, look no
> > further than Guava.)
> >
> > Julian
> >
> >
> > On Jul 6, 2022, at 1:53 AM, Gian Merlino  wrote:
> >
> > My proposal for the next release is that we merely drop the leading "0."
> > and don't change anything else about our dev process. We'd start the next
> > release at 24.0, and then likely do 25.0 shortly after. Same as today, just
> > no leading '0.".
> >
> > Separately, I'd like to craft a better versioning story around extension
> > API, query API, etc. But I don't think we need to connect these two things.
> > The dropping of the leading "0." is mainly about reflecting the reality
> > that the project is way more stable than a random member of the public
> > would expect for a "0." release. The better versioning story is an effort
> > that is independent from that.
> >
> > On Tue, Jun 7, 2022 at 11:50 AM Xavier Léauté  > >
> > wrote:
> >
> > Extension API: do extensions written for version X run as expected with
> >
> > version Y?
> >
> > One thing I'd like to see us do before we declare to 1.0 and provide
> > backwards compatibility for extensions APIs is
> > to remove some of the crufty Hadoop 2.x and Guava 16 dependency constraints
> > we have (or at least isolate them so
> > extensions and core are not constrained by old versions). Removing those
> > will likely be a breaking change for extensions.
> >
> > I'm also fine declaring 1.0, but that might mean we can't deprecate things
> > until 2.0, and then remove those in 3.0 depending on
> > what our backwards compatibility guarantees are. What I'd like us to avoid
> > is to be further entrenched and bogged down in
> > moving away from those dependencies by declaring a stable API.
> >
> > Xavier
> >
> > On Mon, Jun 6, 2022 at 2:45 PM rahul gidwani 
> > wrote:
> >
> > Hi Gian, this is great.
> >
> > For me what is most important is (2) and (4)
> > Does my current extension work with new releases?
> > Can I do a rolling upgrade of druid to the next version?
> >
> > The more things that are versioned the better, but (2) and (4) have been
> > the things that have been most important to me in the past.
> >
> > Anyone in the community have any thoughts on this?
> > Thank you
> > rahul
> >
> >
> >
> > On Fri, May 27, 2022 at 11:22 AM Gian Merlino  wrote:
> >
> > Yeah, I'd say the next one after 24.0 would be 25.0. The idea is really
> > just to remove the leading zero and thereby communicate the accurate
> >
> > state
> >
> > of the project: it has been stable and production-ready for a long
> >
> > time.
> >
> > Some people see the leading zero and interpret that as a sign of an
> > immature or non-production-ready system. So I think this change is
> >
> > worth
> >
> > doing and beneficial.
> >
> > I do think we can do better at communicating compatibility, but IMO
> > semantic versioning for the whole system isn't the best way to do it.
> > Semantic versioning is good for libraries, where people need one kind
> >
> > of
> >
> > assurance: that they can update to the latest version of the library
> > without needing to make changes in their program. But Drui

Re: Next Druid release version scheme

[Julian Hyde]

Would 24.0 and 25.0 each be regarded as major versions for the purposes of
semantic versioning?

If so, under the rules of semantic versioning, we *can* make breaking API
changes but that doesn’t mean that we *should*. (For an example of a
project that followed the letter of semantic versioning but still
undermined the trust of their users by making too many API changes, look no
further than Guava.)

Julian


On Jul 6, 2022, at 1:53 AM, Gian Merlino  wrote:

My proposal for the next release is that we merely drop the leading "0."
and don't change anything else about our dev process. We'd start the next
release at 24.0, and then likely do 25.0 shortly after. Same as today, just
no leading '0.".

Separately, I'd like to craft a better versioning story around extension
API, query API, etc. But I don't think we need to connect these two things.
The dropping of the leading "0." is mainly about reflecting the reality
that the project is way more stable than a random member of the public
would expect for a "0." release. The better versioning story is an effort
that is independent from that.

On Tue, Jun 7, 2022 at 11:50 AM Xavier Léauté 
wrote:

Extension API: do extensions written for version X run as expected with

version Y?

One thing I'd like to see us do before we declare to 1.0 and provide
backwards compatibility for extensions APIs is
to remove some of the crufty Hadoop 2.x and Guava 16 dependency constraints
we have (or at least isolate them so
extensions and core are not constrained by old versions). Removing those
will likely be a breaking change for extensions.

I'm also fine declaring 1.0, but that might mean we can't deprecate things
until 2.0, and then remove those in 3.0 depending on
what our backwards compatibility guarantees are. What I'd like us to avoid
is to be further entrenched and bogged down in
moving away from those dependencies by declaring a stable API.

Xavier

On Mon, Jun 6, 2022 at 2:45 PM rahul gidwani 
wrote:

Hi Gian, this is great.

For me what is most important is (2) and (4)
Does my current extension work with new releases?
Can I do a rolling upgrade of druid to the next version?

The more things that are versioned the better, but (2) and (4) have been
the things that have been most important to me in the past.

Anyone in the community have any thoughts on this?
Thank you
rahul



On Fri, May 27, 2022 at 11:22 AM Gian Merlino  wrote:

Yeah, I'd say the next one after 24.0 would be 25.0. The idea is really
just to remove the leading zero and thereby communicate the accurate

state

of the project: it has been stable and production-ready for a long

time.

Some people see the leading zero and interpret that as a sign of an
immature or non-production-ready system. So I think this change is

worth

doing and beneficial.

I do think we can do better at communicating compatibility, but IMO
semantic versioning for the whole system isn't the best way to do it.
Semantic versioning is good for libraries, where people need one kind

of

assurance: that they can update to the latest version of the library
without needing to make changes in their program. But Druid is
infrastructure software with many varied senses of compatibility, such

as:


1) Query API: do user queries written for version X return compatible
responses when run against version Y?
2) Extension API: do extensions written for version X run as expected

with

version Y?
3) Storage format: can servers at version X read segments written by
servers at version Y?
4) Intracluster protocol: can a server at version X communicate

properly

with a server at version Y?
5) Server configuration: do server configurations (runtime properties,

jvm

configs) written for version X work as expected for version Y?
6) Ecosystem: does version Y drop support for older versions of

ZooKeeper,

Kafka, Hadoop, etc, which were supported by version X?

In practice we do find good reasons to make such changes in one or more

of

these areas in many of our releases. We try to maximize compatibility
between releases, but it is balanced against the effort to improve the
system while keeping the code maintainable. So if we considered all of
these areas in semantic versioning, we'd be incrementing the major

version

often anyway. The effect would be similar to having a "meaningless"

version

number but with more steps.

IMO a better approach would be to introduce more kinds of version

numbers.

In my experience the two most important kinds of compatibility to most
users are "Query API" and "Extension API". So if we had a "Query API
version" or "Extension API version" then we could semantically version

the

Query and Extension API versions, separately from the main Druid

version.

(Each Druid release would have an associated Extension API version,

and a

list of supported Query API versions that users could choose between

on a

per-query basis.)

Rahul, I wonder what you think about this idea? What kinds of

compatibility

are most important to 

Re: Druid-specific Calcite keywords

[Julian Hyde]

Those sound like good feature ideas. The way to get them into Calcite is to add 
new variables to the template file, provide empty defaults for the variables, 
and then override the variables in Druid’s parser. 

For an example, see the changes to core/src/main/codegen/config.fmpp in 
https://github.com/apache/calcite/commit/e8baf4f4720e07cdce1aa2baabf20042b9353bbb
 
<https://github.com/apache/calcite/commit/e8baf4f4720e07cdce1aa2baabf20042b9353bbb>
 which added support for Postgres-style “::” cast operator in the Babel 
sub-parser but not the core parser.

Julian


> On Nov 5, 2021, at 8:19 AM, Gian Merlino  wrote:
> 
> Thanks for the note! Do you have any pointers to projects that do this in
> sort of a "best practices" way?
> 
> As to specifics: one thing I wanted to explore is adding keywords that do
> some of the same things as our query context parameters, so you don't have
> to set context parameters in order to get the behavior you want. (Sometimes
> people find it difficult to do that due to the abstractions that sit
> between them and the Druid API.) That's stuff like useApproximateTopN,
> which maybe could be "ORDER BY APPROXIMATE " or "ORDER BY 
>  APPROXIMATE".
> 
> I had also wanted to look at adding a PARTITION BY keyword that controls
> partitioning of query result sets.
> 
> On Thu, Nov 4, 2021 at 11:48 PM Julian Hyde  wrote:
> 
>> Some specifics would be useful. But in general, adding a new keyword
>> (reserved or non-reserved) will require changes to the paser. Calcite
>> allows (I won't say it makes it easy) for projects like Druid to
>> create a derived parser by building a parser from the same parser
>> template as Calcite's core parser but with different template
>> variables.
>> 
>> If the keyword is non-reserved, there is an additional grammar rule
>> that transforms the keyword token back into an identifier. It applies
>> in all contexts except the one where the keyword is specifically
>> needed by the grammar.  For example, the non-reserved keyword
>> BERNOULLI can only occur immediately after the keyword TABLESAMPLE. In
>> a location that expects an identifier (e.g. after FROM), BERNOULLI
>> will be converted into an identifier. Thus you can use BERNOULLI as a
>> table name.
>> 
>> Julian
>> 
>> On Thu, Nov 4, 2021 at 2:18 PM Gian Merlino  wrote:
>>> 
>>> Hey Druids,
>>> 
>>> I'm looking into how to add keywords to Druid's SQL dialect, and I wanted
>>> to ask if anyone has enough familiarity with Calcite to point at some
>> info
>>> about how to do that without needing to modify Calcite itself?
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
>> For additional commands, e-mail: dev-h...@druid.apache.org
>> 
>> 

Re: Druid-specific Calcite keywords

[Julian Hyde]

Some specifics would be useful. But in general, adding a new keyword
(reserved or non-reserved) will require changes to the paser. Calcite
allows (I won't say it makes it easy) for projects like Druid to
create a derived parser by building a parser from the same parser
template as Calcite's core parser but with different template
variables.

If the keyword is non-reserved, there is an additional grammar rule
that transforms the keyword token back into an identifier. It applies
in all contexts except the one where the keyword is specifically
needed by the grammar.  For example, the non-reserved keyword
BERNOULLI can only occur immediately after the keyword TABLESAMPLE. In
a location that expects an identifier (e.g. after FROM), BERNOULLI
will be converted into an identifier. Thus you can use BERNOULLI as a
table name.

Julian

On Thu, Nov 4, 2021 at 2:18 PM Gian Merlino  wrote:
>
> Hey Druids,
>
> I'm looking into how to add keywords to Druid's SQL dialect, and I wanted
> to ask if anyone has enough familiarity with Calcite to point at some info
> about how to do that without needing to modify Calcite itself?

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: 欢迎申报｜2021“科创中国”开源创新榜评选活动

[Julian Hyde]

Moderator’s note: as moderator, I allowed this message to dev@ even though it 
is in Chinese and seems to be a mass mailing, because it appears to be on topic 
(namely open source). The automatic translation of the first paragraph is as 
follows:

Dear open source innovation ranking application candidates,
  Hello! Thank you for your support for the 2021 "Technology China" Open 
Source Innovation Ranking! Under the guidance of the China Association for 
Science and Technology, the Science and Technology Communication Center of the 
Chinese Association for Science and Technology, the "Science Innovation China" 
Open Source Innovation Consortium, together with the China Institute of 
Communications, the Software Research Institute of the Chinese Academy of 
Sciences, the Chinese Developer Community (CSDN) and other institutional 
platforms organize the development of "Science The "Creative China" open source 
innovation list selection activity fully mobilized member units of the 
"Technological Innovation China" Open Source Innovation Consortium to 
participate in the review, using open source community databases and 
professional analysis tools for big data selection and recommendation; senior 
experts and scholars were invited to participate in the review. Aims to select 
a group of outstanding open source communities/institutions and products, form 
a demonstration and lead role, promote the prosperity of China's open source 
ecology and the rapid development of the open source industry, and build a 
"consultation, co-construction, sharing, symbiosis, and win-win" Technological 
community. For outstanding open source communities/organizations and products, 
the organizer will organize various forms of publicity reports.


As moderators, we filter out spam and offensive emails, and we prefer emails to 
be in English, but we do not censor emails that are targeted.

Julian


> On Oct 26, 2021, at 5:30 PM, 张凯  wrote:
> 
> 尊敬的开源创新榜评选申报候选人，
> 
>
> 您好！感谢你们对2021“科创中国”开源创新榜评选的支持！在中国科学技术协会的指导下，中国科协科学技术传播中心、“科创中国”开源创新联合体联合中国通信学会、中科院软件研究所、中国开发者社区（CSDN）等机构平台组织开展“科创中国”开源创新榜单评选活动，充分发动“科创中国”开源创新联合体成员单位等参与评审，利用开源社区数据库和专业分析工具进行大数据遴选推荐；邀请资深专家学者参与评审。旨在选树一批优秀的开源社区/机构和产品，形成示范引领带动作用，促进我国开源生态的繁荣和开源产业的快速发展，建设“共商、共建、共享、共生、共赢”的科技共同体。对于优秀的开源社区/机构和产品，举办单位会组织各种形式的宣传报道。
> 
> 
> 
> 榜单设置
> 
> 
> 
> 
> 此次评选包括三个榜单，分别为：2021年度优秀开源产品（50个）、2021年度优秀开源社区（10家）及2021年度突出贡献开源机构（10家），其中2021年度优秀开源产品（50个）来自：人工智能、数据库、操作系统、云原生、大数据、物联网、芯片、区块链、工业互联网、软件研发十大国内开源重点领域。
> 
> 
> 
> 
> 申报条件
> 
> 
> 
> 
> 面向中国开源行业领域，选出一批具有创新性、贡献度和影响力的开源产品、社区、机构。
> 
> （一）开源产品
> 
> 
> 
> 
> 由中国企事业单位、高等院校、科研院所、社团组织或行业个人发起或主导的开源项目。
> 
> 必须有明确的开源许可证项目：采用OSI认证开源许可证。
> 
> 托管在第三方托管平台上。
> 
> 
> 
> 
> （二）开源社区
> 
> 
> 
> 
> 由中国企事业单位、高等院校、科研院所、社团组织或行业个人发起或主导的开源社区。
> 
> 
> 
> 
> （三）开源机构
> 
> 
> 
> 
> 由中国企事业单位、高等院校、科研院所或社团组织发起或主导的开源机构。
> 
> 
> 
> 
> 
> 
> 申报截止：2021年11月15日
> 
> 详细请查看：2021“科创中国”开源创新榜评选官网及申报入口https://cccst.org.cn/subpage/osis
> 
> 
> 
> 有任何问题请通过邮件签名的联系方式及时沟通，预祝申报顺利！
> 
> 以上。
> 
> 
>  ---
> 张凯
> CSDN 开源运营总监
> Kai Zhang
> CSDN OSS Operations Director
> 
> 
> 成就一亿技术人成为技术人成长和交流的家园
> 北京市朝阳区酒仙桥路10号恒通商务园B8座2层 100015
> 2/F, B8B Block, Universal Business Park, 10 Jiuxianqiao Road, Chaoyang 
> District, Beijing, China 100015
> 
> e.zhang...@csdn.net
> 
> M.+86 13691490568
> 
> Wechat: 13691490568
> 
> 
> 
> 
> 

Re: Enabling dependabot in our github repository

[Julian Hyde]

I agree that PRs should not be committed immediately and unconditionally when 
the dependabot finds them. But if we defer, there is a concern that good PRs 
will be forgotten. How about making a particular person (say the release 
manager) or triggering event (say voting on an RC) responsible for checking all 
applicable PRs have been applied?

> On Jun 8, 2021, at 6:58 AM, Gian Merlino  wrote:
> 
> Here's a running list of PRs opened by the dependabot:
> https://github.com/apache/druid/pulls?q=is%3Apr+author%3Aapp%2Fdependabot
> 
> On Mon, Jun 7, 2021 at 12:22 PM Gian Merlino  wrote:
> 
>> There's been some extra discussion this PR:
>> https://github.com/apache/druid/pull/11079
>> 
>> I just +1'ed it, but I wanted to come back here to say that IMO, we should
>> avoid getting in the habit of blindly applying these updates without
>> testing. There's been lots of situations in the past where a
>> harmless-looking dependency upgrade broke something. Sometimes the new
>> dependency version had a regression in it, and sometimes even without
>> regressions it can introduce compatibility problems.
>> 
>> So, I think it'd be good to apply the updates when we're confident in our
>> ability to test them, and add ignores (or tests!) for the rest.
>> 
>> On Thu, Apr 8, 2021 at 12:35 PM Xavier Léauté 
>> wrote:
>> 
>>> Thanks Maytas, I asked in that thread. They seemed concerned about write
>>> access requested by dependabot,
>>> but that should no longer be required as far as I can tell, now that it is
>>> natively integrated into GitHub.
>>> It should only be a matter of adding the config file to the repo, similar
>>> to what we do to automate closing stale issues / PR.
>>> 
>>> On Tue, Apr 6, 2021 at 2:50 PM Maytas Monsereenusorn 
>>> wrote:
>>> 
 I remember seeing someone asked about Dependabot in asfinfra slack
>>> channel
 a few weeks ago. However, asfinfra said they cannot allow it.
 Here is the link:
 https://the-asf.slack.com/archives/CBX4TSBQ8/p1616539376210800
 I think this is the same as Github's dependabot.
 
 Best Regards,
 Maytas
 
 
 On Tue, Apr 6, 2021 at 2:37 PM Xavier Léauté  wrote:
 
> Hi folks, as you know Druid has a lot of dependencies, and keeping up
 with
> the latest versions of everything, whether it relates to fixing CVEs
>>> or
> other improvements is a lot of manual work.
> 
> I suggest we enable Github's dependabot in our repository to keep our
> dependencies up to date. The bot is also helpful in providing a short
> commit log summary to understand changes.
> This might yield a flurry of PRs initially, but we can configure it to
> exclude libraries or version ranges that we know are unsafe for us to
> upgrade to.
> 
> It looks like some other ASF repos have this enabled already (see
> https://github.com/apache/commons-imaging/pull/126), so hopefully
>>> this
> only
> requires filing an INFRA ticket.
> 
> Happy to take care of it if folks are on board.
> 
> Thanks!
> Xavier
> 
 
>>> 
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Push-down of operations for SystemSchema tables

[Julian Hyde]

Jason,

> I'm new to Calcite (and Druid) so if I have some terminology
> incorrect, please point it out.

From a Calcite perspective, I can tell you that your terminology (and ideas) 
seem spot on.

I can’t say whether they make sense in Druid (or are easy to achieve).

Julian


> On May 13, 2021, at 4:21 PM, Jason Koch  wrote:
> 
> Hi all,
> 
> I'm looking to implement push-down for some operations in the
> SystemSchema class, and looking for your input on the best way to
> tackle this.
> 
> With profiling, we have found some UI slowness related to large
> segment counts and task counts. Inspecting the code, it seems that
> much of the data is fully materialized before often being discarded
> [1], which makes it a good opportunity for a pushdown optimization.
> This would make for a more snappy UI experience for segments, tasks
> and so on. It would also I believe address #6827 [2].
> 
> In looking at the code I can see a couple of approaches that might be 
> sensible:
> 
> - Modify the tables to support the linq4j Queryable interface, and
> have all inputs provided in a single pass. This would be a
> (relatively) straightforward way of fixing this specific problem,
> however I am not sure how extensible/reusable this is, and whether I
> have a correct understanding of Queryable.
> 
> - Build up a custom RelNode structure for the sys. tables, along with
> some rules, that could perform the required Logical operations in a
> single pass on the underlying structures. This seems that perhaps some
> of the rules would be more reusable and more in line with existing
> Druid query architecture, however, seems like a more complex solution.
> I think a starting point would be to convert these tables to
> `ProjectableFilterableTable` and then develop a RelOptRule to match a
> LogicalSort+tables and pushing down the required additional sort
> comparators in. On scan(), then, the sort, projection, and filter
> details would all be available to perform a single pass.
> 
> - Any other suggestions or pointers?
> 
> Other thoughts:
> - Are there any common query use cases in these schemas that you think
> we should target as a goal for opt rules? If I know in advance then I
> can use that to guide the work.
> - If complexity goes up a lot, especially for the second option, it
> might be beneficial to move the rules and configuration to a new
> package (org.apache.druid.sql.calcite.schema.sys?).
> - It seems that these queries are currently performed in the Bindable
> convention which would be a little slower than the Enumerable
> convention. Is there any appetite to switch? I did not identify any
> negative consequences from my reading.
> 
> I'm new to Calcite (and Druid) so if I have some terminology
> incorrect, please point it out.
> 
> [1] 
> https://github.com/apache/druid/blob/master/sql/src/main/java/org/apache/druid/sql/calcite/schema/SystemSchema.java#L292-L373
> - for ex, a "select * from sys.segments order by date desc limit 25"
> requires full materialization of all fields of all objects
> (.toString()) in order to correctly sort, at which point we pick first
> 25 rows, and then most data is not needed. Ideally we could perform a
> sort based on underlying timestamp, and only materialize the results
> for the first 25 discovered rows.
> [2] https://github.com/apache/druid/issues/6827
> 
> Thanks
> Jason
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> For additional commands, e-mail: dev-h...@druid.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Regarding sharing the official Druid Twitter account

[Julian Hyde]

Your proposal does have the effect of elevating the Chair above the
PMC, elevating committers and non-committer contributors, and making
the process more difficult for PMC members.

We trust PMC members, and we would like them to tweet more, so let's
not impose any more process on them.

Yes, we should add a process for committers and contributors to tweet
on behalf of the project. It could be a PR, but it could be as simple
as emailing private@ and asking someone to tweet.

Julian



On Mon, May 3, 2021 at 7:06 PM Jihoon Son  wrote:
>
> Thanks Julian,
>
> I'm not suggesting giving more rights to the PMC chair, but rather
> suggesting limiting the number of people who share the password for
> better security and maintenance. On TweetDeck, there is no difference
> between "owners" and "admins" except that owners can manage password
> and login verification settings. Perhaps it doesn't have to be only
> the PMC chair who owns the account, but a small number of PMC members
> who are willing as you suggested.
>
> > Are you suggesting that tweets from PMC members would have to go through 
> > the review process? That would, if anything, tend to reduce the number of 
> > tweets from PMC members.
>
> Yes, that's what I'm suggesting. I don't imagine it would do as the
> review process must be simple enough. We are not going to review the
> content of the tweets. Rather, we should review whether the tweets
> follow the ASF branding policy. Committers will have to review those
> tweets based on the guidelines we will make. Given that we usually
> review ~10 PRs every day, reviewing a few extra tweets will not be a
> big burden to the community. This will also become better as we invite
> more non-developers as committers.
>
>
>
> On Mon, May 3, 2021 at 6:39 PM Julian Hyde  wrote:
> >
> > Any PMC member who is willing could be the owner of the Twitter account. 
> > The PMC chair has a few extra responsibilities but no more 
> > rights/privileges than other PMC members.
> >
> > Are you suggesting that tweets from PMC members would have to go through 
> > the review process? That would, if anything, tend to reduce the number of 
> > tweets from PMC members.
> >
> > Julian
> >
> >
> > > On May 3, 2021, at 6:06 PM, Jihoon Son  wrote:
> > >
> > > Hi all,
> > >
> > > The official Druid Twitter account [1] is currently being managed by a
> > > small group of PMC members who have direct access to the account. This
> > > is causing a couple of issues.
> > >
> > > - Only PMC members can access the account. However, most of the
> > > current PMC members are developers who primarily write codes rather
> > > than tweets.
> > > - Because non-PMC members cannot access the account, non-developers
> > > cannot make contributions using Twitter.
> > > - We currently cannot invite non-developers as PMC members since we
> > > are not acknowledging non-coding work.
> > > - Sharing the password among many people can cause security problems.
> > >
> > > To address these issues, I suggest making a new process for managing
> > > the Twitter account and tweets as below.
> > >
> > > 1. Sharing the account
> > >
> > > The official Druid Twitter account is owned by the PMC chair. Only the
> > > PMC chair can access the account directly. Other contributors,
> > > including PMCs, share the account using TweetDeck [2]. TweetDeck
> > > natively allows sharing the account among different people without
> > > sharing the password. TweetDeck also has embedded roles, i.e.,
> > > "owner", "admin", and "contributor", that have different permissions.
> > > As these roles are similar to the Apache PMC members and committers,
> > > respectively, I suggest
> > >  - The PMC chair invites PMC members as "admins" on TweetDeck.
> > >  - The PMC chair or PMC members invite Committers as "contributors"
> > > on TweetDeck.
> > >  - The "admins" and "contributors" can log in with their "own"
> > > account on TweetDeck and write tweets on behalf of the Druid account.
> > >
> > > See [3] to learn more details of the "owner", "admin", and "contributor" 
> > > roles.
> > >
> > > 2. A new review process for tweets
> > >
> > > I suggest setting up a "review process" for tweets similar to what we
> > > do for the code review. Every tweet should go through the review
> > > process before

Re: Regarding sharing the official Druid Twitter account

[Julian Hyde]

Any PMC member who is willing could be the owner of the Twitter account. The 
PMC chair has a few extra responsibilities but no more rights/privileges than 
other PMC members.

Are you suggesting that tweets from PMC members would have to go through the 
review process? That would, if anything, tend to reduce the number of tweets 
from PMC members.

Julian


> On May 3, 2021, at 6:06 PM, Jihoon Son  wrote:
> 
> Hi all,
> 
> The official Druid Twitter account [1] is currently being managed by a
> small group of PMC members who have direct access to the account. This
> is causing a couple of issues.
> 
> - Only PMC members can access the account. However, most of the
> current PMC members are developers who primarily write codes rather
> than tweets.
> - Because non-PMC members cannot access the account, non-developers
> cannot make contributions using Twitter.
> - We currently cannot invite non-developers as PMC members since we
> are not acknowledging non-coding work.
> - Sharing the password among many people can cause security problems.
> 
> To address these issues, I suggest making a new process for managing
> the Twitter account and tweets as below.
> 
> 1. Sharing the account
> 
> The official Druid Twitter account is owned by the PMC chair. Only the
> PMC chair can access the account directly. Other contributors,
> including PMCs, share the account using TweetDeck [2]. TweetDeck
> natively allows sharing the account among different people without
> sharing the password. TweetDeck also has embedded roles, i.e.,
> "owner", "admin", and "contributor", that have different permissions.
> As these roles are similar to the Apache PMC members and committers,
> respectively, I suggest
>  - The PMC chair invites PMC members as "admins" on TweetDeck.
>  - The PMC chair or PMC members invite Committers as "contributors"
> on TweetDeck.
>  - The "admins" and "contributors" can log in with their "own"
> account on TweetDeck and write tweets on behalf of the Druid account.
> 
> See [3] to learn more details of the "owner", "admin", and "contributor" 
> roles.
> 
> 2. A new review process for tweets
> 
> I suggest setting up a "review process" for tweets similar to what we
> do for the code review. Every tweet should go through the review
> process before being published. We should also provide guidelines for
> what "good" tweets are. Good tweets should promote only the Apache
> Druid brand. The review process should be done based on the
> guidelines. What should be in the review guidelines would be an
> interesting discussion. I would like to discuss it later if we all
> agree on having the review process.
> 
> For the review tool, I suggest using GitHub. What I'm imagining is
> that contributors make Pull Requests for tweets they want to publish.
> We can organize tweets on GitHub in a structured way such as
> organizing based on the date when they are published. I am open to
> other suggestions as well if there are better alternatives.
> 
> This review process can be also useful for acknowledging non-coding
> work. We can collect metrics to identify active contributors and give
> them credits. For example, contributors who write tweets can be
> invited as Apache committers based on their activity.
> 
> 3. Metrics for the Druid account
> 
> Some metrics such as impressions for the Druid account are available
> in [4]. This will be available only for the account owner which will
> be the PMC chair. I'm not sure if there are other services where you
> can see those metrics without accessing the account.
> 
> Love to hear from others.
> Jihoon
> 
> [1] https://twitter.com/druidio
> [2] https://tweetdeck.twitter.com
> [3] https://help.twitter.com/en/using-twitter/tweetdeck-teams
> [4] https://analytics.twitter.com
> 
> PS. I think this Twitter account issue is a small piece of the problem
> of non-coding work being not acknowledged. I will start another thread
> to discuss it.
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> For additional commands, e-mail: dev-h...@druid.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: About maintaining the Helm's Chart of Apache Druid

[Julian Hyde]

This code was developed outside of the ASF, so it’s possible that we need to go 
through the IP clearance process [1]. Can a PMC member please figure out the 
answer to that question, and answer on this list.

Has the Helm project given any indication whether they approve or disapprove of 
the code being copied into Druid?

Does Druid intend to take ownership of the code? I.e. be the one and only copy 
of this code, do necessary maintenance work (especially including security 
fixes) and accepting patches.

Julian

[1] https://incubator.apache.org/ip-clearance/



> On Apr 26, 2021, at 7:33 PM, Benedict Jin  wrote:
> 
> Hi Senlan,
> 
> Thank you very much for your message and support. I have created a PR to do 
> the migration, referring to the experience of Apache Superset. FYI, 
> https://github.com/apache/druid/pull/11163 and 
> https://github.com/apache/superset/tree/master/helm/superset .
> 
> Regards,
> Benedict Jin
> 
> On 2021/04/26 12:59:12, Senlan Yao  wrote: 
>> Thanks @Benedict Jin,
>> It is a good idea to maintain the Druid Helm's Chart in the Apache 
>> repository.
>> Since "https://github.com/helm/charts; repo has been deprecation, we can't 
>> maintain the chart, and the k8s user can't find druid chart package from 
>> "https://artifacthub.io/packages/search?page=1_query_web=druid; any more.
>> If we can maintain chart in the Apache repository, it will solve 
>> https://github.com/apache/druid/issues/5582, ans also the the k8s user can 
>> install Druid from helm chart package. 
>> 
>> On 2021/04/22 01:58:46, Benedict Jin  wrote: 
>>> Hi all,
>>> 
>>> Should we maintain the Helm's Chart in the Apache Druid repository? Now the 
>>> development and maintenance of Helm's Chart has been stalled. It was 
>>> previously maintained by @maver1ck @AWaterColorPen and me (@asdf2014). 
>>> Currently, https://github.com/helm/charts/tree/master/incubator/druid 
>>> cannot be maintained. I recommend that we just copy this directory directly 
>>> to the root directory of https://github.com/apache/druid and maintain it. 
>>> But I'm not so sure whether there is a license issue. What do you think?
>>> 
>>> Regards,
>>> Benedict Jin
>>> 
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
>>> For additional commands, e-mail: dev-h...@druid.apache.org
>>> 
>>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
>> For additional commands, e-mail: dev-h...@druid.apache.org
>> 
>> 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> For additional commands, e-mail: dev-h...@druid.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Removing Druid support for JDK 8 and adding support for JDK 11

[Julian Hyde]

I have no objections to dropping support for JDK 8, but why only support
JDK 11? What would it take to support JDK 11 and all newer JDKs?

There's an analogy with starting and stopping a train. Once you have
stopped a train (settled on a particular JDK release for a number of years)
starting it is hard, because of static friction. So it's easier to just
keep the train rolling.

In Calcite we have a policy to support all JDK versions, and it means that
we have never become dug into one version. Supporting a new version becomes
a small incremental effort. Sometimes we have had to build shims to protect
us against changes, but it was never too onerous in terms of maintenance or
performance. Obviously Druid is a very different project to Calcite, but
"keeping the train rolling" has worked well for us.

Also, remember that Druid is a project, not a product. It's fine if Druid
compiles, and builds, and passes the basic test suite, on JDK 12 even
though you declare that is not 'supported' in production.

Julian


On Mon, Nov 9, 2020 at 2:41 PM Suneet Saldanha 
wrote:

> Hi Druids!
>
> I've been thinking about what it would take for us to remove support for
> JDK 8 in the 0.21 release, and officially add support for JDK 11.
>
> I see that unit tests for jdk11 were added about 1+ year ago in Aug 2019 -
> https://github.com/apache/druid/pull/8400
> And integration tests were added ~ 8 months ago in Feb 2020 -
> https://github.com/apache/druid/pull/9249
>
> Since the jdk11 integration tests were added, I can't remember seeing any
> tests fail for a jdk11 test, but pass for jdk 8 or vice versa. Given that
> it's been about 1 year that these tests have been running, is there any
> opposition to officially removing support for JDK 8 and adding support for
> JDK 11?
>
> There are a few big advantages I can think of for this:
>
>1. Java 8 is end of life, so officially supporting java 11 will allow
>users to run Druid with at least java 11 which is slotted for support till
>September 2023 according to
>https://www.oracle.com/java/technologies/java-se-support-roadmap.html
>2. We can reduce our Travis usage by almost half, and lighten the load
>on ASF's Travis account - today we are the second highest consumer of
>travis resources by project https://infra-reports.apache.org/cistats/
>[image: Screen Shot 2020-11-09 at 2.34.45 PM.png]
>
> The only limiting factor I can think of are full scale performance tests
> that prove there is no major regression in moving from jdk 8 to jdk 11. Are
> there any other issues that should be considered before adding support for
> java 11?
>

Re: Druid not listed in Apache project list by category?

[Julian Hyde]

Have we registered a DOAP (description of a project) file [1] for Druid? That 
might be the problem.

For comparison, Calcite has a DOAP [2] and its project page is much richer [3].

Julian

[1] https://projects.apache.org/doap.html

[2] https://github.com/apache/calcite/blob/master/site/doap_calcite.rdf

[3] https://projects.apache.org/project.html?calcite

On 2020/07/31 18:49:51, Gian Merlino  wrote: 
> That's a good point. We must be missing some metadata. I'm not sure how
> this page works — does anyone else know?
> 
> On Fri, Jul 31, 2020 at 11:49 AM Will Lauer 
> wrote:
> 
> > I was browsing the list of Apache projects today looking for something, and
> > while I was there, I noticed that Druid was missing. While it showed up in
> > the list of all projects (https://projects.apache.org/projects.html), I
> > don't see it listed anywhere in the list of projectes grouped by category (
> > https://projects.apache.org/projects.html?category). I think this second
> > place (projects grouped by category) is the more useful of the two for
> > exploring Apache projects, so I was surprised to see Druid missing from
> > that list. Perhaps some metadata is missing on the druid project to get it
> > to be categorized correctly?
> >
> > Will
> >
> > 
> >
> > Will Lauer
> >
> > Senior Principal Architect, Audience & Advertising Reporting
> > Data Platforms & Systems Engineering
> >
> > M 508 561 6427
> > 1908 S. First St
> > Champaign, IL 61822
> >
> >    
> > 
> > 
> >
> 

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Druid Namespacing Proposal

[Julian Hyde]

Reading the blog post, I was sorry to hear that "performance bottlenecks in the 
SQL parser” prevented you from making SQL your main interface to Druid. Can you 
tell us more about the issues? Is there a bug logged against Calcite or Druid?

In the blog post, thanks for qualifying the Druid, Calcite and HBase trademarks 
with the word “Apache”. Can you please do the same for Hadoop and Kafka?

Julian

> On Mar 9, 2020, at 4:48 PM, Julian Jaffe  wrote:
> 
> Hey all,
> 
> I recently wrote a proposal 
> to add namespacing to Druid segments to allow for more flexible ingestion.
> I'm planning on giving it a little more polish in the next few days, but
> I'd be very interested in any community feedback. We've been running a
> similar system for a year now with great success.
> 
> There have also been a number of people who have reached out after
> attending the meetup we hosted or reading our blog post
> 
> and
> asked for more details about how we implemented some of our customizations,
> and so I'm also hoping that even if there isn't enough community appetite
> for this change in Druid, the proposal can serve as a guide for people to
> follow if they wish to implement this on their own.
> 
> Julian
> 
> 
> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Podling Druid Report Reminder - December 2019

[Julian Hyde]

Druid plans to graduate, so I hope there aren’t 3 items!

My comment on the proposed report is that it doesn’t mention the graduation 
resolution. It should also say that why the resolution was previously shelved - 
IP issues that have now been resolved.

Julian


> On Dec 4, 2019, at 11:25 AM, Justin Mclean  wrote:
> 
> Hi,
> 
> Currently the report will not be accpeted, under "Three most important 
> unfinished issues to address before graduating:" you need to list 3 items. I 
> can also see there is important information that has been omitted from the 
> report.The incubator needs this information so that it can have an accurate 
> report to the board.
> 
> Thanks,
> Justin
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> For additional commands, e-mail: dev-h...@druid.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Dec 2019 podling report draft

[Julian Hyde]

Does Druid intend to put a resolution to graduate on the agenda of the Dec 2019 
board meeting?

If so that should definitely be in the report.

Julian


> On Dec 3, 2019, at 11:49 PM, Jonathan Wei  wrote:
> 
> Updated the section about trademarks on the wiki to the following:
> 
> The PPMC has from time to time reached out to have naming corrected on
> third party sites. VP Brand has approved graduating as Apache Druid.
> 
> On Tue, Dec 3, 2019 at 10:11 PM Jonathan Wei  wrote:
> 
>> I've updated the wiki with the contents posted above, for the new section
>> "Is the PPMC managing the podling's brand / trademarks?", I've put the
>> following:
>> 
>>> There has been recent discussion/activity regarding trademarks on the
>> podling's private mailing list.
>> 
>> On Tue, Dec 3, 2019 at 5:22 PM Jonathan Wei  wrote:
>> 
>>> Hi all,
>>> 
>>> I've written a draft for the upcoming podling report, please comment if
>>> you have any feedback.
>>> 
>>> 
>>> 
>>> Druid is a high-performance, column-oriented, distributed data store.
>>> 
>>> Druid has been incubating since 2018-02-28.
>>> 
>>> Three most important issues to address in the move towards graduation:
>>> 
>>> 1. No major issues
>>> 
>>> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
>>> aware of?
>>> 
>>> None
>>> 
>>> How has the community developed since the last report?
>>> - 2 new committers have been added
>>> - A healthy, constant flow of bug fixes, quality improvements and new
>>> features are still ongoing at https://github.com/apache/incubator-druid.
>>> - Several community meetups have been held.
>>> 
>>> How has the project developed since the last report?
>>> - Since the last report, we have had a total of 187 commits from 34
>>> contributors.
>>> - We have released 1 version, 0.16.0.
>>> - We currently have a vote open for 0.16.1, second release candidate
>>> 
>>> How would you assess the podling's maturity?
>>> Please feel free to add your own commentary.
>>> 
>>> [ ] Initial setup
>>> [ ] Working towards first release
>>> [ ] Community building
>>> [X] Nearing graduation
>>> [ ] Other:
>>> 
>>> Date of last release:
>>> 
>>> - Druid 0.16.0-incubating was released on Sep 24, 2019.
>>> 
>>> When were the last committers or PPMC members elected?
>>> 
>>> The Druid PPMC elected 2 new committers to the project on September 22,
>>> 2019.
>>> 
>>> The previous Sep 2019 report was incorrect here, as the two committers
>>> were not yet officially elected then.
>>> 
>>> Have your mentors been helpful and responsive or are things falling
>>> through the cracks? In the latter case, please list any open issues
>>> that need to be addressed.
>>> 
>>> As always, Julian has been quite helpful.
>>> 
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [VOTE] Release Apache Druid (incubating) 0.16.0 [RC3]

[Julian Hyde]

Full checksum. An attacker can easily generate a binary that matches a given 32 
bit bit (8 digit) hash. That’s why we use SHA-256 or SHA-512.

If it helps, here is a typical Calcite vote email:

http://mail-archives.apache.org/mod_mbox/calcite-dev/201906.mbox/%3cCA+EpF8vwOceAeUjv+DJU=zqrkoqu3dwckwsypqhrj6crw9e...@mail.gmail.com%3e
 
<http://mail-archives.apache.org/mod_mbox/calcite-dev/201906.mbox/%3CCA+EpF8vwOceAeUjv+DJU=zqrkoqu3dwckwsypqhrj6crw9e...@mail.gmail.com%3E>
 




> On Sep 16, 2019, at 1:43 AM, Clint Wylie  wrote:
> 
> Ah, oops, yes indeed they are reversed, my bad! I certainly agree with all
> your points on why it is a good idea, and will update our template after
> the release to make sure we do it in the future. Is it better practice to
> include the full checksum, or would truncated to the first 8 or so
> characters be preferable to play nice with email?
> 
> On Sun, Sep 15, 2019 at 8:34 PM Julian Hyde  wrote:
> 
>> Sorry for my rather terse -1 vote. I had assumed that we had been
>> following the policy for a while, so when I noticed that we were not I
>> assumed it was a mistake by the release manager.
>> 
>> Actually I am not sure whether it is policy, but there's definitely a
>> strong case for including hashes. The point is this: we are voting on
>> artifacts, principally apache-druid-0.16.0-incubating-src.tar.gz.
>> 
>> Suppose we all vote on the current
>> apache-druid-0.16.0-incubating-src.tar.gz, the vote passes, and then
>> someone replaces it with a similar file that contains some bad stuff.
>> How are we to know whether that is the file we voted on?
>> 
>> Putting the file hash in the vote email guarantees that we are all
>> voting on the same set of artifacts, and that set of artifacts is
>> recorded.
>> 
>> I think you reversed the hashes (I got 0c4b71f0 for bin, 1f25c55e for
>> src), but that's close enough, so let's proceed.
>> 
>> 
>> +1 (binding)
>> 
>> Checked hashes, LICENSE, NOTICE, DISCLAIMER; ran RAT; compiled
>> (skipping tests) using JDK 8 on Ubuntu. Checked that src.tar.gz
>> matches git commit.
>> 
>> Julian
>> 
>> 
>> Julian
>> 
>> On Sun, Sep 15, 2019 at 7:24 PM Clint Wylie  wrote:
>>> 
>>>> The vote email must contain the checksums of the artifacts we are
>> voting
>>> on.
>>> 
>>> Apologies, I wasn't aware of this requirement since we haven't put them
>> in
>>> our prior incubating release vote threads and I was just copying the same
>>> basic template I and others have previously used. Out of curiosity is
>> this
>>> a new-ish requirement that I missed, or one we just didn't notice or have
>>> just been turning a blind eye to? Regardless, since we are now
>> maintaining
>>> a 'how to ASF release' guide in the github repo that includes templates
>> for
>>> voting threads,
>>> 
>> https://github.com/apache/incubator-druid/blob/master/distribution/asf-release-process-guide.md#body
>> ,
>>> I'll
>>> be sure to update it, thanks!
>>> 
>>>> No need for a new RC; I change my vote if the release manager sends an
>>>> email with the checksums.
>>> 
>>> If this thread is ok, here they are:
>>> 
>>> artifact checksums
>>> src:
>>> 
>> 0c4b71f077e28d2f4d3bc3f072543374570b98ec6a1918a5e1828e1da7e3871b5efb04070a8bcdbc172a817e43254640ce28a99757984be7d8dd3d607f1d870e
>>> bin:
>>> 
>> 1f25c55e83069cf7071a97c1e0d56732437dbac4ef373ed1ed72b5b618021b74c107269642226e80081354c8da2e92dc26f1541b01072a4720fd6cfe8dc161a8
>>> docker: df9b900d3726ce123a5c054768da1ea08eba6efe635ced5abc3ad72d6c835e2c
>>> 
>>> Thanks!
>>> Clint
>>> 
>>> On Sun, Sep 15, 2019 at 6:22 PM Julian Hyde  wrote:
>>> 
>>>> -1
>>>> 
>>>> The vote email must contain the checksums of the artifacts we are
>> voting
>>>> on.
>>>> 
>>>> No need for a new RC; I change my vote if the release manager sends an
>>>> email with the checksums.
>>>> 
>>>> Julian
>>>> 
>>>> On Fri, Sep 13, 2019 at 11:57 PM Clint Wylie 
>> wrote:
>>>>> 
>>>>> Hi all,
>>>>> 
>>>>> I have created a build for Apache Druid (incubating) 0.16.0, release
>>>>> candidate 3.
>>>>> 
>>>>> Thanks for everyone who has helped contribute to the release! You can
>>>> read
>>>>> the proposed release notes here:
>&g

Re: [VOTE] Release Apache Druid (incubating) 0.16.0 [RC3]

[Julian Hyde]

Sorry for my rather terse -1 vote. I had assumed that we had been
following the policy for a while, so when I noticed that we were not I
assumed it was a mistake by the release manager.

Actually I am not sure whether it is policy, but there's definitely a
strong case for including hashes. The point is this: we are voting on
artifacts, principally apache-druid-0.16.0-incubating-src.tar.gz.

Suppose we all vote on the current
apache-druid-0.16.0-incubating-src.tar.gz, the vote passes, and then
someone replaces it with a similar file that contains some bad stuff.
How are we to know whether that is the file we voted on?

Putting the file hash in the vote email guarantees that we are all
voting on the same set of artifacts, and that set of artifacts is
recorded.

I think you reversed the hashes (I got 0c4b71f0 for bin, 1f25c55e for
src), but that's close enough, so let's proceed.


+1 (binding)

Checked hashes, LICENSE, NOTICE, DISCLAIMER; ran RAT; compiled
(skipping tests) using JDK 8 on Ubuntu. Checked that src.tar.gz
matches git commit.

Julian


Julian

On Sun, Sep 15, 2019 at 7:24 PM Clint Wylie  wrote:
>
> > The vote email must contain the checksums of the artifacts we are voting
> on.
>
> Apologies, I wasn't aware of this requirement since we haven't put them in
> our prior incubating release vote threads and I was just copying the same
> basic template I and others have previously used. Out of curiosity is this
> a new-ish requirement that I missed, or one we just didn't notice or have
> just been turning a blind eye to? Regardless, since we are now maintaining
> a 'how to ASF release' guide in the github repo that includes templates for
> voting threads,
> https://github.com/apache/incubator-druid/blob/master/distribution/asf-release-process-guide.md#body,
> I'll
> be sure to update it, thanks!
>
> > No need for a new RC; I change my vote if the release manager sends an
> > email with the checksums.
>
> If this thread is ok, here they are:
>
> artifact checksums
> src:
> 0c4b71f077e28d2f4d3bc3f072543374570b98ec6a1918a5e1828e1da7e3871b5efb04070a8bcdbc172a817e43254640ce28a99757984be7d8dd3d607f1d870e
> bin:
> 1f25c55e83069cf7071a97c1e0d56732437dbac4ef373ed1ed72b5b618021b74c107269642226e80081354c8da2e92dc26f1541b01072a4720fd6cfe8dc161a8
> docker: df9b900d3726ce123a5c054768da1ea08eba6efe635ced5abc3ad72d6c835e2c
>
> Thanks!
> Clint
>
> On Sun, Sep 15, 2019 at 6:22 PM Julian Hyde  wrote:
>
> > -1
> >
> > The vote email must contain the checksums of the artifacts we are voting
> > on.
> >
> > No need for a new RC; I change my vote if the release manager sends an
> > email with the checksums.
> >
> > Julian
> >
> > On Fri, Sep 13, 2019 at 11:57 PM Clint Wylie  wrote:
> > >
> > > Hi all,
> > >
> > > I have created a build for Apache Druid (incubating) 0.16.0, release
> > > candidate 3.
> > >
> > > Thanks for everyone who has helped contribute to the release! You can
> > read
> > > the proposed release notes here:
> > > https://github.com/apache/incubator-druid/issues/8369
> > >
> > > The release candidate has been tagged in GitHub as
> > > druid-0.16.0-incubating-rc3 (54d29e438a4df34d75e2385af6cefd1092c4ebb3),
> > > available here:
> > >
> > https://github.com/apache/incubator-druid/releases/tag/druid-0.16.0-incubating-rc3
> > >
> > > The artifacts to be voted on are located here:
> > >
> > https://dist.apache.org/repos/dist/dev/incubator/druid/0.16.0-incubating-rc3/
> > >
> > > Staged druid.apache.org website documentation is available here:
> > > https://druid.staged.apache.org/docs/0.16.0-incubating/design/index.html
> > >
> > > A Docker image containing the binary of the release candidate can be
> > > retrieved via:
> > > docker pull apache/incubator-druid:0.16.0-incubating-rc3
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/cwylie.asc
> > >
> > > This key and the key of other committers can also be found in the
> > project's
> > > KEYS file here:
> > > https://dist.apache.org/repos/dist/release/incubator/druid/KEYS
> > >
> > > (If you are a committer, please feel free to add your own key to that
> > file
> > > by following the instructions in the file's header.)
> > >
> > >
> > > Verify checksums:
> > > diff <(shasum -a512 apache-druid-0.16.0-incubating-src.tar.gz | \
> > > cut -d ' ' -f1) \
> > > <(cat apache-druid-0.16.0-incubating-src.tar.gz.sha512 ; echo)
> > >
> >

Re: [VOTE] Release Apache Druid (incubating) 0.16.0 [RC3]

[Julian Hyde]

-1

The vote email must contain the checksums of the artifacts we are voting on.

No need for a new RC; I change my vote if the release manager sends an
email with the checksums.

Julian

On Fri, Sep 13, 2019 at 11:57 PM Clint Wylie  wrote:
>
> Hi all,
>
> I have created a build for Apache Druid (incubating) 0.16.0, release
> candidate 3.
>
> Thanks for everyone who has helped contribute to the release! You can read
> the proposed release notes here:
> https://github.com/apache/incubator-druid/issues/8369
>
> The release candidate has been tagged in GitHub as
> druid-0.16.0-incubating-rc3 (54d29e438a4df34d75e2385af6cefd1092c4ebb3),
> available here:
> https://github.com/apache/incubator-druid/releases/tag/druid-0.16.0-incubating-rc3
>
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/incubator/druid/0.16.0-incubating-rc3/
>
> Staged druid.apache.org website documentation is available here:
> https://druid.staged.apache.org/docs/0.16.0-incubating/design/index.html
>
> A Docker image containing the binary of the release candidate can be
> retrieved via:
> docker pull apache/incubator-druid:0.16.0-incubating-rc3
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/cwylie.asc
>
> This key and the key of other committers can also be found in the project's
> KEYS file here:
> https://dist.apache.org/repos/dist/release/incubator/druid/KEYS
>
> (If you are a committer, please feel free to add your own key to that file
> by following the instructions in the file's header.)
>
>
> Verify checksums:
> diff <(shasum -a512 apache-druid-0.16.0-incubating-src.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-0.16.0-incubating-src.tar.gz.sha512 ; echo)
>
> diff <(shasum -a512 apache-druid-0.16.0-incubating-bin.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-0.16.0-incubating-bin.tar.gz.sha512 ; echo)
>
> Verify signatures:
> gpg --verify apache-druid-0.16.0-incubating-src.tar.gz.asc \
> apache-druid-0.16.0-incubating-src.tar.gz
>
> gpg --verify apache-druid-0.16.0-incubating-bin.tar.gz.asc \
> apache-druid-0.16.0-incubating-bin.tar.gz
>
> Please review the proposed artifacts and vote. Note that Apache has
> specific requirements that must be met before +1 binding votes can be cast
> by PMC members. Please refer to the policy at
> http://www.apache.org/legal/release-policy.html#policy for more details.
>
> As part of the validation process, the release artifacts can be generated
> from source by running:
> mvn clean install -Papache-release,dist -Dgpg.skip
>
> The RAT license check can be run from source by:
> mvn apache-rat:check -Prat
>
> This vote will be open for at least 72 hours. The vote will pass if a
> majority of at least three +1 PMC votes are cast.
>
> Once the vote has passed, the second stage vote will be called on the
> Apache Incubator mailing list to get approval from the Incubator PMC.
>
> [ ] +1 Release this package as Apache Druid (incubating) 0.16.0
> [ ] 0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
> Thanks!
>
> Apache Druid (incubating) is an effort undergoing incubation at The Apache
> Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is
> required of all newly accepted projects until a further review indicates
> that the infrastructure, communications, and decision making process have
> stabilized in a manner consistent with other successful ASF projects. While
> incubation status is not necessarily a reflection of the completeness or
> stability of the code, it does indicate that the project has yet to be
> fully endorsed by the ASF.

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: September 2019 Podling Report (Draft)

[Julian Hyde]

Oops! We need to change our invite process so that we do not announce 
committers before they have accepted. Sorry about that, Fokko.

Julian


> On Sep 5, 2019, at 2:54 AM, Driesprong, Fokko  wrote:
> 
> Not to make things awkward, but I think it is already out in the open on
> the mailing list:
> https://lists.apache.org/thread.html/b3a158e04b7ed2f4f958b6e86554e4a6794eca193736f1cf774eb8a9@%3Cgeneral.incubator.apache.org%3E
> 
> Process wise, I haven't gotten any formal invite, so this makes it
> difficult to accept or decline. We might want to formalize this process a
> bit more. The process is described here
> <https://community.apache.org/newcommitter.html#new-committer-process>.
> Apart from that, I feel as well that Druid is nearing graduation.
> 
> Cheers, Fokko Driesprong
> 
> Op wo 4 sep. 2019 om 21:58 schreef Julian Hyde :
> 
>> It’s probably better to wait until they have accepted. It would be
>> embarrassing if they declined.
>> 
>> As every CFO knows, it’s good to defer the reporting of revenue… you have
>> something positive in the bag to add to the next report.
>> 
>> Julian
>> 
>> 
>>> On Sep 3, 2019, at 8:40 PM, David Lim  wrote:
>>> 
>>> Thanks Jon, looks good.
>>> 
>>> I think it's okay to mention that we have voted on and extended
>> invitations
>>> to two additional committers and are awaiting their acceptance (without
>>> mentioning them by name).
>>> 
>>> On Tue, Sep 3, 2019 at 5:13 PM Jonathan Wei  wrote:
>>> 
>>>> Hi all,
>>>> 
>>>> I've prepared the following podling report draft, and I am planning to
>>>> submit it before the deadline tomorrow.
>>>> 
>>>> Please take a look and let me know if you have any feedback.
>>>> 
>>>> --
>>>> 
>>>> Druid is a high-performance, column-oriented, distributed data store.
>>>> 
>>>> Druid has been incubating since 2018-02-28.
>>>> 
>>>> Three most important issues to address in the move towards graduation:
>>>> 
>>>> 1. Resolve trademark issues
>>>> 
>>>> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
>>>> aware of?
>>>> 
>>>> We will need to resolve the trademark issues before graduation, which
>> the
>>>> board has been made aware of privately. We are grateful for the board's
>>>> support, and we will work with trademarks@ to resolve the issues.
>>>> 
>>>> How has the community developed since the last report?
>>>> 
>>>> - A healthy, constant flow of bug fixes, quality improvements and new
>>>> features are still ongoing at https://github.com/apache/incubator-druid
>> .
>>>> - Two Druid community meetups have been scheduled on 9/17 and 9/19.
>>>> 
>>>> How has the project developed since the last report?
>>>> 
>>>> - Since the last report, we have had a total of 293 commits from 30
>>>> contributors.
>>>> - The project website has been migrated to Apache infrastructure.
>>>> - We have released 2 versions, 0.15.0 and 0.15.1.
>>>> - We have code frozen the upcoming 0.16.0 release branch and are
>> preparing
>>>> the first release candidate.
>>>> 
>>>> How would you assess the podling's maturity?
>>>> Please feel free to add your own commentary.
>>>> 
>>>> [ ] Initial setup
>>>> [ ] Working towards first release
>>>> [ ] Community building
>>>> [X] Nearing graduation
>>>> [ ] Other:
>>>> 
>>>> Date of last release:
>>>> 
>>>> - Druid 0.15.1-incubating was released on Aug 15, 2019.
>>>> 
>>>> When were the last committers or PPMC members elected?
>>>> 
>>>> The Druid PPMC elected 10 new committers to the project on November 20,
>>>> 2018.
>>>> 
>>>> Have your mentors been helpful and responsive or are things falling
>>>> through the cracks? In the latter case, please list any open issues
>>>> that need to be addressed.
>>>> 
>>>> Julian has continued to be very helpful.
>>>> 
>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
>> For additional commands, e-mail: dev-h...@druid.apache.org
>> 
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: September 2019 Podling Report (Draft)

[Julian Hyde]

It’s probably better to wait until they have accepted. It would be embarrassing 
if they declined.

As every CFO knows, it’s good to defer the reporting of revenue… you have 
something positive in the bag to add to the next report. 

Julian


> On Sep 3, 2019, at 8:40 PM, David Lim  wrote:
> 
> Thanks Jon, looks good.
> 
> I think it's okay to mention that we have voted on and extended invitations
> to two additional committers and are awaiting their acceptance (without
> mentioning them by name).
> 
> On Tue, Sep 3, 2019 at 5:13 PM Jonathan Wei  wrote:
> 
>> Hi all,
>> 
>> I've prepared the following podling report draft, and I am planning to
>> submit it before the deadline tomorrow.
>> 
>> Please take a look and let me know if you have any feedback.
>> 
>> --
>> 
>> Druid is a high-performance, column-oriented, distributed data store.
>> 
>> Druid has been incubating since 2018-02-28.
>> 
>> Three most important issues to address in the move towards graduation:
>> 
>> 1. Resolve trademark issues
>> 
>> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
>> aware of?
>> 
>> We will need to resolve the trademark issues before graduation, which the
>> board has been made aware of privately. We are grateful for the board's
>> support, and we will work with trademarks@ to resolve the issues.
>> 
>> How has the community developed since the last report?
>> 
>> - A healthy, constant flow of bug fixes, quality improvements and new
>> features are still ongoing at https://github.com/apache/incubator-druid.
>> - Two Druid community meetups have been scheduled on 9/17 and 9/19.
>> 
>> How has the project developed since the last report?
>> 
>> - Since the last report, we have had a total of 293 commits from 30
>> contributors.
>> - The project website has been migrated to Apache infrastructure.
>> - We have released 2 versions, 0.15.0 and 0.15.1.
>> - We have code frozen the upcoming 0.16.0 release branch and are preparing
>> the first release candidate.
>> 
>> How would you assess the podling's maturity?
>> Please feel free to add your own commentary.
>> 
>> [ ] Initial setup
>> [ ] Working towards first release
>> [ ] Community building
>> [X] Nearing graduation
>> [ ] Other:
>> 
>> Date of last release:
>> 
>> - Druid 0.15.1-incubating was released on Aug 15, 2019.
>> 
>> When were the last committers or PPMC members elected?
>> 
>> The Druid PPMC elected 10 new committers to the project on November 20,
>> 2018.
>> 
>> Have your mentors been helpful and responsive or are things falling
>> through the cracks? In the latter case, please list any open issues
>> that need to be addressed.
>> 
>> Julian has continued to be very helpful.
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: September 2019 Podling Report (Draft)

[Julian Hyde]

Looks good. Thanks for the shout-out. :)

> On Sep 3, 2019, at 4:13 PM, Jonathan Wei  wrote:
> 
> Hi all,
> 
> I've prepared the following podling report draft, and I am planning to
> submit it before the deadline tomorrow.
> 
> Please take a look and let me know if you have any feedback.
> 
> --
> 
> Druid is a high-performance, column-oriented, distributed data store.
> 
> Druid has been incubating since 2018-02-28.
> 
> Three most important issues to address in the move towards graduation:
> 
> 1. Resolve trademark issues
> 
> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
> aware of?
> 
> We will need to resolve the trademark issues before graduation, which the
> board has been made aware of privately. We are grateful for the board's
> support, and we will work with trademarks@ to resolve the issues.
> 
> How has the community developed since the last report?
> 
> - A healthy, constant flow of bug fixes, quality improvements and new
> features are still ongoing at https://github.com/apache/incubator-druid.
> - Two Druid community meetups have been scheduled on 9/17 and 9/19.
> 
> How has the project developed since the last report?
> 
> - Since the last report, we have had a total of 293 commits from 30
> contributors.
> - The project website has been migrated to Apache infrastructure.
> - We have released 2 versions, 0.15.0 and 0.15.1.
> - We have code frozen the upcoming 0.16.0 release branch and are preparing
> the first release candidate.
> 
> How would you assess the podling's maturity?
> Please feel free to add your own commentary.
> 
> [ ] Initial setup
> [ ] Working towards first release
> [ ] Community building
> [X] Nearing graduation
> [ ] Other:
> 
> Date of last release:
> 
> - Druid 0.15.1-incubating was released on Aug 15, 2019.
> 
> When were the last committers or PPMC members elected?
> 
> The Druid PPMC elected 10 new committers to the project on November 20,
> 2018.
> 
> Have your mentors been helpful and responsive or are things falling
> through the cracks? In the latter case, please list any open issues
> that need to be addressed.
> 
> Julian has continued to be very helpful.


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Graduation

[Julian Hyde]

> On Jun 26, 2019, at 10:23 PM, Gian Merlino  wrote:
> Speaking for myself I would be happy to have any of you (Julian,
> Jun, Taylor) if you're interested. It sounds like you might be, Julian,
> which if so is great.

Yes, I would be happy to serve on the PMC, if that is what the community wants.

Julian


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Stalebot for issues

[Julian Hyde]

I claim that features have a different lifecycle to bugs. There may not be a 
strong case for doing a particular feature today, but in a year, there may be a 
greater demand. If a bugs are not fixed, their importance usually declines over 
time.

Are people able to vote for features in GitHub issues? Are they able to vote to 
them if they are closed? I think it’s useful for people to continue to chime in 
on features, and eventually build consensus about what should be built.

Perhaps a label “not on roadmap” on a feature is all that is necessary to 
manage people’s expectations. I agree that closing bugs makes sense because 
(for some reason!) users assume that developers intend to fix every single bug.

Julian



> On Jun 25, 2019, at 8:55 AM, Gian Merlino  wrote:
> 
> To me it makes sense to close even "Feature" ideas that have no
> constituency, since it is just clutter to have a bunch of feature ideas
> around that nobody is actively pushing. However this starts to remind me of
> the Wikipedia "deletionism vs. inclusionism" debate:
> https://en.wikipedia.org/wiki/Deletionism_and_inclusionism_in_Wikipedia which
> simmers even to this day.
> 
> "Performance" and "Refactoring" makes more sense to consider evergreen,
> although there may still be some benefit in stalebotting them anyway, since
> the bot bumps things periodically to encourage reconsideration. Without
> that, some perfectly good ideas might be totally forgotten, open forever
> but never looked at. I'm ok either way on these two labels, I suppose.
> 
> 
> On Mon, Jun 24, 2019 at 11:36 AM Roman Leventov 
> wrote:
> 
>> I wrote previous messages in this thread before I've discovered that the
>> stalebot send me more than 100 messages. (That shouldn't be surprising
>> since I'm the author of 174 open issues in Druid:
>> 
>> https://github.com/apache/incubator-druid/search?p=1=is%3Aopen+author%3Aleventov+is%3Aissue=Issues
>> ).
>> As an experiment, I'll try to go over all notifications and post here how
>> many of them can actually be closed now.
>> 
>> On the other hand, I've realized that a big and a legitimate case for
>> stalebot closing issues are the issues of "Problem report" kind (
>> 
>> https://github.com/apache/incubator-druid/issues/new?assignees==problem_report.md=
>> ).
>> The reasoning is that
>> - As time passes, the issue may be fixed in the newer Druid versions.
>> - The report may be irreproducible or hardly reproducible, especially if
>> the Druid version used is unspecified or there is otherwise too little
>> information in the issue.
>> 
>> "Flaky test" issues are somewhat similar, too.
>> 
>> Jon once suggested to add a "Problem report" tag:
>> 
>> https://lists.apache.org/thread.html/61068635cc338dd0da6d43bfca16adf9ccdd3d61e267b598124ca3ad@%3Cdev.druid.apache.org%3E
>> .
>> We could revive this idea in the form of "Uncategorized problem report". It
>> would be a committer's duty to reassign either to "bug", "invalid", or
>> "won't fix" upon verification.
>> 
>> Then, I suggest that the stalebot only watches "Uncategorized problem
>> report", "Flaky test", and issues without any tags (that would sweep all
>> old issues which are essentially uncategorized problem reports, as well as
>> new issues when the authors use the "Other" button instead of "Problem
>> report" button).
>> 
>> I think that the majority of "Feature/Change request", "Feature",
>> "Refactoring", "Performance", etc. issues would be "evergreen", so it's
>> more practically to close them only by occasion when someone visits these
>> old issues.
>> 
>> On Fri, 21 Jun 2019 at 21:57, Gian Merlino  wrote:
>> 
>>> The core idea is that it's good for someone or something to go through
>> old
>>> issues periodically and clean up anything that's no longer relevant,
>> since
>>> having a bunch of irrelevant issues lying around is poor project hygiene.
>>> No human is really volunteering for this, hence the bot. The fact that it
>>> bumps things before closing them is useful too, since it sort of forces
>>> periodic re-consideration of relevancy.
>>> 
> The effect should be giving us an
> open issues list that more accurately respects the issues that people
>>> in
> the community feel are important.
> 
 The list would still be too long to be comprehensible or digestible for
 anybody, nor that anyone is expected to go through the full list at any
 time.
>>> 
>>> Maybe so, but I would really hope that with a shorter list, it could
>>> potentially be more digestible. At least wouldn't have a large amount of
>>> irrelevant issues. If you look through our older issues, so many of them
>>> are irrelevant or questionably relevant to today's Druid. This is fine if
>>> nobody ever looks at them, which is probably the case for regular
>>> contributors, who I assume mostly interact with issues through
>>> notifications. But it can be misleading to those that don't pay attention
>>> to the project every day.
>>> 
 Personally, I open many issues

Re: Graduation

[Julian Hyde]

Instead of just the initial committers, how about making all committers members 
of the PMC?

Some of the names on your list have not been active since incubation, so maybe 
give them a chance to decline membership of the PMC?

As you can tell, I skew towards recent engagement rather than merit earned in 
the project’s pre-Apache history.

Most projects invite their mentors to join the PMC, as they are PPMC members. 
Also they are ASF members, and it is useful to have one or two members fighting 
your corner.

There is very little harm to having a large PMC from a diversity of companies, 
and much to be gained.

Julian


For reference, here is Gian’s proposed list:

* Charles Allen
* David Lim
* Eric Tschetter
* Fangjin Yang
* Gian Merlino
* Himanshu Gupta
* Jihoon Son
* Jonathan Wei
* Kurt Young
* Lijin Bin
* Maxime Beauchemin
* Nishant Bangarwa
* Parag Jain
* Roman Leventov
* Slim Bouguerra
* Xavier Léauté

And here is the list of current committers:

* Benedict Jin
* Lijin Bin
* Slim Bouguerra
* Eric Tschetter
* Charles Allen
* Clint Wylie
* David Lim
* Dylan Wylie
* Egor Ryashin
* Fangjin Yang
* Dayue Gao
* Gian Merlino
* Himanshu Gupta
* Julian Hyde
* Jihoon Son
* Jonathan Wei
* Jun Rao
* Kaijian Ding
* Kurt Young
* Roman Leventov
* Maxime Beauchemin
* Niketh Sabbineni
* Nishant Bangarwa
* Parag Jain
* P. Taylor Goetz
* Mingming Qiu
* Surekha Saharan
* Xavier Léauté
* Xinyu Zhang


Julian


> On Jun 25, 2019, at 8:53 AM, Fangjin Yang  wrote:
> 
> +1 to start the formal vote
> 
> On Tue, Jun 25, 2019 at 8:45 AM Gian Merlino  wrote:
> 
>> Hey all,
>> 
>> I've prepared a draft resolution based on the above discussion:
>> https://gist.github.com/gianm/373153126ecab63cf2ea6596e1468f46
>> 
>> And also self assessed us against the maturity model:
>> https://gist.github.com/gianm/33ae4d61ebaa8b8714d9e2f51a11e7f7
>> 
>> In this self-assessment, three things that come up as areas for improvement
>> are better documenting our release process (RE50), publicly posting a place
>> where security issues can be confidentially reported (QU30), and posting a
>> list of PMC members online (CS10), perhaps once we have a proper PMC.
>> 
>> Could people please review the above documents? Assuming they look in order
>> then I will start a formal VOTE.
>> 
>> On Fri, Jun 21, 2019 at 11:59 AM Gian Merlino  wrote:
>> 
>>> I've updated the status page:
>>> http://incubator.apache.org/projects/druid.html. I think it's pretty
>>> accurate now. I will start going through the other stuff on the
>> graduation
>>> list too. Is anyone interested in self-assessing us against the maturity
>>> model? Like Julian said it isn't required, but, maybe nice to have
>> anyway.
>>> 
>>> On Wed, Jun 19, 2019 at 9:34 PM Slim Bouguerra >> 
>>> wrote:
>>> 
>>>> Thanks Gian for taking the lead on this!
>>>> 
>>>> 
>>>> On Tue, Jun 18, 2019 at 11:50 PM Himanshu  wrote:
>>>> 
>>>>> SGTM
>>>>> 
>>>>> On Tue, Jun 18, 2019 at 6:16 PM Gian Merlino  wrote:
>>>>> 
>>>>>> I just read through that guide. It seems there is still some work to
>>>> do,
>>>>>> including updating our status file, etc. It doesn't look like much
>>>> and I
>>>>>> should be able to get those things done today & tomorrow.
>>>>>> 
>>>>>> PMC chair seems like mostly a secretarial job, is that right? Based
>>>> on:
>>>>>> https://www.apache.org/dev/pmc.html#chair the responsibilities are
>>>> stuff
>>>>>> like writing reports for the board, monitoring lists for
>>>> communications
>>>>>> from the board, and recordkeeping related to new committers &
>>>> committer
>>>>>> roster. I can volunteer for this if that works for people, & also
>>>> write
>>>>> up
>>>>>> a charter. I checked out a few examples, they seem to be pretty
>>>>>> straightforward.
>>>>>> 
>>>>>> As to who will be on the PMC, I figure same as the current PPMC is a
>>>>>> reasonable place to start.
>>>>>> 
>>>>>> On Thu, Jun 13, 2019 at 5:30 PM Julian Hyde 
>> wrote:
>>>>>> 
>>>>>>> Before starting a vote, read through
>>>>>>> https://incubator.apache.org/guides/graduation.html <
>>>>>>> https://incubator.apache.org/guides/graduation.html>. E.g. you
>>>> need a
>

Re: [VOTE] Release Apache Druid (incubating) 0.15.0 [RC2]

[Julian Hyde]

+1 (binding)

Checked hashes and signatures, LICENSE, NOTICE, DISCLAIMER, README, ran RAT, 
compiled on Linux/JDK 8.

Compared to git, there is one generated file in the src distribution, 
sql-function-doc.ts. Not sure whether this is intended to be in the src 
distribution.

Julian



> On Jun 17, 2019, at 1:41 PM, David Lim  wrote:
> 
> +1 (binding)
> 
> src package:
> - verified signature/hash
> - compared source distribution contents against git tag (44c9323)
> - LICENSE, NOTICE, and DISCLAIMER are present
> - unit tests passed
> - RAT check passed
> - built binary distribution
> - ran quickstart batch tutorial
> 
> bin package:
> - verified signature/hash
> - verified META-INF/MANIFEST.MF:Build-Revision tag in JAR files matches
> source distribution git.version:Build-Revision (44c9323)
> - LICENSE, NOTICE, and DISCLAIMER are present
> - ran quickstart batch tutorial
> 
> On Wed, Jun 12, 2019 at 6:13 PM Jihoon Son  wrote:
> 
>> Hi all,
>> 
>> I have created a build for Apache Druid (incubating) 0.15.0, release
>> candidate 2.
>> 
>> Thanks to everyone who has contributed to this release! You can read the
>> proposed release notes here:
>> https://github.com/apache/incubator-druid/issues/7854
>> 
>> The release candidate has been tagged in GitHub as
>> druid-0.15.0-incubating-rc2 (44c9323), available here:
>> 
>> https://github.com/apache/incubator-druid/releases/tag/druid-0.15.0-incubating-rc2
>> 
>> The artifacts to be voted on are located here:
>> 
>> https://dist.apache.org/repos/dist/dev/incubator/druid/0.15.0-incubating-rc2/
>> 
>> Release artifacts are signed with the following key:
>> https://people.apache.org/keys/committer/jihoonson.asc. This key and the
>> key
>> of other committers can also be found in the project's KEYS file here:
>> 
>> https://dist.apache.org/repos/dist/release/incubator/druid/KEYS
>> 
>> (If you are a committer, please feel free to add your own key to that file
>> by following the instructions in the file's header.)
>> 
>> Please review the proposed artifacts and vote. Please note that Apache has
>> specific
>> requirements that must be met before +1 binding votes can be cast by PMC
>> members. Please refer to the policy at
>> http://www.apache.org/legal/release-policy.html#policy for more details.
>> 
>> As part of the validation process, the release artifacts can be generated
>> from source by running: mvn clean install -Papache-release -Dtar
>> 
>> This vote will be open for at least 72 hours but likely more, in following
>> the Druid community's practice of deploying the RC to larger clusters and
>> allowing it to soak for a period of time to flush out any remaining issues.
>> The vote will pass if a majority of at least three +1 PMC votes are cast.
>> 
>> Once the vote has passed, the second stage vote will be called on the
>> Apache Incubator mailing list to get approval from the Incubator PMC.
>> 
>> [ ] +1 Release this package as Apache Druid (incubating) 0.15.0
>> [ ]  0 I don't feel strongly about it, but I'm okay with the release
>> [ ] -1 Do not release this package because...
>> 
>> Thanks!
>> Jihoon
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [ANNOUNCE] Apache Druid (incubating) 0.14.1 release

[Julian Hyde]

No reason to wait til next release; you can send to
annou...@apache.org this release.

On Sun, May 12, 2019 at 11:13 PM Clint Wylie  wrote:
>
> Thanks Julian, I'll be sure to get these things fixed up before the next
> release!
>
> On Sat, May 11, 2019 at 1:30 PM Julian Hyde  wrote:
>
> > Thanks for being release manager and making the announcement, Clint.
> >
> > Most projects announce by sending to annou...@apache.org. This reaches
> > a wider audience (I know several journalists and analysts who follow
> > it). So it is useful to have an explanatory paragraph "Apache Druid
> > (incubating) is ...".
> >
> > This search gives some examples:
> >
> > https://lists.apache.org/list.html?annou...@apache.org:gte=1d:incubator%20released
> > .
> >
> > Apache policy is for only the latest release (of each active dev
> > branch) to be on the mirrors. Old releases are served from the
> > archives. So, going forward, plan to delete an old release from
> > dist/release each time you make a release. Also fix up the history
> > page to point to the artifacts in the archive.
> >
> > Julian
> >
> > On Thu, May 9, 2019 at 1:11 AM Clint Wylie  wrote:
> > >
> > > Druid 0.14.1-incubating is a small patch release that includes a handful
> > of
> > > bug and documentation fixes.
> > >
> > > Source and binary distributions can be downloaded from:
> > > https://druid.apache.org/downloads.html
> > >
> > > Important notice:
> > > This release fixes an issue with druid-datasketches extension with
> > quantile
> > > sketches, but introduces another one with theta sketches that was
> > confirmed
> > > after the release was finalized. If you utilize theta sketches, we
> > > recommend not upgrading to this release. This will be fixed in the next
> > > release of Druid.
> > >
> > > See the release notes for additional details:
> > >
> > https://github.com/apache/incubator-druid/releases/tag/druid-0.14.1-incubating
> > >
> > > Thanks to everyone who contributed to this release!
> > >
> > > 
> > >
> > > Disclaimer: Apache Druid is an effort undergoing incubation at The Apache
> > > Software Foundation (ASF), sponsored by the Apache Incubator. Incubation
> > is
> > > required of all newly accepted projects until a further review indicates
> > > that the infrastructure, communications, and decision making process have
> > > stabilized in a manner consistent with other successful ASF projects.
> > While
> > > incubation status is not necessarily a reflection of the completeness or
> > > stability of the code, it does indicate that the project has yet to be
> > > fully endorsed by the ASF.
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> > For additional commands, e-mail: dev-h...@druid.apache.org
> >
> >

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [ANNOUNCE] Apache Druid (incubating) 0.14.1 release

[Julian Hyde]

Thanks for being release manager and making the announcement, Clint.

Most projects announce by sending to annou...@apache.org. This reaches
a wider audience (I know several journalists and analysts who follow
it). So it is useful to have an explanatory paragraph "Apache Druid
(incubating) is ...".

This search gives some examples:
https://lists.apache.org/list.html?annou...@apache.org:gte=1d:incubator%20released.

Apache policy is for only the latest release (of each active dev
branch) to be on the mirrors. Old releases are served from the
archives. So, going forward, plan to delete an old release from
dist/release each time you make a release. Also fix up the history
page to point to the artifacts in the archive.

Julian

On Thu, May 9, 2019 at 1:11 AM Clint Wylie  wrote:
>
> Druid 0.14.1-incubating is a small patch release that includes a handful of
> bug and documentation fixes.
>
> Source and binary distributions can be downloaded from:
> https://druid.apache.org/downloads.html
>
> Important notice:
> This release fixes an issue with druid-datasketches extension with quantile
> sketches, but introduces another one with theta sketches that was confirmed
> after the release was finalized. If you utilize theta sketches, we
> recommend not upgrading to this release. This will be fixed in the next
> release of Druid.
>
> See the release notes for additional details:
> https://github.com/apache/incubator-druid/releases/tag/druid-0.14.1-incubating
>
> Thanks to everyone who contributed to this release!
>
> 
>
> Disclaimer: Apache Druid is an effort undergoing incubation at The Apache
> Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is
> required of all newly accepted projects until a further review indicates
> that the infrastructure, communications, and decision making process have
> stabilized in a manner consistent with other successful ASF projects. While
> incubation status is not necessarily a reflection of the completeness or
> stability of the code, it does indicate that the project has yet to be
> fully endorsed by the ASF.

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: deprecating the old repo

[Julian Hyde]

One trick that people sometimes use is to create a new branch called “obsolete” 
or similar, update the README in that branch to point to the new project 
location, and make that branch the default branch in GitHub.

> On Apr 26, 2019, at 11:02 AM, Gian Merlino  wrote:
> 
> That makes sense to me. Are you interested in doing a PR to the
> docker-druid repo to make its README point to the new Dockerfiles? If so,
> that should do it.
> 
> On Fri, Apr 26, 2019 at 3:33 AM Jokin Cuadrado  wrote:
> 
>> Hi, I was searching for a way to run druid on docker for some
>> experimentation, and the first results has been a repo on the droid-io
>> organization https://github.com/druid-io/docker-druid
>> 
>> I found after (Thanks to dylan pointing out in slack) that there are newer
>> dockerfiles commited on the apache incubatin repo.
>> 
>> As the ol repo sits unmodified and with some open pull request, I think
>> that cleaning the old repo, marking as deprecated and pointing to the new
>> repo would be nice. The https://github.com/druid-io organization should
>> maybe also point to the incubator project, as the only active project it's
>> the website code.
>> 
>> Regards,
>> Jokin.
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Code review

[Julian Hyde]

+1

As someone who reviews Druid release candidates but does not develop Druid 
code, I wish that “mvn test” would “just work”. I can imagine that first-time 
and occasional contributors are in a similar boat to me.

I know this is not an easy thing to achieve, because the test suite also has to 
work for more experienced contributors.

Julian


> On Apr 25, 2019, at 11:42 AM, David Glasser  wrote:
> 
> As a new contributor, I've actually really appreciated the careful and high
> quality code reviews I've received (primarily from Jihoon).
> 
> The real source of friction for me has not been the feedback from
> developers, but the painfulness of running the tests.
> 
> - Figuring out how to locally run the full suite of tests that CI will run
> is not super documented.
> - Understanding how to just run the tests that are most relevant to your
> change (either from CLI or IntelliJ) isn't well documented.  It's
> especially unclear what things can be perfectly successfully run from
> IntelliJ's own test runners and what things really require you to run them
> through Maven.  (Druid is also the first time I've used Maven directly and
> it's a huge challenge for me to understand what's going on and how to run
> it properly.)
> - Many of the tests are just very slow and maybe could be parallelized
> better?
> 
> It's enough of a pain that I often when iterating wouldn't even bother
> trying to run tests manually but would just push to GitHub and let Travis
> run it instead. But Travis itself is very slow and while it's somewhat
> parallelized, it's not super parallelized — and as a lowly PR author I
> can't even kill the overall job if I push a new version of the PR or if a
> sub-job already failed.  So this would frequently add "round trip" times of
> half an hour or more in the middle of trying to take the good advice from
> reviewers to heart.
> 
> So while I agree that it would be great to move as many of the checks as
> possible into CI from "core dev teams mind", I'd also encourage folks with
> more time and expertise than I have now to put effort into making the CI
> experience faster and easier.  (I wonder if some of the issues could just
> be solved with money, eg running on more powerful Travis machines or
> parallelizing the slower tests even further onto a larger number of
> containers.  I've also generally found in my own work that CircleCI at
> least feels faster and easier to work with than Travis FWIW.)
> 
> --dave
> 
> On Mon, Apr 22, 2019 at 7:44 PM Gian Merlino  wrote:
> 
>> Hey Druids,
>> 
>> Sometimes I feel like this too:
>> https://twitter.com/julianhyde/status/1108502471830204416.
>> 
>> I believe our code review process today has too much friction in it, and
>> that we can work to reduce it. The two main frictions I see are code
>> reviews not happening in a timely manner, and code reviews sometimes
>> descending into a swamp of nit-picks. The good news, at least, is that we
>> are not the first development team to have these problems, and that they
>> can be solved. I've written some thoughts below about principles that I
>> think might help. I am not necessarily proposing making these the law of
>> the land, but am hoping to start a discussion about how we can generally
>> improve things.
>> 
>> 1) "Let robots handle style checks." Encode Druid's code style in
>> checkstyle or other tools, and avoid making subjective style comments
>> directly as humans. If we can't figure out how to set up automated
>> verification for a particular style point, then give it up. Rationale:
>> authors can self-check style when checking is automated. Also, it's better
>> for robots to enforce style, because software development is a social
>> endeavor, and people don't mind style nit-picking from robots as much as
>> they do from humans.
>> 
>> 2) "Write down what really matters." I suggest we put together a short,
>> highly visible list of things that should be considered commit-blockers for
>> patches. Not a list of best practices, but a list of true blockers. "Short"
>> is in service of point (3), below. "Highly visible" is so it can act as a
>> shared set of values. My suggested list would be correctness of
>> implementation, documentation of new or changed functionality, tests for
>> reasonably testable functionality, avoidance of excessive additional
>> maintenance burden, and avoidance of breaking existing use cases (including
>> things that would break clusters that run at large scale, or that would
>> break rolling updates). Some of these points are subjective but I think
>> it's still a good start. Rationale: authors will know what is expected of
>> them, which should generally improve PR quality, and speed up review. Also,
>> similar to the previous point: people are generally happy to follow what
>> they perceive as community-maintained standards, but not as happy to follow
>> what they perceive as the opinion of a single reviewer.
>> 
>> 3) "Minimize hoop-jumping." We should make an effort 

Re: Running UNION ALL queries in parallel

[Julian Hyde]

Yes it does. Only ORDER BY causes order. 

Julian

> On Apr 20, 2019, at 12:17, Gian Merlino  wrote:
> 
> Hey Julian,
> 
> I think it'd be fine to issue the queries in parallel with a few
> adjustments. We'd want to avoid buffering, meaning we'd want to allow query
> results to be mixed together (return rows in the order they become
> available, rather than in query order sequence). I believe the SQL standard
> allows "UNION ALL" to behave this way, but we should double check that too.
> 
> On Fri, Apr 19, 2019 at 2:56 PM Julian Jaffe 
> wrote:
> 
>> Hey all,
>> 
>> Druid currently executes UNION ALL queries sequentially (
>> 
>> https://github.com/apache/incubator-druid/blob/master/sql/src/main/java/org/apache/druid/sql/calcite/rel/DruidUnionRel.java#L98
>> ).
>> There's a comment in that method that restates this, but does not explain
>> why. Is there a reason why each subquery of a union all query can't be
>> executed in parallel?
>> 
>> Thanks,
>> Julian
>> 

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [ANNOUNCE] Apache Druid (incubating) 0.14.0 released

[Julian Hyde]

Congratulations on the release, everyone!

The release notes [1] have some branding issues. The project’s name is “Apache 
Druid (incubating)”, but the notes call it “Druid” throughout; in fact they 
don’t mention “Apache” at all. It isn’t helped by the fact that the links point 
to druid.io  rather than druid.apache.org 
. 

Hadoop, Parquet, Kafka and (probably) Kinesis also need to be qualified.

Looking over other documentation, talks, and blog posts, it’s clear that Druid 
developers need to get into the habit of using the Apache brand.

Julian

[1] 
https://github.com/apache/incubator-druid/releases/tag/druid-0.14.0-incubating 

 


> On Apr 9, 2019, at 7:02 PM, Jonathan Wei  wrote:
> 
> Druid 0.14.0-incubating contains over 200 new features,
> performance/stability/documentation improvements, and bug fixes from 54
> contributors. Major new features and improvements include:
> 
> - New web console
> - Kinesis indexing service
> - Decommissioning mode for Historicals
> - Published segment cache in Broker
> - Bloom filter aggregator and expression
> - Updated Parquet extension
> - Force push down option for nested GroupBy queries
> - Better segment handoff and drop rule handling
> - Automatically kill MapReduce jobs when Hadoop ingestion tasks are killed
> - DogStatsD tag support for statsd emitter
> - New API for retrieving all lookup specs
> - New compaction options
> - More efficient cachingCost segment balancing strategy
> 
> Source and binary distributions can be downloaded from:
> https://druid.apache.org/downloads.html
> 
> Release notes are at:
> https://github.com/apache/incubator-druid/releases/tag/druid-0.14.0-incubating
> 
> A big thank you to all the contributors in this milestone release!
> 
> 
> 
> Disclaimer: Apache Druid is an effort undergoing incubation at The Apache
> Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is
> required of all newly accepted projects until a further review indicates
> that the infrastructure, communications, and decision making process have
> stabilized in a manner consistent with other successful ASF projects. While
> incubation status is not necessarily a reflection of the completeness or
> stability of the code, it does indicate that the project has yet to be
> fully endorsed by the ASF.
> 
> 
> On Tue, Apr 9, 2019 at 2:53 PM sebb  wrote:
> 
>> Announcements of podling release should please include the standard
>> Incubation disclaimer.
>> 
>> See for example:
>> 
>> 
>> https://lists.apache.org/thread.html/d94b89e6c4e83f2364607b47020ca715aaa7fae1a3b97b374b93ca7f@%3Cgeneral.incubator.apache.org%3E
>> 
>> https://lists.apache.org/thread.html/ec3d6b90a2b0054eb2f0a2e4ba2e09a594463720e2b28e75b315c693@%3Cgeneral.incubator.apache.org%3E
>> 
>> https://lists.apache.org/thread.html/8993ecb33e0f73e69faabba8c1f07d3a9965a178df1698c42fc9f4b2@%3Cgeneral.incubator.apache.org%3E
>> 
>> Thanks.
>> 
>> On Tue, 9 Apr 2019 at 22:20, Jonathan Wei  wrote:
>>> 
>>> Druid 0.14.0-incubating contains over 200 new features,
>>> performance/stability/documentation improvements, and bug fixes from 54
>>> contributors. Major new features and improvements include:
>>> 
>>> - New web console
>>> - Kinesis indexing service
>>> - Decommissioning mode for Historicals
>>> - Published segment cache in Broker
>>> - Bloom filter aggregator and expression
>>> - Updated Parquet extension
>>> - Force push down option for nested GroupBy queries
>>> - Better segment handoff and drop rule handling
>>> - Automatically kill MapReduce jobs when Hadoop ingestion tasks are
>> killed
>>> - DogStatsD tag support for statsd emitter
>>> - New API for retrieving all lookup specs
>>> - New compaction options
>>> - More efficient cachingCost segment balancing strategy
>>> 
>>> Source and binary distributions can be downloaded from:
>>> https://druid.apache.org/downloads.html
>>> 
>>> Release notes are at:
>>> 
>> https://github.com/apache/incubator-druid/releases/tag/druid-0.14.0-incubating
>>> 
>>> A big thank you to all the contributors in this milestone release!
>> 
>> -
>> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
>> For additional commands, e-mail: general-h...@incubator.apache.org
>> 
>> 

Re: [VOTE] Release Apache Druid (incubating) 0.14.0 [RC3]

[Julian Hyde]

+1 (binding)

Checked hashes, LICENSE, NOTICE, copyright date, DISCLAIMER, built (skipping 
tests) on JDK 8 Ubuntu. Ran RAT.

Checked that contents of src.tar.gz correspond to commit f169ada.

Did not verify bin package other than checking hashes.

Thanks for the nice clear build instructions in README.md.

Julian


> On Apr 1, 2019, at 1:50 PM, David Lim  wrote:
> 
> +1
> 
> src package:
> - verified signature/hash
> - ran RAT check
> - compared source distribution contents against git tag (f169ada)
> - LICENSE, NOTICE, and DISCLAIMER are present
> - compiled and ran unit tests
> - built binary distribution
> - ran quickstart batch tutorial
> 
> bin package:
> - verified signature/hash
> - verified META-INF/MANIFEST.MF:Build-Revision tag in JAR files matches
> source distribution git.version:Build-Revision (f169ada)
> - ran quickstart batch tutorial
> 
> On Sun, Mar 31, 2019 at 4:48 PM Jihoon Son  wrote:
> 
>> +1
>> 
>> src package:
>> - verified signature/hash
>> - compiled and ran unit tests, passed
>> 
>> bin package:
>> - verified signature/hash
>> - ran Kafka/Kinesis Indexing service for a couple of days, verified query
>> results
>> 
>> Jihoon
>> 
>> On Fri, Mar 29, 2019 at 4:08 PM Clint Wylie  wrote:
>> 
>>> +1
>>> 
>>> src package:
>>> - verified signature/hash
>>> - ran RAT check, passed
>>> - compiled and ran unit tests, passed
>>> - ran integration tests, passed
>>> - built binary distribution, ran quickstart batch tutorial with compiled
>>> artifacts, stuff loaded and and could be queried
>>> 
>>> bin package:
>>> - verified signature/hash
>>> - ran quickstart batch tutorial, stuff loaded and and could be queried
>>> 
>>> On Fri, Mar 29, 2019 at 12:30 AM Jonathan Wei  wrote:
>>> 
 Hi all,
 
 I have created a build for Apache Druid (incubating) 0.14.0, release
 candidate 3.
 
 A list of the patches applied since release candidate 1 can be found
>>> here:
 
 
>>> 
>> https://github.com/apache/incubator-druid/pulls?utf8=%E2%9C%93=is%3Apr+base%3A0.14.0-incubating+merged%3A%3E2019-03-23T05+
 
 Thanks to everyone who has contributed to this release! You can read
>> the
 proposed release notes here:
 https://github.com/apache/incubator-druid/issues/7126
 
 The release candidate has been tagged in GitHub as
 druid-0.14.0-incubating-rc3 (f169ada), available here:
 
 
>>> 
>> https://github.com/apache/incubator-druid/releases/tag/druid-0.14.0-incubating-rc3
 
 The artifacts to be voted on are located here:
 
 
>>> 
>> https://dist.apache.org/repos/dist/dev/incubator/druid/0.14.0-incubating-rc3/
 
 Release artifacts are signed with the following key:
 https://people.apache.org/keys/committer/jonwei.asc. This key and the
>>> key
 of other committers can also be found in the project's KEYS file here:
 
 https://dist.apache.org/repos/dist/release/incubator/druid/KEYS
 
 (If you are a committer, please feel free to add your own key to that
>>> file
 by following the instructions in the file's header.)
 
 Please review the proposed artifacts and vote. Note that Apache has
 specific
 requirements that must be met before +1 binding votes can be cast by
>> PMC
 members. Please refer to the policy at
 http://www.apache.org/legal/release-policy.html#policy for more
>> details.
 
 As part of the validation process, the release artifacts can be
>> generated
 from source by running: mvn clean install -Papache-release,dist
 
 The RAT license check can be run from source by:
 mvn apache-rat:check -Prat
 
 This vote will be open for at least 72 hours. The vote will pass if a
 majority of at least three +1 PMC votes are cast.
 
 Once the vote has passed, the second stage vote will be called on the
 Apache Incubator mailing list to get approval from the Incubator PMC.
 
 [ ] +1 Release this package as Apache Druid (incubating) 0.14.0
 [ ]  0 I don't feel strongly about it, but I'm okay with the release
 [ ] -1 Do not release this package because...
 
 Thanks!
 
>>> 
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Policy for accepting new modules and large contributions. Protocol for removing modules

[Julian Hyde]

One line of this policy is problematic:

> Be "endorsed" by at least two committers who are recently active in
> Druid. (Read: supporting Druid is part of their job duty, at least
> part-time.)

It creates two tiers of committers.

Sustainability is a concern for all code contributions. The project can decide 
to reject contributions on that basis. But we have to trust that all committers 
are acting in the best interests of the project. Their employment status is 
irrelevant.

Julian


> On Mar 20, 2019, at 3:48 PM, Roman Leventov  wrote:
> 
> Historically, Druid has been exceptionally inclusive for external
> contributors, but support costs are not taken into account.
> 
> Does anybody see this as a problem?
> 
> We may require for large "feature" PRs to core modules, including PRs into
> some "core" extensions (such as kafka-indexing and sql) (those that
> increase number of classes and complexity) to either
> - Be authored by a committer. So if a non-committer want to push such a
> big change to the core, he needs to become a committer first.
> - Be "endorsed" by at least two committers who are recently active in
> Druid. (Read: supporting Druid is part of their job duty, at least
> part-time.)
> 
> Regarding removing old modules, we may revisit all extensions in
> extensions-core and extensions-contrib.
> - The new contrib may not be tested in CI builds unless the contrib code
> is changed in the PR (note: I failed to quickly find if Travis supports
> such thing. It may not.)
> - Contrib modules may not be included in the general Druid distribution
> (but some vendors may include some contrib modules into their
> distributions).
> - Updating contrib modules' code (even making them compile against the
> core) may become the sole responsibility of people who care about these
> contrib modules.
> - If we see that some contrib module "falls off" and nobody updates it
> for, say, one year, we remove the module from the codebase completely.
> 
> Note: both ideas presented above are merely ideas, not firm proposals.


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [VOTE] Release Apache Druid (incubating) 0.14.0 [RC1]

[Julian Hyde]

-1 due to wrong copyright year in NOTICE file.

Checked hashes, LICENSE, NOTICE, DISCLAIMER; compiled on Ubuntu JDK 8 (skipping 
tests); checked that file contents match git commit; ran apache-rat.

Copyright year in NOTICE is 2018.

I recommend that you obsolete 
https://dist.apache.org/repos/dist/dev/incubator/druid/KEYS 
. Keys must live 
in https://dist.apache.org/repos/dist/release/incubator/druid/KEYS 
, and it is 
error-prone to have them in two locations. I wasted some time because I 
imported from the latter.

I recommend that you add a README file that describes the java version 
required. I tried building on JDK 11 and JDK 9 before I happened on JDK 8. The 
audience of a source release is a developer, possibly years in the future, 
trying to build from source, and by then instructions in web sites may be out 
of date. We know this use case is not typical, but it is important.

Julian


> On Mar 15, 2019, at 6:25 PM, Jonathan Wei  wrote:
> 
> Hi all,
> 
> I have created a build for Apache Druid (incubating) 0.14.0, release
> candidate 1.
> 
> Thanks to everyone who has contributed to this release! You can read the
> proposed release notes here:
> https://github.com/apache/incubator-druid/issues/7126
> 
> The release candidate has been tagged in GitHub as
> druid-0.14.0-incubating-rc1 (88ef108), available here:
> https://github.com/apache/incubator-druid/releases/tag/druid-0.14.0-incubating-rc1
> 
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/incubator/druid/0.14.0-incubating-rc1/
> 
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/jonwei.asc. This key and the key
> of other committers can also be found in the project's KEYS file here:
> 
> https://dist.apache.org/repos/dist/dev/incubator/druid/KEYS
> 
> (If you are a committer, please feel free to add your own key to that file
> by following the instructions in the file's header.)
> 
> Please review the proposed artifacts and vote. Please note that Apache has
> specific
> requirements that must be met before +1 binding votes can be cast by PMC
> members. Please refer to the policy at
> http://www.apache.org/legal/release-policy.html#policy for more details.
> 
> As part of the validation process, the release artifacts can be generated
> from source by running: mvn clean install -Papache-release -Dtar
> 
> This vote will be open for at least 72 hours but likely more, in following
> the Druid community's practice of deploying the RC to larger clusters and
> allowing it to soak for a period of time to flush out any remaining issues.
> The vote will pass if a majority of at least three +1 PMC votes are cast.
> 
> Once the vote has passed, the second stage vote will be called on the
> Apache Incubator mailing list to get approval from the Incubator PMC.
> 
> [ ] +1 Release this package as Apache Druid (incubating) 0.14.0
> [ ]  0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
> 
> Thanks!

Re: [VOTE] Release Apache Druid (incubating) 0.14.0 [RC1]

[Julian Hyde]

My bad. I was looking apache-druid-0.13.0-incubating-rc1 when I should have 
been looking at 0.14.0-incubating-rc1.


> On Mar 16, 2019, at 9:47 AM, Julian Hyde  wrote:
> 
> Jonathan,
> 
> I’m reviewing now. The release seems to be signed by David Lim. Did David 
> make this release?
> 
> $ gpg --verify apache-druid-0.13.0-incubating-src.tar.gz{.asc,}
> gpg: Signature made Sat 20 Oct 2018 08:50:20 PM PDT
> gpg:using RSA key 58B5D669D2FFD83B37D88DF8BB64B3727183DE56
> gpg: Good signature from "David Lim " [unknown]
> gpg: WARNING: This key is not certified with a trusted signature!
> gpg:  There is no indication that the signature belongs to the owner.
> Primary key fingerprint: 58B5 D669 D2FF D83B 37D8  8DF8 BB64 B372 7183 DE56
> 
> Julian
> 
> 
> 
>> On Mar 15, 2019, at 6:25 PM, Jonathan Wei  wrote:
>> 
>> Hi all,
>> 
>> I have created a build for Apache Druid (incubating) 0.14.0, release
>> candidate 1.
>> 
>> Thanks to everyone who has contributed to this release! You can read the
>> proposed release notes here:
>> https://github.com/apache/incubator-druid/issues/7126
>> 
>> The release candidate has been tagged in GitHub as
>> druid-0.14.0-incubating-rc1 (88ef108), available here:
>> https://github.com/apache/incubator-druid/releases/tag/druid-0.14.0-incubating-rc1
>> 
>> The artifacts to be voted on are located here:
>> https://dist.apache.org/repos/dist/dev/incubator/druid/0.14.0-incubating-rc1/
>> 
>> Release artifacts are signed with the following key:
>> https://people.apache.org/keys/committer/jonwei.asc. This key and the key
>> of other committers can also be found in the project's KEYS file here:
>> 
>> https://dist.apache.org/repos/dist/dev/incubator/druid/KEYS
>> 
>> (If you are a committer, please feel free to add your own key to that file
>> by following the instructions in the file's header.)
>> 
>> Please review the proposed artifacts and vote. Please note that Apache has
>> specific
>> requirements that must be met before +1 binding votes can be cast by PMC
>> members. Please refer to the policy at
>> http://www.apache.org/legal/release-policy.html#policy for more details.
>> 
>> As part of the validation process, the release artifacts can be generated
>> from source by running: mvn clean install -Papache-release -Dtar
>> 
>> This vote will be open for at least 72 hours but likely more, in following
>> the Druid community's practice of deploying the RC to larger clusters and
>> allowing it to soak for a period of time to flush out any remaining issues.
>> The vote will pass if a majority of at least three +1 PMC votes are cast.
>> 
>> Once the vote has passed, the second stage vote will be called on the
>> Apache Incubator mailing list to get approval from the Incubator PMC.
>> 
>> [ ] +1 Release this package as Apache Druid (incubating) 0.14.0
>> [ ]  0 I don't feel strongly about it, but I'm okay with the release
>> [ ] -1 Do not release this package because...
>> 
>> Thanks!
> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [VOTE] Release Apache Druid (incubating) 0.14.0 [RC1]

[Julian Hyde]

Jonathan,

I’m reviewing now. The release seems to be signed by David Lim. Did David make 
this release?

$ gpg --verify apache-druid-0.13.0-incubating-src.tar.gz{.asc,}
gpg: Signature made Sat 20 Oct 2018 08:50:20 PM PDT
gpg:using RSA key 58B5D669D2FFD83B37D88DF8BB64B3727183DE56
gpg: Good signature from "David Lim " [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:  There is no indication that the signature belongs to the owner.
Primary key fingerprint: 58B5 D669 D2FF D83B 37D8  8DF8 BB64 B372 7183 DE56

Julian



> On Mar 15, 2019, at 6:25 PM, Jonathan Wei  wrote:
> 
> Hi all,
> 
> I have created a build for Apache Druid (incubating) 0.14.0, release
> candidate 1.
> 
> Thanks to everyone who has contributed to this release! You can read the
> proposed release notes here:
> https://github.com/apache/incubator-druid/issues/7126
> 
> The release candidate has been tagged in GitHub as
> druid-0.14.0-incubating-rc1 (88ef108), available here:
> https://github.com/apache/incubator-druid/releases/tag/druid-0.14.0-incubating-rc1
> 
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/incubator/druid/0.14.0-incubating-rc1/
> 
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/jonwei.asc. This key and the key
> of other committers can also be found in the project's KEYS file here:
> 
> https://dist.apache.org/repos/dist/dev/incubator/druid/KEYS
> 
> (If you are a committer, please feel free to add your own key to that file
> by following the instructions in the file's header.)
> 
> Please review the proposed artifacts and vote. Please note that Apache has
> specific
> requirements that must be met before +1 binding votes can be cast by PMC
> members. Please refer to the policy at
> http://www.apache.org/legal/release-policy.html#policy for more details.
> 
> As part of the validation process, the release artifacts can be generated
> from source by running: mvn clean install -Papache-release -Dtar
> 
> This vote will be open for at least 72 hours but likely more, in following
> the Druid community's practice of deploying the RC to larger clusters and
> allowing it to soak for a period of time to flush out any remaining issues.
> The vote will pass if a majority of at least three +1 PMC votes are cast.
> 
> Once the vote has passed, the second stage vote will be called on the
> Apache Incubator mailing list to get approval from the Incubator PMC.
> 
> [ ] +1 Release this package as Apache Druid (incubating) 0.14.0
> [ ]  0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
> 
> Thanks!


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [VOTE] Release Apache Druid (incubating) 0.14.0 [RC1]

[Julian Hyde]

Hey Don,

Your comments are most welcome, but I’d like to make a point of order. In 
Apache, vote threads are strictly business - voting +1 or -1 in the release, 
with reasons why. If you have something to discuss about the content of the 
release (or otherwise), it’s best to start a new thread. That way, the 
discussion doesn’t get in the way of the main business of the vote thread.

Julian


> On Mar 16, 2019, at 7:32 AM, Don Bowman  wrote:
> 
> On Fri, 15 Mar 2019 at 21:26, Jonathan Wei  wrote:
> 
>> Hi all,
>> 
>> I have created a build for Apache Druid (incubating) 0.14.0, release
>> candidate 1.
>> 
>> Thanks to everyone who has contributed to this release! You can read the
>> proposed release notes here:
>> https://github.com/apache/incubator-druid/issues/7126
>> 
>> The release candidate has been tagged in GitHub as
>> druid-0.14.0-incubating-rc1 (88ef108), available here:
>> 
>> https://github.com/apache/incubator-druid/releases/tag/druid-0.14.0-incubating-rc1
>> 
>> 
>> 
> Although its a bit late now, I am hoping that we make the docker image a
> standard artifact. This really helps adoption.
> The dockerfile is now in .14, but I didn't really see how that translates
> into 'built in the pipeline as an artifact'
> like other projects do.
> 
> Any comment on what i can do to help this? after the PR was accepted etc,
> it kind of vanished.


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Incubator report

[Julian Hyde]

Hi Druid PPMC,

The March incubator report is due tomorrow. Is someone working on it?

Julian

https://wiki.apache.org/incubator/March2019 

Re: Datasketches

[Julian Hyde]

I don’t know how a project can formally track another project, but individuals 
certainly can.

If any Druid committers are ASF members then they could volunteer to help as 
mentors of the Data Sketches podling. 

If any Druid committers are past or current contributors to the DataSketches 
they could ask to be put onto the initial contributors list.

And any of us could join the podling’s dev list, monitor it to see if anything 
is of interest to Druid, and report back on this list. And vice versa, telling 
the DataSketches community what’s going on in Druid.

Lastly, if you find DataSketches interesting, just offer to help. I’m sure 
there’s plenty to do, and they would love your help. Your experience with Druid 
and the ASF incubation process will be useful to them.

Julian


> On Feb 25, 2019, at 9:56 AM, Charles Allen  
> wrote:
> 
> There are a lot of here and there discussions on how to handle sketching /
> hll / histograms / other-stats, and it is getting kind of hard to keep
> track of them all.
> 
> In addition, looks like Datasketches is in an incubating proposal stage for
> Apache
> http://mail-archives.apache.org/mod_mbox/incubator-general/201902.mbox/%3CCA%2BUaPnt%3DUvbLr_v-4%2BYbAmHsAM-GqQG%2Bb%3DgOw3BL3Cemj%2BOwSA%40mail.gmail.com%3E
> 
> 
> I think it is important enough and wide spread enough to have a top level
> consideration within the druid project. Either a label or a "github
> project" or something so that things can be tracked easier.
> 
> Anyone have any opinions or desires here?
> 
> Thanks,
> Charles Allen


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: TeamCity having problems

[Julian Hyde]

The page [1] requires a login. So, a Druid contributor has to get a TeamCity 
account?

That would be OK, if TeamCity provides a lot of value. But it raises the bar 
for occasional and first-time contributors. 

Julian

[1] 
https://hub.jetbrains.com/auth/login?response_type=code_id=7b63a7f6-a4ca-4ad2-99d6-ecede5a769a5_uri=https:%2F%2Fteamcity.jetbrains.com%2FhubPlugin%2Flogin.html=0-0-0-0-0%207b63a7f6-a4ca-4ad2-99d6-ecede5a769a5=%2FviewLog.html%3FbuildId%3D1998017%26buildTypeId%3DOpenSourceProjects_Druid_InspectionsPullRequests_type=online

> On Feb 22, 2019, at 10:59 AM, Gian Merlino  wrote:
> 
> It's one of the two integrations we have on GitHub that run to validate
> pull requests. We have TeamCity doing static analysis; and Travis running
> unit tests, integration tests, and checkstyle. You can see an example on
> this pull request here: https://github.com/apache/incubator-druid/pull/7118,
> under "Inspections: pull requests (Druid)".
> 
> On Fri, Feb 22, 2019 at 10:17 AM Julian Hyde  wrote:
> 
>> What is TeamCity? How does Druid use it? Is it accessible to all Druid
>> contributors? Are all Druid contributors required to use it? Is its use
>> documented somewhere? (I searched
>> https://www.google.com/search?q=apache+druid+teamcity <
>> https://www.google.com/search?q=apache+druid+teamcity> but couldn’t find
>> any mention.)
>> 
>> 
>> 
>>> On Feb 22, 2019, at 8:26 AM, Furkan KAMACI 
>> wrote:
>>> 
>>> Hi Gian,
>>> 
>>> There maybe a problem with permissions after upgrading Apache parent POM
>>> version to 21. This maybe a related issue:
>>> https://jira.apache.org/jira/browse/INFRA-10286
>>> 
>>> Kind Regards,
>>> Furkan KAMACI
>>> 
>>> On Fri, Feb 22, 2019 at 7:13 PM Gian Merlino  wrote:
>>> 
>>>> It looks like TeamCity is having problems since a few days ago - builds
>> of
>>>> master and PRs are flaky. Here's the link to recent builds of master:
>>>> 
>>>> 
>> https://teamcity.jetbrains.com/viewType.html?buildTypeId=OpenSourceProjects_Druid_Inspections=buildTypeHistoryList_OpenSourceProjects_Druid=__all_branches__
>>>> 
>>>> It looks like the Windows agents generally work fine and the Ubuntu
>> agents
>>>> are flaky. I can't see any recent commits that seem likely to have
>> caused
>>>> this so am assuming something is going wrong on the TeamCity server
>> side.
>>>> 
>>>> I looked at the logs of a few successful and a few failed builds and
>>>> noticed the failed ones have a lot of these messages, but the successful
>>>> ones didn't:
>>>> 
>>>> [Step 1/1] [  43336]   WARN - ution.rmi.RemoteProcessSupport - [RMI TCP
>>>> Connection(3)-127.0.0.1] WARN
>>>> org.eclipse.aether.internal.impl.DefaultUpdateCheckManager - Failed to
>>>> create parent directories for tracking file
>>>> 
>>>> 
>> /home/teamcity/.m2/repository/org/apache/apache/21/apache-21.pom.lastUpdated
>>>> 
>>>> I'm not sure what that might indicate or what to do about it. Maybe we
>> need
>>>> to… clear… some caches? Anyone have any ideas?
>>>> 
>> 
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Knowledge sharing between Druid developers via technical talks

[Julian Hyde]

I like that idea. I always wish that meetups were more about the community of 
contributors (people writing code, answering questions, writing documentation, 
and pushing the product to new places). But sadly meetups are usually organized 
by marketing departments.

Some conferences (e.g. Hadoop summit, and I suspect FOSDEM, OSCON and Berlin 
Buzzwords) have BoFs (birds-of-a-feather meetings) that occur in the evening 
after the main conference sessions. They are extremely free format, and anyone 
who shows up can speak. If Druid contributors are heading to such conferences, 
it’s worth sounding out on this list a few days before. There might be other 
Druid contributors attending the same conference.

Julian



> On Feb 22, 2019, at 10:45 AM, Gian Merlino  wrote:
> 
> Could be nice for the last talk in a meetup to be one of these, that way
> anyone that isn't interested could leave early.
> 
> On Fri, Feb 22, 2019 at 9:51 AM Eyal Yurman 
> wrote:
> 
>> Thanks for the response, that sounds great!
>> 
>> Since the meetups are user-focused, perhaps a separate "track" which is
>> open to all but is dev-focused? This could be before/after the main event.
>> 
>> I promise that once I get enough experience with the code base, I'd
>> volunteer to present, but hopefully, there are much better candidates at
>> the moment :)
>> 
>> On Mon, Feb 18, 2019 at 1:36 PM Gian Merlino 
>> wrote:
>> 
>>> I am interested especially if the format is something live. An in-person
>>> meetup with a recording distributed afterwards would be my preference, if
>>> people are into that. Maybe something at one of the Druid meetups?
>>> 
>>> On Wed, Feb 13, 2019 at 8:38 PM Eyal Yurman
>>>  wrote:
>>> 
 Hi,
 
 This is something usually being done in companies, but I think it is
 useful
 for any community, especially our community which is so distributed.
 
 I think it would be absolutely wonderful if we can find people willing to
 share their knowledge with other contributors via the form of a
 tech-talk.
 I.e. it would be very useful if someone could take a subject (Just for
 example, groupBy query) and present the high-level
 architecture/implementation.
 
 I know this requires significant effort, but I hope to convince you of
 the
 benefits it would provide to the Druid project:
 - Helping any newcomer being more effective, thus providing better
 contribution ROI against work effort.
 - Serving as a high-quality medium of communication within the group of
 committers, which would lead to more trust and understanding.
 
 Recording and uploaded such sessions will make them Apache-Way compatible
 (Along with serving future viewers).
 
 So, anyone up to the challenge? :)
 
 Eyal.
 
>>> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: TeamCity having problems

[Julian Hyde]

What is TeamCity? How does Druid use it? Is it accessible to all Druid 
contributors? Are all Druid contributors required to use it? Is its use 
documented somewhere? (I searched 
https://www.google.com/search?q=apache+druid+teamcity 
 but couldn’t find any 
mention.)



> On Feb 22, 2019, at 8:26 AM, Furkan KAMACI  wrote:
> 
> Hi Gian,
> 
> There maybe a problem with permissions after upgrading Apache parent POM
> version to 21. This maybe a related issue:
> https://jira.apache.org/jira/browse/INFRA-10286
> 
> Kind Regards,
> Furkan KAMACI
> 
> On Fri, Feb 22, 2019 at 7:13 PM Gian Merlino  wrote:
> 
>> It looks like TeamCity is having problems since a few days ago - builds of
>> master and PRs are flaky. Here's the link to recent builds of master:
>> 
>> https://teamcity.jetbrains.com/viewType.html?buildTypeId=OpenSourceProjects_Druid_Inspections=buildTypeHistoryList_OpenSourceProjects_Druid=__all_branches__
>> 
>> It looks like the Windows agents generally work fine and the Ubuntu agents
>> are flaky. I can't see any recent commits that seem likely to have caused
>> this so am assuming something is going wrong on the TeamCity server side.
>> 
>> I looked at the logs of a few successful and a few failed builds and
>> noticed the failed ones have a lot of these messages, but the successful
>> ones didn't:
>> 
>> [Step 1/1] [  43336]   WARN - ution.rmi.RemoteProcessSupport - [RMI TCP
>> Connection(3)-127.0.0.1] WARN
>> org.eclipse.aether.internal.impl.DefaultUpdateCheckManager - Failed to
>> create parent directories for tracking file
>> 
>> /home/teamcity/.m2/repository/org/apache/apache/21/apache-21.pom.lastUpdated
>> 
>> I'm not sure what that might indicate or what to do about it. Maybe we need
>> to… clear… some caches? Anyone have any ideas?
>> 

Re: LICENSE question

[Julian Hyde]

That sounds right.

What is the license of these files?

Julian


> On Feb 8, 2019, at 11:17 AM, Jonathan Wei  wrote:
> 
> Hi mentors,
> 
> I am updating Druid's LICENSE and NOTICE file for our next release, and had
> a question on handling some our dependencies.
> 
> We have a web console (
> https://github.com/apache/incubator-druid/tree/0.14.0-incubating/web-console)
> that pulls some modules from the NPM registry and bundles them using
> webpack at build time.
> 
> Is the following understanding correct?
> - The pulled NPM modules don't need to be in LICENSE for our source
> distribution, since they're not pulled and bundled yet
> - In a binary distribution, we'll need to include those modules in LICENSE
> 
> Thanks,
> Jon


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Forbiddenapis Plugin

[Julian Hyde]

I’ve seen a similar problem in Calcite, which also uses forbiddenApis. It only 
seems to occurs in a “bad build”; when you do “mvn clean” the problems 
disappear.

My hypothesis is that the code is generated by javac, for example for the 
messages from “assert”, or when concatenating string literals separated by “+", 
and it really is not something to worry about.

Julian


> On Jan 31, 2019, at 9:40 AM, Gian Merlino  wrote:
> 
> Good question. I'm not sure. They are at least doing String.format on
> _something_ with no default locale.
> 
> On Thu, Jan 31, 2019 at 9:36 AM Charles Allen
>  wrote:
> 
>> Is this indicative of latent bugs the generated sources have?
>> 
>> On Thu, Jan 31, 2019 at 8:55 AM Gian Merlino  wrote:
>> 
>>> I get those sometimes with generated sources -- typically doing a "mvn
>>> clean" beforehand clears it up. We might be able to add exclusions for
>> the
>>> generated source directories in order to avoid the need to do this.
>>> 
>>> On Thu, Jan 31, 2019 at 5:15 AM Furkan KAMACI 
>>> wrote:
>>> 
 I try to run forbiddenapis plugin at Druid. However I get that errors
>> but
 does not know where they actually points:
 
 [INFO] Scanning classes for violations...
 [ERROR] Forbidden method invocation:
 java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses
>>> default
 locale]
 [ERROR]   in org.apache.druid.math.expr.BinaryEvalOpExprBase
>> (Expr.java,
 method body of '$$$reportNull$$$0(int)')
 [ERROR] Forbidden method invocation:
 java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses
>>> default
 locale]
 [ERROR]   in org.apache.druid.math.expr.LongExpr (Expr.java, method
>> body
>>> of
 '$$$reportNull$$$0(int)')
 [ERROR] Forbidden method invocation:
 java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses
>>> default
 locale]
 [ERROR]   in org.apache.druid.math.expr.FunctionExpr (Expr.java, method
 body of '$$$reportNull$$$0(int)')
 [ERROR] Forbidden method invocation:
 java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses
>>> default
 locale]
 [ERROR]   in org.apache.druid.data.input.impl.InputRowParser
 (InputRowParser.java, method body of '$$$reportNull$$$0(int)')
 [ERROR] Forbidden method invocation:
 java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses
>>> default
 locale]
 [ERROR]   in org.apache.druid.math.expr.BinAndExpr (Expr.java, method
>>> body
 of '$$$reportNull$$$0(int)')
 [ERROR] Forbidden method invocation:
 java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses
>>> default
 locale]
 [ERROR]   in org.apache.druid.java.util.common.concurrent.Execs
 (Execs.java, method body of '$$$reportNull$$$0(int)')
 [ERROR] Forbidden method invocation:
 java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses
>>> default
 locale]
 [ERROR]   in org.apache.druid.math.expr.BinOrExpr (Expr.java, method
>> body
 of '$$$reportNull$$$0(int)')
 [ERROR] Forbidden method invocation:
 java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses
>>> default
 locale]
 [ERROR]   in org.apache.druid.math.expr.StringExpr (Expr.java, method
>>> body
 of '$$$reportNull$$$0(int)')
 [ERROR] Forbidden method invocation:
 java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses
>>> default
 locale]
 [ERROR]   in org.apache.druid.math.expr.DoubleExpr (Expr.java, method
>>> body
 of '$$$reportNull$$$0(int)')
 [ERROR] Forbidden method invocation:
 java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses
>>> default
 locale]
 [ERROR]   in org.apache.druid.math.expr.UnaryMinusExpr (Expr.java,
>> method
 body of '$$$reportNull$$$0(int)')
 [ERROR] Forbidden method invocation:
 java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses
>>> default
 locale]
 [ERROR]   in org.apache.druid.math.expr.UnaryNotExpr (Expr.java, method
 body of '$$$reportNull$$$0(int)')
 [ERROR] Forbidden method invocation:
 java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses
>>> default
 locale]
 [ERROR]   in org.apache.druid.math.expr.IdentifierExpr (Expr.java,
>> method
 body of '$$$reportNull$$$0(int)')
 [ERROR] Scanned 714 class file(s) for forbidden API invocations (in
>>> 0.65s),
 12 error(s).
 
 Do you have any idea?
 
 Kind Regards,
 Furkan KAMACI
 
>>> 
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Off list major development

[Julian Hyde]

just because a handful of the committers have this attitude,
>> it
>>>>> isn't fair to expect the wider community to also, that's why I am in
>>> favor
>>>>> of formalizing the process.
>>>>> 
>>>>> Can you please explain what is overbearing ? what can be changed to
>> make
>>>>> it
>>>>>> easy ?
>>>>>> Most of the points are kind of the actual questions that you want to
>>>>>> address before hand anyway isn't ?
>>>>>> 
>>>>> 
>>>>> Sorry for the confusion, I said it's "not overbearing", I think it's
>>> fine.
>>>>> 
>>>>> What are the complaints ?
>>>>> 
>>>>> 
>>>>> Is this and other previous threads not a complaint about opening a
>> large
>>>>> PR
>>>>> without a proposal? :) I just mean that formalizing the process, even
>>> if a
>>>>> proposal has a reference PR opened with it near concurrently, could
>>>>> prevent
>>>>> these discussions from happening in the future because ground rules
>> have
>>>>> been set and we are all on the same page I guess.
>>>>> 
>>>>> I think we should also probably consider renaming the "design review"
>>>>> label
>>>>> to "proposal" or something to make it more clear that a PR is
>> associated
>>>>> with a proposal. It might also be worth considering using github
>>> projects
>>>>> for particularly large things that involve multiple follow up PRs,
>> but I
>>>>> haven't used them in much depth to know if they add anything.
>>>>> 
>>>>> It seems like we are all converging on agreement to do a github issue
>>>>> proposal for larger changes (I would vote for announcing them on the
>> dev
>>>>> list too for more visibility), so that design review is separated from
>>>>> code
>>>>> review. I guess my main concern was not wanting to discourage
>>>>> experimentation by walling it off behind mandatory discussions, but
>> the
>>>>> more I think about it,  it doesn't technically change much about this
>>>>> process, it just requires a more formal proposal to accompany any
>>>>> experiments that are shared as a PR.
>>>>> 
>>>>> 
>>>>> On Tue, Jan 8, 2019 at 12:28 PM Gian Merlino  wrote:
>>>>> 
>>>>>> I think for us, choosing to use GitHub issues as discussion threads
>>> for
>>>>>> potential 'major' contributions would be a good idea, especially if
>> we
>>>>>> encourage people to start them before PRs show up. Definitely agree
>>> that
>>>>>> all contributors should go through the same process -- I couldn't
>> see
>>> it
>>>>>> working well any other way.
>>>>>> 
>>>>>> On Mon, Jan 7, 2019 at 12:23 PM Julian Hyde 
>> wrote:
>>>>>> 
>>>>>>> Statically, yes, GitHub PRs are the same as GitHub cases. But
>>>>>> dynamically,
>>>>>>> they are different, because you can only log a PR when you have
>>>>> finished
>>>>>>> work.
>>>>>>> 
>>>>>>> A lot of other Apache projects use JIRA, so there is a clear
>>>>> distinction
>>>>>>> between cases and contributions. JIRA cases, especially when
>> logged
>>>>> early
>>>>>>> in the lifecycle of a contribution, become long-running
>> conversation
>>>>>>> threads with a lot of community participation. If the Druid chose
>> to
>>>>> do
>>>>>> so,
>>>>>>> GitHub cases could be the same.
>>>>>>> 
>>>>>>> Be careful that you do not treat “potential contributors” (by
>> which
>>> I
>>>>>>> presume you mean non-committers) differently from committers and
>> PMC
>>>>>>> members. Anyone starting a major piece of work should follow the
>>> same
>>>>>>> process. (Experienced committers probably have a somewhat better
>>> idea
>>>>>> what
>>>>>>> work will turn out to be “major”, so they get a little more
>> leeway.)
>>>

Re: Off list major development

[Julian Hyde]

Statically, yes, GitHub PRs are the same as GitHub cases. But dynamically, they 
are different, because you can only log a PR when you have finished work.

A lot of other Apache projects use JIRA, so there is a clear distinction 
between cases and contributions. JIRA cases, especially when logged early in 
the lifecycle of a contribution, become long-running conversation threads with 
a lot of community participation. If the Druid chose to do so, GitHub cases 
could be the same.

Be careful that you do not treat “potential contributors” (by which I presume 
you mean non-committers) differently from committers and PMC members. Anyone 
starting a major piece of work should follow the same process. (Experienced 
committers probably have a somewhat better idea what work will turn out to be 
“major”, so they get a little more leeway.)

Julian


> On Jan 7, 2019, at 12:10 PM, Gian Merlino  wrote:
> 
> I don't think there's a need to raise issues for every change: a small bug
> fix or doc fix should just go straight to PR. (GitHub PRs show up as issues
> in the issue-search UI/API, so it's not like this means the patch has no
> corresponding issue -- in a sense the PR _is_ the issue.)
> 
> I do think it makes sense to encourage potential contributors to write to
> the dev list or raise an issue if they aren't sure if something would need
> to go through a more heavy weight process.
> 
> Fwiw we do have a set of 'design review' criteria already (we had a
> discussion about this a couple years ago) at:
> http://druid.io/community/#getting-your-changes-accepted. So we wouldn't be
> starting from zero on defining that. We set it up back when we were trying
> to _streamline_ our process -- we used to require two non-author +1s for
> _every_ change, even minor ones. The introduction of design review criteria
> was meant to classify which PRs need that level of review and which ones
> are minor and can be merged with less review. I do think it helped with
> getting minor PRs merged more quickly. The list of criteria is,
> 
> - Major architectural changes or API changes
> - HTTP requests and responses (e. g. a new HTTP endpoint)
> - Interfaces for extensions
> - Server configuration (e. g. altering the behavior of a config property)
> - Emitted metrics
> - Other major changes, judged by the discretion of Druid committers
> 
> Some of it is subjective, but it has been in place for a while, so it's at
> least something we are relatively familiar with.
> 
> On Mon, Jan 7, 2019 at 11:32 AM Julian Hyde  wrote:
> 
>> Small contributions don’t need any design review, whereas large
>> contributions need significant review. I don’t think we should require an
>> additional step for those (many) small contributions. But who decides
>> whether a contribution fits into the small or large category?
>> 
>> I think the solution is for authors to log a case (or send an email to
>> dev) before they start work on any contribution. Then committers can
>> request a more heavy-weight process if they think it is needed.
>> 
>> Julian
>> 
>> 
>>> On Jan 7, 2019, at 11:24 AM, Gian Merlino  wrote:
>>> 
>>> It sounds like splitting design from code review is a common theme in a
>> few
>>> of the posts here. How does everyone feel about making a point of
>>> encouraging design reviews to be done as issues, separate from the pull
>>> request, with the expectations that (1) the design review issue
>>> ("proposal") should generally appear somewhat _before_ the pull request;
>>> (2) pull requests should _not_ have design review happen on them, meaning
>>> there should no longer be PRs with design review tags, and we should move
>>> the design review approval process to the issue rather than the PR.
>>> 
>>> For (1), even if we encourage design review discussions to start before a
>>> pull request appears, I don't see an issue with them running concurrently
>>> for a while at some point.
>>> 
>>> On Thu, Jan 3, 2019 at 5:35 PM Jonathan Wei  wrote:
>>> 
>>>> Thanks for raising these concerns!
>>>> 
>>>> My initial thoughts:
>>>> - I agree that separation of design review and code-level review for
>> major
>>>> changes would be more efficient
>>>> 
>>>> - I agree that a clear, more formalized process for handling major
>> changes
>>>> would be helpful for contributors:
>>>> - Define what is considered a major change
>>>> - Define a standard proposal structure, KIP-style proposal format
>> sounds
>>>> good to me
>>>> 
>>>> - I think it's too

Re: Off list major development

[Julian Hyde]

 of noise and will
>>>>> ultimately make  the GitHub page unreadable.
>>>>> Avoid overlapping of work.
>>>>> Once code is written hard to think abstract.
>>>>> Separate page for Design review later can always be used it as a
>> Design
>>>>> document that is readable and code free-ish.
>>>>> As i said the goal of this first round is to see if the community
>> agree
>>>>> about such change, then make the process of design more inclusive
>> thus
>>>>> other contributors can submit a counter proposals.
>>>>> 
>>>>> [Step 2] IF everybody agree about that point Step 2 is to define
>> which
>>>>> medium is used to Publish a primitive form of a CODE FREE Abstract
>>>> Proposal
>>>>> containing at least the following bullet points.
>>>>> - The problem description and motivation
>>>>> - Overview of the proposed change
>>>>> - Operational impact (compatibility/ plans to upgrades) public API
>>>> changes,
>>>>> configuration changes, algorithm, and so on
>>>>> - Expected benefits and drawbacks
>>>>> - Rationale and alternatives
>>>>> - Estimate Time to Deliver if possible.
>>>>> 
>>>>> The way i think this can be is a Github issue where member of the
>>>> community
>>>>> will interact via comments and the author will be updating the
>>>> description
>>>>> in the light of comments provided by the community.
>>>>> 
>>>>> During and near the end of the design discussions the author/s can
>>> start
>>>>> writing POCs to help guide the review process this naturally will be
>> a
>>>> Pull
>>>>> request with actual code.
>>>>> 
>>>>> *Now the most important thing is that we need to agree that any work
>>> that
>>>>> does not align with this formal process will be ignored and the
>> author
>>>> will
>>>>> be asked to start with a DIP*
>>>>> *That is what i meant with  “If it didn’t happen on the mailing list,
>>> it
>>>>> didn’t happen.”*
>>>>> 
>>>>> Thanks and happy coding!
>>>>> 
>>>>> On Wed, Jan 2, 2019 at 6:47 PM Gian Merlino  wrote:
>>>>> 
>>>>>> One of the advantages I see with a more formal process is (like
>> Kafka
>>>>> KIPs)
>>>>>> is that it levels the playing field a bit and sets some ground
>> rules
>>>> for
>>>>>> working together. In a way it can help encourage contributions by
>>>> making
>>>>> it
>>>>>> clear what is expected of potential contributors.
>>>>>> 
>>>>>> We have a design review process today that is not as formal as
>> KIPs,
>>>> but
>>>>> is
>>>>>> somewhat heavier than the one you describe. Maybe we could tweak
>> our
>>>>>> current one by starting to do design reviews separately from PRs.
>>> i.e.,
>>>>> for
>>>>>> anything that meets our 'design review' criteria, do that on the
>> dev
>>>> list
>>>>>> or in a separate issue, and keep the PR focused on code-level
>> stuff.
>>>> That
>>>>>> way we don't end up trying to do both at once. And it makes it
>> easier
>>>> to
>>>>>> start talking about design before the code is ready, which would be
>>>>> better.
>>>>>> 
>>>>>> On Wed, Jan 2, 2019 at 6:10 PM Julian Hyde 
>> wrote:
>>>>>> 
>>>>>>> It’s really hard to say no to a contribution when someone has put
>>> in
>>>> a
>>>>>>> significant amount of work.
>>>>>>> 
>>>>>>> The following approach is simple and works really well: Before
>> you
>>>>> start
>>>>>>> work, log a case, describing the problem. When you have some
>> ideas
>>>>> about
>>>>>>> design, add those to the case. When you have a code branch, add
>> its
>>>> URL
>>>>>> to
>>>>>>> the case. And so forth. At any point in the proceedings, people
>> can
>>>>> chime
>>>>>>> in with their opinio

Re: Off list major development

[Julian Hyde]

in cases
>>> where an approach still needs proven to be a viable idea *to the author*,
>>> so that a much more productive discussion can be had in the first place.
>>> 
>>> I think there is a trade off, I don't think we want to discourage
>>> experimentation by walling it off behind mandatory discussions before it
>>> can even start, but I do think formalizing the process for large changes
>> is
>>> a good thing, especially since we probably can't expect the wider
>> community
>>> to have the same attitude towards a large PR getting discarded as a
>>> committer might. I think the Kafka approach is reasonable, a bit more
>>> formal than our design review process but not overbearing.
>> 
>> 
>> Can you please explain what is overbearing ? what can be changed to make it
>> easy ?
>> Most of the points are kind of the actual questions that you want to
>> address before hand anyway isn't ?
>> 
>> 
>>> Going code first
>>> should be in general discouraged, but when it does happen, it seems like
>>> opening DIP/an issue/starting a mailing list thread or whatever we go
>> with
>>> to have a more high level design discussion alongside the reference PR
>>> could alleviate some of these complaints?
>> 
>> 
>> What are the complaints ?
>> 
>> 
>>> +1 for "DIP" heh, I think making
>>> them in the form of github issues is probably appropriate, with a dev
>> list
>>> thread to announce them perhaps?
>>> 
>> 
>> I think  github issue with [Proposal] header like
>> https://github.com/apache/incubator-druid/issues/4349 is good to me,
>> 
>> Thanks!
>> 
>> 
>>> On Thu, Jan 3, 2019 at 10:28 AM Slim Bouguerra  wrote:
>>> 
>>>> Thanks everyone for interacting with this thread.
>>>> 
>>>> The fact that i, Roman, Jihoon  and others in the past (FJ
>>>> 
>>> 
>> https://groups.google.com/forum/#!msg/druid-user/gkUEsAYIfBA/6B2GJdLkAgAJ)
>>>> raised this point indicates that PRs without a proposal are indeed an
>>> issue
>>>> and we need to solve it.
>>>> 
>>>> Something Similar to KIP maybe called DIPs is fine with me.
>>>> What i strive to see is the following:
>>>> 
>>>> [Step 1] formalize what is the kind of work that needs a formal
>>> Proposal, I
>>>> think Roman and Jihoon has already covered that pretty well. am +1 on
>>> that.
>>>> 
>>>> 
>>> 
>> https://lists.apache.org/thread.html/9b30d893bdb6bb633cf6a9a700183ffb5b98f115330531a55328ac77@%3Cdev.druid.apache.org%3E
>>>> 
>>>> I am strongly in favor of the separation of Proposal Review and (later)
>>>> Code review PRs. My  main reasons:
>>>> Most importantly code reviewing will introduce lot of noise and will
>>>> ultimately make  the GitHub page unreadable.
>>>> Avoid overlapping of work.
>>>> Once code is written hard to think abstract.
>>>> Separate page for Design review later can always be used it as a Design
>>>> document that is readable and code free-ish.
>>>> As i said the goal of this first round is to see if the community agree
>>>> about such change, then make the process of design more inclusive thus
>>>> other contributors can submit a counter proposals.
>>>> 
>>>> [Step 2] IF everybody agree about that point Step 2 is to define which
>>>> medium is used to Publish a primitive form of a CODE FREE Abstract
>>> Proposal
>>>> containing at least the following bullet points.
>>>> - The problem description and motivation
>>>> - Overview of the proposed change
>>>> - Operational impact (compatibility/ plans to upgrades) public API
>>> changes,
>>>> configuration changes, algorithm, and so on
>>>> - Expected benefits and drawbacks
>>>> - Rationale and alternatives
>>>> - Estimate Time to Deliver if possible.
>>>> 
>>>> The way i think this can be is a Github issue where member of the
>>> community
>>>> will interact via comments and the author will be updating the
>>> description
>>>> in the light of comments provided by the community.
>>>> 
>>>> During and near the end of the design discussions the author/s can
>> start
>>>> writing POCs to help guide the review process this naturally will be a
>>> Pull
>>>

Re: Off list major development

[Julian Hyde]

Slim,

I agree with your points that offline development is bad for community. But I 
don’t think you need much mentor help. You have raised valid issues and the 
Druid community needs to decide what its development practices should be.

Julian


> On Jan 2, 2019, at 10:29 AM, Slim Bouguerra  wrote:
> 
> Hello everyone and hope you all have very good holidays.
> 
> First, this email is not directed on the author or the PR
> https://github.com/apache/incubator-druid/pull/6794  it self, but i see
> this PR as a perfect example.
> 
> One of the foundation of Apache Way or what i would simply call open source
> community driven development is that "Technical decisions are discussed,
> decided, and archived publicly.
> developpement"
> Which means that big technical  changes such as the one brought by #/6794
> should have started as a proposal and round of discussions about the major
> changes designs not as 11K line of code.
> I believe such openness will promote a lot of good benefits such as:
> 
> - ensures community health and growth.
> - ensures everyone can participate not only the authors and his co-workers.
> - ensures that the project is driven by the community and not a given
> company or an individual.
> - ensures that there is consensus (not saying 100% agreement;) however it
> means that all individuals will accept the current progress on the project
> until some better proposal is put forth.
> 
> Personally such BIG offline PR makes me feel excluded and doesn't give me a
> sense that i belong to  a community at all.
> 
> To prevent such off list development i think as a Druid Community we need
> to stick to the apache way “If it didn’t happen on the mailing list, it
> didn’t happen.”
> 
> I would appreciate if some of the Apache mentor help with this.
> Thanks


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Mentoring Apache Druid

[Julian Hyde]

Ping! Do we want an additional mentor or not?

> On Dec 17, 2018, at 1:01 PM, Julian Hyde  wrote:
> 
> I had to look up the process for adding a mentor to an existing podling. From 
> the PPMC guide [1]:
> 
>> Adding new mentors
>> 
>> At times, it may be desirable to add a new mentor to a podling. Under all 
>> circumstances, a mentor must be an IPMC member.
>> 
>> IPMC members are free to volunteer to be a mentor to a podling as they see 
>> fit. To do so, they should mail the podling stating their intentions. The 
>> podling should then decide if it wants to add the new mentor or not. If a 
>> decision has been reached to add the mentor, then all podling documentation 
>> should be updated to reflect the change (podlings.xml, 
>> projects/{podling}.xml) and the incubator be notified of the change as well.
>> 
>> If a podling is in a position where they feel they need to add a new mentor, 
>> they may want to drop a mail on the general incubator mailing list to try to 
>> recruit a new mentor.
> 
> Furkan is an IPMC member (and an ASF member). The Druid community needs to 
> decide whether they would like a new mentor. It’s up to you whether to use a 
> vote - but in my opinion consensus would be sufficient.
> 
> Julian
> 
> [1] https://incubator.apache.org/guides/ppmc.html#adding_new_mentors
> 
>> On Dec 12, 2018, at 11:58 AM, Furkan KAMACI  wrote:
>> 
>> Hi Julian,
>> 
>> Thanks! I believe that Druid is promising and it would be great to assist
>> it through its path graduating from Incubation.
>> 
>> Kind Regards,
>> Furkan KAMACI
>> 
>> On Tue, Dec 11, 2018 at 12:11 PM Julian Hyde  wrote:
>> 
>>> Furkan,
>>> 
>>> Thanks for reaching out. We would be delighted to have some extra help! I
>>> am one of the mentors, and Taylor Goetz and Jun Rao are also active on this
>>> list.
>>> 
>>> Julian
>>> 
>>> 
>>>> On Dec 7, 2018, at 11:51 PM, Furkan KAMACI 
>>> wrote:
>>>> 
>>>> Hi All,
>>>> 
>>>> I'm Furkan KAMACI who is a person loves open source, Java, and Machine
>>>> Learning. I'm a member of The Apache Software Foundation, PMC member of
>>>> Apache Incubator, Committer and PMC member of Apache Gora, Committer and
>>>> PMC member of Apache Nutch, Committer of Apache ManifoldCF and mail list
>>>> moderator of Apache Solr. I'm also a member of W3C (World Wide Web
>>>> Consortium).
>>>> 
>>>> I've developed software and managed teams which creates products on
>>>> analyzing Petabytes of data to run efficient Machine Learning and Search
>>>> algorithms on Big Data. I have a work experience +10 years including
>>>> companies as like Alcatel-Lucent/Nokia and has an academical background.
>>>> Currently, I have a company named as LAGOM which works on Big Data and
>>>> Machine Learning and contributes to open source projects.
>>>> 
>>>> I've contributed to many ASF projects throughout the years including
>>> Gora,
>>>> ManifoldCF, Nutch, Solr/Lucene, Stanbol, Clerezza, Commons CSV, Fluo and
>>>> Druid.
>>>> 
>>>> I believe the power of Druid and would like to take part in its journey
>>>> into becoming a TLP of Apache and want to take responsibility as a
>>> Mentor.
>>>> 
>>>> PS: My Apache and GitHub id are kamaci.
>>>> 
>>>> Kind Regards,
>>>> Furkan KAMACI
>>> 
>>> 
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
>>> For additional commands, e-mail: dev-h...@druid.apache.org
>>> 
>>> 
> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Drop 0. from the version

[Julian Hyde]

Our messages crossed, so I didn’t see the message where you said “versioning 
doesn’t need to be semantic”.

I did not say that Druid should do SemVer; I said that you should have a 
policy, and discuss how that policy differs from SemVer.

Julian

> On Dec 21, 2018, at 10:57 AM, Roman Leventov  wrote:
> 
> Julian, I mentioned semantic versioning in the previous message. The
> comparison with IntelliJ may seem shocking at first, but actually Druid may
> be semantically closer to IntelliJ than to Hadoop or Spark. Druid is a
> sever-side *application*, not a library or framework. Like Druid has
> extensions API, IntelliJ has plugin API, that is also very unstable and
> broken in every IntelliJ release, as far as I know.
> 
> Comparison with Guava doesn't make a lot of sense - exactly because of
> that. Guava is a library, half of the Java world depends on it. Almost
> nothing depends on Druid.
> 
> On Fri, 21 Dec 2018 at 19:46, Julian Hyde  wrote:
> 
>> No one has mentioned Semantic Versioning (semver [1]) yet, so I will.
>> I don't know whether the Druid developers think in terms of semver,
>> but a lot of your audience will. In semver, the shift from 0. to 1. is
>> a big event, because the "only remove APIs in major releases" rule
>> does not apply for versions < 1.0.
>> 
>> It would be good if Druid had a policy for how long APIs and will be
>> around. It does not need to be based on semver, but if it isn't, it
>> should explain how it is different than semver.
>> 
>> It should also spell out the planned release cadence. It sounds like
>> you're thinking of two major versions per year, which sounds great.
>> But note that Guava did exactly that, and got flak from a lot of
>> people because features would move from supported to deprecated to
>> gone in just six months.
>> 
>> Regarding combining release 1.0 with Druid's graduation. My gut says
>> no. A lot of people mistakenly think that graduation is a sign of
>> product maturity, whereas it's actually a sign of *community*
>> maturity. You don't want to play into those misconceptions and make
>> people think that Druid is less mature than it really is. (For the
>> record, I think Druid's product and community are both very mature.) I
>> think of 1.0 and graduation as two separate opportunities to generate
>> some news. For 1.0 you can talk about how Druid is industry strength,
>> has wide adoption at large scales; for graduation you can talk about
>> community and what that brings to governance, and adapting to market
>> needs.
>> 
>> Also regarding that "1.0" thing. How about going straight to 14.0?
>> 
>> Julian
>> 
>> [1] https://semver.org/
>> On Fri, Dec 21, 2018 at 10:10 AM Charles Allen  wrote:
>>> 
>>> If I'm greedily honest, I don't want to maintain that many backport
>>> channels. I'd rather have "If you want XYZ backport for version 14, then
>>> you have to take the latest minor version for 14" and have a policy to
>>> where someone can upgrade from 14.x --> 14.latest with (hopefully) no
>>> config changes.
>>> 
>>> 
>>> 
>>> 
>>> On Fri, Dec 21, 2018 at 9:03 AM David Glasser >> 
>>> wrote:
>>> 
>>>> One nice advantage to moving out of 0.x is that it frees up a digit on
>> the
>>>> right side to more cleanly differentiate between "minor release (a
>> random
>>>> assortment of bug fixes, small features, etc)" and "patch release
>>>> (literally the minimum delta to give you a security fix or high impact
>> bug
>>>> fix)".
>>>> 
>>>> --dave
>>>> 
>>>> On Fri, Dec 21, 2018 at 8:58 AM Gian Merlino  wrote:
>>>> 
>>>>> I'm not too fussy around whether we do a 1.0 or simply drop the 0.
>> and
>>>> have
>>>>> it be a 14.0 or 15.0 or 16.0 or wherever we are at the time we do
>> it. I
>>>>> also like the quarterly cadence of release-from-master we had before
>> we
>>>> got
>>>>> blocked on the ASF transition, and would like to pick that back up
>> again
>>>>> (with the next branch cut from master at the end of January, since
>> we did
>>>>> the 0.13.0 branch cut in late October).
>>>>> 
>>>>> Seems to me that good points of discussion are, what should we use
>> as the
>>>>> rule for incrementing the major version? Do we do what we've been
>> doing
>>>>> (incrementing whenever there'

Re: Drop 0. from the version

[Julian Hyde]

No one has mentioned Semantic Versioning (semver [1]) yet, so I will.
I don't know whether the Druid developers think in terms of semver,
but a lot of your audience will. In semver, the shift from 0. to 1. is
a big event, because the "only remove APIs in major releases" rule
does not apply for versions < 1.0.

It would be good if Druid had a policy for how long APIs and will be
around. It does not need to be based on semver, but if it isn't, it
should explain how it is different than semver.

It should also spell out the planned release cadence. It sounds like
you're thinking of two major versions per year, which sounds great.
But note that Guava did exactly that, and got flak from a lot of
people because features would move from supported to deprecated to
gone in just six months.

Regarding combining release 1.0 with Druid's graduation. My gut says
no. A lot of people mistakenly think that graduation is a sign of
product maturity, whereas it's actually a sign of *community*
maturity. You don't want to play into those misconceptions and make
people think that Druid is less mature than it really is. (For the
record, I think Druid's product and community are both very mature.) I
think of 1.0 and graduation as two separate opportunities to generate
some news. For 1.0 you can talk about how Druid is industry strength,
has wide adoption at large scales; for graduation you can talk about
community and what that brings to governance, and adapting to market
needs.

Also regarding that "1.0" thing. How about going straight to 14.0?

Julian

[1] https://semver.org/
On Fri, Dec 21, 2018 at 10:10 AM Charles Allen  wrote:
>
> If I'm greedily honest, I don't want to maintain that many backport
> channels. I'd rather have "If you want XYZ backport for version 14, then
> you have to take the latest minor version for 14" and have a policy to
> where someone can upgrade from 14.x --> 14.latest with (hopefully) no
> config changes.
>
>
>
>
> On Fri, Dec 21, 2018 at 9:03 AM David Glasser 
> wrote:
>
> > One nice advantage to moving out of 0.x is that it frees up a digit on the
> > right side to more cleanly differentiate between "minor release (a random
> > assortment of bug fixes, small features, etc)" and "patch release
> > (literally the minimum delta to give you a security fix or high impact bug
> > fix)".
> >
> > --dave
> >
> > On Fri, Dec 21, 2018 at 8:58 AM Gian Merlino  wrote:
> >
> > > I'm not too fussy around whether we do a 1.0 or simply drop the 0. and
> > have
> > > it be a 14.0 or 15.0 or 16.0 or wherever we are at the time we do it. I
> > > also like the quarterly cadence of release-from-master we had before we
> > got
> > > blocked on the ASF transition, and would like to pick that back up again
> > > (with the next branch cut from master at the end of January, since we did
> > > the 0.13.0 branch cut in late October).
> > >
> > > Seems to me that good points of discussion are, what should we use as the
> > > rule for incrementing the major version? Do we do what we've been doing
> > > (incrementing whenever there's either an incompatible change in extension
> > > APIs, or in query APIs, or when necessary to preserve the ability to
> > always
> > > be able to roll forward/back one major release). Or do we do something
> > else
> > > (Roman seems to be suggesting dropping extension APIs from
> > consideration).
> > >
> > > And also, what does 1.0 or 14.0 or 15.0 or what-have-you mean to us? Is
> > it
> > > something that should be tied to ASF graduation? Completeness of vision?
> > > Stability of APIs or operational characteristics? Something else? You are
> > > right that it is sort of a marketing/mentality thing, so it's an
> > > opportunity for us to declare that we feel Druid has reached some
> > > milestone. My feeling at this time is probably ASF graduation or
> > > completeness of vision (see my earlier mail for thoughts there) are the
> > > ones that make most sense to me.
> > >
> > > On Fri, Dec 21, 2018 at 10:41 AM Charles Allen 
> > wrote:
> > >
> > > > Is there any feeling in the community that the logic behind the
> > releases
> > > > needs to change?
> > > >
> > > > If so then I think we should discuss what that release cadence needs to
> > > > look like.
> > > >
> > > > If not then dropping the 0. prefix is a marketing / mental item. Kind
> > of
> > > > like the 3.x->4.x Linux kernel upgrade. If this is the case then would
> > we
> > > > even want to go with 1.x? I think Roman's proposal would work fine in
> > > this
> > > > case. Where we just call it Apache Druid 14 (or 15 or whatever it is
> > when
> > > > we get there) and just keep the same logic for when we release stuff,
> > > which
> > > > has been something like:
> > > >
> > > > For a X.Y release, going to a X.? release should be very straight
> > forward
> > > > for anyone running stock Druid.
> > > > For a X.Y release, going to a (X+1).? or from a (X+1).? back to an X.Y
> > > > release should be feasible. It might require running a 

Re: Mentoring Apache Druid

[Julian Hyde]

I had to look up the process for adding a mentor to an existing podling. From 
the PPMC guide [1]:

> Adding new mentors
> 
> At times, it may be desirable to add a new mentor to a podling. Under all 
> circumstances, a mentor must be an IPMC member.
> 
> IPMC members are free to volunteer to be a mentor to a podling as they see 
> fit. To do so, they should mail the podling stating their intentions. The 
> podling should then decide if it wants to add the new mentor or not. If a 
> decision has been reached to add the mentor, then all podling documentation 
> should be updated to reflect the change (podlings.xml, 
> projects/{podling}.xml) and the incubator be notified of the change as well.
> 
> If a podling is in a position where they feel they need to add a new mentor, 
> they may want to drop a mail on the general incubator mailing list to try to 
> recruit a new mentor.

Furkan is an IPMC member (and an ASF member). The Druid community needs to 
decide whether they would like a new mentor. It’s up to you whether to use a 
vote - but in my opinion consensus would be sufficient.

Julian

[1] https://incubator.apache.org/guides/ppmc.html#adding_new_mentors

> On Dec 12, 2018, at 11:58 AM, Furkan KAMACI  wrote:
> 
> Hi Julian,
> 
> Thanks! I believe that Druid is promising and it would be great to assist
> it through its path graduating from Incubation.
> 
> Kind Regards,
> Furkan KAMACI
> 
> On Tue, Dec 11, 2018 at 12:11 PM Julian Hyde  wrote:
> 
>> Furkan,
>> 
>> Thanks for reaching out. We would be delighted to have some extra help! I
>> am one of the mentors, and Taylor Goetz and Jun Rao are also active on this
>> list.
>> 
>> Julian
>> 
>> 
>>> On Dec 7, 2018, at 11:51 PM, Furkan KAMACI 
>> wrote:
>>> 
>>> Hi All,
>>> 
>>> I'm Furkan KAMACI who is a person loves open source, Java, and Machine
>>> Learning. I'm a member of The Apache Software Foundation, PMC member of
>>> Apache Incubator, Committer and PMC member of Apache Gora, Committer and
>>> PMC member of Apache Nutch, Committer of Apache ManifoldCF and mail list
>>> moderator of Apache Solr. I'm also a member of W3C (World Wide Web
>>> Consortium).
>>> 
>>> I've developed software and managed teams which creates products on
>>> analyzing Petabytes of data to run efficient Machine Learning and Search
>>> algorithms on Big Data. I have a work experience +10 years including
>>> companies as like Alcatel-Lucent/Nokia and has an academical background.
>>> Currently, I have a company named as LAGOM which works on Big Data and
>>> Machine Learning and contributes to open source projects.
>>> 
>>> I've contributed to many ASF projects throughout the years including
>> Gora,
>>> ManifoldCF, Nutch, Solr/Lucene, Stanbol, Clerezza, Commons CSV, Fluo and
>>> Druid.
>>> 
>>> I believe the power of Druid and would like to take part in its journey
>>> into becoming a TLP of Apache and want to take responsibility as a
>> Mentor.
>>> 
>>> PS: My Apache and GitHub id are kamaci.
>>> 
>>> Kind Regards,
>>> Furkan KAMACI
>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
>> For additional commands, e-mail: dev-h...@druid.apache.org
>> 
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Mentoring Apache Druid

[Julian Hyde]

Furkan,

Thanks for reaching out. We would be delighted to have some extra help! I am 
one of the mentors, and Taylor Goetz and Jun Rao are also active on this list.

Julian


> On Dec 7, 2018, at 11:51 PM, Furkan KAMACI  wrote:
> 
> Hi All,
> 
> I'm Furkan KAMACI who is a person loves open source, Java, and Machine
> Learning. I'm a member of The Apache Software Foundation, PMC member of
> Apache Incubator, Committer and PMC member of Apache Gora, Committer and
> PMC member of Apache Nutch, Committer of Apache ManifoldCF and mail list
> moderator of Apache Solr. I'm also a member of W3C (World Wide Web
> Consortium).
> 
> I've developed software and managed teams which creates products on
> analyzing Petabytes of data to run efficient Machine Learning and Search
> algorithms on Big Data. I have a work experience +10 years including
> companies as like Alcatel-Lucent/Nokia and has an academical background.
> Currently, I have a company named as LAGOM which works on Big Data and
> Machine Learning and contributes to open source projects.
> 
> I've contributed to many ASF projects throughout the years including Gora,
> ManifoldCF, Nutch, Solr/Lucene, Stanbol, Clerezza, Commons CSV, Fluo and
> Druid.
> 
> I believe the power of Druid and would like to take part in its journey
> into becoming a TLP of Apache and want to take responsibility as a Mentor.
> 
> PS: My Apache and GitHub id are kamaci.
> 
> Kind Regards,
> Furkan KAMACI


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: PR Milestone policy

[Julian Hyde]

FJ,

What you are proposing sounds suspiciously like project management. If a 
contributor makes a contribution, that contribution should be given a fair 
review in a timely fashion and be committed based on its merits. You overstate 
the time-sensitivity of contributions. I would imagine that there are only a 
few days preceding each release where stability is a major concern. At any 
other times, contributions can go in after a review.

Remember that in Apache, no one person or group of people determines the 
technical direction of the project, nor the timing of the releases. 
Contributions are accepted based on merit, and release timing is determined by 
consensus.

Let’s be sure not to overuse milestone policy. Milestones should be for 
guidance only.

Julian


> On Dec 6, 2018, at 10:12 AM, Fangjin Yang  wrote:
> 
> Roman - one of the roles of a committer is to make decisions on what is
> best for Druid and the Druid community. If a committer feels that their PR
> should be included in the next release, they should make an argument of why
> that is. Conversely, if folks in the community feel that a PR should not be
> included, they should be free to voice their opinion as well.
> 
> Many of the community contributions I see today are adding value to the
> project and we should try to include them in upcoming releases. The PRs I
> see adding no value are unnecessary refactoring of that serve no real
> purpose. They don't make the code stable, easier to maintain, or add new
> features, and look to be submitted only to increase total contribution line
> count to Druid. I think we should aim to prevent these types of PRs in any
> release because they don't serve to benefit the community.
> 
> On Tue, Dec 4, 2018 at 5:24 AM Roman Leventov  wrote:
> 
>> Fangjin, what you suggest will lead to just one thing - all committers will
>> always assign their PRs to the next release milestone. In addition, you
>> also assign PRs from non-committers to the next release milestone. So
>> nearly 100% of new PRs will have that milestone. It will make this whole
>> activity pointless, because the milestone will not tell release managers
>> anything. Except maybe creating unneeded sense of rush.
>> 
>> Currently in Druid, there is a good fraction of PRs that are merged very
>> quickly (in a matter of days and sometimes hours), but there are also quite
>> some less lucky PRs that linger for months. For contributors, it's not very
>> important that the PR is merged in 1 hour, it's more important that it
>> appears in the next release. Therefore we need to optimize for the fraction
>> of PRs that are merged in 1 month or less (the average time between
>> creation of a new release branch and a final release date). Reviewers
>> should schedule their time so that there are less PRs that are merged in
>> less than one day, but more PRs that are merged in less than one month.
>> 
>> On Tue, 4 Dec 2018 at 04:28, Julian Hyde  wrote:
>> 
>>> I agree with you that merging PRs promptly is very important for growing
>>> community. Or, if the PR is inadequate, promptly explain to the
>> contributor
>>> what they can do to improve it.
>>> 
>>> Assigning target milestones to bugs and issues that don’t yet have PRs
>> can
>>> be problematic. The person assigning the milestone has stepped into the
>>> role of project manager, unless they are committing to fix the issue
>>> personally. And even then, they are implicitly saying “hold the release
>>> while I work on this code”, which should really be the responsibility of
>>> the release manager alone.
>>> 
>>> Julian
>>> 
>>> 
>>> 
>>> 
>>>> On Dec 3, 2018, at 1:57 PM, Fangjin Yang  wrote:
>>>> 
>>>> Committers. In general I think we should try to be more inclusive of
>> the
>>>> community and people that are interested in contributing to Druid and
>> try
>>>> to get their PRs in as much as possible. This helps to grow the
>>> community.
>>>> To me, this means assigning milestones to contributions, not being
>> overly
>>>> picky on code (if it has no real impact on functionality/performance).
>> If
>>>> we need to push PRs back to a later release because they are
>> complicated
>>>> and require more review, we can always do that.
>>>> 
>>>> On Tue, Nov 27, 2018 at 4:45 PM Julian Hyde  wrote:
>>>> 
>>>>> Fangjin,
>>>>> 
>>>>> You wrote
>>>>> 
>>>>>> we should try to assign milestones to PRs we want
>>>>

Re: [VOTE] Release Apache Druid (incubating) 0.13.0 [RC4]

[Julian Hyde]

+1 (binding)

Downloaded, built and ran tests on Java 8/Ubuntu. Checked LICENSE, NOTICE, 
DISCLAIMER. Ran RAT. Checked that git matches contents of src tarball.

The issues I noticed last time with GPL licenses, files missing headers have 
been fixed.

Julian


> On Dec 5, 2018, at 1:47 PM, Jihoon Son  wrote:
> 
> +1
> 
> - Verified signatures and checksums of src and bin.
> - Unit tests and integration tests passed.
> 
> Jihoon
> 
> On Tue, Dec 4, 2018 at 10:49 AM Gian Merlino  wrote:
> 
>> +1
>> 
>> - Verified signatures and checksums of both src and bin packages.
>> - Source tarball matches tag.
>> - Source tarball builds and tests pass.
>> - Ran through quickstart on binary tarball.
>> 
>> On Mon, Dec 3, 2018 at 5:57 PM benedictjin2...@gmail.com <
>> benedictjin2...@gmail.com> wrote:
>> 
>>> 
>>> 
>>> On 2018/12/02 09:50:55, David Lim  wrote:
 Hi all,
 
 I have created a build for Apache Druid (incubating) 0.13.0, release
 candidate 4.
 
 A list of the patches applied can be found here:
  Since rc1:
 
>>> 
>> https://github.com/apache/incubator-druid/pulls?utf8=%E2%9C%93=is%3Apr+base%3A0.13.0-incubating+merged%3A%3C2018-11-17T06+
  Since rc2:
 
>>> 
>> https://github.com/apache/incubator-druid/pulls?utf8=%E2%9C%93=is%3Apr+base%3A0.13.0-incubating+merged%3A2018-11-16T05..2018-11-17T06+
  Since rc3:
 
>>> 
>> https://github.com/apache/incubator-druid/pulls?utf8=%E2%9C%93=is%3Apr+base%3A0.13.0-incubating+merged%3A2018-11-30..2018-12-02+
 
 Thanks to everyone who has contributed to this release! You can read
>> the
 proposed release notes here:
 https://github.com/apache/incubator-druid/issues/6442
 
 The release candidate has been tagged in GitHub as
 druid-0.13.0-incubating-rc4 (cf15aac), available here:
 
>>> 
>> https://github.com/apache/incubator-druid/releases/tag/druid-0.13.0-incubating-rc4
 
 The artifacts to be voted on are located here:
 
>>> 
>> https://dist.apache.org/repos/dist/dev/incubator/druid/0.13.0-incubating-rc4
 
 A staged Maven repository is available for review at:
 
>> https://repository.apache.org/content/repositories/orgapachedruid-1002/
 
 Release artifacts are signed with the key [7183DE56]:
 https://people.apache.org/keys/committer/davidlim.asc (also available
>>> here:
 http://pgp.mit.edu/pks/lookup?search=davidlim%40apache.org=index)
 
 This key and the key of other committers can also be found in the
>>> project's
 KEYS file here:
>>> https://dist.apache.org/repos/dist/dev/incubator/druid/KEYS
 
 Please review the proposed artifacts and vote. As this is our first
>>> release
 under the Apache Incubator program, note that Apache has specific
 requirements that must be met before +1 binding votes can be cast by
>> PMC
 members. Please refer to the policy at
 http://www.apache.org/legal/release-policy.html#policy for more
>> details.
 
 As part of the validation process, the release artifacts can be
>> generated
 from source by running: mvn clean install -Papache-release,dist,rat
 
 This vote will be open for at least 72 hours. The vote will pass if a
 majority of at least three +1 PMC votes are cast.
 
 Once the vote has passed, the second stage vote will be called on the
 Apache Incubator mailing list to get approval from the Incubator PMC.
 
 [ ] +1 Release this package as Apache Druid (incubating) 0.13.0
 [ ]  0 I don't feel strongly about it, but I'm okay with the release
 [ ] -1 Do not release this package because...
 
 Starting with my +1 (binding)
 
 Thanks!
 +1
>>> 
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
>>> For additional commands, e-mail: dev-h...@druid.apache.org
>>> 
>>> 
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: PR Milestone policy

[Julian Hyde]

I agree with you that merging PRs promptly is very important for growing 
community. Or, if the PR is inadequate, promptly explain to the contributor 
what they can do to improve it.

Assigning target milestones to bugs and issues that don’t yet have PRs can be 
problematic. The person assigning the milestone has stepped into the role of 
project manager, unless they are committing to fix the issue personally. And 
even then, they are implicitly saying “hold the release while I work on this 
code”, which should really be the responsibility of the release manager alone.

Julian




> On Dec 3, 2018, at 1:57 PM, Fangjin Yang  wrote:
> 
> Committers. In general I think we should try to be more inclusive of the
> community and people that are interested in contributing to Druid and try
> to get their PRs in as much as possible. This helps to grow the community.
> To me, this means assigning milestones to contributions, not being overly
> picky on code (if it has no real impact on functionality/performance). If
> we need to push PRs back to a later release because they are complicated
> and require more review, we can always do that.
> 
> On Tue, Nov 27, 2018 at 4:45 PM Julian Hyde  wrote:
> 
>> Fangjin,
>> 
>> You wrote
>> 
>>> we should try to assign milestones to PRs we want
>>> to get in
>> 
>> Can you please define “we”? Do you mean committers, PMC members, release
>> managers, everyone?
>> 
>> Julian
>> 
>> 
>>> On Nov 26, 2018, at 8:43 AM, Roman Leventov  wrote:
>>> 
>>> About a year ago, Gian wrote (
>>> 
>> https://groups.google.com/forum/#!msg/druid-development/QPZUIzLtZ2I/eZyw8BoVCgAJ
>>> ):
>>> 
>>> "For milestones, I think it would work to use them only for PRs/issues
>> that
>>> are truly release blockers -- should be limited to critical bugs
>> discovered
>>> after a release branch is cut. We could make release notes the way you
>>> suggest, by walking the commit history."
>>> 
>>> Today, Fangjin wrote (
>>> 
>> https://github.com/apache/incubator-druid/pull/6656#issuecomment-441698159
>> ):
>>> 
>>> "I think where possible we should try to assign milestones to PRs we want
>>> to get in and aim to have the PR reviewed and merged before then. If the
>> PR
>>> needs to be pushed back to a future release we can always do that."
>>> 
>>> Personally I don't agree with the second take and differentiating non-bug
>>> fixing PRs by their "importance". I think the proportion of PRs that are
>>> assigned the next milestone by committer will depend on self-confidence
>> of
>>> the committer and politics, not the objective importance of the PRs. It
>>> will also make possible for some minor PRs to be sidetracked for
>> extremely
>>> long time if not forever, because there always other more important PRs
>> to
>>> work on. While true in the short and mid run, this is very frustrating
>> for
>>> the authors and could turn them away from contributing into Druid, that
>> is
>>> bad in the long run. Actually this thing happens already sometimes and we
>>> should think how to address that, but differentiating PRs could
>>> only exacerbate this effect.
>>> 
>>> Instead, I think the importance of PR should grow with the time passed
>>> since the author addressed all comments (or just created the PR) and the
>> PR
>>> passed automated checks. I. e. a freshly created PR may be not super
>>> important, but if it passes all checks and is open for two months without
>>> reviews, the PR becomes more important to review. This should help to
>>> reduce the variance in PR's time-to-merge and improve the average
>>> contributor experience. In the long run I think it's healthier than
>>> squeezing one extra feature into the very next release.
>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
>> For additional commands, e-mail: dev-h...@druid.apache.org
>> 
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [VOTE] Release Apache Druid (incubating) 0.13.0 [RC3]

[Julian Hyde]

David,

It is customary to close every vote with a either a “cancel” or “result” email. 
In this case, can you send an email with the subject line

  [CANCEL] [VOTE] Release Apache Druid (incubating) 0.13.0 [RC3]

Julian


> On Dec 2, 2018, at 1:53 AM, David Lim  wrote:
> 
> Hello,
> 
> Voting on this thread is now closed in favor of RC4. Please take a look at
> the proposed RC4 artifacts and vote!
> 
> https://lists.apache.org/thread.html/17b509c359509ff450904ab53500a5b08067bae9640f669056d06884@%3Cdev.druid.apache.org%3E
> 
> On Thu, Nov 29, 2018 at 5:19 PM David Lim  wrote:
> 
>> Thanks Jihoon, I called off the IPMC vote on the general mailing list.
>> 
>> On Thu, Nov 29, 2018 at 4:35 PM Jihoon Son  wrote:
>> 
>>> Hi all,
>>> 
>>> I recently found some bugs reported in
>>> https://github.com/apache/incubator-druid/issues/6684.
>>> Since some overlord APIs and UI function don't work because of these bugs,
>>> I think they are the blocker for 0.13.0 release.
>>> 
>>> Jihoon
>>> 
>>> On Tue, Nov 27, 2018 at 11:17 PM David Lim  wrote:
>>> 
 IPMC voting thread:
 
 
>>> https://lists.apache.org/thread.html/bdca6f6d757361cd599e35e7a201ade67d776474496a7c927e20609c@%3Cgeneral.incubator.apache.org%3E
 
 On Tue, Nov 27, 2018 at 11:38 PM David Lim  wrote:
 
> Thank you all for participating in the release process!
> 
> The 0.13.0-rc3 vote has now been open for 10 days (38 days since
> 0.13.0-rc1) and I am proposing that we move onto the next step, which
>>> is
 to
> present the rc3 release artifacts for approval as Apache Druid
 (incubating)
> 0.13.0 by the Incubator PMC. The results from the community vote
>>> (adding
 my
> vote as a +1):
> 
> David Lim: +1 (binding)
> Fangjin Yang: +1 (binding)
> Jihoon Son: +1 (binding)
> Yuvaraj Loganathan: +1
> Benedict Jin: +1
> Jonathan Wei: +1 (binding)
> Furkan Kamaci: +1
> Gian Merlino: +1 (binding)
> Nishant Bangarwa: +1 (binding)
> Clint Wylie: +1
> 
> I will be starting a voting thread on the general incubator mailing
>>> list
> to request an IPMC vote.
> 
> Regards,
> David
> 
> On Tue, Nov 27, 2018 at 1:59 PM Clint Wylie 
 wrote:
> 
>> +1
>> 
>> src package:
>> - verified signature/hash
>> - verify LICENSE, NOTICE, DISCLAIMER file present and contents look
>>> good
>> - compiled and ran unit tests, passed
>> - ran integration tests, passed
>> - built binary distribution, started local cluster with compiled
>> artifacts,
>> tested indexing/query with native indexing, wikipedia example data,
>> everything functional
>> 
>> bin package:
>> - verified signature/hash
>> - started local cluster, tested indexing/query with native indexing,
>> wikipedia example data, everything functional
>> 
>> On Mon, Nov 26, 2018 at 7:19 AM Furkan KAMACI <
>>> furkankam...@gmail.com>
>> wrote:
>> 
>>> Hi,
>>> 
>>> - Compiled source code
>>> - Run unit tests
>>> - Run integration tests
>>> 
>>> +1 (non-binding)
>>> 
>>> Kind Regards,
>>> Furkan KAMACI
>>> 
>>> On Mon, Nov 26, 2018 at 10:44 AM Jonathan Wei 
>> wrote:
>>> 
 +1
 
 - Verified signature
 - Ran unit tests and integration tests
 - Built the distribution and ran all of the quickstart examples
 
 On Sun, Nov 25, 2018 at 11:31 PM Jonathan Wei 
>> wrote:
 
> +1
> 
> - Verified signature
> - Ran unit tests and integration tests
> - Built the distribution and ran all of the quickstart examples
> 
> On Sun, Nov 25, 2018 at 8:01 PM 金嘉怡 >>> 
>> wrote:
> 
>> +1
>> 
>> On 2018/11/17 07:07:23, David Lim  wrote:
>>> Hi all,>
>>> 
>>> I have created a build for Apache Druid (incubating) 0.13.0,
>>> release>
>>> candidate 3.>
>>> 
>>> A list of the patches applied can be found here:>
>>>  Since rc1:>
>>> 
>> 
 
>>> 
>> 
 
>>> https://github.com/apache/incubator-druid/pulls?utf8=%E2%9C%93=is%3Apr+base%3A0.13.0-incubating+merged%3A%3C2018-11-17T06+
> 
>> 
>>>  Since rc2:>
>>> 
>> 
 
>>> 
>> 
 
>>> https://github.com/apache/incubator-druid/pulls?utf8=%E2%9C%93=is%3Apr+base%3A0.13.0-incubating+merged%3A2018-11-16T05..2018-11-17T06+
> 
>> 
>>> 
>>> Thanks to everyone who has contributed to this release! You
>>> can
>> read
>> the>
>>> proposed release notes here:>
>>> https://github.com/apache/incubator-druid/issues/6442>
>>> 
>>> The release candidate has been tagged in GitHub as>
>>> druid-0.13.0-incubating-rc3 (56c97b2), available here:>
>>> 
>> 
 

Re: PR Milestone policy

[Julian Hyde]

Fangjin,

You wrote

> we should try to assign milestones to PRs we want
> to get in

Can you please define “we”? Do you mean committers, PMC members, release 
managers, everyone?

Julian


> On Nov 26, 2018, at 8:43 AM, Roman Leventov  wrote:
> 
> About a year ago, Gian wrote (
> https://groups.google.com/forum/#!msg/druid-development/QPZUIzLtZ2I/eZyw8BoVCgAJ
> ):
> 
> "For milestones, I think it would work to use them only for PRs/issues that
> are truly release blockers -- should be limited to critical bugs discovered
> after a release branch is cut. We could make release notes the way you
> suggest, by walking the commit history."
> 
> Today, Fangjin wrote (
> https://github.com/apache/incubator-druid/pull/6656#issuecomment-441698159):
> 
> "I think where possible we should try to assign milestones to PRs we want
> to get in and aim to have the PR reviewed and merged before then. If the PR
> needs to be pushed back to a future release we can always do that."
> 
> Personally I don't agree with the second take and differentiating non-bug
> fixing PRs by their "importance". I think the proportion of PRs that are
> assigned the next milestone by committer will depend on self-confidence of
> the committer and politics, not the objective importance of the PRs. It
> will also make possible for some minor PRs to be sidetracked for extremely
> long time if not forever, because there always other more important PRs to
> work on. While true in the short and mid run, this is very frustrating for
> the authors and could turn them away from contributing into Druid, that is
> bad in the long run. Actually this thing happens already sometimes and we
> should think how to address that, but differentiating PRs could
> only exacerbate this effect.
> 
> Instead, I think the importance of PR should grow with the time passed
> since the author addressed all comments (or just created the PR) and the PR
> passed automated checks. I. e. a freshly created PR may be not super
> important, but if it passes all checks and is open for two months without
> reviews, the PR becomes more important to review. This should help to
> reduce the variance in PR's time-to-merge and improve the average
> contributor experience. In the long run I think it's healthier than
> squeezing one extra feature into the very next release.


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Weekly dev sync minutes (2018-11-27)

[Julian Hyde]

Thanks for writing up the minutes. I know it’s a considerable effort to write 
minutes, but it allows me to keep up with what’s going on in the project. Also, 
an email to the dev list is an excellent opportunity to spawn follow-up 
discussions. And discussion on the dev list or in GitHub issues is exactly what 
we want.

Here’s a query to find the minutes: 
https://lists.apache.org/list.html?dev@druid.apache.org:gte=0d:dev%20sync%20minutes
 

 

Julian


> On Nov 27, 2018, at 11:17 AM, Gian Merlino  wrote:
> 
> Thanks, Dave!
> 
> On Tue, Nov 27, 2018 at 10:51 AM David Lim  wrote:
> 
>> Attendees: David Lim, Jihoon Son, Atul, Clint Wylie, Eyal Yurman, Roman
>> Leventov
>> 
>> Following the discussion after last week's sync, we will be taking meeting
>> minutes of the weekly dev sync going forward so that everyone in the
>> community can be informed. At the start of each meeting, the moderator
>> should issue a call for a volunteer who will be responsible for taking the
>> minutes and posting them to the dev mailing list afterwards.
>> 
>> On the call today, we decided that posting the minutes to the dev mailing
>> list (as opposed to adding them to a wiki, Google docs, etc.) would be
>> beneficial through increased visibility and community engagement. We may
>> also want to create an index page with links to the mail archive so that
>> previous minutes can be located easily.
>> 
>> Other matters that were brought up:
>> 
>> 0.13.0-rc3 has been out for about 10 days now with no major issues
>> reported. Imply has been running that build for the past week; Atul
>> mentioned that he has deployed 0.13.0-rc1 to their cluster and it has been
>> running without issues (they plan to upgrade to rc3 this week).
>> 
>> If no issues come up in the next few days, we will present 0.13.0-rc3 to
>> the incubator PMC for their vote on releasing it as Apache Druid 0.13.0.
>> 
>> Roman wanted to highlight issue
>> https://github.com/apache/incubator-druid/issues/ 
>>  which identifies the
>> PasswordProvider abstraction as a potential source of race condition
>> related issues. We decided on the call that this is not a blocker for
>> 0.13.0.
>> 

Re: Sync up this week

[Julian Hyde]

Apache doesn’t have a policy on video per se, but we do have a policy that 
conversations should be open to all. Due to time-zones etc. some people may not 
be able to attend the dev syncs, and those people must not feel excluded.

If consent is preventing you from recording the dev syncs, perhaps you can 
state in the invite that attendance implies consent?

Or do what the Drill community does: send out minutes after each meeting. See 
“Hangout” on https://drill.apache.org/community-resources/ 
.

Also remember that in Apache, decisions must be made on-list. The dev sync is a 
great place to have discussions and share information, but you must bring the 
discussion onto the dev list if there are decisions to be made.

Julian


> On Nov 13, 2018, at 4:43 PM, Charles Allen  
> wrote:
> 
> We had been recording it into youtube, but something broke and the youtube
> channel didn't work anymore. No one had the time or willpower to fix it. I
> know I won't be able to release video that I record without getting a few
> forms filled out by everyone involved. I don't know if ASF has any policy
> on releasing videos.
> 
> 
> 
> On Tue, Nov 13, 2018 at 4:33 PM Jun Rao  wrote:
> 
>> Hi, Charles,
>> 
>> Are those dev sync meetings being recorded? It would be useful to make the
>> recordings or at least the meeting notes available for public access (e.g.
>> Apache wiki).
>> 
>> Thanks,
>> 
>> Jun
>> 
>> On Tue, Nov 13, 2018 at 9:15 AM Charles Allen
>>  wrote:
>> 
>>> Hi all!
>>> 
>>> I have an off-site today so will not be able to host the sync up. Is
>> anyone
>>> else able to host?
>>> 
>>> Thank you,
>>> Charles Allen
>>> 
>> 

Re: Druid PR review checklist

[Julian Hyde]

Thanks for starting this thread, Roman. It’s a great discussion to be having.

A word of caution about google docs. Since this one can be edited by anyone who 
has the link, and the link is posted in a public archive, then at some point 
this doc will fall victim to spam or vandalism. I suggest that after this 
discussion has died down (say a week or so?) you move the content to a more 
protected medium, say a GitHub PR, and remove the doc or make it read-only.

Julian


> On Nov 12, 2018, at 2:42 PM, Roman Leventov  wrote:
> 
> A lot of new committers are expected to enter the projects with rights to
> review and merge PRs.
> 
> I suggest to create a PR review checklist to help new (and old!) reviewers
> (and PR authors, for self-review before even publishing a PR) not to forget
> something.
> 
> I think a PR (because it's not editable by many people) or a Wiki page
> (because it's not commentable) on Github is not an ideal form of
> collaboration for creating an original version of such document, so I
> created a Google document (commentable, editable):
> https://docs.google.com/document/d/17EEKT6fih9Dd5NfXjBoECcKbVp1eOB2vb3jKqTF9pPc/edit?usp=sharing
> 
> Developers are welcome to add comments and list things that they look at
> when doing reviews.
> 
> Note: the list is going to be huge and people are not realistically
> expected to pedantically follow all of it's items on every PR review, but
> IMO such "gold standard" should help to keep the quality of reviews high.


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [VOTE] Release Apache Druid (incubating) 0.13.0 [RC1]

[Julian Hyde]

> On Oct 30, 2018, at 12:04 PM, Gian Merlino  wrote:
> 
> It says
> to use http://static.druid.io/, but we probably won't put the artifacts
> there.

Indeed. It is mandatory that the artifacts and their checksums are hosted on 
apache.org . (There can be mirrors.)

Julian

Re: [VOTE] Release Apache Druid (incubating) 0.13.0 [RC1]

[Julian Hyde]

Your’e right. After I imported the keys, using

$ gpg --recv-keys 58B5D669D2FFD83B37D88DF8BB64B3727183DE56

it worked. I could have done ‘gpg —import KEYS’ also.

Changing my vote to +0 (binding). I would give +1 but the artifacts I am asked 
to review contains a bin.tar.gz and I have no idea how to review binary 
artifacts. Maybe some other reviewers can chime in.

I do know how to find out online how to build Druid. I think it is important 
that src.tar.gz is self-contained, and that includes the necessary build 
instructions. (Just as I really appreciate when a box of pasta has “boil for 12 
minutes” written on the side.)

I don’t know whether Apache has guidelines for PaxHeaders. I’m just saying it 
looked weird to me. And by the way, emacs tar-mode choked on the tar file. Not 
a blocker, just friction.

There is a way to add headers to .md files. And I claim there is just as much 
creativity in these as in source code. Please fix in the next release.

Julian


> On Oct 22, 2018, at 10:04 PM, David Lim  wrote:
> 
> Hi Julian,
> 
> I believe the PaxHeader files are the result of extracting a tarball built
> with POSIX tar with a GNU or other variant. I believe it is a result of
> this configuration:
> https://github.com/apache/incubator-druid/blob/master/distribution/pom.xml#L195
> 
> Does Apache have guidelines on what variant of tar should be used in
> generating release artifacts?
> 
> Also, just thought I would note that we do have published documentation on
> building from source here:
> https://github.com/apache/incubator-druid/blob/master/docs/content/development/build.md
> - but there are a few statements that should be updated, hence my comment
> that I'll make sure the docs get updated.
> 
> Regards,
> David
> 
> 
> On Mon, Oct 22, 2018 at 10:20 PM David Lim  wrote:
> 
>> Hi Julian,
>> 
>> Thank you for the thorough review!
>> 
>> For the GPG key, my understanding is that it was expected that users would
>> fetch the key by either running 'gpg --import KEYS' on the KEYS file (as
>> per https://www.apache.org/dev/release-signing.html#keys-policy) or by
>> importing it from the Apache phonebook (
>> https://people.apache.org/keys/committer/davidlim.asc) or grabbing it
>> from a well-known key server (e.g.
>> http://pgp.mit.edu/pks/lookup?search=davidlim%40apache.org=index). Did
>> this not work for you?
>> 
>>> src.tar.gz file contains files such as
>> ./PaxHeaders.X/apache-druid-0.13.0-incubating-src_indexing-service_src_test_java_org_apache_druid_s
>> I will check to see whether these files are expected or not.
>> 
>> The files you identified from the diff against the git tag are all either
>> expected to be omitted or were generated as part of the source packaging.
>> 
>> For instructions on building the release from source, I mentioned the
>> command in the original post but it may have been somewhat buried. The
>> command I used was:
>> 
>> mvn clean install -Papache-release -Dtar
>> 
>> I'll make sure that this is updated in the docs.
>> 
>> As for the file exclusions, maybe the committers who helped set up the
>> exclusions can comment. I suspect that many of them are either a) files
>> which cannot properly support the licensing header and remain syntactically
>> valid, or b) deemed to be insignificant in its creative content and
>> therefore not protected by copyright law, as per:
>> http://www.apache.org/legal/src-headers.html#faq-exceptions.
>> 
>> Thanks again!
>> 
>> David
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> On Mon, Oct 22, 2018 at 9:32 PM Julian Hyde  wrote:
>> 
>>> -1 due to GPG error; but also see other concerns below and fix them if
>>> there are genuine issues.
>>> 
>>> As this is Druid’s first Apache release candidate, I am impressed by how
>>> many things you got right.
>>> 
>>> I have completely ignored bin.tar.gz. It’s not possible to “release”
>>> binaries in Apache because it’s not possible to audit binaries.
>>> 
>>> I downloaded and checked signatures. .sha512 is ok. GPG failed with “No
>>> public key” error:
>>> 
>>> $ gpg --verify apache-druid-0.13.0-incubating-src.tar.gz.asc
>>> apache-druid-0.13.0-incubating-src.tar.gz
>>> gpg: Signature made Sat Oct 20 20:50:20 2018 PDT
>>> gpg:using RSA key 58B5D669D2FFD83B37D88DF8BB64B3727183DE56
>>> gpg: Can't check signature: No public key
>>> 
>>> src.tar.gz file contains files such as
>>> ./PaxHeaders.X/apache-druid-0.13.0-incubating-src_indexing-service_src_test_java_org_apache_dr

Re: [VOTE] Release Apache Druid (incubating) 0.13.0 [RC1]

[Julian Hyde]

ntegration-tests/docker/middlemanager.conf
integration-tests/docker/overlord.conf
integration-tests/docker/router-no-client-auth-tls.conf
integration-tests/docker/router-permissive-tls.conf
integration-tests/docker/router.conf
integration-tests/docker/run-mysql.sh
integration-tests/docker/sample-data.sql
integration-tests/docker/supervisord.conf
integration-tests/docker/tls/generate-client-certs-and-keystores.sh
integration-tests/docker/tls/generate-expired-client-cert.sh
integration-tests/docker/tls/generate-good-client-cert.sh
integration-tests/docker/tls/generate-incorrect-hostname-client-cert.sh
integration-tests/docker/tls/generate-invalid-intermediate-client-cert.sh
integration-tests/docker/tls/generate-root-certs.sh
integration-tests/docker/tls/generate-server-certs-and-keystores.sh
integration-tests/docker/tls/generate-to-be-revoked-client-cert.sh
integration-tests/docker/tls/generate-untrusted-root-client-cert.sh
integration-tests/docker/tls/generate-valid-intermediate-client-cert.sh
integration-tests/docker/tls/root.cnf
integration-tests/docker/tls/root2.cnf
integration-tests/docker/zookeeper.conf
publications/demo/Makefile
publications/demo/druid_demo.aux
publications/demo/druid_demo.bbl
publications/demo/druid_demo.bib
publications/demo/druid_demo.blg
publications/demo/druid_demo.out
publications/demo/druid_demo.tex
publications/demo/vldb.cls
publications/radstack/.gitignore
publications/radstack/Makefile
publications/radstack/README.md
publications/radstack/radstack.bib
publications/radstack/radstack.tex
publications/radstack/src/druid_plot.R
publications/radstack/src/druid_tables.R
publications/radstack/vldb.cls
publications/whitepaper/.gitignore
publications/whitepaper/Makefile
publications/whitepaper/README.md
publications/whitepaper/acm_proc_article-sp.cls
publications/whitepaper/druid.bib
publications/whitepaper/druid.tex
publications/whitepaper/dummy.ps
publications/whitepaper/modii658-yang.bib
publications/whitepaper/modii658-yang.tex
publications/whitepaper/sig-alternate-2013.cls
publications/whitepaper/src/druid_plot.R
publications/whitepaper/src/druid_tables.R
server/src/main/resources/static/old-console/css/demo_table.css
server/src/main/resources/static/old-console/css/jquery-ui-1.9.2.css
server/src/main/resources/static/old-console/images/favicon.ico
server/src/main/resources/static/old-console/js/jquery-1.11.0.min.js
server/src/main/resources/static/old-console/js/jquery-ui-1.9.2.js
server/src/main/resources/static/old-console/js/jquery.dataTables-1.8.2.js
server/src/main/resources/static/old-console/js/underscore-1.2.2.js







> On Oct 22, 2018, at 10:58 AM, Julian Hyde  wrote:
> 
> Thanks for finding that list, David. There are a lot of things to check. 
> Therefore, before voting +1 you need to do some due diligence, and with your 
> vote you should describe how you validated the release.
> 
> For example, in a recent Calcite release thread[1] a typical vote looked like 
> this:
> 
>> +1 (non-binding)
>> - downloaded, checked gpg and sha256
>> - compiled and ran tests ("mvn clean install") using JDK 8_162, 10.0.1 on 
>> Fedora and Windows
>> - ran simple queries via sqlline
> 
> Everyone should download the artifacts, check sha256 and gpg (asc) 
> signatures, and compile the code.
> 
> Julian
> 
> [1] 
> https://lists.apache.org/thread.html/c85d5f3cf1bbd9e28d76acd6905dee83cb54334c8b5d8979e1894648@%3Cdev.calcite.apache.org%3E
> 
>> On Oct 22, 2018, at 10:47 AM, David Lim  wrote:
>> 
>> I believe what Julian wanted to highlight was this line in the announcement:
>> 
>>> As this is our first release under the Apache Incubator program, note
>> that Apache has specific requirements that must be met before +1 binding
>> votes can be cast by PMC members. Please refer to the policy at
>> http://www.apache.org/legal/release-policy.html#policy for more details.
>> 
>> Some of the statements in that document:
>> 
>> - Before casting +1 binding votes, individuals are REQUIRED to:
>>   - download all signed source code packages onto their own hardware
>>   - verify that they meet all requirements of ASF policy on releases, for
>> example:
>>   - Every ASF release MUST contain one or more source packages, which
>> MUST be sufficient for a user to build and test the release provided they
>> have access to the appropriate platform and tools
>>   - All supplied packages MUST be cryptographically signed by the
>> Release Manager with a detached signature
>>   - Binary/bytecode package MUST have the same version number as the
>> source release and MUST only add binary/bytecode files that are the result
>> of compiling that version of the source code release and its dependencies
>>   - Each package MUST provide a LICENSE file and a NOTICE fi

Re: [VOTE] Release Apache Druid (incubating) 0.13.0 [RC1]

[Julian Hyde]

Thanks for finding that list, David. There are a lot of things to check. 
Therefore, before voting +1 you need to do some due diligence, and with your 
vote you should describe how you validated the release.

For example, in a recent Calcite release thread[1] a typical vote looked like 
this:

> +1 (non-binding)
>  - downloaded, checked gpg and sha256
>  - compiled and ran tests ("mvn clean install") using JDK 8_162, 10.0.1 on 
> Fedora and Windows
>  - ran simple queries via sqlline

Everyone should download the artifacts, check sha256 and gpg (asc) signatures, 
and compile the code.

Julian

[1] 
https://lists.apache.org/thread.html/c85d5f3cf1bbd9e28d76acd6905dee83cb54334c8b5d8979e1894648@%3Cdev.calcite.apache.org%3E

> On Oct 22, 2018, at 10:47 AM, David Lim  wrote:
> 
> I believe what Julian wanted to highlight was this line in the announcement:
> 
>> As this is our first release under the Apache Incubator program, note
> that Apache has specific requirements that must be met before +1 binding
> votes can be cast by PMC members. Please refer to the policy at
> http://www.apache.org/legal/release-policy.html#policy for more details.
> 
> Some of the statements in that document:
> 
> - Before casting +1 binding votes, individuals are REQUIRED to:
>- download all signed source code packages onto their own hardware
>- verify that they meet all requirements of ASF policy on releases, for
> example:
>- Every ASF release MUST contain one or more source packages, which
> MUST be sufficient for a user to build and test the release provided they
> have access to the appropriate platform and tools
>- All supplied packages MUST be cryptographically signed by the
> Release Manager with a detached signature
>- Binary/bytecode package MUST have the same version number as the
> source release and MUST only add binary/bytecode files that are the result
> of compiling that version of the source code release and its dependencies
>- Each package MUST provide a LICENSE file and a NOTICE file which
> account for the package's exact content. LICENSE and NOTICE MUST NOT
> provide unnecessary information about materials which are not bundled in
> the package, such as separately downloaded dependencies. For source
> packages, LICENSE and NOTICE MUST be located at the root of the
> distribution. For additional packages, they MUST be located in the
> distribution format's customary location for licensing materials, such as
> the META-INF directory of Java "jar" files.
>- validate all cryptographic signatures
>- compile as provided
>- test the result on their own platform
> 
> Additionally, as an incubator project, we are required to have a DISCLAIMER
> file indicating that we are undergoing incubation.
> 
> One question I have: for the binary tarball package, we have a LICENSE and
> NOTICE file in the root of the distribution which is what we have always
> done, but I have not also included these files in the individual JAR files
> under META-INF. I thought that having them in the root would be sufficient,
> but now I'm thinking they might actually also need to be in each JAR file
> since those files will be made available through Maven independent of our
> tarball packaging. I checked the Maven artifacts for previous versions of
> Druid and they don't include the LICENSE and NOTICE file in the JAR, but it
> feels to me like this will be required. Thoughts welcome.
> 
> David
> 
> 
> 
> On Mon, Oct 22, 2018 at 9:04 AM Slim Bouguerra 
> wrote:
> 
>> Hey Julian
>> Thanks for pointing that out.
>> 
>> For the Apache related major changes please carefully review
>> https://github.com/apache/incubator-druid/labels/Apache
>> For bugs/features the release note is what you want to check
>> https://github.com/apache/incubator-druid/issues/6442/
>> 
>> 
>> On Sun, Oct 21, 2018 at 6:38 PM Fangjin Yang  wrote:
>> 
>>> +1
>>> 
>>> On Sun, Oct 21, 2018 at 3:34 PM Julian Hyde  wrote:
>>> 
>>>> Hey Slim,
>>>> 
>>>> Since this is an Apache release, and you've voted on Apache releases
>>>> before in Calcite and Hive, can you explain what you checked before
>>>> you voted "+1". There are many folks here who have not been through
>>>> the release process, and we veterans should show them the ropes.
>>>> 
>>>> Julian
>>>> 
>>>> On Sun, Oct 21, 2018 at 1:16 PM Slim Bouguerra <
>> slim.bougue...@gmail.com
>>>> 
>>>> wrote:
>>>>> 
>>>>> +1
>>>>> 
>>>>>> On Oct 21, 2018, at 8:41 AM

Re: [VOTE] Release Apache Druid (incubating) 0.13.0 [RC1]

[Julian Hyde]

Hey Slim,

Since this is an Apache release, and you've voted on Apache releases
before in Calcite and Hive, can you explain what you checked before
you voted "+1". There are many folks here who have not been through
the release process, and we veterans should show them the ropes.

Julian

On Sun, Oct 21, 2018 at 1:16 PM Slim Bouguerra  wrote:
>
> +1
>
> > On Oct 21, 2018, at 8:41 AM, David Lim  wrote:
> >
> > Hi all,
> >
> > I have created a build for Apache Druid (incubating) 0.13.0, release
> > candidate 1.
> >
> > Thanks to everyone who has contributed to this release! You can read the
> > proposed release notes here:
> > https://github.com/apache/incubator-druid/issues/6442
> >
> > The release candidate has been tagged in GitHub as
> > druid-0.13.0-incubating-rc1 (acf15b4), available here:
> > https://github.com/apache/incubator-druid/releases/tag/druid-0.13.0-incubating-rc1
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/incubator/druid/apache-druid-0.13.0-incubating-rc1/
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/davidlim.asc. This key and the key
> > of other committers can also be found in the project's KEYS file here:
> >
> > https://dist.apache.org/repos/dist/dev/incubator/druid/KEYS
> >
> > (If you are a committer, please feel free to add your own key to that file
> > by following the instructions in the file's header.)
> >
> > Please review the proposed artifacts and vote. As this is our first release
> > under the Apache Incubator program, note that Apache has specific
> > requirements that must be met before +1 binding votes can be cast by PMC
> > members. Please refer to the policy at
> > http://www.apache.org/legal/release-policy.html#policy for more details.
> >
> > As part of the validation process, the release artifacts can be generated
> > from source by running: mvn clean install -Papache-release -Dtar
> >
> > This vote will be open for at least 72 hours but likely more, in following
> > the Druid community's practice of deploying the RC to larger clusters and
> > allowing it to soak for a period of time to flush out any remaining issues.
> > The vote will pass if a majority of at least three +1 PMC votes are cast.
> >
> > Once the vote has passed, the second stage vote will be called on the
> > Apache Incubator mailing list to get approval from the Incubator PMC.
> >
> > [ ] +1 Release this package as Apache Druid (incubating) 0.13.0
> > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Thanks!
> > David
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> For additional commands, e-mail: dev-h...@druid.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [VOTE] Tranquility 0.8.3 release

[Julian Hyde]

Can someone please clarify what is going on here. Am I correct that Tranquility 
is not an Apache project? Who is allowed to vote for this release - Druid PPMC 
members?

What is being voted upon? A particular set of artifacts to be released, the 
latest commit in github, or something else? (If it’s not an Apache release, I 
guess I shouldn’t complain that the vote doesn’t follow Apache protocol.)

Julian


> On Oct 15, 2018, at 7:51 PM, David Lim  wrote:
> 
> +1
> 
> On Mon, Oct 15, 2018 at 6:08 PM Fangjin Yang  wrote:
> 
>> +1
>> 
>> On Mon, Oct 15, 2018 at 4:40 PM Jihoon Son  wrote:
>> 
>>> +1
>>> 
>>> Thanks Jon!
>>> 
>>> Jihoon
>>> 
>>> On Tue, Oct 16, 2018 at 5:22 AM Jonathan Wei  wrote:
>>> 
 Hi all,
 
 I'd like to open a vote for a new Tranquility release, 0.8.3. The new
 release would have the following improvements and bug fixes:
 
 Improvements:
 * Update Curator and Scala. (#213)
 * support rollup function in druid 0.9.2 (#210)
 * Allow customization of zookeeper path through properties. (#215)
 * Update MMX libraries and replace scala_tools.time (#220)
 * Exclude deps with *GPL licenses. (#223)
 * expose sslContext and prefer tlsPort if present (#257)
 * Support Basic HTTP auth with druid, TLS support for server (#277)
 
 Bug fixes:
 * remove data type and input row parser type binding (#193)
 * Change default host/port for DruidNode and FlinkBeam (#266)
 * Thread-safe samza BeamProducer (#228)
 
 Notably, this release would allow Tranquility to work with TLS-secured
 Druid clusters and support Basic HTTP user/pass authentication.
 
 Thanks,
 Jon
 
>>> 
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: First Apache release of Druid

[Julian Hyde]

The amount of work won’t be huge, because you’ve already done a lot of 
groundwork. But you may find the latency frustrating.

One idea to reduce latency is to produce a “dummy” release candidate that you 
know will fail the vote. You could do this right now, if you like. But you can 
check the signing and staging process, have people check the contents of the 
tar.gz, and so forth. It will generate a bunch of JIRA cases that you can fix 
for the second RC.

Julian


> On Sep 19, 2018, at 12:05 PM, Gian Merlino  wrote:
> 
> I think we want this release to be seen by the user community as a 'real'
> release, and in that case it seems best to satisfy both sets of
> requirements at once. The legal requirements: confirming we have all our
> ducks in a row Apache/IP-wise. And the community requirements: producing a
> release that is at least as high quality as our users have become
> accustomed to in the past, and follows past precedent for deciding what
> goes in a release and what doesn't. If that means more work for us, then in
> my view, so be it. We will strive to prepare something for the IPMC to take
> a look at as soon as possible. I think it's doable on a relatively short
> timeframe from now.
> 
> On Mon, Sep 17, 2018 at 2:07 PM Julian Hyde  <mailto:jh...@apache.org>> wrote:
> 
>> An ASF release is fundamentally a legal transaction. Issuing a bunch of
>> source code under a license, having made sure that the contributions to
>> that source code are in order. There’s not a strict requirement that it
>> works, or even compiles.
>> 
>> Now obviously we, as diligent software engineers who are hoping to build a
>> community, want to deliver something that is functional, well-documented
>> and a delight to use. But those are all secondary to the purpose of
>> launching a blob of intellectual property into the world.
>> 
>> Be advised that the “issues” that the IPMC will find with your release
>> will likely have nothing to do with code bugs, testing or documentation.
>> So, you need to find a balance between the technical tasks and the other
>> aspects of the release process.
>> 
>> To your question. It makes a lot of sense to rename java packages before
>> the release, for the benefit of Druid’s community. But it’s not an absolute
>> requirement.
>> 
>> Julian
>> 
>> 
>>> On Sep 17, 2018, at 1:16 PM, Xavier Léauté  wrote:
>>> 
>>> Julian, maybe the requirements for an ASF release aren't clear to
>> everyone.
>>> It seems we are trying to move all our artifacts to be under org.apache
>> in
>>> order to meet ASF requirements for a release. Doing so would imply a
>> major
>>> release for us since those changes wouldn't be backwards compatible. Are
>>> you saying that we would be able to do a release without renaming
>> artifacts?
>>> 
>>> On Mon, Sep 17, 2018 at 12:42 PM Julian Hyde >> <mailto:jh...@apache.org> > jh...@apache.org>> wrote:
>>> 
>>>> I’m probably guilty of not spending enough time reading through dev@
>>>> archives to find the plans. I hadn’t figured out that the first ASF
>> release
>>>> was going to be a major release (i.e. numbered 0.x) or that the release
>>>> cadence for such releases is about every six months. Sorry about that.
>>>> 
>>>> I saw this thread [1] but the end-of-September timescale isn’t explicit.
>>>> 
>>>> It may be challenging if your first Apache release is also a major
>> release
>>>> (e.g. the two rounds of voting take a while, especially if each vote
>> fails
>>>> a couple of times). So, if you are planning say a beta release before a
>>>> 0.13 then that might be a better first apache release.
>>>> 
>>>> Julian
>>>> 
>>>> [1]
>>>> 
>> https://lists.apache.org/thread.html/e6a378201f7e7ab6da2493fe6ee4ae276768c461ea5c676a953d8139@%3Cdev.druid.apache.org%3E
>>  
>> <https://lists.apache.org/thread.html/e6a378201f7e7ab6da2493fe6ee4ae276768c461ea5c676a953d8139@%3Cdev.druid.apache.org%3E>
>>>> <
>>>> 
>> https://lists.apache.org/thread.html/e6a378201f7e7ab6da2493fe6ee4ae276768c461ea5c676a953d8139@%3Cdev.druid.apache.org%3E
>> <
>> https://lists.apache.org/thread.html/e6a378201f7e7ab6da2493fe6ee4ae276768c461ea5c676a953d8139@%3Cdev.druid.apache.org%3E
>>> 
>>>>> 
>>>> 
>>>> 
>>>>> On Sep 17, 2018, at 12:19 PM, Gian Merlino  wrote:
>>>>> 
>>>>> Hi Julian,
>>>>> 
>>>>&

Re: First Apache release of Druid

[Julian Hyde]

An ASF release is fundamentally a legal transaction. Issuing a bunch of source 
code under a license, having made sure that the contributions to that source 
code are in order. There’s not a strict requirement that it works, or even 
compiles.

Now obviously we, as diligent software engineers who are hoping to build a 
community, want to deliver something that is functional, well-documented and a 
delight to use. But those are all secondary to the purpose of launching a blob 
of intellectual property into the world.

Be advised that the “issues” that the IPMC will find with your release will 
likely have nothing to do with code bugs, testing or documentation. So, you 
need to find a balance between the technical tasks and the other aspects of the 
release process.

To your question. It makes a lot of sense to rename java packages before the 
release, for the benefit of Druid’s community. But it’s not an absolute 
requirement.

Julian


> On Sep 17, 2018, at 1:16 PM, Xavier Léauté  wrote:
> 
> Julian, maybe the requirements for an ASF release aren't clear to everyone.
> It seems we are trying to move all our artifacts to be under org.apache in
> order to meet ASF requirements for a release. Doing so would imply a major
> release for us since those changes wouldn't be backwards compatible. Are
> you saying that we would be able to do a release without renaming artifacts?
> 
> On Mon, Sep 17, 2018 at 12:42 PM Julian Hyde  <mailto:jh...@apache.org>> wrote:
> 
>> I’m probably guilty of not spending enough time reading through dev@
>> archives to find the plans. I hadn’t figured out that the first ASF release
>> was going to be a major release (i.e. numbered 0.x) or that the release
>> cadence for such releases is about every six months. Sorry about that.
>> 
>> I saw this thread [1] but the end-of-September timescale isn’t explicit.
>> 
>> It may be challenging if your first Apache release is also a major release
>> (e.g. the two rounds of voting take a while, especially if each vote fails
>> a couple of times). So, if you are planning say a beta release before a
>> 0.13 then that might be a better first apache release.
>> 
>> Julian
>> 
>> [1]
>> https://lists.apache.org/thread.html/e6a378201f7e7ab6da2493fe6ee4ae276768c461ea5c676a953d8139@%3Cdev.druid.apache.org%3E
>> <
>> https://lists.apache.org/thread.html/e6a378201f7e7ab6da2493fe6ee4ae276768c461ea5c676a953d8139@%3Cdev.druid.apache.org%3E
>>  
>> <https://lists.apache.org/thread.html/e6a378201f7e7ab6da2493fe6ee4ae276768c461ea5c676a953d8139@%3Cdev.druid.apache.org%3E>
>>> 
>> 
>> 
>>> On Sep 17, 2018, at 12:19 PM, Gian Merlino  wrote:
>>> 
>>> Hi Julian,
>>> 
>>> I am surprised to read that you feel the project hasn't come up with a
>> plan
>>> for an Apache release yet. I feel like we do have a plan. I wonder if
>> your
>>> message means that our plan is no good, or just that it isn't clear.
>>> 
>>> From my perspective, as a community, we have decided that our next
>> release
>>> from master (0.13) is going to be an Apache release. And we're treating
>> it
>>> the same way we've treated all our other from-master releases in the past
>>> (0.10, 0.11, 0.12, etc). That is to say, we have tagged a set of issues
>>> with the release number (
>>> https://github.com/apache/incubator-druid/milestone/25) and we are
>> working
>>> to get that list down to zero so we can start doing RCs: either by
>>> finishing the tasks or by punting them to future releases. We have some
>>> extra Apache stuff in this release, and have an "Apache" label in github
>>> that we've been tagging those issues and PRs with. Some relevant changes
>>> include the following,
>>> 
>>> 1) https://github.com/apache/incubator-druid/pull/5976 (Update license
>>> headers.)
>>> 2) https://github.com/apache/incubator-druid/pull/6266 (Rename io.druid
>> to
>>> org.apache.druid.)
>>> 3) https://github.com/apache/incubator-druid/pull/6215 (Adding licenses
>> and
>>> enable apache-rat-plugin.)
>>> 
>>> Based on the tempo so far, I am hoping that we will get this release
>>> branched off and start doing RCs later in September.
>>> 
>>> We haven't modified our NOTICE file yet, although I think we'll need to,
>>> based on what I've seen on the Incubator site. If you have any advice
>> about
>>> what's the minimal set of tasks we should get done before starting to
>>> generate and vote on RCs, that would be helpful towards getting it done
>>> faster

Re: First Apache release of Druid

[Julian Hyde]

I’m probably guilty of not spending enough time reading through dev@ archives 
to find the plans. I hadn’t figured out that the first ASF release was going to 
be a major release (i.e. numbered 0.x) or that the release cadence for such 
releases is about every six months. Sorry about that.

I saw this thread [1] but the end-of-September timescale isn’t explicit.

It may be challenging if your first Apache release is also a major release 
(e.g. the two rounds of voting take a while, especially if each vote fails a 
couple of times). So, if you are planning say a beta release before a 0.13 then 
that might be a better first apache release.

Julian

[1] 
https://lists.apache.org/thread.html/e6a378201f7e7ab6da2493fe6ee4ae276768c461ea5c676a953d8139@%3Cdev.druid.apache.org%3E
 
<https://lists.apache.org/thread.html/e6a378201f7e7ab6da2493fe6ee4ae276768c461ea5c676a953d8139@%3Cdev.druid.apache.org%3E>


> On Sep 17, 2018, at 12:19 PM, Gian Merlino  wrote:
> 
> Hi Julian,
> 
> I am surprised to read that you feel the project hasn't come up with a plan
> for an Apache release yet. I feel like we do have a plan. I wonder if your
> message means that our plan is no good, or just that it isn't clear.
> 
> From my perspective, as a community, we have decided that our next release
> from master (0.13) is going to be an Apache release. And we're treating it
> the same way we've treated all our other from-master releases in the past
> (0.10, 0.11, 0.12, etc). That is to say, we have tagged a set of issues
> with the release number (
> https://github.com/apache/incubator-druid/milestone/25) and we are working
> to get that list down to zero so we can start doing RCs: either by
> finishing the tasks or by punting them to future releases. We have some
> extra Apache stuff in this release, and have an "Apache" label in github
> that we've been tagging those issues and PRs with. Some relevant changes
> include the following,
> 
> 1) https://github.com/apache/incubator-druid/pull/5976 (Update license
> headers.)
> 2) https://github.com/apache/incubator-druid/pull/6266 (Rename io.druid to
> org.apache.druid.)
> 3) https://github.com/apache/incubator-druid/pull/6215 (Adding licenses and
> enable apache-rat-plugin.)
> 
> Based on the tempo so far, I am hoping that we will get this release
> branched off and start doing RCs later in September.
> 
> We haven't modified our NOTICE file yet, although I think we'll need to,
> based on what I've seen on the Incubator site. If you have any advice about
> what's the minimal set of tasks we should get done before starting to
> generate and vote on RCs, that would be helpful towards getting it done
> faster.
> 
> 
> On Mon, Sep 17, 2018 at 10:45 AM Julian Hyde  wrote:
> 
>> Druid has been in incubation for several months and has not yet produced
>> an Apache release. There were initially some issues with IP transfer that
>> prevented that release, but they are now solved. The release is becoming
>> urgent, because the code is still not been released under the Apache
>> license. Can the project please come up with a plan for that release?
>> 
>> I have seen the following in other incubating projects. They want their
>> first release to be a “major” release, and then they start asking product
>> managers to dictate the content and timing of the release, and they ask
>> their marketing people what they could do to make it a “big splash". Don’t
>> do that. A release is nothing more than a snapshot of whatever is on the
>> master branch. Releases must be driven by the community.
>> 
>> The first Apache release is always more effort than it seems. My advice is
>> to start as soon as possible, and make its goals as limited as possible.
>> 
>> Julian (wearing my “mentor” hat)
>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
>> For additional commands, e-mail: dev-h...@druid.apache.org
>> 
>> 

First Apache release of Druid

[Julian Hyde]

Druid has been in incubation for several months and has not yet produced an 
Apache release. There were initially some issues with IP transfer that 
prevented that release, but they are now solved. The release is becoming 
urgent, because the code is still not been released under the Apache license. 
Can the project please come up with a plan for that release?

I have seen the following in other incubating projects. They want their first 
release to be a “major” release, and then they start asking product managers to 
dictate the content and timing of the release, and they ask their marketing 
people what they could do to make it a “big splash". Don’t do that. A release 
is nothing more than a snapshot of whatever is on the master branch. Releases 
must be driven by the community.

The first Apache release is always more effort than it seems. My advice is to 
start as soon as possible, and make its goals as limited as possible.

Julian (wearing my “mentor” hat)


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Druid as a OLTP solution

[Julian Hyde]

Shushant,

Gian is definitely correct that Druid is not designed for OLTP.

However the scenario you describe does not sound like OLTP. You talk of 
querying data, but you don’t mention inserting/updating individual records with 
low latency, which is a hallmark of OLTP, and something that Druid is not good 
at.

One thing Druid is very good at is near-realtime OLAP. If you have a 
transactional system and a means (such as Kafka) to flow the inserts to that 
transactional system into Druid with low latency, then Druid can perform 
queries on a snapshot of your transactional system that is only slightly 
delayed from reality. If that is your scenario, it is worth looking more 
closely at Druid.

Julian


> On Aug 19, 2018, at 6:30 PM, Gian Merlino  wrote:
> 
> Hi Shushant,
> 
> Druid is definitely not intended for OLTP processing. It's generally meant
> for storing, querying, and analyzing event streams. Check out
> http://druid.io/ for some more info about what Druid is good at.
> 
> On Sat, Aug 18, 2018 at 10:19 PM Shushant Arora 
> wrote:
> 
>> Hi
>> 
>> I have a requirement where I have a large scale data and each record has a
>> mandatory visitorId field and one or more optional dimension
>> and records of same visitorId can come after few days time gap.So I have
>> records in system like
>> visitorId1,Timestamp1,Dim1=val1
>> visitorId1,Timestamp1+20days,Dim2=val2
>> 
>> Now I have a requirement to count distinct visitorIds where Dim1=val1 and
>> Dim2=val2(i.,e any row of same visitor has Dim1=val1 and any row of same
>> vsisitor has Dim2=val2).
>> 
>> 
>> Can I do grpupBy visitorId and filter on 2 dimensions Dim=val1 and
>> Dim2=val2 both being in separate row and count on visitorId .
>> Since druid needs Timestamp in each event so I used currentTs for that.
>> Will this return result as 1.
>> 
>> Or druid is not for OLTP processimg?
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Please add me to mailing list

[Julian Hyde]

It's incorrect to say that you subscribe to a mailing list by sending
a message to it.

To subscribe to dev@druid, send a message to dev-subscribe@druid.
Similarly unsubscribe. Similarly users@ and commits@.

Details here:
* https://mail-archives.apache.org/mod_mbox/druid-dev/
* https://mail-archives.apache.org/mod_mbox/druid-users/
* https://mail-archives.apache.org/mod_mbox/druid-commits/


On Thu, Aug 16, 2018 at 10:33 AM Jihoon Son  wrote:
>
> Hi artyom!
>
> you can subscribe those mailing list by sending a message to them.
> Please check https://apache.org/foundation/mailinglists.html for more
> details.
>
> Best,
> Jihoon
>
> On Thu, Aug 16, 2018 at 9:25 AM Artyom Nazarenko 
> wrote:
>
> >
> > Hi!
> >
> > Please add me to  dev@druid.apache.org   and druid-u...@googlegroups.com
> > mailing lists.
> >
> > Thank you!
> >
> > --
> > Artyom Nazarenko

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Changing release process for release candidates

[Julian Hyde]

If it’s not clear, “release manager” is an important role. The RM is 
responsible for the key events in the release - building a release candidate 
(including signatures), putting it in a place that it can be viewed, writing 
release notes, starting a vote, closing the vote, writing the release 
announcement.

There MUST be a vote on the release, but for other parts of the process it’s 
best if the the RM doesn’t hang around waiting for permission.



> On Aug 6, 2018, at 4:52 PM, Gian Merlino  wrote:
> 
> It sounds good to me, it streamlines things a bit and seems to be what
> other projects are doing. As Julian pointed out in the other thread it
> still pays to have someone "managing" the release and to have some
> discussion about when's the right time to start a release branch. The
> "release manager" job has rotated through a few different people over the
> past few major releases, which is good.
> 
> On Mon, Aug 6, 2018 at 3:20 PM Jihoon Son  wrote:
> 
>> Hi all,
>> 
>> Our current release process for RCs begins with a vote. It usually takes up
>> a few days, but is actually not a mandatory process for creating RCs. If we
>> can reach consensus without explicit votes, we can expect the faster
>> release in the future.
>> 
>> The original discussion is available at
>> 
>> https://lists.apache.org/thread.html/d887f0c6e23f1625e549389c08a9a5e74a7a24db4d5e007b6e8d10f6@%3Cdev.druid.apache.org%3E
>> .
>> 
>> Welcome any idea.
>> 
>> Best,
>> Jihoon
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: About creating 0.12.2-rc1

[Julian Hyde]

Gian is correct. Creating an RC doesn’t require a vote. It does require a 
release manager. Usually in Calcite we determine the timeframe of the release, 
and choose an RM, by a discussion that reaches consensus without an explicit 
vote. 

The RM may do a little “traffic control”, asking whether people consider the 
branch is in good shape, and perhaps asking people to stop pushing, again by a 
non-vote email thread.

Julian

> On Aug 5, 2018, at 8:56 AM, Gian Merlino  wrote:
> 
> +1, and fwiw, it looks like Apache projects don't always need to do votes
> for creating release candidates. For example on the Calcite mailing list I
> see votes for _final_ releases, but the release candidates seem to be
> created and uploaded without a vote. There is generally some discussion on
> the list about whether it's a good time to do a release candidate, but I
> don't generally see formal votes. I think something similar could work for
> us in the future and could help us get releases out quicker.
> 
> On Fri, Aug 3, 2018 at 9:38 PM Prashant Deva 
> wrote:
> 
>> +1
>> Prashant
>> 
>> 
>> On Fri, Aug 3, 2018 at 7:11 PM Niketh Sabbineni <
>> niketh.sabbin...@gmail.com>
>> wrote:
>> 
>>> +1
>>> 
>>> Looking forward to this
>>> 
 On Fri, Aug 3, 2018 at 7:09 PM Jihoon Son  wrote:
 
 Hi folks,
 
 Releasing 0.12.2 has been delayed because, fortunately, we could find
>>> more
 bugs to be fixed before release.
 
 Currently, there remains only one PR (
 https://github.com/apache/incubator-druid/pull/6106 ) to be merged for
 0.12.2. Once the Travis CI passes, I'll merge that PR shortly. Then,
>>> we're
 ready for 0.12.2-rc1 release.
 
 So, I think it's time to ask your opinion about creating 0.12.2-rc1
>>> without
 the release vote. I think it makes sense because we have already had
>> two
 votes (
 
 
>>> 
>> https://lists.apache.org/thread.html/a96f2e39506118be26184bd950bc51d360107d75e9ac547d8597817a@%3Cdev.druid.apache.org%3E
 ,
 
 
>>> 
>> https://lists.apache.org/thread.html/11a50f22e7669a527625e190bebbe50b7586dd72733c3bf6a1024c02@%3Cdev.druid.apache.org%3E
 )
 for 0.12.2-rc1 release and there's no objection.
 
 If there's no objection for this for 48 hours, I'll start 0.12.2-rc1
 release.
 
 Best,
 Jihoon
 
>>> --
>>> Niketh Sabbineni
>>> 
>> 

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Podling Report (August 2018)

[Julian Hyde]

Please email the mentors (or the dev list) when it is posted to the wiki and is 
ready for sign-off.

Julian


> On Aug 1, 2018, at 7:19 PM, Jonathan Wei  wrote:
> 
> I don't see the incubator wiki page for August 2018 up yet, so I'll post
> the current report here for now:
> 
> 
> Druid Podling Report (August 2018)
> 
> 
> Druid is a high-performance, column-oriented, distributed data store.
> 
> Druid has been incubating since 2018-02-28.
> 
> Three most important issues to address in the move towards graduation:
> 
> 1. Plan and execute our first Apache release.
> 2. Move the website to Apache infrastructure.
> 3. Expanding the community and adding more committers
> 
> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware
> of?
> 
> - None.
> 
> How has the community developed since the last report?
> 
> - A healthy, constant flow of bug fixes, quality improvements and new
> features
>  are still ongoing at https://github.com/apache/incubator-druid.
> - Our next community meetup has been scheduled for August 8.
> 
> How has the project developed since the last report?
> 
> - This report covers activity since the May 2018 report
> - Source has been migrated to Apache infrastructure
> - License header updates are almost complete
> - Since the last report there have been 93 commits from 20 individuals.
> - We have released 0.12.1, a non-incubator release.
> - We currently voting on the 0.12.2 non-incubator bug fix release. This
> will be our final non-incubator release.
> 
> How would you assess the podling's maturity?
> Please feel free to add your own commentary.
> 
>  [ ] Initial setup
>  [X] Working towards first release
>  [ ] Community building
>  [ ] Nearing graduation
>  [ ] Other:
> 
> Date of last release:
> 
> - Druid 0.12.1 on 2018-06-08 (non-Apache release)
> - No official Apache release yet since beginning Apache Incubation
> 
> When were the last committers or PPMC members elected?
> 
> - Project is still functioning with the initial set of committers.
> 
> On Tue, Jul 31, 2018 at 4:02 PM, Jonathan Wei  wrote:
> 
>> Thanks for reviewing.
>> 
>> Our last meetup was in March, but we have an upcoming meetup on August 8
>> (after the report is due I assume), should we mention that in this report?
>> 
>> I noticed the incubator wiki page for August 2018 hasn't been created yet,
>> does anyone know if that's expected to be up soon? (Not sure on what exact
>> date our podling report is due)
>> 
>> - Jon
>> 
>> 
>> On Sun, Jul 29, 2018 at 10:30 AM, Julian Hyde 
>> wrote:
>> 
>>> Thanks - this looks good.
>>> 
>>> I’d not include the url for the case to replace license headers. Reports
>>> rarely include urls whose sole purpose is to prove statements in the
>>> report.
>>> 
>>> If there have been meet ups/talks about Druid, mention them. Druid has a
>>> vibrant community, as a result of your ongoing community building
>>> activities such as talks, but still, you should take credit for them.
>>> 
>>> Julian
>>> 
>>>> On Jul 28, 2018, at 10:24 AM, Jonathan Wei  wrote:
>>>> 
>>>> Hi all,
>>>> 
>>>> I'm posting a draft of the August 2018 report (which covers activity
>>> since
>>>> our last report in May):
>>>> 
>>>> 
>>>> 
>>>> Druid Podling Report (August 2018)
>>>> 
>>>> 
>>>> Druid is a high-performance, column-oriented, distributed data store.
>>>> 
>>>> Druid has been incubating since 2018-02-28.
>>>> 
>>>> Three most important issues to address in the move towards graduation:
>>>> 
>>>> 1. Plan and execute our first Apache release.
>>>> 2. Move the website to Apache infrastructure.
>>>> 3. Expanding the community and adding more committers
>>>> 
>>>> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
>>> aware
>>>> of?
>>>> 
>>>> - None.
>>>> 
>>>> How has the community developed since the last report?
>>>> 
>>>> - A healthy, constant flow of bug fixes, quality improvements and new
>>>> features
>>>> are still ongoing at https://github.com/apache/incubator-druid.
>>>> 
>>>> How has the project developed since the last report?
>>>> 
>>>> - This report covers activity since the May 2018 report
>>>> - Sou

Re: GitBox review comments

[Julian Hyde]

Ah, I see. Thanks for clarifying.

> On Jul 24, 2018, at 6:43 PM, Gian Merlino  wrote:
> 
> There is a github feature to make multiple comments as a single "review",
> although from what I can see, gitbox splits those up into multiple emails
> anyway, so it doesn't help. I have poked Infra again on our ticket:
> https://issues.apache.org/jira/browse/INFRA-16674
> 
> On Tue, Jul 24, 2018 at 5:55 PM Julian Hyde  wrote:
> 
>> I know there is an open request with INFRA to route git review comments to
>> a list other than dev. But until then, we are still getting a lot of
>> messages on the list each day. A lot of them come in groups, as a reviewer
>> makes multiple comments on a particular PR.
>> 
>> I believe git has a feature called “review” where a reviewer can make
>> multiple comments and they all appear in the same email. Is this feature
>> supported in gitbox? If so, could reviewers consider using it, in order to
>> reduce the email volume?
>> 
>> Julian
>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
>> For additional commands, e-mail: dev-h...@druid.apache.org
>> 
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

GitBox review comments

[Julian Hyde]

I know there is an open request with INFRA to route git review comments to a 
list other than dev. But until then, we are still getting a lot of messages on 
the list each day. A lot of them come in groups, as a reviewer makes multiple 
comments on a particular PR.

I believe git has a feature called “review” where a reviewer can make multiple 
comments and they all appear in the same email. Is this feature supported in 
gitbox? If so, could reviewers consider using it, in order to reduce the email 
volume?

Julian


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: License headers and NOTICE file

[Julian Hyde]

Do your best with the NOTICE file. We will scrutinize it during the release 
process.

I know it seems impolite not to mention contributors and copied code in the 
NOTICE, but there is a good reason to keep its contents absolutely minimal. 
Downstream projects are required to reproduce the NOTICE file, therefore 
everything we put in it places a burden on those projects. 

Lastly note that a binary artifacts often include much more than a source 
release. If so, their NOTICE files may need to include extra things. To keep 
things simple, we strongly recommend that the first Apache release is source 
only. 

Julian
 

> On Jul 11, 2018, at 1:00 PM, Gian Merlino  wrote:
> 
> I am looking at sorting out our license headers and NOTICE file.
> 
> For license headers the only real question I have at this point is
> https://github.com/apache/incubator-druid/issues/5835#issuecomment-404257393.
> If anyone with experience in this sort of thing could chime in then that
> would be very useful.
> 
> For NOTICE, it looks like
> http://www.apache.org/dev/licensing-howto.html#overview-of-files governs
> what the NOTICE file should contain. Here is our current one:
> https://github.com/apache/incubator-druid/blob/master/NOTICE. It has the
> following features,
> 
> 1) Copyright notices up top for three major contributors.
> 2) A notice for each project where we've copied code directly into the
> Druid codebase. They're all Apache licensed.
> 
> From reading the licensing-howto it seems like the file needs some tweaks.
> A bolded principle in the guide is "Do not add anything to NOTICE which is
> not legally required." The copyright notices for major contributors don't
> seem necessary given they are all covered by SGA / CLAs. The howto also
> says "If the dependency supplies a NOTICE file, its contents must be
> analyzed and the relevant portions bubbled up into the top-level NOTICE
> file." We haven't been doing that -- we have just been listing the projects
> themselves.
> 
> For projects where we've copied code, and for which those projects don't
> have a NOTICE, should we remove them completely from our NOTICE? And I
> suppose we should double check that we have source files marked
> appropriately in cases where they're copied from another project.
> 
> Is there anyone with more experience in writing NOTICE files that could
> chime in with thoughts please?


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Gitbox notifications

[Julian Hyde]

As it happens, all 3 of Druid's mentors (including myself) are ASF
members. Their names are bold in
https://people.apache.org/phonebook.html?podling=druid.

I am busy, I will try to get to it today.

On Wed, Jul 11, 2018 at 10:20 AM, Roman Leventov  wrote:
> It writes "Access restricted to ASF members and PMC chairs only!" to me.
> "ASF members" != any Apache project committers, this is a special title,
> little people have it.
>
> On Wed, 11 Jul 2018 at 11:50, Gian Merlino  wrote:
>
>> Infra wrote back asking us to add a comm...@druid.apache.org list via
>> https://selfserve.apache.org/. It sounds like once we do that, they can
>> move the notifications. I tried logging in with my apache id, but the tool
>> wouldn't let me in.
>>
>> Does anyone else have the ability to use this tool? Maybe one of our
>> mentors?
>>
>> On Sat, Jul 7, 2018 at 8:20 AM Gian Merlino  wrote:
>>
>> > I asked yesterday in our repo migration Infra ticket to adjust the
>> mailing
>> > lists:
>> >
>> >
>> https://issues.apache.org/jira/browse/INFRA-16674?focusedCommentId=16535589=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16535589
>> .
>> > I am not sure if we can expect Infra to adjust things on the weekend but
>> > hopefully it can be done soon.
>> >
>> > Discussion in this thread is also relevant:
>> >
>> https://lists.apache.org/thread.html/dac18141e34cd6f49d8b4136046d6d8e2fd237986c9c6b9a2d12948d@%3Cdev.druid.apache.org%3E
>> .
>> > I linked some filters there that I'm using to manage the spam a bit.
>> >
>> >
>> > On Sat, Jul 7, 2018 at 4:37 AM Roman Leventov 
>> wrote:
>> >
>> >> Could we forward GitBox notifications to a dedicated mailing list? I'm
>> >> afraid that the huge volume of spam will discourage people from
>> >> subscribing
>> >> to this list.
>> >>
>> >
>>

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Druid repo migration plan

[Julian Hyde]

I see there are notifications not just for commits but for every comment on 
every issue. That’s going to be overwhelming. I can’t imagine anyone wanting to 
subscribe to the gitbox list at the current volume.

How about only sending comments to people who have watched a particular case?

Julian


> On Jul 6, 2018, at 1:08 PM, Julian Hyde  wrote:
> 
> I like the idea of a separate list for commits. Keeps the noise down on the 
> dev list.
> 
> Other projects (e.g. calcite) would name that list “commits”, i.e. 
> comm...@druid.apache.org <mailto:comm...@druid.apache.org>. If you were to 
> add other repos (e.g. a subversion repo for web site) you could send its 
> commits there too.
> 
>> On Jul 6, 2018, at 12:24 PM, Samarth Jain > <mailto:samarth.j...@gmail.com>> wrote:
>> 
>> +1 to sending to git...@druid.apache.org <mailto:git...@druid.apache.org>
>> 
>> On Fri, Jul 6, 2018 at 12:17 PM, Gian Merlino > <mailto:g...@apache.org>> wrote:
>> 
>>> The repo move has happened: https://github.com/apache/incubator-druid 
>>> <https://github.com/apache/incubator-druid>
>>> 
>>> My understanding is that now we can and should start prepping license,
>>> notice, etc for a release. And also making sure that our integrations still
>>> work right (Travis / TeamCity).
>>> 
>>> Also: does anyone else find the gitbox mails sent to dev@druid.apache.org 
>>> <mailto:dev@druid.apache.org>
>>> annoying? I'm planning to set up a mail filter to put them into another
>>> folder (they're redundant to notifications I already get from GitHub). If
>>> people generally feel the same way we could ask Infra to move them to a
>>> separate mailing list, maybe git...@druid.apache.org 
>>> <mailto:git...@druid.apache.org>.
>>> 
>>> On Tue, Jul 3, 2018 at 12:57 PM Gian Merlino >> <mailto:g...@apache.org>> wrote:
>>> 
>>>> Here is the ticket, btw: https://issues.apache.org/ 
>>>> <https://issues.apache.org/>
>>> jira/browse/INFRA-16674.
>>>> Repo move should be happening real soon now!
>>>> 
>>>> On Mon, Jul 2, 2018 at 11:55 PM Gian Merlino >>> <mailto:g...@apache.org>> wrote:
>>>> 
>>>>> Our infra ticket is progressing along and it looks like we're just about
>>>>> ready to pull the trigger on moving the repo. So, committers, please
>>> make
>>>>> sure your ASF gitbox stuff is working: https://gitbox.apache.org/setup/ 
>>>>> <https://gitbox.apache.org/setup/>
>>>>> 
>>>>> On Fri, Jun 22, 2018 at 1:22 PM Gian Merlino >>>> <mailto:g...@apache.org>> wrote:
>>>>> 
>>>>>> Thanks for the tips, Max!! I think we are, hopefully, doing okay on
>>> some
>>>>>> of these. My thoughts inline.
>>>>>> 
>>>>>>> Since you need elevated rights on both
>>>>>> orgs to move the repo (say airbnb and apache) and that both parties
>>>>>> aren't
>>>>>> ok with that, it's typical to use a middleman org like `apacheinfra`.
>>>>>> 
>>>>>> Luckily, our org is limited to just Druid stuff (
>>>>>> https://github.com/druid-io <https://github.com/druid-io>) so we should 
>>>>>> be OK to add Apache Infra
>>>>>> people with elevated rights.
>>>>>> 
>>>>>>> * make merge hook checks optional, so that if coverage, travis, or
>>> code
>>>>>> quality checks do not prevent merging, since it's likely those check
>>>>>> won't
>>>>>> trigger and as a non-admin you won't be able to force-merge
>>>>>> 
>>>>>> We have a couple (Travis and TeamCity) and they're already optional.
>>>>>> 
>>>>>>> * consider unprotecting protected branches so that you can push to
>>>>>> master
>>>>>> if controlling master is important in your workflow. This way you can
>>>>>> effectively merge PRs without clicking the button on GH.
>>>>>> 
>>>>>> Master _is_ important. Although I think if we can't do PRs, then
>>> pushing
>>>>>> directly to master is probably not going to be too helpful anyway (the
>>> PRs
>>>>>> are essential to our code review workflow). So I think we have to hope
>>> for
>>>>>> t

Re: Druid repo migration plan

[Julian Hyde]

 a period of instability for us
>>>>>> and
>>>>>> a lot of slow back and forth with Apache infra. Hopefully the process
>>>>>> has
>>>>>> been ironed out since then. Be prepared and go into it knowing that
>> you
>>>>>> may
>>>>>> not be able to merge PRs for days/weeks.
>>>>>> https://issues.apache.org/jira/browse/INFRA-14267
>>>>>> 
>>>>>> On the ticket you open with INFRA, make it really clear what your GH
>>>>>> integrations are and validate that they are all approved/supported by
>>>>>> Apache prior to the move. Some integrations (like codeclimate) require
>>>>>> rights on the GH org (Apache) and INFRA is categoric against that. If
>>>>>> some
>>>>>> services aren't supported make sure to disable the integrations prior
>> to
>>>>>> the move, find replacement services. Also make sure INFRA will
>>>>>> adjust/tweak
>>>>>> the integration post move as you likely need admin rights to do so.
>>>>>> 
>>>>>> A caveat is around the "redirect chain" on GH. This is what allows
>>>>>> hitting `
>>>>>> github.com/airbnb/superset` <http://github.com/airbnb/superset> to
>>>>>> redirect to `
>>>>>> github.com/apache/incubator-superset`
>>>>>> <http://github.com/apache/incubator-superset> to redirect to the
>> right
>>>>>> place. This
>>>>>> also allows `git remote`s to just work post move. This redirect chain
>> is
>>>>>> fragile and can break in some cases. Since you need elevated rights on
>>>>>> both
>>>>>> orgs to move the repo (say airbnb and apache) and that both parties
>>>>>> aren't
>>>>>> ok with that, it's typical to use a middleman org like `apacheinfra`.
>>>>>> They
>>>>>> grant you admin right to that org and you move the repo to there, and
>>>>>> they
>>>>>> do the second part. If, post move, the middleman was to fork a repo
>> with
>>>>>> the same name, or create one, it would break the redirect chain.
>>>>>> Something
>>>>>> INFRA should be aware of at this point and cautious around it. Also
>> note
>>>>>> that some GH integrations work through the redirect chain and some
>> don't
>>>>>> and require re-pointing/configuring the service to the new location.
>>>>>> 
>>>>>> Thing you can do to prepare and mitigate risks:
>>>>>> * make merge hook checks optional, so that if coverage, travis, or
>> code
>>>>>> quality checks do not prevent merging, since it's likely those check
>>>>>> won't
>>>>>> trigger and as a non-admin you won't be able to force-merge
>>>>>> * consider unprotecting protected branches so that you can push to
>>>>>> master
>>>>>> if controlling master is important in your workflow. This way you can
>>>>>> effectively merge PRs without clicking the button on GH.
>>>>>> * make sure core committers have their Gitbox access setup, I think it
>>>>>> can
>>>>>> be a bit tricky and may involve your mentor / infra pulling some
>> levers
>>>>>> on
>>>>>> whimsy
>>>>>> 
>>>>>> I hope this helps!
>>>>>> 
>>>>>> Max
>>>>>> 
>>>>>> 
>>>>>> On Thu, Jun 21, 2018 at 6:27 PM Jonathan Wei 
>> wrote:
>>>>>> 
>>>>>>> This is the JIRA issue for the Druid migration:
>>>>>>> https://issues.apache.org/jira/browse/INFRA-16674
>>>>>>> 
>>>>>>> - Jon
>>>>>>> 
>>>>>>> 
>>>>>>> On Thu, Jun 21, 2018 at 6:10 PM, Jonathan Wei 
>>>>>> wrote:
>>>>>>> 
>>>>>>>>> I generally approve of this idea, as long as Apache INFRA is
>>>>>> willing
>>>>>>> and
>>>>>>>> able to make it happen. I know it’s straightforward to move a git
>>>>>>>> repository from a GitHub project to any other place, but I’m not
>>>>>> sure
>&g

Re: Podling Report Reminder - June 2018

[Julian Hyde]

Yes, the mentors are active and I’m one of them.

As a mentor I am happy to give advice, and do so frequently. But I’m not going 
to nag the PPMC to file reports. They need to do that for themselves.

Julian

> On Jun 19, 2018, at 18:00, Justin Mclean  wrote:
> 
> Hi,
> 
> I noticed the podling has failed to report this month and so will need two 
> report next month. You're a relatively new podling so I wonder are you just 
> not aware of the process or is it that you need more help from your mentors? 
> Are your mentors currently active?
> 
> I also notice on your dev list a number of votes for non ASF releases. I 
> can’t see (but may of missed it) any discussion to why these non ASF releases 
> are needed or what is holding you up making an ASF release. It seems a little 
> odd to me that you would vote on these and then not get the IPMC to look at 
> them, so it may be good idea to have the IPMC look at these releases in the 
> future, but hopefully your next release will be an ASF one and will be voted 
> on by the IPMC.
> 
> Thanks,
> Justin
> -
> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> For additional commands, e-mail: dev-h...@druid.apache.org
> 

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Podling Report Reminder - May 2018

[Julian Hyde]

Thanks. I have signed off.

> On May 3, 2018, at 4:22 PM, Gian Merlino <g...@apache.org> wrote:
> 
> I have posted this on https://wiki.apache.org/incubator/May2018.
> 
> On Thu, May 3, 2018 at 1:03 PM, Gian Merlino <g...@apache.org> wrote:
> 
>> Here's a draft (below). Since it's already overdue, I'll post it on the
>> wiki in a few hours.
>> 
>> Druid
>> 
>> Druid is a high-performance, column-oriented, distributed data store.
>> 
>> Druid has been incubating since 2018-02-28.
>> 
>> Three most important issues to address in the move towards graduation:
>> 
>> 1. Complete SGA for current sources and ICLAs for current committers.
>> 2. Move the source code and website to Apache infrastructure.
>> 3. Plan and execute our first Apache release.
>> 
>> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware
>> of?
>> 
>> - None.
>> 
>> How has the community developed since the last report?
>> 
>> - We have disabled posting on our pre-Apache development list.
>> - We have updated our pre-Apache community page (
>> http://druid.io/community/) to
>>  include information about incubation.
>> - We have set up a placeholder site on https://druid.apache.org/.
>> - A healthy, constant flow of bug fixes, quality improvements and new
>> features
>>  are still ongoing on https://github.com/druid-io/druid.
>> 
>> How has the project developed since the last report?
>> 
>> - Since the last report there have been 69 commits from 23 individuals.
>> - We have conducted a vote to put out a release candidate 0.12.1-rc1. This
>>  release candidate is being done outside the Incubator. We also
>> anticipate the
>>  0.12.1 release to be done outside the Incubator.
>> 
>> How would you assess the podling's maturity?
>> Please feel free to add your own commentary.
>> 
>>  [X] Initial setup
>>  [ ] Working towards first release
>>  [ ] Community building
>>  [ ] Nearing graduation
>>  [ ] Other:
>> 
>> Date of last release:
>> 
>> - Druid 0.12.0 on 2018-03-06 (non-Apache release)
>> - No official Apache release yet since beginning Apache Incubation
>> 
>> When were the last committers or PPMC members elected?
>> 
>> - Project is still functioning with the initial set of committers.
>> 
>> Signed-off-by:
>> 
>>  [ ](druid) Julian Hyde
>>  [ ](druid) P. Taylor Goetz
>>  [ ](druid) Jun Rao
>> 
>> On Thu, May 3, 2018 at 11:06 AM, P. Taylor Goetz <ptgo...@gmail.com>
>> wrote:
>> 
>>> Great, thanks!
>>> 
>>> If you have any trouble posting to the wiki, you can forward the report
>>> to me and I will post it.
>>> 
>>> -Taylor
>>> 
>>>> On May 3, 2018, at 2:01 PM, Gian Merlino <g...@apache.org> wrote:
>>>> 
>>>> I can take up this one.
>>>> 
>>>> On Thu, May 3, 2018 at 10:48 AM, P. Taylor Goetz <ptgo...@gmail.com>
>>> wrote:
>>>> 
>>>>> The druid podling report was due yesterday. Is anyone working on it or
>>>>> willing to take this up?
>>>>> 
>>>>> -Taylor
>>>>> 
>>>>>> On Apr 27, 2018, at 11:01 PM, johndam...@apache.org wrote:
>>>>>> 
>>>>>> Dear podling,
>>>>>> 
>>>>>> This email was sent by an automated system on behalf of the Apache
>>>>>> Incubator PMC. It is an initial reminder to give you plenty of time to
>>>>>> prepare your quarterly board report.
>>>>>> 
>>>>>> The board meeting is scheduled for Wed, 16 May 2018, 10:30 am PDT.
>>>>>> The report for your podling will form a part of the Incubator PMC
>>>>>> report. The Incubator PMC requires your report to be submitted 2 weeks
>>>>>> before the board meeting, to allow sufficient time for review and
>>>>>> submission (Wed, May 02).
>>>>>> 
>>>>>> Please submit your report with sufficient time to allow the Incubator
>>>>>> PMC, and subsequently board members to review and digest. Again, the
>>>>>> very latest you should submit your report is 2 weeks prior to the
>>> board
>>>>>> meeting.
>>>>>> 
>>>>>> Candidate names should not be made public before people are actually
>>>>>> elected, so please do not include the names of poten

Re: Druid 0.12.1-rc1 vote

[Julian Hyde]

You should make it clear that this is a non-ASF release. Such releases are 
allowed during incubation but not encouraged.

Also be sure to mention it in your next report.

Julian


> On Apr 24, 2018, at 2:49 PM, Jihoon Son  wrote:
> 
> Hi all,
> 
> We currently have no open issues/PRs for 0.12.1 (
> https://github.com/druid-io/druid/milestone/26), so I created a branch for
> 0.12.1 (https://github.com/druid-io/druid/tree/0.12.1).
> 
> Let's vote on releasing RC1. Here is my +1.
> 
> Best,
> Jihoon


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Dev list migration

[Julian Hyde]

You should also update https://github.com/druid-io/druid/blob/master/README.md 
.


> On Apr 24, 2018, at 8:08 AM, Gian Merlino  wrote:
> 
> Today (well, yesterday) is the day we had decided on for migrating the dev
> list. If there are no objections I'll update the community page, send out
> another note to the old list, and put the old list into read only mode.
> 
> On Tue, Apr 17, 2018 at 8:28 PM, Himanshu  wrote:
> 
>> +1
>> 
>> On Tue, Apr 17, 2018 at 3:20 PM, Nishant Bangarwa <
>> nbanga...@hortonworks.com
>>> wrote:
>> 
>>> +1
>>> --
>>> Nishant Bangarwa
>>> Hortonworks
>>> (M): +91-9729200044
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On 4/17/18, 10:19 PM, "Jonathan Wei"  wrote:
>>> 
 +1
 
 On 2018/04/17 18:09:37, David Lim  wrote:
> +1
> 
> On Tue, Apr 17, 2018 at 11:49 AM, Gian Merlino 
>> wrote:
> 
>> Hi all,
>> 
>> In the dev sync today there was some general agreement around
>>> migrating the
>> dev list to Apache next week. Please +1 or -1 in this thread as
>>> desired.
>> 
>> I think ideally we'd want an autoresponder but I don't see a way to
>>> set
>> one. So the concrete plan I'd propose instead is to do the following
>>> next
>> Monday, April 23,
>> 
>> 1) Post a message to the list that we have migrated to
>> dev@druid.apache.org
>> 2) Sticky that post on
>> https://groups.google.com/forum/#!forum/druid-development
>> 3) Update the list link on the web site at
>> http://druid.io/community/
>> 4) Disable posting on druid-developm...@googlegroups.com
>> 
>> This message has been cross posted to both lists, but please reply
>> on
>>> the
>> Apache list. If you haven't signed up for it yet, you can do that by
>> emailing dev-subscr...@druid.apache.org.
>> 
> 
 
 -
 To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
 For additional commands, e-mail: dev-h...@druid.apache.org
 
>>> 
>> 

Re: Legal implications of using JMH

[Julian Hyde]

We use JMH in Calcite for microbenchmarking. My understanding was that our use 
is OK. We are not linking it into the product; during tests, we link against 
JMH, run the test, then delete the executable.

In addition to Calcite, it is used in Hive [1] Log4j2 [2], and Camel [3].

I think we should ask for a decision from Legal before changing anything.

Julian

[1] https://github.com/apache/hive/blob/master/itests/hive-jmh/pom.xml 


[2] 
https://github.com/apache/logging-log4j2/tree/master/log4j-perf/src/main/java/org/apache/logging/log4j/perf/jmh
 


[3] https://github.com/apache/camel/blob/master/tests/camel-jmh/pom.xml 


> On Apr 23, 2018, at 2:48 PM, Owen Omalley  wrote:
> 
> The relevant page from Apache is:
> 
> https://www.apache.org/licenses/GPL-compatibility.html and 
> https://www.apache.org/legal/resolved.html#category-x
> 
> GPL software can not be linked to from Apache projects at all. The problem is 
> that the GPL is viral and you’d have to license the rest of the code under 
> the GPL.
> 
> .. Owen
> 
> Begin forwarded message:
> 
> From: Gian Merlino >
> Subject: Re: Legal implications of using JMH
> Date: April 23, 2018 at 2:14:31 PM PDT
> To: dev@druid.apache.org
> Reply-To: dev@druid.apache.org
> 
> I'm not sure why the ORC folks decided this was necessary. I found these
> two links,
> 
> - https://issues.apache.org/jira/browse/ORC-298
> - https://lists.apache.org/thread.html/c27267a6a3df9b76c5e036fc181c86
> 5150639f4cf15ffc5cca27b690@%3Cdev.orc.apache.org%3E
> 
> It looks like they ended up moving the code to a different, non-Apache
> repository.
> 
> The relevant Apache policy, I think, is: http://www.apache.org/
> legal/resolved.html#optional. From what I've seen it comes up most often
> with MySQL drivers (which we also use). See also https://issues.apache.org/
> jira/browse/LEGAL-200.
> 
> It seems to me like JMH and MySQL are in the same boat. Both are optional
> dependencies -- there is no reason that a "normal" user needs to run
> druid-benchmarks. If this understanding is correct, then we should just
> make sure we aren't distributing JMH (or MySQL drivers) as part of binary
> releases. But I would be interested in understanding the thought process of
> the ORC folks in more detail.
> 
> On Mon, Apr 23, 2018 at 1:44 PM, Roman Leventov 
> >
> wrote:
> 
> See this:
> http://mail.openjdk.java.net/pipermail/jmh-dev/2018-April/002743.html JMH
> might be not welcome in an Apache-licensed project.
> 
> 
> 

Re: Legal implications of using JMH

[Julian Hyde]

I agree with Gian. 

Calcite uses JMH. We don’t distribute the binaries it produces — because we 
only need them for running benchmarks. 

It’s not an identical situation to MySQL because while there are other 
implementations of JDBC — e.g. Postgres — there is only one implementation of 
JMH. But JMH is only used during testing, not production.

So, let’s not do anything rash, like starting to remove code. I think we will 
have clarity in a few days as various folks in Apache react to what Owen has 
done. We can revisit this as part of the first Druid release.

Julian


> On Apr 23, 2018, at 2:14 PM, Gian Merlino  wrote:
> 
> I'm not sure why the ORC folks decided this was necessary. I found these
> two links,
> 
> - https://issues.apache.org/jira/browse/ORC-298
> - https://lists.apache.org/thread.html/c27267a6a3df9b76c5e036fc181c86
> 5150639f4cf15ffc5cca27b690@%3Cdev.orc.apache.org%3E
> 
> It looks like they ended up moving the code to a different, non-Apache
> repository.
> 
> The relevant Apache policy, I think, is: http://www.apache.org/
> legal/resolved.html#optional. From what I've seen it comes up most often
> with MySQL drivers (which we also use). See also https://issues.apache.org/
> jira/browse/LEGAL-200.
> 
> It seems to me like JMH and MySQL are in the same boat. Both are optional
> dependencies -- there is no reason that a "normal" user needs to run
> druid-benchmarks. If this understanding is correct, then we should just
> make sure we aren't distributing JMH (or MySQL drivers) as part of binary
> releases. But I would be interested in understanding the thought process of
> the ORC folks in more detail.
> 
> On Mon, Apr 23, 2018 at 1:44 PM, Roman Leventov 
> wrote:
> 
>> See this:
>> http://mail.openjdk.java.net/pipermail/jmh-dev/2018-April/002743.html JMH
>> might be not welcome in an Apache-licensed project.
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Dependencies licenses Report

[Julian Hyde]

The main tool to use is Apache RAT. Definitely use that. 

One of the hardest tasks is getting the contents of LICENSE and NOTICE right. 
That is a manual task I’m afraid. 

Julian

> On Apr 18, 2018, at 08:34, Gian Merlino  wrote:
> 
> Hi Slim,
> 
> Do you know if ORC & Hive use this tool as part of their release process?
> And if it's considered a good tool by itself for verifying we meet all of
> the Apache licensing requirements, or if we'll need something else too?
> 
>> On Tue, Apr 17, 2018 at 9:15 PM, Slim Bouguerra  wrote:
>> 
>> One of the question last dev synch was about the generation of dependency
>> licenses.
>> Some projects (ORC and Hive) use the maven site plugin that can generates
>> reports with all the dependencies and licenses details.
>> I have run it on Druid and this is how it looks for Druid Api Module.
>> cmd
>> 
>> mvn project-info-reports:dependencies
>> 
>> The site directory can be found under target/site
>> here is an example for one module
>> https://drive.google.com/file/d/1P8R0kZjp8zP4WSOVrKdlJF7Xr8-
>> OI7Oe/view?usp=sharing
>> 
>> Also no fancy tools used to detect unwanted licenses, it is done while
>> reviewing PR
>> 
>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
>> For additional commands, e-mail: dev-h...@druid.apache.org
>> 

-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: Web site

[Julian Hyde]

3 repos makes sense to me. I believe that ASF infra can handle it.

And you have a one-time opportunity to clean out cruft from your source code 
repo as you move to ASF.

Julian


> On Apr 16, 2018, at 9:58 AM, Gian Merlino <g...@apache.org> wrote:
> 
> We have a bit of a hybrid setup today: the docs (a big part of the site)
> are in the main "druid" repo. The rest of the site (landing page, news
> page, download page) are in a separate website repo. It makes sense to me
> because we want to version the docs along with the code, but we _don't_
> want to version stuff like the landing page and news as part of the code,
> we'd rather they "float" as a separate thing.
> 
> The one thing I don't like about how we do the site today is the problem
> you mentioned: the built site branch does get large, since it has docs for
> every version we've ever released.
> 
> Maybe it makes sense to have three repos: one for the sources and
> docs/tutorials (which are tied to the sources), one for the "floating"
> parts of the site that are untethered to any particular release, and one
> for the built site?
> 
> On Mon, Apr 16, 2018 at 9:44 AM, Julian Hyde <jh...@apache.org> wrote:
> 
>> (Speaking not as a mentor, just someone who has deployed sites on ASF
>> infrastructure.)
>> 
>> What makes most sense to me is to put the web site source in master along
>> with the source code, and to put the generated site in a different git repo
>> (not just a different branch). It allows you to make a commit that changes
>> both source code and the web site.
>> 
>> And it prevents the source repo from becoming really large (generated web
>> sites can be large if you deploy 100M of generated java doc each release).
>> You don’t want every contributor to have to download a huge git repo.
>> 
>> Julian
>> 
>> 
>>> On Apr 16, 2018, at 8:48 AM, Gian Merlino <g...@apache.org> wrote:
>>> 
>>> A technical note that I also posted in the "migration logistics" thread:
>>> the sources for the site are at
>>> https://github.com/apache/incubator-druid-website. The branch "asf-git"
>> is
>>> served on the site https://druid.incubator.apache.org/. I think once we
>>> migrate http://druid.io/, we could do something similar to what we do on
>>> https://github.com/druid-io/druid-io.github.io, where sources are in
>> "src"
>>> and the built site is in "master". Except when using ASF infra, it makes
>>> more sense to put the sources in "master" and the built site in
>> "asf-git".
>>> 
>>> 
>>> On Sun, Apr 15, 2018 at 3:58 PM, Julian Hyde <jh...@apache.org> wrote:
>>> 
>>>> There has been some back-channel discussion about the web site.
>>>> 
>>>> Druid has a good and successful web site outside of Apache, namely
>>>> druid.io <http://druid.io/>. We cannot start transitioning that site
>>>> until the legal IP transfer has completed. In the mean time, we were
>> left
>>>> without a web site: requests to http://druid.apache.org/ <
>>>> http://druid.apache.org/> and http://druid.incubator.apache.org/ <
>>>> http://druid.incubator.apache.org/> would receive an HTTP 404.
>>>> 
>>>> Gian has created a simple web site in Apache that has hyperlinks to
>>>> druid.io <http://druid.io/> and references the current user list
>>>> druid-u...@googlegroups.com <mailto:druid-u...@googlegroups.com>. Links
>>>> to outside Apache regarded as a breach of Apache branding policy and are
>>>> frowned upon; but as a mentor I totally understand why they are
>> necessary:
>>>> Druid has a great community, and we must protect that community during
>> the
>>>> transition to Apache.
>>>> 
>>>> The current web site is good enough for the short-term, but let’s get a
>>>> proper branding-compliant web site up and running as soon as we can.
>> Let’s
>>>> make it one of the “top three” tasks listed in each board report.
>>>> 
>>>> I see [1] that Gian is pushing to move traffic from
>>>> druid-...@googlegroups.com <mailto:druid-...@googlegroups.com> to this
>>>> dev list. That effort is most welcome, also.
>>>> 
>>>> Julian
>>>> 
>>>> [1] https://groups.google.com/d/msg/druid-development/q1ip-
>>>> L8xpBk/hDCYaIsQCgAJ <https://groups.google.com/d/
>>>> msg/druid-development/q1ip-L8xpBk/hDCYaIsQCgAJ>
>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
>> For additional commands, e-mail: dev-h...@druid.apache.org
>> 
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Web site

[Julian Hyde]

There has been some back-channel discussion about the web site.

Druid has a good and successful web site outside of Apache, namely druid.io 
. We cannot start transitioning that site until the legal IP 
transfer has completed. In the mean time, we were left without a web site: 
requests to http://druid.apache.org/  and 
http://druid.incubator.apache.org/  would 
receive an HTTP 404.

Gian has created a simple web site in Apache that has hyperlinks to druid.io 
 and references the current user list 
druid-u...@googlegroups.com . Links to 
outside Apache regarded as a breach of Apache branding policy and are frowned 
upon; but as a mentor I totally understand why they are necessary: Druid has a 
great community, and we must protect that community during the transition to 
Apache.

The current web site is good enough for the short-term, but let’s get a proper 
branding-compliant web site up and running as soon as we can. Let’s make it one 
of the “top three” tasks listed in each board report.

I see [1] that Gian is pushing to move traffic from druid-...@googlegroups.com 
 to this dev list. That effort is most 
welcome, also.

Julian

[1] https://groups.google.com/d/msg/druid-development/q1ip-L8xpBk/hDCYaIsQCgAJ 
 

Re: [druid-dev] Apache migration logistics

[Julian Hyde]

Per https://incubator.apache.org/guides/transitioning_asf.html 
<https://incubator.apache.org/guides/transitioning_asf.html> we need to get 
SGA/CCLA on file, then we should file a JIRA case similar to 
https://issues.apache.org/jira/browse/INFRA-15735 
<https://issues.apache.org/jira/browse/INFRA-15735> or 
https://issues.apache.org/jira/browse/INFRA-12261 
<https://issues.apache.org/jira/browse/INFRA-12261> to do the import.

Julian


> On Mar 27, 2018, at 5:20 PM, Gian Merlino <gianmerl...@gmail.com> wrote:
> 
> Hi, today I found myself wondering about migration of source repos.
> 
> Does anyone know if we've got our GitBox git repo set up? And who we need
> to talk to about getting the druid-io repos transferred to the apache org
> in github such that we can do source control in an Apache-certified-okay
> way? It sounded like we are going to be able to keep using GitHub PRs and
> issues. So I'm hoping we can do this process:
> https://help.github.com/articles/about-repository-transfers/ which lets us
> keep all the issues, watchers, & stars intact.
> 
> On Mon, Mar 12, 2018 at 3:25 PM, Xavier Léauté <xav...@confluent.io> wrote:
> 
>> FYI, to update your information on the status page you need to check out
>> https://svn.apache.org/repos/asf/incubator/public/trunk/content/projects/
>> with
>> your Apache credentials and update the druid.xml file in that directory.
>> 
>> On Mon, Mar 12, 2018 at 2:49 PM Gian Merlino <g...@imply.io> wrote:
>> 
>>> Committers: please,
>>> 
>>> 1) If you don't have an apache id already, fill out an ICLA:
>>> https://www.apache.org/dev/new-committers-guide.html#
>> guide-for-new-committers and
>>> then post here and hopefully someone can figure out how to get you an id?
>>> 
>>> 2) When you have an id, post it here if it's not in
>>> http://incubator.apache.org/projects/druid.html so someone can figure
>> out
>>> how to add you to that, and then also try to sign up to
>>> private-subscr...@druid.apache.org (+ dev-subscr...@druid.apache.org
>>> which you should be on already). If you can't, then also post here, and
>>> hopefully someone can figure _that_ out.
>>> 
>>> Gian
>>> 
>>> On Fri, Mar 9, 2018 at 11:28 AM, Xavier Léauté <xav...@confluent.io>
>>> wrote:
>>> 
>>>> This thread is already going to both lists, and it looks like responses
>>>> automatically go to both. Would be good to check what happens if we
>>>> subscribe dev@ to the google group. If responding from the apache list
>>>> doesn't automatically add the google group as well, it will be hard to
>> keep
>>>> the group useful.
>>>> 
>>>> Agree with Julian a cutoff is necessary anyway, since the google group
>>>> inherently becomes less useful over time, as some information only ends
>> up
>>>> in the apache list.
>>>> 
>>>> On Fri, Mar 9, 2018 at 11:14 AM Nishant Bangarwa <
>>>> nishant.mon...@gmail.com> wrote:
>>>> 
>>>>> We can register dev@druid.apache.org and us...@druid.apache.org as a
>>>>> user in druid user groups so that going forward any mails that are
>> sent to
>>>>> druid google groups are also received on the apache lists and is on the
>>>>> record. This would be to bridge the gap during the migration only.
>>>>> 
>>>>> @Julian, I go ahead and try setting this up, If this seems reasonable ?
>>>>> 
>>>>> On Sat, 10 Mar 2018 at 00:09 Julian Hyde <jhyde.apa...@gmail.com>
>> wrote:
>>>>> 
>>>>>> I don’t know. I don’t think it’s easy.
>>>>>> 
>>>>>> 
>>>>>> On Mar 9, 2018, at 7:31 AM, Roman Leventov <
>>>>>> roman.leven...@metamarkets.com> wrote:
>>>>>> 
>>>>>> Could archives of druid-dev and druid-users mailing lists be
>>>>>> transferred to the new lists?
>>>>>> 
>>>>>> On Thu, Mar 8, 2018 at 8:48 AM, Julian Hyde <jh...@apache.org> wrote:
>>>>>> 
>>>>>>> I’m working on it. It turns that I don’t have sufficient karma to
>>>>>>> create a git repo, so I’ve put in a request on the incubator list.
>>>>>>> 
>>>>>>> 
>>>>>>> On Mar 6, 2018, at 10:12 AM, Xavier Léauté <xav...@confluent.io>
>>>>>>> wrote:
>>>>>>> 
>&g

Re: Roll call?

[Julian Hyde]

Ok, thanks.

> On Mar 25, 2018, at 8:15 AM, Charles Allen <char...@allen-net.com> wrote:
> 
> Its been called out in the weekly sync up and sent to the mailing lists. I 
> think it is safe to move things here in the sense that there’s not much else 
> we can do for anyone who may have not moved over yet.
> 
> 
> 
> Cheers,
> 
> Charles Allen
> 
> 
> 
> 
> 
> 
> From: Julian Hyde <jh...@apache.org>
> Sent: Saturday, March 24, 2018 5:29:44 PM
> To: dev@druid.apache.org
> Subject: Roll call?
> 
> Do we believe that all PPMC members (initial committers and mentors) have now 
> joined the dev and private lists?
> 
> (As a moderator of the mailing lists, I ought to know how to check who is on 
> the list, but I don't. Maybe we should have a roll call.)
> 
> If everyone is on the lists, I think we should move all business to the 
> Apache lists.
> 
> Julian
> 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
> For additional commands, e-mail: dev-h...@druid.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Roll call?

[Julian Hyde]

Do we believe that all PPMC members (initial committers and mentors) have now 
joined the dev and private lists?

(As a moderator of the mailing lists, I ought to know how to check who is on 
the list, but I don't. Maybe we should have a roll call.)

If everyone is on the lists, I think we should move all business to the Apache 
lists.

Julian


-
To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org
For additional commands, e-mail: dev-h...@druid.apache.org

Re: [druid-dev] Apache migration logistics

[Julian Hyde]

I don’t know. I don’t think it’s easy.

> On Mar 9, 2018, at 7:31 AM, Roman Leventov <roman.leven...@metamarkets.com> 
> wrote:
> 
> Could archives of druid-dev and druid-users mailing lists be transferred to 
> the new lists?
> 
> On Thu, Mar 8, 2018 at 8:48 AM, Julian Hyde <jh...@apache.org 
> <mailto:jh...@apache.org>> wrote:
> I’m working on it. It turns that I don’t have sufficient karma to create a 
> git repo, so I’ve put in a request on the incubator list.
> 
> 
>> On Mar 6, 2018, at 10:12 AM, Xavier Léauté <xav...@confluent.io 
>> <mailto:xav...@confluent.io>> wrote:
>> 
>> Julian, it looks like you or one of the mentors has to request the source 
>> code repos. Could you request a gitbox enabled repo?
>> 
>> Based on https://incubator.apache.org/guides/transitioning_asf.html 
>> <https://incubator.apache.org/guides/transitioning_asf.html>, for the 
>> initial migration, we need to involve infra to import the initial git 
>> history and grant them admin rights to the github repo.
>> 
>> Charles, it also sound like we won't be able to do any code migration util 
>> legal signs off on the software grant, could you drive that?
>> 
>> On Mon, Mar 5, 2018 at 12:52 PM Julian Hyde <jhyde.apa...@gmail.com 
>> <mailto:jhyde.apa...@gmail.com>> wrote:
>> The dev, users and private mailing lists now exist. You can see the archives:
>> 
>> * https://lists.apache.org/list.html?dev@druid.apache.org 
>> <https://lists.apache.org/list.html?dev@druid.apache.org>
>> * https://lists.apache.org/list.html?us...@druid.apache.org 
>> <https://lists.apache.org/list.html?us...@druid.apache.org>
>> * https://lists.apache.org/list.html?priv...@druid.apache.org 
>> <https://lists.apache.org/list.html?priv...@druid.apache.org>
>> 
>> To see the last of these, you need to log in.
>> 
>> There will also be archives at 
>> https://mail-archives.apache.org/mod_mbox/druid-dev/ 
>> <https://mail-archives.apache.org/mod_mbox/druid-dev/> etc. (you might need 
>> to wait a few minutes for the archiver to catch up).
>> 
>> If you are an initial committer or mentor, you are a member of the Druid 
>> PPMC and you must be on both dev and private lists now. Send a message to 
>> dev-subscr...@druid.apache.org <mailto:dev-subscr...@druid.apache.org> and 
>> private-subscr...@geode.apache.org 
>> <mailto:private-subscr...@geode.apache.org>.
>> 
>> Everyone else is welcome to join dev and/or users.
>> 
>> Julian
>> 
>>> On Mar 5, 2018, at 11:58 AM, Nishant Bangarwa <nishant.mon...@gmail.com 
>>> <mailto:nishant.mon...@gmail.com>> wrote:
>>> 
>>> Apache Incubator Superset is another example which uses github issues - 
>>> https://github.com/apache/incubator-superset/issues 
>>> <https://github.com/apache/incubator-superset/issues>
>>> For Superset it works as all the github issue interactions are captured in 
>>> ASF owned mailing list via Gitbox Integration. 
>>> See - https://lists.apache.org/list.html?d...@superset.apache.org 
>>> <https://lists.apache.org/list.html?d...@superset.apache.org>
>>> 
>>> For Druid, If everyone agrees we can also choose to capture interactions on 
>>> github issues at an Apache Owned mailing list e.g iss...@druid.apache.org 
>>> <mailto:iss...@druid.apache.org> and continue to use github issues. 
>>> 
>>> @Jihoon, Thanks for the Airflow migration link, super helpful.
>>> 
>>> 
>>> 
>>> On Tue, 6 Mar 2018 at 00:44 Jihoon Son <ghoon...@gmail.com 
>>> <mailto:ghoon...@gmail.com>> wrote:
>>> Gian,
>>> 
>>> there was a discussion for using a third-party issue tracker 
>>> (https://issues.apache.org/jira/browse/LEGAL-249 
>>> <https://issues.apache.org/jira/browse/LEGAL-249>). I think the point is
>>> 
>>> > Okay, it looks like the requirement is just to capture the intent to 
>>> > contribute in ASF-owned infrastructure. That means that the automated 
>>> > process that adds PR information to a JIRA issue or sends it to a mailing 
>>> > list is fine.
>>> 
>>> In short, it looks like allowed (Apache Fluo is using Github issue tracker 
>>> (https://github.com/apache/fluo/issues 
>>> <https://github.com/apache/fluo/issues>)), but it should be captured by 
>>> another issue tracker system owned by ASF. If so, I think it's better to 
>>> use ASF Jira.
>>&g

Re: [druid-dev] Apache migration logistics

[Julian Hyde]

I’m working on it. It turns that I don’t have sufficient karma to create a git 
repo, so I’ve put in a request on the incubator list.

> On Mar 6, 2018, at 10:12 AM, Xavier Léauté <xav...@confluent.io> wrote:
> 
> Julian, it looks like you or one of the mentors has to request the source 
> code repos. Could you request a gitbox enabled repo?
> 
> Based on https://incubator.apache.org/guides/transitioning_asf.html 
> <https://incubator.apache.org/guides/transitioning_asf.html>, for the initial 
> migration, we need to involve infra to import the initial git history and 
> grant them admin rights to the github repo.
> 
> Charles, it also sound like we won't be able to do any code migration util 
> legal signs off on the software grant, could you drive that?
> 
> On Mon, Mar 5, 2018 at 12:52 PM Julian Hyde <jhyde.apa...@gmail.com 
> <mailto:jhyde.apa...@gmail.com>> wrote:
> The dev, users and private mailing lists now exist. You can see the archives:
> 
> * https://lists.apache.org/list.html?dev@druid.apache.org 
> <https://lists.apache.org/list.html?dev@druid.apache.org>
> * https://lists.apache.org/list.html?us...@druid.apache.org 
> <https://lists.apache.org/list.html?us...@druid.apache.org>
> * https://lists.apache.org/list.html?priv...@druid.apache.org 
> <https://lists.apache.org/list.html?priv...@druid.apache.org>
> 
> To see the last of these, you need to log in.
> 
> There will also be archives at 
> https://mail-archives.apache.org/mod_mbox/druid-dev/ 
> <https://mail-archives.apache.org/mod_mbox/druid-dev/> etc. (you might need 
> to wait a few minutes for the archiver to catch up).
> 
> If you are an initial committer or mentor, you are a member of the Druid PPMC 
> and you must be on both dev and private lists now. Send a message to 
> dev-subscr...@druid.apache.org <mailto:dev-subscr...@druid.apache.org> and 
> private-subscr...@geode.apache.org 
> <mailto:private-subscr...@geode.apache.org>.
> 
> Everyone else is welcome to join dev and/or users.
> 
> Julian
> 
>> On Mar 5, 2018, at 11:58 AM, Nishant Bangarwa <nishant.mon...@gmail.com 
>> <mailto:nishant.mon...@gmail.com>> wrote:
>> 
>> Apache Incubator Superset is another example which uses github issues - 
>> https://github.com/apache/incubator-superset/issues 
>> <https://github.com/apache/incubator-superset/issues>
>> For Superset it works as all the github issue interactions are captured in 
>> ASF owned mailing list via Gitbox Integration. 
>> See - https://lists.apache.org/list.html?d...@superset.apache.org 
>> <https://lists.apache.org/list.html?d...@superset.apache.org>
>> 
>> For Druid, If everyone agrees we can also choose to capture interactions on 
>> github issues at an Apache Owned mailing list e.g iss...@druid.apache.org 
>> <mailto:iss...@druid.apache.org> and continue to use github issues. 
>> 
>> @Jihoon, Thanks for the Airflow migration link, super helpful.
>> 
>> 
>> 
>> On Tue, 6 Mar 2018 at 00:44 Jihoon Son <ghoon...@gmail.com 
>> <mailto:ghoon...@gmail.com>> wrote:
>> Gian,
>> 
>> there was a discussion for using a third-party issue tracker 
>> (https://issues.apache.org/jira/browse/LEGAL-249 
>> <https://issues.apache.org/jira/browse/LEGAL-249>). I think the point is
>> 
>> > Okay, it looks like the requirement is just to capture the intent to 
>> > contribute in ASF-owned infrastructure. That means that the automated 
>> > process that adds PR information to a JIRA issue or sends it to a mailing 
>> > list is fine.
>> 
>> In short, it looks like allowed (Apache Fluo is using Github issue tracker 
>> (https://github.com/apache/fluo/issues 
>> <https://github.com/apache/fluo/issues>)), but it should be captured by 
>> another issue tracker system owned by ASF. If so, I think it's better to use 
>> ASF Jira.
>> 
>> BTW, here is a link to a doc how Airflow migrated to Apache 
>> (https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=62693993 
>> <https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=62693993>).
>>  I think it's helpful.
>> 
>> Jihoon
>> 
>> 2018년 3월 5일 (월) 오전 10:47, Nishant Bangarwa <nishant.mon...@gmail.com 
>> <mailto:nishant.mon...@gmail.com>>님이 작성:
>> > So i would propose to setup forwarding to apache lists as first step and 
>> > request everyone to migrate to apache lists, migrate source code then 
>> > website. After sufficient time has passed when everyone has migrated to 
>> > apache mailing lis

Fwd: Podling Report Reminder - March 2018

[Julian Hyde]

Usually podlings get an automated reminder, like the one below, for the IPMC 
report. There was no reminder for Druid this month, because I didn’t get the 
dev list set up in time. Nishant filed a report in time anyway (well done!) but 
I thought you’d be interested in the blurb anyhow.

Julian


> Begin forwarded message:
> 
> From: johndam...@apache.org
> Subject: Podling Report Reminder - March 2018
> Date: March 6, 2018 at 5:32:29 PM PST
> To: d...@quickstep.incubator.apache.org
> Reply-To: d...@quickstep.incubator.apache.org
> 
> Dear podling,
> 
> This email was sent by an automated system on behalf of the Apache
> Incubator PMC. It is an initial reminder to give you plenty of time to
> prepare your quarterly board report.
> 
> The board meeting is scheduled for Wed, 21 March 2018, 10:30 am PDT.
> The report for your podling will form a part of the Incubator PMC
> report. The Incubator PMC requires your report to be submitted 2 weeks
> before the board meeting, to allow sufficient time for review and
> submission (Wed, March 07).
> 
> Please submit your report with sufficient time to allow the Incubator
> PMC, and subsequently board members to review and digest. Again, the
> very latest you should submit your report is 2 weeks prior to the board
> meeting.
> 
> Thanks,
> 
> The Apache Incubator PMC
> 
> Submitting your Report
> 
> --
> 
> Your report should contain the following:
> 
> *   Your project name
> *   A brief description of your project, which assumes no knowledge of
>the project or necessarily of its field
> *   A list of the three most important issues to address in the move
>towards graduation.
> *   Any issues that the Incubator PMC or ASF Board might wish/need to be
>aware of
> *   How has the community developed since the last report
> *   How has the project developed since the last report.
> *   How does the podling rate their own maturity.
> 
> This should be appended to the Incubator Wiki page at:
> 
> https://wiki.apache.org/incubator/March2018
> 
> Note: This is manually populated. You may need to wait a little before
> this page is created from a template.
> 
> Mentors
> ---
> 
> Mentors should review reports for their project(s) and sign them off on
> the Incubator wiki page. Signing off reports shows that you are
> following the project - projects that are not signed may raise alarms
> for the Incubator PMC.
> 
> Incubator PMC