Re: Impala Board Report due by Wed Aug 9th - Initial Reminder

[Jim Apple]

I just posted the following board report:

## Description:
The mission of Apache Impala is the creation and maintenance of software
related to a high-performance distributed SQL engine

## Project Status:
Current project status: Ongoing with high activity
Issues for the board: none

## Membership Data:
Apache Impala was founded 2017-11-14 (6 years ago)
There are currently 67 committers and 38 PMC members in this project.
The Committer-to-PMC ratio is roughly 9:5.

Community changes, past quarter:
- No new PMC members. Last addition was Andrew Sherman on 2023-04-20.
- Kurt Deschler was added as committer on 2023-07-03

## Project Activity:

The latest release was 4.1.2 on 2023-04-10.

Over the last three months, the Impala community has implemented the
following:

- Fixed numerous race conditions and one null pointer exception
- Fixed several build failures (flaky pre-merge tests)
- Improved compatibility with JDK, aarch64, cgroups, Redhat and Ubuntu,
LLVM,
  OpenSSL, and Spring
- Improved compatibility with Apache projects Ozone, Maven, Hive, Iceberg,
  Avro, Hadoop, Atlas, Ranger, Thrift, Kudu, and ORC
- Added high availability to the catalog service
- Added support for building DEB and RPM packages
- Improved performance on TPC-DS
- Fixed two query correctness bugs
- Made many improvements to cardinality estimations

## Community Health:

In answer to the question about community health from the last board report,
each patch produces several emails - every review, commit, new version of a
patch, and bot linter run produces an email.

reviews@ is the most reliable metric of Impala community activity level.
There
were 4042 emails to that list in June, July, and August.

On Wed, Aug 2, 2023 at 6:48 PM Sander Striker  wrote:

> Hello,
>
> According to our records, you are listed as the chair of Impala,
> a committee that is due to submit a report by Wed Aug 9th
> for the next ASF board meeting. This is an initial reminder to prepare a
> report for Impala and submit it as described below.
>
> In the Apache governance model, the ASF board delegates responsibility for
> managing projects to PMCs. This allows projects to govern themselves, in
> terms of their own development goals, guidelines, and volunteer spirit,
> within the scope of our purpose as an open source foundation. The state
> allows us to provide an umbrella of corporate protection to our projects
> and volunteers, but only to the extent that we retain active and effective
> oversight of each project's operation on behalf of the public's interest.
>
> To enable the board to provide oversight across the foundation, each PMC
> is tasked with providing the board a quarterly report on the health of
> their project. This allows us to hear your heartbeat, to see the project
> through your eyes, and to inform the public through our meeting minutes.
>
> Thank you for being a responsible project chair and helping us maintain
> oversight over the Apache Software Foundation. If, for whatever reason,
> a full report is not possible by the deadline, please report just that.
> It's okay to postpone a report by a month.
>
> Please note that we will be reviewing and approving these reports,
> asynchronously, during the week prior to our scheduled board meeting.
> Reports received after Wed Aug 9th will be postponed to the
> next regular meeting. Please try to submit your report early.
>
>
> Submitting your report
> --
>
> Full details about reporting to the board are at
>
>   https://www.apache.org/foundation/board/reporting
>
> Please be aware that the board is looking for your personal observations,
> assessment, and ideas, not just raw statistics.
>
> Chairs may use one of several mechanisms to submit or edit their report:
>
>  a) the Apache Reporter Service
> https://reporter.apache.org/
>
>  b) the Whimsy online agenda tool
> https://whimsy.apache.org/board/agenda/2023-08-16/Impala
>
>  c) carefully editing and committing changes to the dated agenda in
> https://svn.apache.org/repos/private/foundation/board
>
>  d) or, if none of the above work, send an email to bo...@apache.org with
> Subject: [REPORT] Impala
>
> If you believe it won't be possible to prepare a report before the
> deadline,
> or if the PMC is aware that the Chair is unavailable, please report that
> and we can reschedule or have someone else report on your behalf.
>
>
> Attending the Board Meeting
> ---
>
> The formal board meeting (usually an online videoconference) will be held
> at
>
>   Wed, 16 Aug 2023 at 21:00 UTC
>
> which in other time zones is
>
>   https://timeanddate.com/s/4357
>
> As always, chairs and ASF members are welcome to attend the board meeting.
> However, in most cases, we will not be using meeting time to discuss
> reports
> unless you specifically request time to speak in person.
>
> During the week prior to the meeting, the directors will read the received
> reports, make comments (if any) within the agenda 

Re: [DISCUSS] Maintenance Release for Impala 4.1.2

[Jim Apple]

SGTM

On Fri, Feb 17, 2023 at 2:44 PM Quanlong Huang 
wrote:

> Hi all,
>
> Impala 4.1.1 was released in Oct. 2022. There are several bug fixes that
> would be nice to be backported, e.g.
>
> d3ece4433 IMPALA-11857: Connect join build fragment to join in graphical
> plan
>
> ff7b5db60 IMPALA-11081: Fix incorrect results in partition key scan
>
> 32536ba25 IMPALA-11845: (Addendum) Don't specify db name in the new struct
> tests
>
> 0c1bd9eff IMPALA-11845: Fix incorrect check of struct STAR path in
> resolvePathWithMasking
>
> b0009db40 IMPALA-11843: Fix IndexOutOfBoundsException in analytic limit
> pushdown
>
> 11068d9ae IMPALA-11811: Avoid storing unregistered predicate objects in a
> Map
>
> b816b6e39 IMPALA-11753: CatalogD OOMkilled due to natively allocated memory
>
> 6838e988a IMPALA-11696: Fix incorrect warnings of ignoring delimiters on
> text/sequence tables
>
> 3c9d23a23 IMPALA-11707: Fix global runtime IN-list filter of numeric types
> are AlwaysFalse
>
> 367378438 IMPALA-11744: Table mask view should preserve the original column
> order in Hive
>
> f8819ac7c IMPALA-11751: (Addendum) fix test for Ozone
>
> bd9f44838 IMPALA-11751: Template tuple of Avro header should be transferred
> to ScanRangeSharedState
>
> 77d80aeda IMPALA-11812: Deduplicate column schema in hmsPartitions
>
> 64efb7695 IMPALA-11779: Fix crash in TopNNode due to slots in null type
>
> Though we have released 4.2.0 in Dec. 2022, the 4.1 branch can be
> considered as a stable release since it's a better target for 3.x users to
> upgrade to 4.x, e.g. Impala 4.1 depends on kudu 1.16, while Impala 4.2
> depends on a snapshot version of Kudu.
>
> I propose that we do a maintenance release for 4.1.2. What do you think? If
> there are no objections, I volunteer to be the release manager of Impala
> 4.1.2.
>
> Thanks,
> Quanlong
>

Fwd: Impala Board Report due by Wed Feb 8th - Initial Reminder

[Jim Apple]

Here is the report I prepared for the board:

## Description:
The mission of Apache Impala is the creation and maintenance of software
related to a high-performance distributed SQL engine

## Issues:
There are no issues requiring board attention.

## Membership Data:
Apache Impala was founded 2017-11-14 (5 years ago)
There are currently 65 committers and 37 PMC members in this project.
The Committer-to-PMC ratio is roughly 9:5.

Community changes, past quarter:
- Daniel Becker was added to the PMC on 2023-01-16
- Yida Wu was added as committer on 2023-01-20
- Li Penglin was added as committer on 2022-12-20
- Michael Smith was added as committer on 2022-11-07

## Project Activity:

The Impala community had implemented the following over the last three
months:

- Improved support for Apache projects including Iceberg, Hadoop,
  Ozone, Thrift, Ranger, Hive, Parquet, Avro, Hudi, and Kudu
- Multiple dependency upgrades due to their CVEs
- Multiple tracing and debugging improvements
- Multiple fixes for flaky tests
- Numerous documentation fixes
- Some tightening of authorization constraints
- DDL support for bucketed tables
- Improved support for Docker and for Ubuntu 16.04
- Made the docs much prettier
- Added support for Aliyun Object Storage Service
- Fixed multiple crashes


## Community Health:

4.2.0 was released 2022-12-12.

reviews@ is the most reliable metric of Impala community activity
level. There were 2980 emails to that list in November, December, and
January. Impala remains a vibrant project.

-- Forwarded message -
From: Sander Striker 
Date: Wed, Jan 25, 2023 at 12:48 PM
Subject: Impala Board Report due by Wed Feb 8th - Initial Reminder
To: Jim Apple 
Cc: 


Hello,

According to our records, you are listed as the chair of Impala,
a committee that is due to submit a report by Wed Feb 8th
for the next ASF board meeting. This is an initial reminder to prepare a
report for Impala and submit it as described below.

In the Apache governance model, the ASF board delegates responsibility for
managing projects to PMCs. This allows projects to govern themselves, in
terms of their own development goals, guidelines, and volunteer spirit,
within the scope of our purpose as an open source foundation. The state
allows us to provide an umbrella of corporate protection to our projects
and volunteers, but only to the extent that we retain active and effective
oversight of each project's operation on behalf of the public's interest.

To enable the board to provide oversight across the foundation, each PMC
is tasked with providing the board a quarterly report on the health of
their project. This allows us to hear your heartbeat, to see the project
through your eyes, and to inform the public through our meeting minutes.

Thank you for being a responsible project chair and helping us maintain
oversight over the Apache Software Foundation. If, for whatever reason,
a full report is not possible by the deadline, please report just that.
It's okay to postpone a report by a month.

Please note that we will be reviewing and approving these reports,
asynchronously, during the week prior to our scheduled board meeting.
Reports received after Wed Feb 8th will be postponed to the
next regular meeting. Please try to submit your report early.


Submitting your report
--

Full details about reporting to the board are at

  https://www.apache.org/foundation/board/reporting

Please be aware that the board is looking for your personal observations,
assessment, and ideas, not just raw statistics.

Chairs may use one of several mechanisms to submit or edit their report:

 a) the Apache Reporter Service
https://reporter.apache.org/

 b) the Whimsy online agenda tool
https://whimsy.apache.org/board/agenda/2023-02-15/Impala

 c) carefully editing and committing changes to the dated agenda in
https://svn.apache.org/repos/private/foundation/board

 d) or, if none of the above work, send an email to bo...@apache.org with
Subject: [REPORT] Impala

If you believe it won't be possible to prepare a report before the deadline,
or if the PMC is aware that the Chair is unavailable, please report that
and we can reschedule or have someone else report on your behalf.


Attending the Board Meeting
---

The formal board meeting (usually an online videoconference) will be held at

  Wed, 15 Feb 2023 at 22:00 UTC

which in other time zones is

  https://timeanddate.com/s/4303

As always, chairs and ASF members are welcome to attend the board meeting.
However, in most cases, we will not be using meeting time to discuss reports
unless you specifically request time to speak in person.

During the week prior to the meeting, the directors will read the received
reports, make comments (if any) within the agenda tool, discuss those
comments on the board and/or private committee lists, and vote to approve.

If we have comments on a report, we will forward them during the
review

Re: CWiki Access

[Jim Apple]

Done

On Thu, Dec 15, 2022 at 12:51 PM Jason Fehr  wrote:

> Hello,
>
> Please grant my cwiki user "jasonmfehr" write access to the Impala wiki at
> https://cwiki.apache.org/confluence/display/IMPALA/Impala+Home
>
> Thanks,
> Jason
>

Re: Apache Jira Account

[Jim Apple]

What do you want your username to be?

On Fri, Nov 18, 2022 at 10:11 PM Jason Fehr  wrote:

> Greetings,
>
> I recently joined Cloudera working on the Impala backend team reporting to
> Abhishek Rawat.  I am going to do Impala engine related work and thus need
> an account on the Apache Jira instance and also access to the "IMPALA" Jira
> project.  Can someone please set that up for me?
>
> Thanks,
> Jason
>

Fwd: Impala Board Report due by Wed Nov 9th - Initial Reminder

[Jim Apple]

Here is the board report I just posted:

## Description:
The mission of Apache Impala is the creation and maintenance of
software related to a high-performance distributed SQL engine

## Issues:
There are no issues requiring board attention.

## Membership Data:
Apache Impala was founded 2017-11-14 (5 years ago)
There are currently 63 committers and 36 PMC members in this project.
The Committer-to-PMC ratio is roughly 8:5.

Community changes, past quarter:
- No new PMC members. Last addition was Tamás Máté on 2022-07-10.
- No new committers in August, September, and October. Michael Smith
  was added as committer on 2022-11-07

## Project Activity:

 - Improved support for Apache projects including Hadoop, Iceberg,
   Hive, Ozone, Commons, Kudu, Ranger, ORC, Parquet, Tez, and Thrift
 - Improved support for Guava, Jackson, AWS S3, Tencent COS, Ubuntu
   18+, log4j 1.x -> reload 4j, Docker, Java 11, Redhat and
   Redhat-based Linux distributions, Spring, flatbuffers, GCC 10.4,
   Docker, zlib, and zstd
 - Reduced compile times and built binaries' size
 - Improved debugging support
 - Increased decimal performance
 - Added support for TBLPROPERTIES on views
 - Fixed multiple flaky tests
 - Fixed multiple memory leaks
 - Added support for map type in SELECT list
 - Added support for TLS 1.3
 - Added support for BINARY columns
 - Made multiple improvements to code review tooling

## Community Health:

reviews@ is the most reliable metric of Impala community activity
level. There were 3847 emails to that list in August, September, and
October. Impala remains a vibrant project.

4.1.1 was released on 2022-10-20.

-- Forwarded message -
From: Sander Striker 
Date: Wed, Nov 2, 2022 at 3:56 AM
Subject: Impala Board Report due by Wed Nov 9th - Initial Reminder
To: Jim Apple 
Cc: 


Hello,

According to our records, you are listed as the chair of Impala,
a committee that is due to submit a report by Wed Nov 9th
for the next ASF board meeting. This is an initial reminder to prepare a
report for Impala and submit it as described below.

In the Apache governance model, the ASF board delegates responsibility for
managing projects to PMCs. This allows projects to govern themselves, in
terms of their own development goals, guidelines, and volunteer spirit,
within the scope of our purpose as an open source foundation. The state
allows us to provide an umbrella of corporate protection to our projects
and volunteers, but only to the extent that we retain active and effective
oversight of each project's operation on behalf of the public's interest.

To enable the board to provide oversight across the foundation, each PMC
is tasked with providing the board a quarterly report on the health of
their project. This allows us to hear your heartbeat, to see the project
through your eyes, and to inform the public through our meeting minutes.

Thank you for being a responsible project chair and helping us maintain
oversight over the Apache Software Foundation. If, for whatever reason,
a full report is not possible by the deadline, please report just that.
It's okay to postpone a report by a month.

Please note that we will be reviewing and approving these reports,
asynchronously, during the week prior to our scheduled board meeting.
Reports received after Wed Nov 9th will be postponed to the
next regular meeting. Please try to submit your report early.


Submitting your report
--

Full details about reporting to the board are at

  https://www.apache.org/foundation/board/reporting

Please be aware that the board is looking for your personal observations,
assessment, and ideas, not just raw statistics.

Chairs may use one of several mechanisms to submit or edit their report:

 a) the Apache Reporter Service
https://reporter.apache.org/

 b) the Whimsy online agenda tool
https://whimsy.apache.org/board/agenda/2022-11-16/Impala

 c) carefully editing and committing changes to the dated agenda in
https://svn.apache.org/repos/private/foundation/board

 d) or, if none of the above work, send an email to bo...@apache.org with
Subject: [REPORT] Impala

If you believe it won't be possible to prepare a report before the deadline,
or if the PMC is aware that the Chair is unavailable, please report that
and we can reschedule or have someone else report on your behalf.


Attending the Board Meeting
---

The formal board meeting (usually an online videoconference) will be held at

  Wed, 16 Nov 2022 at 22:00 UTC

which in other time zones is

  https://timeanddate.com/s/42q5

As always, chairs and ASF members are welcome to attend the board meeting.
However, in most cases, we will not be using meeting time to discuss reports
unless you specifically request time to speak in person.

During the week prior to the meeting, the directors will read the received
reports, make comments (if any) within the agenda tool, discuss those
comments on the board and/or private commit

Re: [VOTE] 4.1.1 release candidate 2

[Jim Apple]

+1 (binding)

On Mon, Oct 17, 2022 at 5:31 AM Xiang Yang  wrote:

> +1 (non-binding)
>
>
> - Verified signature, checksum
>
> - Built from source on CentOS 7.9
>
> - Verified the version
>
> - Ran validate-java-pom-versions.sh
>
>
>
>
>
>
>
> Thanks,
>
> Xiang Yang
>
>
>
>
> At 2022-10-17 19:20:39,"xiabaike" , said:
> >+1 (non-binding)
> >
> >
> >- Checked out the code for 4.1.1-rc2 and ran 'buildall.sh -notests
> -skiptests -release_and_debug'
> >on a Ubuntu 20.04.4 machine successfully.
> >- Checked version with 'SELECT version()'.
> >- Ran 'pushd fe && mvn -fae test' and all tests passed.
> >
> >
> >
> >
> >在 2022年10月17日 17:02，Zhang Yifan 写道：
> >
> >
> >+1 (non-binding) - Ran release.sh and all checks passed. - Downloaded the
> code for 4.1.1-rc2 and ran 'build.sh -notests -release_and_debug' on a
> CentOS 7.9 machine successfully. - Checked version with 'SELECT version()'
> 在 2022-10-17 14:28:28，"skyyws"  写道： >+1 (non-binding) > >
> >- Ran release.sh script, all checks passed(VERSION=4.1.1
> RELEASE_CANDIDATE=2 TREE_HASH=1ed5cadb58a95f8b307a4e4494099a8d4811525d
> RELEASE_MANAGER=stigahuang) >- Downloading apache-impala-4.1.1.tar.gz<">
> https://dist.apache.org/repos/dist/dev/impala/4.1.1/RC1/apache-impala-4.1.1.tar.gz>
> on CentOS7.2 and running 'buildall.sh -notests’ success. > > > > >
> Replied Message  >| From | tangzhi@qq...<2433038...@qq.com> | >| Date
> | 10/13/2022 16:39 | >| To | dev | >| Subject |
> 回复：[VOTE] 4.1.1 release candidate 2 | >+1. Thanks Quanlong! > > > >
> >--原始邮件-- >发件人: "dev" <
> huangquanl...@gmail.com; >发送时间:2022年10月13日(星期四) 下午2:10
> >收件人:"dev" >主题:[VOTE] 4.1.1
> release candidate 2 > > > >Hi all, > >This is a vote for a maintenance
> release of Impala 4.1.1. > >The artifacts for testing can be downloaded
> from: >https://dist.apache.org/repos/dist/dev/impala/4.1.1/RC2/ >Git tag:
> 4.1.1-rc2 https://github.com/apache/impala/releases/tag/4.1.1-rc2 >Tree
> hash: 1ed5cadb58a95f8b307a4e4494099a8d4811525d > >Please vote +1 or -1. -1
> votes should be accompanied by an explanation of >the reason. Only PMC
> members have binding votes, but other community >members are encouraged to
> cast non-binding votes. This vote will pass if >there are 3 binding +1
> votes and more binding +1 votes than -1 votes. > >This wiki page describes
> how to check the release before you vote: >
> https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release#HowtoRelease-HowtoVoteonaReleaseCandidate
> > >I tested this by the release test job: >
> https://jenkins.impala.io/job/release-test-ub1604/68/ >Also verified the
> build from the tarball locally. > >Previous discussion threads: >
> https://lists.apache.org/thread/flqpgynfd20scjdzh40n4wltl7xf51wx >
> https://lists.apache.org/thread/6z0cld1snoj4q50mp66ns17xhdct1jm2 >
> >Previous vote thread: >
> https://lists.apache.org/thread/46m3rcdxqr0xp7rc22wcptqjnsh1hmkz >
> >Thanks, >Quanlong
>

Re: Impala Board Report due by Fri Aug 12th - Reminder

[Jim Apple]

I have submitted the following board report:

## Description:
The mission of Apache Impala is the creation and maintenance of software
related to a high-performance distributed SQL engine

## Issues:
There are no issues requiring board attention.

## Membership Data:
Apache Impala was founded 2017-11-14 (5 years ago)
There are currently 62 committers and 36 PMC members in this project.
The Committer-to-PMC ratio is roughly 8:5.

Community changes, past quarter:
- Tamás Máté was added to the PMC on 2022-07-10
- Riza Suminto was added as committer on 2022-05-27

## Project Activity:

 - Improved support for Apache projects including Iceberg, Parquet,
   Ozone, Kudu, Hive, Avro, HBase, ORC, Ranger, Thrift, Tez, YARN, and
   Hadoop
 - Improved support for re2, Google Cloud, Ubuntu 20, Kerberos,
   CentOS, and Tlinux
 - Multiple improvements to the build system
 - Improved support for timestamps
 - Improved support for views
 - Fix multiple undefined behaviors in C++ code
 - Multiple flaky test improvements
 - Multiple improvements to our Python test and shell environments,
   including transposed result printing
 - Increase security of transport protocols by eliminating default
   support for RC4
 - Support for various statistical UDAFs

## Community Health:

reviews@ is the most reliable metric of Impala community activity
level. There were 3083 emails to that list in May, June, and
July. Impala remains a vibrant project.

4.1.0 was released on 2022-06-01.

On Mon, Aug 8, 2022 at 11:54 AM Sander Striker  wrote:

> Hello,
>
> According to our records, you are listed as the chair of Impala,
> a committee that is due to submit a report by Wed Aug 10th
> for the next ASF board meeting. This is a reminder to prepare a
> report for Impala and submit it as described below.  As this
> reminder reaches you later than usual, please submit the report
> by Fri Aug 12th.
>
> In the Apache governance model, the ASF board delegates responsibility for
> managing projects to PMCs. This allows projects to govern themselves, in
> terms of their own development goals, guidelines, and volunteer spirit,
> within the scope of our purpose as an open source foundation. The state
> allows us to provide an umbrella of corporate protection to our projects
> and volunteers, but only to the extent that we retain active and effective
> oversight of each project's operation on behalf of the public's interest.
>
> To enable the board to provide oversight across the foundation, each PMC
> is tasked with providing the board a quarterly report on the health of
> their project. This allows us to hear your heartbeat, to see the project
> through your eyes, and to inform the public through our meeting minutes.
>
> Thank you for being a responsible project chair and helping us maintain
> oversight over the Apache Software Foundation. If, for whatever reason,
> a full report is not possible by the deadline, please report just that.
> It's okay to postpone a report by a month.
>
> Please note that we will be reviewing and approving these reports,
> asynchronously, during the week prior to our scheduled board meeting.
> Reports received after Wed Aug 10th will be postponed to the
> next regular meeting. Please try to submit your report early.
>
>
> Submitting your report
> --
>
> Full details about reporting to the board are at
>
>   https://www.apache.org/foundation/board/reporting
>
> Please be aware that the board is looking for your personal observations,
> assessment, and ideas, not just raw statistics.
>
> Chairs may use one of several mechanisms to submit or edit their report:
>
>  a) the Apache Reporter Service
> https://reporter.apache.org/
>
>  b) the Whimsy online agenda tool
> https://whimsy.apache.org/board/agenda/2022-08-17/Impala
>
>  c) carefully editing and committing changes to the dated agenda in
> https://svn.apache.org/repos/private/foundation/board
>
>  d) or, if none of the above work, send an email to bo...@apache.org with
> Subject: [REPORT] Impala
>
> If you believe it won't be possible to prepare a report before the
> deadline,
> or if the PMC is aware that the Chair is unavailable, please report that
> and we can reschedule or have someone else report on your behalf.
>
>
> Attending the Board Meeting
> ---
>
> The formal board meeting (usually an online videoconference) will be held
> at
>
>   Wed, 17 Aug 2022 at 15:00 UTC
>
> which in other time zones is
>
>   https://timeanddate.com/s/42nr
>
> As always, chairs and ASF members are welcome to attend the board meeting.
> However, in most cases, we will not be using meeting time to discuss
> reports
> unless you specifically request time to speak in person.
>
> During the week prior to the meeting, the directors will read the received
> reports, make comments (if any) within the agenda tool, discuss those
> comments on the board and/or private committee lists, and vote to approve.
>
> If we have comments on a 

Fwd: Impala Board Report due by Wed May 11st - Initial Reminder

[Jim Apple]

Here is the report I just submitted:

## Description:
The mission of Apache Impala is the creation and maintenance of software
related to a high-performance distributed SQL engine.

## Issues:
There are no issues requiring board attention.

## Membership Data:
Apache Impala was founded 2017-11-14 (4 years ago).
There are currently 61 committers and 35 PMC members in this project.
The Committer-to-PMC ratio is roughly 8:5.

Community changes, past quarter:
- No new PMC members. Last addition was Laszlo Gaal on 2022-01-20.
- No new committers. Last addition was Daniel Becker on 2021-12-07.

## Project Activity:

 - Improved support for compatibility with Apache projects Iceberg,
   ORC, Hive, Ranger, Parquet, Thrift, and Kudu
 - Many fixes for flaky tests
 - Improved support for materialized views
 - Upgraded Spring past three CVEs
 - Upgraded other packages past other CVEs
 - Numerous performance improvements, including some queries improved
   by 50% or more

## Community Health:
reviews@ is the best gauge of Impala community activity. There were
2532 emails to reviews@ in the last three months; Impala remains a
busy community.

The most recent release was Impala 3.4.1, on 2022-04-07.

-- Forwarded message -
Date: Wed, May 4, 2022 at 8:19 AM
Subject: Impala Board Report due by Wed May 11st - Initial Reminder

Hello,

According to our records, you are listed as the chair of Impala,
a committee that is due to submit a report by Wed May 11st
for the next ASF board meeting. This is an initial reminder to prepare a
report for Impala and submit it as described below.

In the Apache governance model, the ASF board delegates responsibility for
managing projects to PMCs. This allows projects to govern themselves, in
terms of their own development goals, guidelines, and volunteer spirit,
within the scope of our purpose as an open source foundation. The state
allows us to provide an umbrella of corporate protection to our projects
and volunteers, but only to the extent that we retain active and effective
oversight of each project's operation on behalf of the public's interest.

To enable the board to provide oversight across the foundation, each PMC
is tasked with providing the board a quarterly report on the health of
their project. This allows us to hear your heartbeat, to see the project
through your eyes, and to inform the public through our meeting minutes.

Thank you for being a responsible project chair and helping us maintain
oversight over the Apache Software Foundation. If, for whatever reason,
a full report is not possible by the deadline, please report just that.
It's okay to postpone a report by a month.

Please note that we will be reviewing and approving these reports,
asynchronously, during the week prior to our scheduled board meeting.
Reports received after Wed May 11st will be postponed to the
next regular meeting. Please try to submit your report early.


Submitting your report
--

Full details about reporting to the board are at

  https://www.apache.org/foundation/board/reporting

Please be aware that the board is looking for your personal observations,
assessment, and ideas, not just raw statistics.

Chairs may use one of several mechanisms to submit or edit their report:

 a) the Apache Reporter Service
https://reporter.apache.org/

 b) the Whimsy online agenda tool
https://whimsy.apache.org/board/agenda/2022-05-18/Impala

 c) carefully editing and committing changes to the dated agenda in
https://svn.apache.org/repos/private/foundation/board

 d) or, if none of the above work, send an email to bo...@apache.org with
Subject: [REPORT] Impala

If you believe it won't be possible to prepare a report before the deadline,
or if the PMC is aware that the Chair is unavailable, please report that
and we can reschedule or have someone else report on your behalf.


Attending the Board Meeting
---

The formal board meeting (usually an online videoconference) will be held at

  Wed, 18 May 2022 at 15:00 UTC

which in other time zones is

  https://timeanddate.com/s/42hh

As always, chairs and ASF members are welcome to attend the board meeting.
However, in most cases, we will not be using meeting time to discuss reports
unless you specifically request time to speak in person.

During the week prior to the meeting, the directors will read the received
reports, make comments (if any) within the agenda tool, discuss those
comments on the board and/or private committee lists, and vote to approve.

If we have comments on a report, we will forward them during the
review and attempt to complete any associated action items as well.
This will allow us to be more responsive to project needs and give you
an opportunity to expand on your report if additional details are
requested prior to the meeting.

Regular board meetings are held monthly, as scheduled at

  https://svn.apache.org/repos/private/committers/board/calendar.txt


Requesting a 

Re: Jira issue assignment

[Jim Apple]

done. Welcome aboard!

On Sat, Apr 23, 2022 at 6:14 PM Jaden Perleoni 
wrote:

> I would like to contribute to your oss. I am completely new to this, and I
> have no idea if this is the way to ask to be assigned to an issue, but the
> contribution guide told me to provide my username and the link. My username
> is: CoolDuck and the issue that I would like to be assigned to is this:
> https://issues.apache.org/jira/browse/IMPALA-11215?filter=12341668.
>

Fwd: Impala Board Report due by Wed Nov 10th - Reminder

[Jim Apple]

Here's my report this quarter. It's skinnier than usual:

## Description:
The mission of Apache Impala is the creation and maintenance of software
related to a high-performance distributed SQL engine

## Issues:
There are no issues requiring board attention

## Membership Data:
Apache Impala was founded 2017-11-14 (4 years ago)
There are currently 59 committers and 34 PMC members in this project.
The Committer-to-PMC ratio is roughly 8:5.

Community changes, past quarter:
- No new PMC members. Last addition was Vihang Karajgaonkar on 2021-06-03.
- No new committers. Last addition was Wenzhe Zhou on 2021-07-09.

## Project Activity:

During August, September, and October, the Impala community:

- Improved support for integrations with Apache projects Ozone, ORC,
  Iceberg, Hive, Ranger, Kudu, DataSketches, Parquet, and HDFS

- Improved integration with non-Apache projects, formats, or protocols
  S3, CentOS 7, PyPi, flame graphs, Docker, and LDAP

This quarter few new features landed that weren't integrations as
mentioned above. Most other patches were bug fixes.

## Community Health:

Perhaps the most stable indicator of Impala activity is reviews@,
which registers an email for each code review, each submit, and each
Jenkins job completion. This decreased this quarter to 2415 from 2819,
a 14% decline. Impala is still a thriving community.

-- Forwarded message -
From: Sander Striker 
Date: Sat, Nov 6, 2021 at 3:53 AM
Subject: Impala Board Report due by Wed Nov 10th - Reminder
To: Jim Apple 
Cc: 


Hello,

According to our records, you are listed as the chair of Impala,
a committee that is due to submit a report by Wed Nov 10th
for the next ASF board meeting. This is an initial reminder to prepare a
report for Impala and submit it as described below.

In the Apache governance model, the ASF board delegates responsibility for
managing projects to PMCs. This allows projects to govern themselves, in
terms of their own development goals, guidelines, and volunteer spirit,
within the scope of our purpose as an open source foundation. The state
allows us to provide an umbrella of corporate protection to our projects
and volunteers, but only to the extent that we retain active and effective
oversight of each project's operation on behalf of the public's interest.

To enable the board to provide oversight across the foundation, each PMC
is tasked with providing the board a quarterly report on the health of
their project. This allows us to hear your heartbeat, to see the project
through your eyes, and to inform the public through our meeting minutes.

Thank you for being a responsible project chair and helping us maintain
oversight over the Apache Software Foundation. If, for whatever reason,
a full report is not possible by the deadline, please report just that.
It's okay to postpone a report by a month.

Please note that we will be reviewing and approving these reports,
asynchronously, during the week prior to our scheduled board meeting.
Reports received after Wed Nov 10th will be postponed to the
next regular meeting. Please try to submit your report early.


Submitting your report
--

Full details about reporting to the board are at

  https://www.apache.org/foundation/board/reporting

Please be aware that the board is looking for your personal observations,
assessment, and ideas, not just raw statistics.

Chairs may use one of several mechanisms to submit or edit their report:

 a) the Apache Reporter Service
https://reporter.apache.org/

 b) the Whimsy online agenda tool
https://whimsy.apache.org/board/agenda/2021-11-17/Impala

 c) carefully editing and committing changes to the dated agenda in
https://svn.apache.org/repos/private/foundation/board

 d) or, if none of the above work, send an email to bo...@apache.org with
Subject: [REPORT] Impala

If you believe it won't be possible to prepare a report before the deadline,
or if the PMC is aware that the Chair is unavailable, please report that
and we can reschedule or have someone else report on your behalf.


Attending the Board Meeting
---

The formal board meeting (usually an online videoconference) will be held at

  Wed, 17 Nov 2021 at 22:00 UTC

which in other time zones is

  https://timeanddate.com/s/42e8

As always, chairs and ASF members are welcome to attend the board meeting.
However, in most cases, we will not be using meeting time to discuss reports
unless you specifically request time to speak in person.

During the week prior to the meeting, the directors will read the received
reports, make comments (if any) within the agenda tool, discuss those
comments on the board and/or private committee lists, and vote to approve.

If we have comments on a report, we will forward them during the
review and attempt to complete any associated action items as well.
This will allow us to be more responsive to project needs and give you
an opportunity to expand on your report

Fwd: Impala Board Report due by Wed Aug 11st - Initial Reminder

[Jim Apple]

Here is the report I filed with the board:

## Description:
The mission of Apache Impala is the creation and maintenance of software
related to a high-performance distributed SQL engine

## Issues:
There are no issues requiring board attention

## Membership Data:
Apache Impala was founded 2017-11-14 (4 years ago)
There are currently 59 committers and 34 PMC members in this project.
The Committer-to-PMC ratio is roughly 8:5.

Community changes, past quarter:
- Vihang Karajgaonkar was added to the PMC on 2021-06-03
- Qifan Chen was added as committer on 2021-06-25
- Tamás Máté was added as committer on 2021-06-11
- Wenzhe Zhou was added as committer on 2021-07-09

## Project Activity:

Impala 4.0.0 was released on 2021-07-12.

CVE-2021-28131 was filed, fixed, and announced.

The Impala community also accomplished:

 * Increased compatibility with other Apache projects, including
   Parquet, Hive, Iceberg, ORC, Ranger, Kudu, DataSketches
 * Improved support for z-order
 * Added functionality to impala-shell (a rarely touched part of the
   codebase)
 * Added support for JSON Web Tokens ("JWT")
 * Added more support for running Impala in containers
 * Fixed multiple DDL race conditions
 * Added multiple planner heuristic improvements to join cardinality
   estimates
 * Added multiple expansions to use of min/max filters
 * Added some support for Alibaba cloud
 * Made multiple fixes to ACID table support

## Community Health:

Perhaps the most stable indicator of Impala activity is reviews@,
which registers an email for each code review, each submit, and each
Jenkins job completion. This decreased this quarter to 2902 from 3153,
an 8% decline. Impala is still a thriving community.


-- Forwarded message -
From: Sander Striker 
Date: Mon, Aug 2, 2021 at 5:30 AM
Subject: Impala Board Report due by Wed Aug 11st - Initial Reminder
To: Jim Apple 
Cc: 


Hello,

According to our records, you are listed as the chair of Impala,
a committee that is due to submit a report by Wed Aug 11st
for the next ASF board meeting. This is an initial reminder to prepare a
report for Impala and submit it as described below.

In the Apache governance model, the ASF board delegates responsibility for
managing projects to PMCs. This allows projects to govern themselves, in
terms of their own development goals, guidelines, and volunteer spirit,
within the scope of our purpose as an open source foundation. The state
allows us to provide an umbrella of corporate protection to our projects
and volunteers, but only to the extent that we retain active and effective
oversight of each project's operation on behalf of the public's interest.

To enable the board to provide oversight across the foundation, each PMC
is tasked with providing the board a quarterly report on the health of
their project. This allows us to hear your heartbeat, to see the project
through your eyes, and to inform the public through our meeting minutes.

Thank you for being a responsible project chair and helping us maintain
oversight over the Apache Software Foundation. If, for whatever reason,
a full report is not possible by the deadline, please report just that.
It's okay to postpone a report by a month.

Please note that we will be reviewing and approving these reports,
asynchronously, during the week prior to our scheduled board meeting.
Reports received after Wed Aug 11st will be postponed to the
next regular meeting. Please try to submit your report early.


Submitting your report
--

Full details about reporting to the board are at

  https://www.apache.org/foundation/board/reporting

Please be aware that the board is looking for your personal observations,
assessment, and ideas, not just raw statistics.

Chairs may use one of several mechanisms to submit or edit their report:

 a) the Apache Reporter Service
https://reporter.apache.org/

 b) the Whimsy online agenda tool
https://whimsy.apache.org/board/agenda/2021-08-18/Impala

 c) carefully editing and committing changes to the dated agenda in
https://svn.apache.org/repos/private/foundation/board

 d) or, if none of the above work, send an email to bo...@apache.org with
Subject: [REPORT] Impala

If you believe it won't be possible to prepare a report before the deadline,
or if the PMC is aware that the Chair is unavailable, please report that
and we can reschedule or have someone else report on your behalf.


Attending the Board Meeting
---

The formal board meeting (usually an online videoconference) will be held at

  Wed, 18 Aug 2021 at 22:00 UTC

which in other time zones is

  https://timeanddate.com/s/42ck

As always, chairs and ASF members are welcome to attend the board meeting.
However, in most cases, we will not be using meeting time to discuss reports
unless you specifically request time to speak in person.

During the week prior to the meeting, the directors will read the received
reports, make comments (if a

Re: Impala 4.0.0 Status & Preparation on Release Note

[Jim Apple]

Thank you, Quanlong!

On Sun, Jul 11, 2021 at 6:46 AM Quanlong Huang 
wrote:

> Hi all,
>
> The vote has passed. We are very close to releasing 4.0.0.
>
> I'm moving the work of release notes to Gerrit:
> https://gerrit.cloudera.org/c/17671/. You can still comment in the Google
> doc. I'll update the gerrit patch as well.
>
> Thanks,
> Quanlong
>
> On Mon, Jul 5, 2021 at 8:41 PM Quanlong Huang 
> wrote:
>
> > Sorry that I was busy last week unfortunately. So the voting was
> delayed..
> > I finally posted it today.
> >
> > For the release note, I'm writing a draft here:
> >
> https://docs.google.com/document/d/1Oee3VPIPJ90uOM5VStcNYt1qEH5-T6F61U5_GM4YHF0/edit?usp=sharing
> > This release has 700+ JIRAs. I try to write short bullet points and
> > ignore trivial JIRAs. However, due to my limitation, I may have missed
> some
> > work. So I wish every contributor can write down his/her remarkable work
> > there with some (optional) short description. Thanks!
> >
> > BTW, some JIRAs are marked as fixed in 4.0.0 but actually should be
> 4.1.0.
> > Please use 4.1.0 as the fixed version in new JIRAs.
> >
> > Thanks,
> > Quanlong
> >
> > On Sun, Jul 4, 2021 at 6:27 AM Jim Apple  wrote:
> >
> >> What's the latest on this? Those two bugs are fixed now, thankfully.
> >>
> >> On Mon, Jun 21, 2021 at 12:28 AM Quanlong Huang <
> huangquanl...@gmail.com>
> >> wrote:
> >>
> >> > Hi all,
> >> >
> >> > I'm still working on starting a RC vote for Impala 4.0.0. It's delayed
> >> > unfortunately since we continually found brokers.
> >> >
> >> > One blocker is we found that some machines don't support AVX2 which
> >> will be
> >> > broken by the previous change in IMPALA-9690. We finally added a new
> >> > flag enable_legacy_avx_support to support machines that only support
> AVX
> >> > but not AVX2. Thank Joe McDonnell for working on this!
> >> >
> >> > Another blocker is a correctness bug:
> >> > https://issues.apache.org/jira/browse/IMPALA-10755
> >> > Aman Sinha is currently working on it. Thank Aman!
> >> >
> >> > There is also a bug fix that I'd like to take in 4.0.0:
> >> > https://gerrit.cloudera.org/c/17604/ (IMPALA-9763). Hopefully, after
> >> these
> >> > two issues are resolved, we can start the RC vote.
> >> >
> >> > On the other hand, I'm thinking about how to improve the readability
> of
> >> our
> >> > release notes. Previously we just group all the JIRAs into different
> >> types.
> >> > But this release has >700 JIRAs. It'd be better to have something like
> >> Kudu
> >> > does: https://kudu.apache.org/docs/release_notes.html
> >> > This would require lots of work. I'm going to create a google doc to
> >> > consolidate the major changes and invite contributors to edit it
> >> together.
> >> > Please let me know if you have any suggestions.
> >> >
> >> > Thanks,
> >> > Quanlong
> >> >
> >>
> >
>

Anybody watching the Aarch64 job?

[Jim Apple]

Hello, all! The Aarch64 job has been failing for weeks with an apparent
problem with the node (not Impala, per se). Does anyone here have the
credential to get this working again?

Thanks!
Jim

Re: [VOTE] 4.0.0 release candidate 7

[Jim Apple]

+1 (binding)

On Tue, Jul 6, 2021 at 5:34 AM Zoltán Borók-Nagy 
wrote:

> So after using 113c0c0f59a3e260884c18cbc5957af41ddffabd as the git tree
> hash my vote is:
>  +1 (binding)
>
> I executed https://jenkins.impala.io/job/release-test-ub1604/46/
>
> I also did the followings manually:
>
>- Downloaded all files from the RC7 download link
>- Manually verified sha512 sum, and asc signature
>- Ran ./buildall.sh -skiptests -notests
>- Started Impala cluster, ran some basic queries
>- Verified version string with SELECT version();
>
> Cheers,
> Zoltan
>
> On Tue, Jul 6, 2021 at 2:24 AM Quanlong Huang 
> wrote:
>
> > Oops, I thought I should use the tree hash of a "real" commit. Thanks for
> > the clarification!
> >
> > On Mon, Jul 5, 2021 at 9:34 PM Zoltán Borók-Nagy 
> > wrote:
> >
> > > Hi Quanlong,
> > >
> > > Thanks for doing all of this.
> > > I executed
> https://jenkins.impala.io/job/release-test-ub1604/45/console
> > > but
> > > it failed when it checked the git tree hash.
> > >
> > > Seems like b51f271f16e7a9c51a7652dc11009bb4193de0fb is the tree hash
> for
> > > the current 'apache/branch-4.0.0 (a702d2d4)'.
> > >
> > > Tag 4.0.0-rc7 is ahead of branch-4.0.0 with one commit (the one that
> > > modified save-version.sh).
> > >
> > > So I think the tree-hash for 4.0.0-rc7 should
> > > be 113c0c0f59a3e260884c18cbc5957af41ddffabd
> > >
> > > Cheers,
> > > Zoltan
> > >
> > >
> > > On Mon, Jul 5, 2021 at 5:24 AM Jim Apple  wrote:
> > >
> > > > Nevermind; found it. I needed to do git fetch -t, not my usual git
> > > > fetch. :-)
> > > >
> > > > On Sun, Jul 4, 2021 at 7:04 PM Jim Apple  wrote:
> > > >
> > > > > I don't see the 4.0.0-rc7 tag in git. Is that just me?
> > > > >
> > > > > On Sun, Jul 4, 2021 at 6:27 PM Quanlong Huang <
> > huangquanl...@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > >> Hi all,
> > > > >>
> > > > >> Finally, this is a vote for Impala 4.0.0! It's the 7th candidate
> > since
> > > > >> previous candidates were blocked for different reasons.
> > > > >>
> > > > >> The artifacts for testing can be downloaded from:
> > > > >> https://dist.apache.org/repos/dist/dev/impala/4.0.0/RC7/
> > > > >> Git tag: 4.0.0-rc7
> > > > >> https://github.com/apache/impala/releases/tag/4.0.0-rc7
> > > > >> Tree hash: b51f271f16e7a9c51a7652dc11009bb4193de0fb
> > > > >>
> > > > >> Please vote +1 or -1. -1 votes should be accompanied by an
> > explanation
> > > > of
> > > > >> the reason. Only PMC members have binding votes, but other
> > > > >> community members are encouraged to cast non-binding votes. This
> > vote
> > > > will
> > > > >> pass if there are 3 binding +1 votes and more binding +1 votes
> than
> > -1
> > > > >> votes.
> > > > >>
> > > > >> This wiki page describes how to check the release before you vote:
> > > > >>
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release#HowtoRelease-HowtoVoteonaReleaseCandidate
> > > > >>
> > > > >> I tested this by running exhaustive jobs for both with/without
> > > > >> LocalCatalog
> > > > >> mode, i.e. ubuntu-16.04-from-scratch and
> > > ubuntu-16.04-dockerised-tests.
> > > > >> (Unfortunately, they have been flushed out from the Jenkins
> history
> > > so I
> > > > >> can't paste the links here)
> > > > >>
> > > > >> Thanks,
> > > > >> Quanlong
> > > > >>
> > > > >
> > > >
> > >
> >
>

Re: [VOTE] 4.0.0 release candidate 7

[Jim Apple]

Nevermind; found it. I needed to do git fetch -t, not my usual git
fetch. :-)

On Sun, Jul 4, 2021 at 7:04 PM Jim Apple  wrote:

> I don't see the 4.0.0-rc7 tag in git. Is that just me?
>
> On Sun, Jul 4, 2021 at 6:27 PM Quanlong Huang 
> wrote:
>
>> Hi all,
>>
>> Finally, this is a vote for Impala 4.0.0! It's the 7th candidate since
>> previous candidates were blocked for different reasons.
>>
>> The artifacts for testing can be downloaded from:
>> https://dist.apache.org/repos/dist/dev/impala/4.0.0/RC7/
>> Git tag: 4.0.0-rc7
>> https://github.com/apache/impala/releases/tag/4.0.0-rc7
>> Tree hash: b51f271f16e7a9c51a7652dc11009bb4193de0fb
>>
>> Please vote +1 or -1. -1 votes should be accompanied by an explanation of
>> the reason. Only PMC members have binding votes, but other
>> community members are encouraged to cast non-binding votes. This vote will
>> pass if there are 3 binding +1 votes and more binding +1 votes than -1
>> votes.
>>
>> This wiki page describes how to check the release before you vote:
>>
>> https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release#HowtoRelease-HowtoVoteonaReleaseCandidate
>>
>> I tested this by running exhaustive jobs for both with/without
>> LocalCatalog
>> mode, i.e. ubuntu-16.04-from-scratch and ubuntu-16.04-dockerised-tests.
>> (Unfortunately, they have been flushed out from the Jenkins history so I
>> can't paste the links here)
>>
>> Thanks,
>> Quanlong
>>
>

Re: [VOTE] 4.0.0 release candidate 7

[Jim Apple]

I don't see the 4.0.0-rc7 tag in git. Is that just me?

On Sun, Jul 4, 2021 at 6:27 PM Quanlong Huang 
wrote:

> Hi all,
>
> Finally, this is a vote for Impala 4.0.0! It's the 7th candidate since
> previous candidates were blocked for different reasons.
>
> The artifacts for testing can be downloaded from:
> https://dist.apache.org/repos/dist/dev/impala/4.0.0/RC7/
> Git tag: 4.0.0-rc7 https://github.com/apache/impala/releases/tag/4.0.0-rc7
> Tree hash: b51f271f16e7a9c51a7652dc11009bb4193de0fb
>
> Please vote +1 or -1. -1 votes should be accompanied by an explanation of
> the reason. Only PMC members have binding votes, but other
> community members are encouraged to cast non-binding votes. This vote will
> pass if there are 3 binding +1 votes and more binding +1 votes than -1
> votes.
>
> This wiki page describes how to check the release before you vote:
>
> https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release#HowtoRelease-HowtoVoteonaReleaseCandidate
>
> I tested this by running exhaustive jobs for both with/without LocalCatalog
> mode, i.e. ubuntu-16.04-from-scratch and ubuntu-16.04-dockerised-tests.
> (Unfortunately, they have been flushed out from the Jenkins history so I
> can't paste the links here)
>
> Thanks,
> Quanlong
>

Re: Impala 4.0.0 Status & Preparation on Release Note

[Jim Apple]

What's the latest on this? Those two bugs are fixed now, thankfully.

On Mon, Jun 21, 2021 at 12:28 AM Quanlong Huang 
wrote:

> Hi all,
>
> I'm still working on starting a RC vote for Impala 4.0.0. It's delayed
> unfortunately since we continually found brokers.
>
> One blocker is we found that some machines don't support AVX2 which will be
> broken by the previous change in IMPALA-9690. We finally added a new
> flag enable_legacy_avx_support to support machines that only support AVX
> but not AVX2. Thank Joe McDonnell for working on this!
>
> Another blocker is a correctness bug:
> https://issues.apache.org/jira/browse/IMPALA-10755
> Aman Sinha is currently working on it. Thank Aman!
>
> There is also a bug fix that I'd like to take in 4.0.0:
> https://gerrit.cloudera.org/c/17604/ (IMPALA-9763). Hopefully, after these
> two issues are resolved, we can start the RC vote.
>
> On the other hand, I'm thinking about how to improve the readability of our
> release notes. Previously we just group all the JIRAs into different types.
> But this release has >700 JIRAs. It'd be better to have something like Kudu
> does: https://kudu.apache.org/docs/release_notes.html
> This would require lots of work. I'm going to create a google doc to
> consolidate the major changes and invite contributors to edit it together.
> Please let me know if you have any suggestions.
>
> Thanks,
> Quanlong
>

Re: Impala 4 Breaking Changes

[Jim Apple]

Once we know the answer to that last question, I'd be interested in reading
more about your vision for what should go in 4.0, Quanlong --
i.e."everything committed on June 2 except patches X and Y".

On Thu, Jun 3, 2021 at 2:10 AM Quanlong Huang 
wrote:

> Sorry to be late here! I'm -1 for merging HEAD into 4.0. Because it will
> bump the CDP_BUILD_NUMBER again which increases the difficulty for 3.x
> users (who may still be using Hive 2.x or 3.x) to adopt Impala-4.0. There
> are two new commits in HEAD that bump the CDP_BUILD_NUMBER.
>
> We may have already depended on some Hive 4.0 APIs. But those features may
> not be needed by 3.x users. They can revert some changes to make Impala-4.0
> compatible with older versions of Hive.
> The lower GBN we have, the easier it is for them to upgrade.
>
> Does those good commits of HEAD depend on bumping the GBN? If not, we can
> just pick them to the 4.0 branch.
>
> Thanks,
> Quanlong
>
> On Wed, May 26, 2021 at 8:52 PM Zoltán Borók-Nagy 
> wrote:
>
> > There's been a handful of good commits since May 1, so +1 for merging
> HEAD
> > into 4.0.
> >
> > Cheers,
> > Zoltan
> >
> > On Wed, May 26, 2021 at 2:11 PM Jim Apple  wrote:
> >
> > > We're only waiting on one patch for the docs, but since I cut the 4.0
> > > branch on May 1,  a number of things have landed. Maybe it would be
> best
> > to
> > > merge HEAD into 4.0.0 once the Sentry doc change lands, thereby
> including
> > > everything we've talked about cherrypicking -- including 9690.
> > >
> > > Thoughts?
> > >
> > > On Wed, May 26, 2021 at 12:40 AM Quanlong Huang <
> huangquanl...@gmail.com
> > >
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > I see IMPALA-9690 is done but still missing in branch-4.0.0 and I
> think
> > > we
> > > > should pick it
> > > >
> > > >- c10e7c9 IMPALA-9690: Set minimum CPU requirement to AVX2 for
> > x86_64
> > > >
> > > > I'd be good to also carry on these fixes for query retry:
> > > >
> > > >    - 42684b4 IMPALA-10413: fix impalad crashes when canceling the
> > > retrying
> > > >query
> > > >- d111443 IMPALA-10704: Fix retried query id not being
> unregistered
> > > when
> > > >retry fails
> > > >
> > > > If you are ok, I can push them to branch-4.0.0.
> > > >
> > > > Thanks,
> > > > Quanlong
> > > >
> > > > On Sun, May 16, 2021 at 6:02 AM Jim Apple 
> wrote:
> > > >
> > > > > Sounds good!
> > > > >
> > > > > On Sat, May 15, 2021 at 1:29 PM Joe McDonnell <
> > > joemcdonn...@cloudera.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Jim,
> > > > > >
> > > > > > I talked with the developer for the Sentry docs bug this past
> week,
> > > and
> > > > > we
> > > > > > went through the list of changes needed for that. The developer
> is
> > > > hoping
> > > > > > to work on it next week.
> > > > > >
> > > > > > If there is any further delay, I'm open to options of working
> > around
> > > > it.
> > > > > >
> > > > > > Thanks,
> > > > > > Joe
> > > > > >
> > > > > > On Sat, May 15, 2021 at 12:43 PM Jim Apple 
> > > wrote:
> > > > > >
> > > > > > > This Sentry one I'm concerned about. The ticket has been open
> > for a
> > > > > year.
> > > > > > > We could do a 4.0 release without the docs, or we could maybe
> > put a
> > > > > note
> > > > > > on
> > > > > > > the docs that Sentry support is no longer available.
> > > > > > >
> > > > > > > Any other ideas on how to tackle this while the ticket waits
> for
> > a
> > > > > patch?
> > > > > > >
> > > > > > > On Thu, May 6, 2021 at 8:36 PM Joe McDonnell <
> > > > > joemcdonn...@cloudera.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > I agree that it is a good idea for us to get going on the 4.0
> > > > > release.
> > > > > > >

Re: Impala 4 Breaking Changes

[Jim Apple]

Oh, now I see this email against cutting a fresh branch. I'll digest this
and respond later.

On Thu, Jun 3, 2021 at 2:10 AM Quanlong Huang 
wrote:

> Sorry to be late here! I'm -1 for merging HEAD into 4.0. Because it will
> bump the CDP_BUILD_NUMBER again which increases the difficulty for 3.x
> users (who may still be using Hive 2.x or 3.x) to adopt Impala-4.0. There
> are two new commits in HEAD that bump the CDP_BUILD_NUMBER.
>
> We may have already depended on some Hive 4.0 APIs. But those features may
> not be needed by 3.x users. They can revert some changes to make Impala-4.0
> compatible with older versions of Hive.
> The lower GBN we have, the easier it is for them to upgrade.
>
> Does those good commits of HEAD depend on bumping the GBN? If not, we can
> just pick them to the 4.0 branch.
>
> Thanks,
> Quanlong
>
> On Wed, May 26, 2021 at 8:52 PM Zoltán Borók-Nagy 
> wrote:
>
> > There's been a handful of good commits since May 1, so +1 for merging
> HEAD
> > into 4.0.
> >
> > Cheers,
> > Zoltan
> >
> > On Wed, May 26, 2021 at 2:11 PM Jim Apple  wrote:
> >
> > > We're only waiting on one patch for the docs, but since I cut the 4.0
> > > branch on May 1,  a number of things have landed. Maybe it would be
> best
> > to
> > > merge HEAD into 4.0.0 once the Sentry doc change lands, thereby
> including
> > > everything we've talked about cherrypicking -- including 9690.
> > >
> > > Thoughts?
> > >
> > > On Wed, May 26, 2021 at 12:40 AM Quanlong Huang <
> huangquanl...@gmail.com
> > >
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > I see IMPALA-9690 is done but still missing in branch-4.0.0 and I
> think
> > > we
> > > > should pick it
> > > >
> > > >- c10e7c9 IMPALA-9690: Set minimum CPU requirement to AVX2 for
> > x86_64
> > > >
> > > > I'd be good to also carry on these fixes for query retry:
> > > >
> > > >- 42684b4 IMPALA-10413: fix impalad crashes when canceling the
> > > retrying
> > > >query
> > > >- d111443 IMPALA-10704: Fix retried query id not being
> unregistered
> > > when
> > > >retry fails
> > > >
> > > > If you are ok, I can push them to branch-4.0.0.
> > > >
> > > > Thanks,
> > > > Quanlong
> > > >
> > > > On Sun, May 16, 2021 at 6:02 AM Jim Apple 
> wrote:
> > > >
> > > > > Sounds good!
> > > > >
> > > > > On Sat, May 15, 2021 at 1:29 PM Joe McDonnell <
> > > joemcdonn...@cloudera.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Jim,
> > > > > >
> > > > > > I talked with the developer for the Sentry docs bug this past
> week,
> > > and
> > > > > we
> > > > > > went through the list of changes needed for that. The developer
> is
> > > > hoping
> > > > > > to work on it next week.
> > > > > >
> > > > > > If there is any further delay, I'm open to options of working
> > around
> > > > it.
> > > > > >
> > > > > > Thanks,
> > > > > > Joe
> > > > > >
> > > > > > On Sat, May 15, 2021 at 12:43 PM Jim Apple 
> > > wrote:
> > > > > >
> > > > > > > This Sentry one I'm concerned about. The ticket has been open
> > for a
> > > > > year.
> > > > > > > We could do a 4.0 release without the docs, or we could maybe
> > put a
> > > > > note
> > > > > > on
> > > > > > > the docs that Sentry support is no longer available.
> > > > > > >
> > > > > > > Any other ideas on how to tackle this while the ticket waits
> for
> > a
> > > > > patch?
> > > > > > >
> > > > > > > On Thu, May 6, 2021 at 8:36 PM Joe McDonnell <
> > > > > joemcdonn...@cloudera.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > I agree that it is a good idea for us to get going on the 4.0
> > > > > release.
> > > > > > > >
> > > > > > > > I'm working on a change to fix IMPALA-9690. I will post an
> > > updat

Re: Impala 4 Breaking Changes

[Jim Apple]

We're only waiting on one patch for the docs, but since I cut the 4.0
branch on May 1,  a number of things have landed. Maybe it would be best to
merge HEAD into 4.0.0 once the Sentry doc change lands, thereby including
everything we've talked about cherrypicking -- including 9690.

Thoughts?

On Wed, May 26, 2021 at 12:40 AM Quanlong Huang 
wrote:

> Hi all,
>
> I see IMPALA-9690 is done but still missing in branch-4.0.0 and I think we
> should pick it
>
>- c10e7c9 IMPALA-9690: Set minimum CPU requirement to AVX2 for x86_64
>
> I'd be good to also carry on these fixes for query retry:
>
>- 42684b4 IMPALA-10413: fix impalad crashes when canceling the retrying
>query
>- d111443 IMPALA-10704: Fix retried query id not being unregistered when
>retry fails
>
> If you are ok, I can push them to branch-4.0.0.
>
> Thanks,
> Quanlong
>
> On Sun, May 16, 2021 at 6:02 AM Jim Apple  wrote:
>
> > Sounds good!
> >
> > On Sat, May 15, 2021 at 1:29 PM Joe McDonnell  >
> > wrote:
> >
> > > Hi Jim,
> > >
> > > I talked with the developer for the Sentry docs bug this past week, and
> > we
> > > went through the list of changes needed for that. The developer is
> hoping
> > > to work on it next week.
> > >
> > > If there is any further delay, I'm open to options of working around
> it.
> > >
> > > Thanks,
> > > Joe
> > >
> > > On Sat, May 15, 2021 at 12:43 PM Jim Apple  wrote:
> > >
> > > > This Sentry one I'm concerned about. The ticket has been open for a
> > year.
> > > > We could do a 4.0 release without the docs, or we could maybe put a
> > note
> > > on
> > > > the docs that Sentry support is no longer available.
> > > >
> > > > Any other ideas on how to tackle this while the ticket waits for a
> > patch?
> > > >
> > > > On Thu, May 6, 2021 at 8:36 PM Joe McDonnell <
> > joemcdonn...@cloudera.com>
> > > > wrote:
> > > >
> > > > > I agree that it is a good idea for us to get going on the 4.0
> > release.
> > > > >
> > > > > I'm working on a change to fix IMPALA-9690. I will post an updated
> > > review
> > > > > once a test run completes. If everything is looking good, I would
> > like
> > > to
> > > > > get this in the 4.0 release.
> > > > >
> > > > > There is also one documentation fix that I think belongs in the
> > Impala
> > > > 4.0
> > > > > release:
> > > > > https://issues.apache.org/jira/browse/IMPALA-9770 - Remove Sentry
> > > > > references
> > > > >
> > > > > Thanks,
> > > > > Joe
> > > > >
> > > > > On Thu, May 6, 2021 at 7:55 PM Jim Apple 
> wrote:
> > > > >
> > > > > > Sure thing! Let us know if you find owners for anything that
> > someone
> > > > > wants
> > > > > > to champion as a 4.0 blocker.
> > > > > >
> > > > > > On Thu, May 6, 2021 at 7:08 PM Quanlong Huang <
> > > huangquanl...@gmail.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Thank Jim! I planned to go through all the JIRAs target at 4.0:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/IMPALA-10690?jql=project%20%3D%20IMPALA%20AND%20resolution%20%3D%20Unresolved%20AND%20%22Target%20Version%22%20%3D%20%22Impala%204.0%22
> > > > > > >
> > > > > > > But due to some other urgent works and the recent holiday, my
> > > > progress
> > > > > is
> > > > > > > slow. Anyway, I think it's time to branch out. We've been
> waiting
> > > too
> > > > > > long.
> > > > > > > Thanks for creating the branch.
> > > > > > >
> > > > > > > Regards,
> > > > > > > Quanlong
> > > > > > >
> > > > > > > On Fri, May 7, 2021 at 3:29 AM Jim Apple 
> > > wrote:
> > > > > > >
> > > > > > > > I've created a 4.0 branch (with all changes through the end
> of
> > > > April)
> > > > > > and
> > > > > > > > star

Re: Impala 4 Breaking Changes

[Jim Apple]

Sounds good!

On Sat, May 15, 2021 at 1:29 PM Joe McDonnell 
wrote:

> Hi Jim,
>
> I talked with the developer for the Sentry docs bug this past week, and we
> went through the list of changes needed for that. The developer is hoping
> to work on it next week.
>
> If there is any further delay, I'm open to options of working around it.
>
> Thanks,
> Joe
>
> On Sat, May 15, 2021 at 12:43 PM Jim Apple  wrote:
>
> > This Sentry one I'm concerned about. The ticket has been open for a year.
> > We could do a 4.0 release without the docs, or we could maybe put a note
> on
> > the docs that Sentry support is no longer available.
> >
> > Any other ideas on how to tackle this while the ticket waits for a patch?
> >
> > On Thu, May 6, 2021 at 8:36 PM Joe McDonnell 
> > wrote:
> >
> > > I agree that it is a good idea for us to get going on the 4.0 release.
> > >
> > > I'm working on a change to fix IMPALA-9690. I will post an updated
> review
> > > once a test run completes. If everything is looking good, I would like
> to
> > > get this in the 4.0 release.
> > >
> > > There is also one documentation fix that I think belongs in the Impala
> > 4.0
> > > release:
> > > https://issues.apache.org/jira/browse/IMPALA-9770 - Remove Sentry
> > > references
> > >
> > > Thanks,
> > > Joe
> > >
> > > On Thu, May 6, 2021 at 7:55 PM Jim Apple  wrote:
> > >
> > > > Sure thing! Let us know if you find owners for anything that someone
> > > wants
> > > > to champion as a 4.0 blocker.
> > > >
> > > > On Thu, May 6, 2021 at 7:08 PM Quanlong Huang <
> huangquanl...@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Thank Jim! I planned to go through all the JIRAs target at 4.0:
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/IMPALA-10690?jql=project%20%3D%20IMPALA%20AND%20resolution%20%3D%20Unresolved%20AND%20%22Target%20Version%22%20%3D%20%22Impala%204.0%22
> > > > >
> > > > > But due to some other urgent works and the recent holiday, my
> > progress
> > > is
> > > > > slow. Anyway, I think it's time to branch out. We've been waiting
> too
> > > > long.
> > > > > Thanks for creating the branch.
> > > > >
> > > > > Regards,
> > > > > Quanlong
> > > > >
> > > > > On Fri, May 7, 2021 at 3:29 AM Jim Apple 
> wrote:
> > > > >
> > > > > > I've created a 4.0 branch (with all changes through the end of
> > April)
> > > > and
> > > > > > started testing a release candidate. Everyone who wants to vote,
> > > > > >
> https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release
> > > is a
> > > > > > guide to both releasing and voting (at the bottom). I'll let you
> > all
> > > > know
> > > > > > when I have something to vote on.
> > > > > >
> > > > > > Thanks!
> > > > > > Jim
> > > > > >
> > > > > > On Fri, Apr 23, 2021 at 12:49 AM Csaba Ringhofer <
> > > > > csringho...@cloudera.com
> > > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > About  IMPALA-9690 (AVX support):
> > > > > > > My preferred solution would be to deprecate support for x64
> > without
> > > > > AVX2
> > > > > > in
> > > > > > > 4.0, but not start removing the related logic yet.
> > > > > > > - We could even add a DCHECK + flag to crash by default if no
> > AVX2
> > > is
> > > > > > > detected, and a message that points them to Impala mailing
> > > > > lists/related
> > > > > > > Jira.
> > > > > > > - After 4.0 is released people would be free to implement
> things
> > > that
> > > > > > > assume AVX2 support on x64, for example compiling Impala/the
> > whole
> > > > > > > toolchain with -mavx2. I think that currently people are held
> > back
> > > > from
> > > > > > > doing this - why put effort in doing+benchmarking a change if
> you
> > > > can't
> > > > > > > push it to master.
> > > >

Re: Impala 4 Breaking Changes

[Jim Apple]

This Sentry one I'm concerned about. The ticket has been open for a year.
We could do a 4.0 release without the docs, or we could maybe put a note on
the docs that Sentry support is no longer available.

Any other ideas on how to tackle this while the ticket waits for a patch?

On Thu, May 6, 2021 at 8:36 PM Joe McDonnell 
wrote:

> I agree that it is a good idea for us to get going on the 4.0 release.
>
> I'm working on a change to fix IMPALA-9690. I will post an updated review
> once a test run completes. If everything is looking good, I would like to
> get this in the 4.0 release.
>
> There is also one documentation fix that I think belongs in the Impala 4.0
> release:
> https://issues.apache.org/jira/browse/IMPALA-9770 - Remove Sentry
> references
>
> Thanks,
> Joe
>
> On Thu, May 6, 2021 at 7:55 PM Jim Apple  wrote:
>
> > Sure thing! Let us know if you find owners for anything that someone
> wants
> > to champion as a 4.0 blocker.
> >
> > On Thu, May 6, 2021 at 7:08 PM Quanlong Huang 
> > wrote:
> >
> > > Thank Jim! I planned to go through all the JIRAs target at 4.0:
> > >
> > >
> >
> https://issues.apache.org/jira/browse/IMPALA-10690?jql=project%20%3D%20IMPALA%20AND%20resolution%20%3D%20Unresolved%20AND%20%22Target%20Version%22%20%3D%20%22Impala%204.0%22
> > >
> > > But due to some other urgent works and the recent holiday, my progress
> is
> > > slow. Anyway, I think it's time to branch out. We've been waiting too
> > long.
> > > Thanks for creating the branch.
> > >
> > > Regards,
> > > Quanlong
> > >
> > > On Fri, May 7, 2021 at 3:29 AM Jim Apple  wrote:
> > >
> > > > I've created a 4.0 branch (with all changes through the end of April)
> > and
> > > > started testing a release candidate. Everyone who wants to vote,
> > > > https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release
> is a
> > > > guide to both releasing and voting (at the bottom). I'll let you all
> > know
> > > > when I have something to vote on.
> > > >
> > > > Thanks!
> > > > Jim
> > > >
> > > > On Fri, Apr 23, 2021 at 12:49 AM Csaba Ringhofer <
> > > csringho...@cloudera.com
> > > > >
> > > > wrote:
> > > >
> > > > > About  IMPALA-9690 (AVX support):
> > > > > My preferred solution would be to deprecate support for x64 without
> > > AVX2
> > > > in
> > > > > 4.0, but not start removing the related logic yet.
> > > > > - We could even add a DCHECK + flag to crash by default if no AVX2
> is
> > > > > detected, and a message that points them to Impala mailing
> > > lists/related
> > > > > Jira.
> > > > > - After 4.0 is released people would be free to implement things
> that
> > > > > assume AVX2 support on x64, for example compiling Impala/the whole
> > > > > toolchain with -mavx2. I think that currently people are held back
> > from
> > > > > doing this - why put effort in doing+benchmarking a change if you
> > can't
> > > > > push it to master.
> > > > > - If a user appears with a valid use case that needs x64 without
> AVX2
> > > > then
> > > > > we can revise our plans, but we shouldn't ignore low hanging fruits
> > for
> > > > the
> > > > > sake of imaginary users :)
> > > > >
> > > > > -Csaba
> > > > >
> > > > > On Sun, Apr 11, 2021 at 5:52 PM Shant Hovsepian <
> > > > sh...@superdupershant.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > +1
> > > > > >
> > > > > > Regarding IMPALA-9690 if someone has the time to clean up the old
> > > > > > conditional cpu logic it would be great, but if the ARM folks
> still
> > > > > > need some of that logic it's not critical enough to block, just a
> > > very
> > > > > > nice to have?
> > > > > >
> > > > > > -Shant
> > > > > >
> > > > > > On Sun, Apr 4, 2021 at 1:09 AM Tim Armstrong <
> > > > tim.g.armstr...@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > I agree that it would be good to get this done even if it's not
> > > > > perfect,
> > &g

Re: Impala 4 Breaking Changes

[Jim Apple]

On Fri, May 7, 2021 at 11:35 PM Csaba Ringhofer 
wrote:

> I also have a commit that I would like to be merged to 4.0:
>
> https://github.com/apache/impala/commit/603091ed772f3f82511fd8fec355fe9b0126933b
> It is a critical fix that is needed wherever the following commit also
> exists:
>
> https://github.com/apache/impala/commit/c65d7861d9ae28f6fc592727ff699a8155dcda2c


SG!

2021-05 board report

[Jim Apple]

Hello all! Here is the report I have prepared for the ASF board for this
quarter:

## Description:
The mission of Apache Impala is the creation and maintenance of software
related to a high-performance distributed SQL engine

## Issues:

No issues require board attention at this time.

## Membership Data:
Apache Impala was founded 2017-11-14 (3 years ago)
There are currently 56 committers and 33 PMC members in this project.
The Committer-to-PMC ratio is roughly 7:5.

Community changes, past quarter:
- No new PMC members. Last addition was Csaba Ringhofer on 2020-02-18.
- No new committers. Last addition was Abhishek Rawat on 2020-12-08.

## Project Activity: During February, March, and April, the Impala
community:

 * Upgrad dependencies: DataSketches, thrift, Impyla, Bzip2, LZ4,
   Snappy, Zlib, ZStd, urllib3, python requests, Paramiko,
   springframework, JacksonDatabing, and slf4j
 * Added improvements to compatibility with ABFS, RHEL 8, Iceberg, S3,
   Ubuntu 20.04, Ranger, Kudu, Calcite, Google Cloud Storage, UTF-8,
   Hive, ORC, and docker hub
 * Addressed reliability for failed nodes and teardowns
 * Added result spooling
 * De-flaked many tests
 * Added most components needed for supporting external frontends
 * Added support for spilling to S3

## Community Health:

The community is overall healthy. This quarter has a common amount of
variability in some previous metrics. It is not infrequent that this
variability has no plainly obvious cause.

 * 157 patches were committed this quarter, vs. 153 the previous
   quarter
 * 212 tickets were opened, up 24%, and 152 tickets were closed, down 67%
 * reviews@ traffic was up 33% to 3288 emails

Re: Impala 4 Breaking Changes

[Jim Apple]

Done!

On Fri, May 7, 2021 at 1:42 AM Zoltán Borók-Nagy 
wrote:

> Hey Jim,
>
> My fix for IMPALA-10482 and IMPALA-10493 (the two issues are fixed in a
> single commit) just got merged after the 4.0 branching:
>
> https://gitbox.apache.org/repos/asf?p=impala.git;a=commit;h=f0f083e45e2c77b1499fa6fa08ff8d9dc4a2785f
>
> Can we add this patch to the 4.0 release?
>
> Thanks,
> Zoltan
>
>
>
> On Fri, May 7, 2021 at 5:36 AM Joe McDonnell 
> wrote:
>
> > I agree that it is a good idea for us to get going on the 4.0 release.
> >
> > I'm working on a change to fix IMPALA-9690. I will post an updated review
> > once a test run completes. If everything is looking good, I would like to
> > get this in the 4.0 release.
> >
> > There is also one documentation fix that I think belongs in the Impala
> 4.0
> > release:
> > https://issues.apache.org/jira/browse/IMPALA-9770 - Remove Sentry
> > references
> >
> > Thanks,
> > Joe
> >
> > On Thu, May 6, 2021 at 7:55 PM Jim Apple  wrote:
> >
> > > Sure thing! Let us know if you find owners for anything that someone
> > wants
> > > to champion as a 4.0 blocker.
> > >
> > > On Thu, May 6, 2021 at 7:08 PM Quanlong Huang  >
> > > wrote:
> > >
> > > > Thank Jim! I planned to go through all the JIRAs target at 4.0:
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/IMPALA-10690?jql=project%20%3D%20IMPALA%20AND%20resolution%20%3D%20Unresolved%20AND%20%22Target%20Version%22%20%3D%20%22Impala%204.0%22
> > > >
> > > > But due to some other urgent works and the recent holiday, my
> progress
> > is
> > > > slow. Anyway, I think it's time to branch out. We've been waiting too
> > > long.
> > > > Thanks for creating the branch.
> > > >
> > > > Regards,
> > > > Quanlong
> > > >
> > > > On Fri, May 7, 2021 at 3:29 AM Jim Apple  wrote:
> > > >
> > > > > I've created a 4.0 branch (with all changes through the end of
> April)
> > > and
> > > > > started testing a release candidate. Everyone who wants to vote,
> > > > > https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release
> > is a
> > > > > guide to both releasing and voting (at the bottom). I'll let you
> all
> > > know
> > > > > when I have something to vote on.
> > > > >
> > > > > Thanks!
> > > > > Jim
> > > > >
> > > > > On Fri, Apr 23, 2021 at 12:49 AM Csaba Ringhofer <
> > > > csringho...@cloudera.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > About  IMPALA-9690 (AVX support):
> > > > > > My preferred solution would be to deprecate support for x64
> without
> > > > AVX2
> > > > > in
> > > > > > 4.0, but not start removing the related logic yet.
> > > > > > - We could even add a DCHECK + flag to crash by default if no
> AVX2
> > is
> > > > > > detected, and a message that points them to Impala mailing
> > > > lists/related
> > > > > > Jira.
> > > > > > - After 4.0 is released people would be free to implement things
> > that
> > > > > > assume AVX2 support on x64, for example compiling Impala/the
> whole
> > > > > > toolchain with -mavx2. I think that currently people are held
> back
> > > from
> > > > > > doing this - why put effort in doing+benchmarking a change if you
> > > can't
> > > > > > push it to master.
> > > > > > - If a user appears with a valid use case that needs x64 without
> > AVX2
> > > > > then
> > > > > > we can revise our plans, but we shouldn't ignore low hanging
> fruits
> > > for
> > > > > the
> > > > > > sake of imaginary users :)
> > > > > >
> > > > > > -Csaba
> > > > > >
> > > > > > On Sun, Apr 11, 2021 at 5:52 PM Shant Hovsepian <
> > > > > sh...@superdupershant.com
> > > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > +1
> > > > > > >
> > > > > > > Regarding IMPALA-9690 if someone has the time to clean up the
> old
> > > > > > >

Re: Impala 4 Breaking Changes

[Jim Apple]

Sure thing! Let us know if you find owners for anything that someone wants
to champion as a 4.0 blocker.

On Thu, May 6, 2021 at 7:08 PM Quanlong Huang 
wrote:

> Thank Jim! I planned to go through all the JIRAs target at 4.0:
>
> https://issues.apache.org/jira/browse/IMPALA-10690?jql=project%20%3D%20IMPALA%20AND%20resolution%20%3D%20Unresolved%20AND%20%22Target%20Version%22%20%3D%20%22Impala%204.0%22
>
> But due to some other urgent works and the recent holiday, my progress is
> slow. Anyway, I think it's time to branch out. We've been waiting too long.
> Thanks for creating the branch.
>
> Regards,
> Quanlong
>
> On Fri, May 7, 2021 at 3:29 AM Jim Apple  wrote:
>
> > I've created a 4.0 branch (with all changes through the end of April) and
> > started testing a release candidate. Everyone who wants to vote,
> > https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release is a
> > guide to both releasing and voting (at the bottom). I'll let you all know
> > when I have something to vote on.
> >
> > Thanks!
> > Jim
> >
> > On Fri, Apr 23, 2021 at 12:49 AM Csaba Ringhofer <
> csringho...@cloudera.com
> > >
> > wrote:
> >
> > > About  IMPALA-9690 (AVX support):
> > > My preferred solution would be to deprecate support for x64 without
> AVX2
> > in
> > > 4.0, but not start removing the related logic yet.
> > > - We could even add a DCHECK + flag to crash by default if no AVX2 is
> > > detected, and a message that points them to Impala mailing
> lists/related
> > > Jira.
> > > - After 4.0 is released people would be free to implement things that
> > > assume AVX2 support on x64, for example compiling Impala/the whole
> > > toolchain with -mavx2. I think that currently people are held back from
> > > doing this - why put effort in doing+benchmarking a change if you can't
> > > push it to master.
> > > - If a user appears with a valid use case that needs x64 without AVX2
> > then
> > > we can revise our plans, but we shouldn't ignore low hanging fruits for
> > the
> > > sake of imaginary users :)
> > >
> > > -Csaba
> > >
> > > On Sun, Apr 11, 2021 at 5:52 PM Shant Hovsepian <
> > sh...@superdupershant.com
> > > >
> > > wrote:
> > >
> > > > +1
> > > >
> > > > Regarding IMPALA-9690 if someone has the time to clean up the old
> > > > conditional cpu logic it would be great, but if the ARM folks still
> > > > need some of that logic it's not critical enough to block, just a
> very
> > > > nice to have?
> > > >
> > > > -Shant
> > > >
> > > > On Sun, Apr 4, 2021 at 1:09 AM Tim Armstrong <
> > tim.g.armstr...@gmail.com>
> > > > wrote:
> > > > >
> > > > > I agree that it would be good to get this done even if it's not
> > > perfect,
> > > > > there's so much good stuff.
> > > > >
> > > > > On Fri, 2 Apr 2021 at 17:40, Joe McDonnell <
> > joemcdonn...@cloudera.com>
> > > > > wrote:
> > > > >
> > > > > > I agree that we should wrap up Impala 4. A large amount of good
> > work
> > > > has
> > > > > > gone in and it belongs in a release.
> > > > > >
> > > > > > We need to decide which breaking changes are truly blockers for
> an
> > > > Impala 4
> > > > > > release. My feeling is that we can't hold the release for
> > > compatibility
> > > > > > breaking changes unless someone signs up to do those changes.
> > > Breaking
> > > > > > compatibility is useful sometimes, but it can't be an indefinite
> > hold
> > > > on
> > > > > > releases. I think a release discussion thread is a good way to
> > > > stimulate
> > > > > > this discussion.
> > > > > >
> > > > > > Apart from the two JIRAs you listed, one other change mentioned
> in
> > > the
> > > > > > original email is switching to use the new on-demand metadata by
> > > > default.
> > > > > >
> > > > > > Thanks,
> > > > > > Joe
> > > > > >
> > > > > > On Fri, Apr 2, 2021 at 5:22 PM Quanlong Huang <
> > > huangquanl...@gmail.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > &

Re: Impala 4 Breaking Changes

[Jim Apple]

I've created a 4.0 branch (with all changes through the end of April) and
started testing a release candidate. Everyone who wants to vote,
https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release is a
guide to both releasing and voting (at the bottom). I'll let you all know
when I have something to vote on.

Thanks!
Jim

On Fri, Apr 23, 2021 at 12:49 AM Csaba Ringhofer 
wrote:

> About  IMPALA-9690 (AVX support):
> My preferred solution would be to deprecate support for x64 without AVX2 in
> 4.0, but not start removing the related logic yet.
> - We could even add a DCHECK + flag to crash by default if no AVX2 is
> detected, and a message that points them to Impala mailing lists/related
> Jira.
> - After 4.0 is released people would be free to implement things that
> assume AVX2 support on x64, for example compiling Impala/the whole
> toolchain with -mavx2. I think that currently people are held back from
> doing this - why put effort in doing+benchmarking a change if you can't
> push it to master.
> - If a user appears with a valid use case that needs x64 without AVX2 then
> we can revise our plans, but we shouldn't ignore low hanging fruits for the
> sake of imaginary users :)
>
> -Csaba
>
> On Sun, Apr 11, 2021 at 5:52 PM Shant Hovsepian  >
> wrote:
>
> > +1
> >
> > Regarding IMPALA-9690 if someone has the time to clean up the old
> > conditional cpu logic it would be great, but if the ARM folks still
> > need some of that logic it's not critical enough to block, just a very
> > nice to have?
> >
> > -Shant
> >
> > On Sun, Apr 4, 2021 at 1:09 AM Tim Armstrong 
> > wrote:
> > >
> > > I agree that it would be good to get this done even if it's not
> perfect,
> > > there's so much good stuff.
> > >
> > > On Fri, 2 Apr 2021 at 17:40, Joe McDonnell 
> > > wrote:
> > >
> > > > I agree that we should wrap up Impala 4. A large amount of good work
> > has
> > > > gone in and it belongs in a release.
> > > >
> > > > We need to decide which breaking changes are truly blockers for an
> > Impala 4
> > > > release. My feeling is that we can't hold the release for
> compatibility
> > > > breaking changes unless someone signs up to do those changes.
> Breaking
> > > > compatibility is useful sometimes, but it can't be an indefinite hold
> > on
> > > > releases. I think a release discussion thread is a good way to
> > stimulate
> > > > this discussion.
> > > >
> > > > Apart from the two JIRAs you listed, one other change mentioned in
> the
> > > > original email is switching to use the new on-demand metadata by
> > default.
> > > >
> > > > Thanks,
> > > > Joe
> > > >
> > > > On Fri, Apr 2, 2021 at 5:22 PM Quanlong Huang <
> huangquanl...@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Sure. If there are no objections, I'll raise a discussion thread
> for
> > the
> > > > > 4.0 release. There are still some unresolved breaking changes, e.g.
> > > > >
> > > > >- IMPALA-2210: Make Parquet the default file format
> > > > >- IMPALA-9690: Bump minimum x86-64 CPU requirements
> > > > >
> > > > > Thanks
> > > > > Quanlong
> > > > >
> > > > > On Tue, Mar 30, 2021 at 11:13 PM Jim Apple 
> > wrote:
> > > > >
> > > > > > Thanks for bringing this up again, Quanlong! I would love to see
> a
> > 4.0
> > > > > > release soon so that 4.1 releases and what not can be prepared.
> > Already
> > > > > the
> > > > > > changlist from the 3.x line is going to be enormous and could be
> > hard
> > > > for
> > > > > > users to digest.
> > > > > >
> > > > > > On Mon, Mar 29, 2021 at 7:22 AM Quanlong Huang <
> > > > huangquanl...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Reviving this thread. I think it's time to create the 4.0.0
> > branch
> > > > and
> > > > > > > prepare for the release now. Any breaking change is landing and
> > we
> > > > > should
> > > > > > > wait?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Quanlong
> > > > > > >
> > > > > &

Design doc for external frontend work?

[Jim Apple]

Hi All! There is a large epic on the JIRA about adding support for external
frontends:

https://issues.apache.org/jira/browse/IMPALA-10513

Is there a design doc for this effort?

Re: Switching Jenkins to Ubuntu 20.04?

[Jim Apple]

Oh, I see now in the logs: the answer is yes, for arm builds, too :-D

On Mon, Apr 5, 2021 at 9:58 AM Jim Apple  wrote:

> Impala builds are still dependent on specific binary packages sourced from
>> Cloudera's CDP builds.
>>
>
> Is this true for arm builds, too?
>

Re: Switching Jenkins to Ubuntu 20.04?

[Jim Apple]

>
> Impala builds are still dependent on specific binary packages sourced from
> Cloudera's CDP builds.
>

Is this true for arm builds, too?

Re: [DISCUSS] 4.0.0 release

[Jim Apple]

Sounds great! One thing I didn't quite understand about your views on 7127,
the fetch-on-demand metadata. Are you saying we should wait on that until
4.1?

On Sun, Apr 4, 2021 at 2:06 AM Quanlong Huang 
wrote:

> Hi all,
>
> Impala-3.4.0 was released in April 2020. It has been a long time since the
> last release. One reason is that this is a major version release so we hope
> it can take all the necessary breaking changes [1]. Listing all the
> unresolved breaking changes here (I may miss some):
>
>- IMPALA-2210: Make Parquet the default file format
>- IMPALA-9690: Bump minimum x86-64 CPU requirements
>- IMPALA-7127: Use the fetch-on-demand metadata (LocalCatalog) mode by
>default
>- I think this one is not visible to end users so maybe we can skip it.
>
> However, we've got lots of amazing features/improvements, e.g.
>
>- Multi-threading
>- New SQL features to support all TPC-DS 99 queries (without manually
>rewriting)
>- Transparent query retry
>- Read support on full ACID ORC tables
>- Apache DataSketches integration
>- Apache Iceberg integration
>- arm64 support
>- ...
>
> I think it's time to release 4.0.0 to make them visible to our users. It'd
> be good if the breaking changes discussed in this thread [1] can land soon.
> Otherwise, I propose leaving them to the next major version release.
>
> Thus, I propose that we release 4.0.0 soon and I volunteer to be its
> release manager. Please speak up and let the community know your ideas,
> especially on the breaking changes. All feedback is welcome!
>
> PS This is not a VOTE thread but only a DISCUSS thread.
>
> Thanks,
> Quanlong
>
> [1]
>
> https://lists.apache.org/thread.html/r91b070c2ddfbdc00b316791611b23816991c68fb0afefd1e18c21c26%40%3Cuser.impala.apache.org%3E
>

Switching Jenkins to Ubuntu 20.04?

[Jim Apple]

Some (or all?) Jenkins pre-commit jobs are now running Ubuntu 16.04. This
is a Long Term Support release, but support ends this month.

What would it take to switch Jenkins jobs to 20.04? This would set up
Jenkins to be able to run for another four years until we need to upgrade
again.

Note I'm not talking about the Jenkins server itself, just the VMs the
jobs run on.

Re: Impala 4 Breaking Changes

[Jim Apple]

Thanks for bringing this up again, Quanlong! I would love to see a 4.0
release soon so that 4.1 releases and what not can be prepared. Already the
changlist from the 3.x line is going to be enormous and could be hard for
users to digest.

On Mon, Mar 29, 2021 at 7:22 AM Quanlong Huang 
wrote:

> Reviving this thread. I think it's time to create the 4.0.0 branch and
> prepare for the release now. Any breaking change is landing and we should
> wait?
>
> Thanks,
> Quanlong
>
> On Thu, Jun 18, 2020 at 5:47 AM Tim Armstrong 
> wrote:
>
> > Another thing that we're looking into is changing the encoded runtime
> > profile representation to be more efficient - see
> > https://issues.apache.org/jira/browse/IMPALA-9378. So that might be a
> > default we will try to change in Impala 4.0.
> >
> > On Fri, Apr 24, 2020 at 12:37 PM Tim Armstrong 
> > wrote:
> >
> > > An addendum to this - we're also considering whether to increase the
> > > minimum CPU version so that we can simplify development and focus more
> on
> > > optimising for the most common CPUs - see
> > > https://issues.apache.org/jira/browse/IMPALA-9690
> > >
> > > Running on commodity hardware is important and we want people to be
> able
> > > to develop on whatever machine they have access to. At most we're
> likely
> > > going to require AVX2 support, which has been supported by the vast
> > > majority of CPUs for the better part of a decade.
> > >
> > > On Tue, Apr 7, 2020 at 11:50 AM Joe McDonnell <
> joemcdonn...@cloudera.com
> > >
> > > wrote:
> > >
> > >> Impala is starting development on Impala 4.0. There are many exciting
> > >> projects underway including full support for Hive ACID tables,
> improved
> > >> multithreading support, and fault tolerance improvements, but Impala
> 4.0
> > >> also serves as a time to alter behavior and platform support. Here is
> > the
> > >> current list of notable breaking changes that are planned in Impala
> 4.0:
> > >>
> > >> Remove support for older operating systems:
> > >>
> > >> Maintaining support for older operating systems requires a
> considerable
> > >> amount of effort, especially with changing security requirements. Each
> > >> extra operating system consumes valuable effort and can conflict with
> > >> support for newer operating systems, so we have decided to remove
> > support
> > >> for some older operating systems. Specifically, we plan to drop
> support
> > for:
> > >>
> > >>-
> > >>
> > >>Centos 6
> > >>-
> > >>
> > >>Ubuntu 14
> > >>-
> > >>
> > >>Debian 8
> > >>
> > >> Each of these is approaching or past its end of life. Since Centos6
> was
> > >> the only operating system still needing Python 2.6 and Python 2.6 is
> > long
> > >> past its end of life, Python 2.6 will no longer be supported. In
> > addition,
> > >> Centos 7 support will be focused on newer versions such as 7.5 and
> > above.
> > >>
> > >> Remove support for Sentry:
> > >>
> > >> Over the past year and a half, Impala's Ranger authorization
> > >> functionality has achieved parity and surpassed the existing Sentry
> > >> authorization functionality. Impala's Sentry support requires ongoing
> > >> effort to test and maintain, including maintenance on Sentry itself to
> > >> address security updates. Given the diminished activity in the Sentry
> > >> community, there is no timeline for addressing these security updates.
> > >> Unless there is a maintainer for Sentry support, Impala plans to focus
> > its
> > >> efforts on Ranger for its authorization system and drop Sentry.
> > >>
> > >> Remove support for Impala-lzo:
> > >>
> > >> Impala-lzo provides code to allow Impala to read the LZO compressed
> > >> tables. LZO is GPL licensed, which is why this support is not included
> > >> directly. The Impala-lzo code interacts with internal Impala code at a
> > >> level that is error prone and intricate. Given the low adoption of LZO
> > and
> > >> the other compression options available, Impala plans to remove
> > Impala-lzo
> > >> support along with the low level interface it used.
> > >>
> > >> Deprecations:
> > >>
> > >> In addition, we also plan to deprecate several existing features so
> that
> > >> they can be removed in a future release. Here is a summary of the most
> > >> notable ones:
> > >>
> > >>-
> > >>
> > >>Impala will default to using on demand metadata management (i.e.
> the
> > >>local catalog implementation) as described in this documentation:
> > >>
> http://impala.apache.org/docs/build/html/topics/impala_metadata.html
> > >>The old metadata system is now deprecated and may be removed in a
> > >>future release.
> > >>-
> > >>
> > >>Impala is deprecating the Beeswax client protocol (i.e. clients
> that
> > >>connect via beeswax_port) in favor of the HiveServer2 client
> > protocol.
> > >>-
> > >>
> > >>Impala is deprecating the old decimal_v2=false behavior from Impala
> > >>2.x. The decimal_v2 query option has defaulted to true since Impala
> > 3.

Fwd: [NOTICE] New anti-spam filtering will be enabled for JIRA tonight

[Jim Apple]

Hello, all! I just got this heads-up from ASF infra noting that JIRA may
reject certain tickets or comments. You can let them know if you encounter
any problems via email or via the #asfinfra channel on Slack:
https://infra.apache.org/slack.html

-- Forwarded message -
From: Daniel Gruno 
Date: Wed, Mar 3, 2021 at 4:36 AM
Subject: [NOTICE] New anti-spam filtering will be enabled for JIRA tonight
To: Users 


Hi folks,
as part of an ongoing effort to block spammers on JIRA, we will be
enabling a brand new version of the old Aardvark system on JIRA.

Aardvark is a mix of honey pots, content matching and (somewhat) naïve
bayesian spam filtering that should catch the bulk of bad content
creators on JIRA.

While it's definitely going to catch some bad guys, it might also cause
some good ones to get blocked, so if your comments or new issues
suddenly cause a 403 response, please to let us know and we can
investigate in detail.

We expect to enable the new scanner at around 11pm UTC tonight, and will
run the test for 2-3 days to see whether it works as intended (and then
figure out if we'll make it permanent).

With regards,
Daniel on behalf of ASF Infra.

Re: Help with task: Rename single letter tables in FE tests

[Jim Apple]

https://helpwanted.apache.org/task.html?2da8576f

You've come to the right place. However, it's very hard to answer questions
like "I want ton help how do I get starteded". Did you try the
instructions? Where did you get stuck, specifically?

Thanks!
Jim

On Sun, Feb 14, 2021 at 11:45 PM I L  wrote:

> I would like to help out with the task listed at /task.html?2da8576f
>
>  I want ton help how do i get starteded
>

Fwd: Impala Board Report due by Wed Feb 10th - Initial Reminder

[Jim Apple]

Hello, all! Here is the report I just pushed. Please LMK if you'd like to
make any corrections or additions.

## Description:
The mission of Apache Impala is the creation and maintenance of software
related to a high-performance distributed SQL engine

## Issues:
There are no issues requiring board attention.

## Membership Data:
Apache Impala was founded 2017-11-14 (3 years ago)
There are currently 56 committers and 33 PMC members in this project.
The Committer-to-PMC ratio is roughly 7:5.

Community changes, past quarter:
- No new PMC members. Last addition was Csaba Ringhofer on 2020-02-18.
- Abhishek Rawat was added as committer on 2020-12-08

## Project Activity: During November, December, and January, the Impala
community added support (or improved support) for:

 - Codegen in the sorter

 - FIPS compliance

 - More sketches from Apache DataSketches

 - Cookie authentication in impala-shell

 - Numerous fixes for flaky tests, including many with timing requirements
that
   were too tight

 - More support for parallelism within a single node ("dop")

 - Role-related statements using Apache Ranger

 - Unicode

 - An admission control daemon

 - More integration with Apache Iceberg

## Community Health:

The community is overall healthy. This quarter has a common amount of
variability in some previous metrics. It is not infrequent that this
variability has no plainly obvious cause, though the US holiday season is
sometimes correlated with lower activity.

 - 2,576 reviews were sent to reviews@, 39% down from the previous
   quarter. This metric is the most notable change.

 - 170 new JIRA tickets were filed, 28% lower than the previous quarter.

 - 153 patches were committed this quarter, 15% down from last quarter.
There
   is a notable dip around Christmas, in which weekly commits increased
from 3
   to 22 within a week.

 - Notable increases in activity are visible in total JIRA traffic as well
as a
   125% increase in JIRAs closed.

-- Forwarded message -
From: Roy T. Fielding 
Date: Wed, Feb 3, 2021 at 6:16 PM
Subject: Impala Board Report due by Wed Feb 10th - Initial Reminder
To: Jim Apple 
Cc: 


Hello,

According to our records, you are listed as the chair of Impala,
a committee that is due to submit a report by Wed Feb 10th
for the next ASF board meeting. This is an initial reminder to prepare a
report for Impala and submit it as described below.

In the Apache governance model, the ASF board delegates responsibility for
managing projects to PMCs. This allows projects to govern themselves, in
terms of their own development goals, guidelines, and volunteer spirit,
within the scope of our purpose as an open source foundation. The state
allows us to provide an umbrella of corporate protection to our projects
and volunteers, but only to the extent that we retain active and effective
oversight of each project's operation on behalf of the public's interest.

To enable the board to provide oversight across the foundation, each PMC
is tasked with providing the board a quarterly report on the health of
their project. This allows us to hear your heartbeat, to see the project
through your eyes, and to inform the public through our meeting minutes.

Thank you for being a responsible project chair and helping us maintain
oversight over the Apache Software Foundation. If, for whatever reason,
a full report is not possible by the deadline, please report just that.
It's okay to postpone a report by a month.

Please note that we will be reviewing and approving these reports,
asynchronously, during the week prior to our scheduled board meeting.
Reports received after Wed Feb 10th will be postponed to the
next regular meeting. Please try to submit your report early.


Submitting your report
--

Full details about reporting to the board are at

  https://www.apache.org/foundation/board/reporting

Please be aware that the board is looking for your personal observations,
assessment, and ideas, not just raw statistics.

Chairs may use one of several mechanisms to submit or edit their report:

 a) the Apache Reporter Service
https://reporter.apache.org/

 b) the Whimsy online agenda tool
https://whimsy.apache.org/board/agenda/2021-02-17/Impala

 c) carefully editing and committing changes to the dated agenda in
https://svn.apache.org/repos/private/foundation/board

 d) or, if none of the above work, send an email to bo...@apache.org with
Subject: [REPORT] Impala

If you believe it won't be possible to prepare a report before the deadline,
or if the PMC is aware that the Chair is unavailable, please report that
and we can reschedule or have someone else report on your behalf.


Attending the Board Meeting
---

The formal board meeting (usually an online videoconference) will be held at

  Wed, 17 Feb 2021 at 20:00 UTC

which in other time zones is

  https://timeanddate.com/s/42ab

As always, chairs and ASF members are welcome

Re: [Black|White]List rename to [Ban|Allow]

[Jim Apple]

I support this idea. It's also more descriptive! 

On Mon, Nov 9, 2020 at 7:00 AM Shant Hovsepian 
wrote:

> Hi All,
>
> Any thoughts on more inclusive terminology for some of our concepts? We
> tend to use coordinator / worker as opposed to master / slave. There aren't
> too many places where we use the term blacklist, but there are a few
> pending features for faulty node and disk handling that might benefit from
> a more inclusive adoption of terminology.
>
> How about using ban or allow lists going forward? We can consider adding an
> alias for the flags blacklisted_dbs, blacklisted_tables, and
> blacklisting_enabled.
>
> -Shant
>

Fwd: Impala Board Report due by Wed Nov 11 - Initial Reminder

[Jim Apple]

Hello, Impalas! I have posted the below report for November. LMK if you'd
like to add anything. Thanks!

## Description:
The mission of Apache Impala is the creation and maintenance of software
related to a high-performance distributed SQL engine

## Issues:
There are no issues requiring board attention.

## Membership Data:
Apache Impala was founded 2017-11-14 (3 years ago)
There are currently 55 committers and 33 PMC members in this project.
The Committer-to-PMC ratio is 5:3.

Community changes, past quarter:
- No new PMC members. Last addition was Csaba Ringhofer on 2020-02-18.
- Aman Sinha was added as committer on 2020-09-08
- Shant Hovsepian was added as committer on 2020-10-13
- Sheng Wang was added as committer on 2020-11-06

## Project Activity: During August, September, and October, the Impala
community
added support (or improved support) for:

 - More Iceberg support, including ALTER TABLE, INSERT INTO for
(non-partitioned
   tables), ORC, and more

 - Movement towards FIPS compliance

 - Error message readability and location improvements

 - System internals visibility improvements into artifacts like like queues
and
   skews

 - Daily aarch64 build-and-test runs

 - Many more patches than a typical quarter about developer
   experience. Eyeballing it, maybe twice as much? This includes fixing some
   long-standing build and test issues.

 - Impala's first patches from contributors at @tencent.com

 - The addition of support for Alluxio

 - First SIMD support outside of the x86-64 family

## Community Health:

The community is overall healthy. This quarter has a common amount of
variability in some previous metrics. It is not infrequent that this
variability has no plainly obvious cause.

 - 4,278 reviews were sent to reviews@, 2% down from the previous quarter

 - 272 new JIRA tickets were filed vs. 315 last quarter

 - 184 patches were committed this quarter, 5% down from last quarter

 - user@ and saw traffic decrease (31 emails to 19), while dev@ saw it
increase
   (73 emails to 89)

-- Forwarded message -
From: Roy T. Fielding 
Date: Mon, Nov 2, 2020 at 5:59 PM
Subject: Impala Board Report due by Wed Nov 11 - Initial Reminder
To: Jim Apple 
Cc: 


Hello,

According to our records, you are listed as the chair of Impala,
a committee that is due to submit a report by Wed Nov 11
for the next ASF board meeting. This is an initial reminder to prepare a
report for Impala and submit it as described below.

In the Apache governance model, the ASF board delegates responsibility for
managing projects to PMCs. This allows projects to govern themselves, in
terms of their own development goals, guidelines, and volunteer spirit,
within the scope of our purpose as an open source foundation. The state
allows us to provide an umbrella of corporate protection to our projects
and volunteers, but only to the extent that we retain active and effective
oversight of each project's operation on behalf of the public's interest.

To enable the board to provide oversight across the foundation, each PMC
is tasked with providing the board a quarterly report on the health of
their project. This allows us to hear your heartbeat, to see the project
through your eyes, and to inform the public through our meeting minutes.

Thank you for being a responsible project chair and helping us maintain
oversight over the Apache Software Foundation. If, for whatever reason,
a full report is not possible by the deadline, please report just that.
It's okay to postpone a report by a month.

Please note that we will be reviewing and approving these reports,
asynchronously, during the week prior to our scheduled board meeting.
Reports received after Wed Nov 11 will be postponed to the
next regular meeting. Please try to submit your report early.


Submitting your report
--

Full details about reporting to the board are at

  https://www.apache.org/foundation/board/reporting

Please be aware that the board is looking for your personal observations,
assessment, and ideas, not just raw statistics.

Chairs may use one of several mechanisms to submit or edit their report:

 a) the Apache Reporter Service
https://reporter.apache.org/

 b) the Whimsy online agenda tool
https://whimsy.apache.org/board/agenda/2020-11-18/Impala

 c) carefully editing and committing changes to the dated agenda in
https://svn.apache.org/repos/private/foundation/board

 d) or, if none of the above work, send an email to bo...@apache.org with
Subject: [REPORT] Impala

If you believe it won't be possible to prepare a report before the deadline,
or if the PMC is aware that the Chair is unavailable, please report that
and we can reschedule or have someone else report on your behalf.


Attending the Board Meeting
---

The formal board meeting (usually an online videoconference) will be held at

  Wed, 18 Nov 2020 at 20:00 UTC

which in other time zones is

  https://timeanddate.com/s/42a2

Re: Discuss Impala ARM CI integration

[Jim Apple]

I did a test on EC2 Graviton 2, and I was able to start from a blank Ubuntu
18.04 image and get all the way to all tests passing. This *included
building the native-toolchain from scratch, with no manual input required*
- it just bootstrapped with no fuss! I used a r6g.2xlarge, which has a list
price of 40 cents per hour, though you have to pay for EBS, too. If we get
to a place where we want this to be in the gerrit merge step, I expect it
to be smooth sailing.

I'm really impressed with those of you who have worked on the pieces of
this to put it together! Thank you for your contributions to the community!

Here, roughly, are how one could duplicate my test:

   1. Launch an r6g.2xlarge using Ubuntu 18.04, Arm64, with a 500GB root
   volume. Leave the following ports open:
  - $YOUR_IP, udp 6-61000 for mosh, in both IPV4 and IPv6
  - $YOUR_IP, tcp 22 for ssh, in both IPV4 and IPv6
  - All TCP and UDP from itself, for loopback
   2. ssh in and
  - apt-get update
  - apt-get install tmux git mosh
   3. Terminate your ssh session and connect again, this time with mosh
   4. Start a tmux session
   5. git clone https://github.com/apache/impala
   6. cd impala
   7. export IMPALA_HOME=$(pwd)
   8. ./bin/bootstrap_development.sh && ./bin/run-all-tests.sh

I'd love to see a performance comparison between two different equal-cost
clusters on AWS: one Arm64, one x86-64, but I'll leave that to the testers
with larger budgets of both time and money! :-D

On Fri, Oct 2, 2020 at 11:05 AM Jim Apple  wrote:

> SG; submitted request for builds@
>
> Tim, I made you and I moderators. You're welcome to decline. :-)
> Volunteers are welcome, but I expect zero moderation work - I set the list
> to "subscribers can post; all others are rejected", so no moderation
> messages should show up.
>
> The form said it will be acted on in 24 hours and the PMC will be informed
> when the mailing list is ready.
>
> Also, FWIW, aside from Docker, AWS supports some ARM EC2 types.
>
> On Thu, Oct 1, 2020 at 4:22 PM Tim Armstrong 
> wrote:
>
>> I think we got stalled here. I never got anywhere with configuring
>> commits@
>> to accept emails from openlab and not sure that it's possible.
>>
>> In order to get something going, my suggestion is to copy what Kudu did
>> and
>> have this go to a custom mailing list. If you follow the thread here it
>> describes what they did
>>
>> https://lists.apache.org/thread.html/rfcd230882ec34171462a0c8c53ef6e8efcad50c9d67f8da18a508d07%40%3Cdev.kudu.apache.org%3E
>>
>> Jim, it looks like the PMC chair is needed to create a new mailing list
>> via
>> https://selfserve.apache.org/ (see
>> https://issues.apache.org/jira/browse/INFRA-20604), so maybe we can go
>> ahead with that route?
>>
>> - Tim
>>
>> I think it would be good to run the tests on impala.jenkins.io but I
>> don't
>> want to hold up things further. Another interesting technology is the ARM
>> emulation support in Docker.
>>
>> On Sun, Sep 20, 2020 at 8:53 PM Jim Apple  wrote:
>>
>> > I'm very excited about this development! I don't see any jobs at
>> > status.openlabtesting.org, though -- is
>> >
>> http://status.openlabtesting.org/builds?job_name=impala-build-test-arm64
>> > the right link? Maybe only the owner of a job can see it?
>> >
>> > As far as the price, based on https://ec2instances.info/, you might be
>> > able
>> > to eyeball the cost. I'm no longer a Jenkins admin, so I'll let the
>> other
>> > community members who are admins speak more to this, if it seems
>> > feasible to them.
>> >
>> > On Sat, Sep 19, 2020 at 2:31 AM Tianhua huang <
>> huangtianhua...@gmail.com>
>> > wrote:
>> >
>> > > I am sorry there is no API with Openlab. But we use ZUUL in our
>> system,
>> > you
>> > > can call ZUUL api to get the job information,
>> > > simple like:  curl
>> > >
>> > >
>> >
>> http://status.openlabtesting.org/api/builds?job_name=impala-build-test-arm64
>> > > | python -m json.tool
>> > > The result we will get is jobs info of 'impala-build-test-arm64',
>> > including
>> > > job status, job log url and so on, I am not sure if we can integrate
>> with
>> > > the job info into impala.jenkins.io? Then the arm job still runs
>> > triggered
>> > > by OpenLab, and jenkins.impala.io only to get the result and show
>> > > the result in it.
>> > >
>> > > Not sure it's worth doing this, maybe let comm...@impala.apache.org
>> > allow
>> &g

Re: Discuss Impala ARM CI integration

[Jim Apple]

OK, we're all set on the builds@ list. According to the confirmation email,
"it will appear on https://lists.apache.org/ within a few minutes of the
first post to it."

On Fri, Oct 2, 2020 at 5:38 PM Tim Armstrong 
wrote:

> Thanks Jim!
>
> On Fri, Oct 2, 2020 at 11:06 AM Jim Apple  wrote:
>
> > SG; submitted request for builds@
> >
> > Tim, I made you and I moderators. You're welcome to decline. :-)
> Volunteers
> > are welcome, but I expect zero moderation work - I set the list to
> > "subscribers can post; all others are rejected", so no moderation
> messages
> > should show up.
> >
> > The form said it will be acted on in 24 hours and the PMC will be
> informed
> > when the mailing list is ready.
> >
> > Also, FWIW, aside from Docker, AWS supports some ARM EC2 types.
> >
> > On Thu, Oct 1, 2020 at 4:22 PM Tim Armstrong 
> > wrote:
> >
> > > I think we got stalled here. I never got anywhere with configuring
> > commits@
> > > to accept emails from openlab and not sure that it's possible.
> > >
> > > In order to get something going, my suggestion is to copy what Kudu did
> > and
> > > have this go to a custom mailing list. If you follow the thread here it
> > > describes what they did
> > >
> > >
> >
> https://lists.apache.org/thread.html/rfcd230882ec34171462a0c8c53ef6e8efcad50c9d67f8da18a508d07%40%3Cdev.kudu.apache.org%3E
> > >
> > > Jim, it looks like the PMC chair is needed to create a new mailing list
> > via
> > > https://selfserve.apache.org/ (see
> > > https://issues.apache.org/jira/browse/INFRA-20604), so maybe we can go
> > > ahead with that route?
> > >
> > > - Tim
> > >
> > > I think it would be good to run the tests on impala.jenkins.io but I
> > don't
> > > want to hold up things further. Another interesting technology is the
> ARM
> > > emulation support in Docker.
> > >
> > > On Sun, Sep 20, 2020 at 8:53 PM Jim Apple  wrote:
> > >
> > > > I'm very excited about this development! I don't see any jobs at
> > > > status.openlabtesting.org, though -- is
> > > >
> > http://status.openlabtesting.org/builds?job_name=impala-build-test-arm64
> > > > the right link? Maybe only the owner of a job can see it?
> > > >
> > > > As far as the price, based on https://ec2instances.info/, you might
> be
> > > > able
> > > > to eyeball the cost. I'm no longer a Jenkins admin, so I'll let the
> > other
> > > > community members who are admins speak more to this, if it seems
> > > > feasible to them.
> > > >
> > > > On Sat, Sep 19, 2020 at 2:31 AM Tianhua huang <
> > huangtianhua...@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > I am sorry there is no API with Openlab. But we use ZUUL in our
> > system,
> > > > you
> > > > > can call ZUUL api to get the job information,
> > > > > simple like:  curl
> > > > >
> > > > >
> > > >
> > >
> >
> http://status.openlabtesting.org/api/builds?job_name=impala-build-test-arm64
> > > > > | python -m json.tool
> > > > > The result we will get is jobs info of 'impala-build-test-arm64',
> > > > including
> > > > > job status, job log url and so on, I am not sure if we can
> integrate
> > > with
> > > > > the job info into impala.jenkins.io? Then the arm job still runs
> > > > triggered
> > > > > by OpenLab, and jenkins.impala.io only to get the result and show
> > > > > the result in it.
> > > > >
> > > > > Not sure it's worth doing this, maybe let
> comm...@impala.apache.org
> > > > allow
> > > > > receiving the result email from Openlab is more easy:)
> > > > >
> > > > >
> > > > >
> > > > > On Sat, Sep 19, 2020 at 11:03 AM Shant Hovsepian <
> > > > > sh...@superdupershant.com>
> > > > > wrote:
> > > > >
> > > > > > Would it be possible to setup an API Key or webhook with
> > > > openlabstesting?
> > > > > >
> > > > > > Then we'd have an Impala Jenkins job that uses the API to call
> > > > > > openlabstesting to do the build and report the results back.
> > > > > >
> > > > > > -Sha

Re: Discuss Impala ARM CI integration

[Jim Apple]

SG; submitted request for builds@

Tim, I made you and I moderators. You're welcome to decline. :-) Volunteers
are welcome, but I expect zero moderation work - I set the list to
"subscribers can post; all others are rejected", so no moderation messages
should show up.

The form said it will be acted on in 24 hours and the PMC will be informed
when the mailing list is ready.

Also, FWIW, aside from Docker, AWS supports some ARM EC2 types.

On Thu, Oct 1, 2020 at 4:22 PM Tim Armstrong 
wrote:

> I think we got stalled here. I never got anywhere with configuring commits@
> to accept emails from openlab and not sure that it's possible.
>
> In order to get something going, my suggestion is to copy what Kudu did and
> have this go to a custom mailing list. If you follow the thread here it
> describes what they did
>
> https://lists.apache.org/thread.html/rfcd230882ec34171462a0c8c53ef6e8efcad50c9d67f8da18a508d07%40%3Cdev.kudu.apache.org%3E
>
> Jim, it looks like the PMC chair is needed to create a new mailing list via
> https://selfserve.apache.org/ (see
> https://issues.apache.org/jira/browse/INFRA-20604), so maybe we can go
> ahead with that route?
>
> - Tim
>
> I think it would be good to run the tests on impala.jenkins.io but I don't
> want to hold up things further. Another interesting technology is the ARM
> emulation support in Docker.
>
> On Sun, Sep 20, 2020 at 8:53 PM Jim Apple  wrote:
>
> > I'm very excited about this development! I don't see any jobs at
> > status.openlabtesting.org, though -- is
> > http://status.openlabtesting.org/builds?job_name=impala-build-test-arm64
> > the right link? Maybe only the owner of a job can see it?
> >
> > As far as the price, based on https://ec2instances.info/, you might be
> > able
> > to eyeball the cost. I'm no longer a Jenkins admin, so I'll let the other
> > community members who are admins speak more to this, if it seems
> > feasible to them.
> >
> > On Sat, Sep 19, 2020 at 2:31 AM Tianhua huang  >
> > wrote:
> >
> > > I am sorry there is no API with Openlab. But we use ZUUL in our system,
> > you
> > > can call ZUUL api to get the job information,
> > > simple like:  curl
> > >
> > >
> >
> http://status.openlabtesting.org/api/builds?job_name=impala-build-test-arm64
> > > | python -m json.tool
> > > The result we will get is jobs info of 'impala-build-test-arm64',
> > including
> > > job status, job log url and so on, I am not sure if we can integrate
> with
> > > the job info into impala.jenkins.io? Then the arm job still runs
> > triggered
> > > by OpenLab, and jenkins.impala.io only to get the result and show
> > > the result in it.
> > >
> > > Not sure it's worth doing this, maybe let comm...@impala.apache.org
> > allow
> > > receiving the result email from Openlab is more easy:)
> > >
> > >
> > >
> > > On Sat, Sep 19, 2020 at 11:03 AM Shant Hovsepian <
> > > sh...@superdupershant.com>
> > > wrote:
> > >
> > > > Would it be possible to setup an API Key or webhook with
> > openlabstesting?
> > > >
> > > > Then we'd have an Impala Jenkins job that uses the API to call
> > > > openlabstesting to do the build and report the results back.
> > > >
> > > > -Shant
> > > >
> > > > On Fri, Sep 18, 2020 at 10:24 PM Tianhua huang <
> > > huangtianhua...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Shant,
> > > > >
> > > > > Thanks for your reply, maybe it's a good idea to trigger arm
> testing
> > by
> > > > > jenkins.impala.io, we can donate an arm instance, but I don't know
> > how
> > > > to
> > > > > do it. Maybe the Jenkins owner of Impala can help me?
> > > > >
> > > > >
> > > > > On Sat, Sep 19, 2020 at 2:46 AM Shant Hovsepian <
> > > > sh...@superdupershant.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Great work Tianhua!
> > > > > >
> > > > > > This is a very exciting enhancement.
> > > > > >
> > > > > > dev@ isn't usually meant for automated test run output. Would it
> > be
> > > > > > possible to get the openlabtesting runs triggered by
> > > jenkins.impala.io
> > > > > > then
> > > > > > have the results of the test run come from that jenkins instance?
>

Re: Discuss Impala ARM CI integration

[Jim Apple]

I'm very excited about this development! I don't see any jobs at
status.openlabtesting.org, though -- is
http://status.openlabtesting.org/builds?job_name=impala-build-test-arm64
the right link? Maybe only the owner of a job can see it?

As far as the price, based on https://ec2instances.info/, you might be able
to eyeball the cost. I'm no longer a Jenkins admin, so I'll let the other
community members who are admins speak more to this, if it seems
feasible to them.

On Sat, Sep 19, 2020 at 2:31 AM Tianhua huang 
wrote:

> I am sorry there is no API with Openlab. But we use ZUUL in our system, you
> can call ZUUL api to get the job information,
> simple like:  curl
>
> http://status.openlabtesting.org/api/builds?job_name=impala-build-test-arm64
> | python -m json.tool
> The result we will get is jobs info of 'impala-build-test-arm64', including
> job status, job log url and so on, I am not sure if we can integrate with
> the job info into impala.jenkins.io? Then the arm job still runs triggered
> by OpenLab, and jenkins.impala.io only to get the result and show
> the result in it.
>
> Not sure it's worth doing this, maybe let comm...@impala.apache.org allow
> receiving the result email from Openlab is more easy:)
>
>
>
> On Sat, Sep 19, 2020 at 11:03 AM Shant Hovsepian <
> sh...@superdupershant.com>
> wrote:
>
> > Would it be possible to setup an API Key or webhook with openlabstesting?
> >
> > Then we'd have an Impala Jenkins job that uses the API to call
> > openlabstesting to do the build and report the results back.
> >
> > -Shant
> >
> > On Fri, Sep 18, 2020 at 10:24 PM Tianhua huang <
> huangtianhua...@gmail.com>
> > wrote:
> >
> > > Hi Shant,
> > >
> > > Thanks for your reply, maybe it's a good idea to trigger arm testing by
> > > jenkins.impala.io, we can donate an arm instance, but I don't know how
> > to
> > > do it. Maybe the Jenkins owner of Impala can help me?
> > >
> > >
> > > On Sat, Sep 19, 2020 at 2:46 AM Shant Hovsepian <
> > sh...@superdupershant.com
> > > >
> > > wrote:
> > >
> > > > Great work Tianhua!
> > > >
> > > > This is a very exciting enhancement.
> > > >
> > > > dev@ isn't usually meant for automated test run output. Would it be
> > > > possible to get the openlabtesting runs triggered by
> jenkins.impala.io
> > > > then
> > > > have the results of the test run come from that jenkins instance?
> This
> > > > should solve the email problem as the notification can come from the
> > > > jenkins account and still give us one centralized place to look at
> > > various
> > > > test statuses.
> > > >
> > > > -Shant
> > > >
> > > > On Thu, Sep 17, 2020 at 8:08 AM Tianhua huang <
> > huangtianhua...@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > Impala can run/test on arm64 platform after our efforts, the issues
> > we
> > > > > fixed you can find in
> > > https://issues.apache.org/jira/browse/IMPALA-9376
> > > > >
> > > > > We are particularly grateful to Tim and Quanlong for their help,
> > thanks
> > > > > very much!
> > > > >
> > > > > And I have integrated Impala arm64 CI with OpenLab(
> > > > > https://openlabtesting.org/ , a community to do open source
> project
> > > > > testing),
> > > > > see
> > > > >
> > >
> http://status.openlabtesting.org/builds?job_name=impala-build-test-arm64
> > > > > the
> > > > > periodic test job runs for a few days already and runs once per
> day.
> > > > >
> > > > > Tim suggested us to send the arm64 test result to
> > > > > comm...@impala.apache.org,
> > > > > but seems it can receive the emails only ncome from @apache.org.
> > > > > Now I wonder if we can send the test result to Impala community
> > > maillist
> > > > > here dev@impala.apache.org, to tell developers that Impala can
> runs
> > on
> > > > > arm64, to find the compatibility in time.
> > > > >
> > > > > What do you think? Or maybe there is any other good suggestion?
> > > > >
> > > > > Waiting for your reply.
> > > > >
> > > > > Thank you very  much!
> > > > >
> > > >
> > >
> >
>

Impala August 2020 board report

[Jim Apple]

Here is the report I pushed just now. Corrections welcome!

## Description:
The mission of Apache Impala is the creation and maintenance of software
related to a high-performance distributed SQL engine

## Issues:
There are no issues requiring board attention.

## Membership Data:
Apache Impala was founded 2017-11-14 (3 years ago)
There are currently 52 committers and 33 PMC members in this project.
The Committer-to-PMC ratio is roughly 7:5.

Community changes, past quarter:
- No new PMC members. Last addition was Csaba Ringhofer on 2020-02-18.
- Anurag Mantripragada was added as committer on 2020-05-13

## Project Activity:
This quarter, the Impala community added support (or improved support) for:
 - GROUPING, INTERSECT DISTINCT, EXCEPT DISTINCT, and uncorrelated
subqueries
   in HAVING
 - Development environment bootstrapping with GCC 7 and on Ubuntu 20.04 and
   SLES12 sp5
 - Sanitizers like ASAN and TSAN in developer testing
 - Asynchronous code execution so a query can start in interpreted mode and
   switch to native code when code generation is complete
 - TPCDS queries in the test suite
 - Running in containerized environments

The Impala community improved compatibility with other Apache projects by:
 - Adopting Apache DataSketches KLL structure for quantile estimation
 - Recognizing the new ASF URL practices when downloading Maven and Ant
 - Improving support for Apache Hive ACID tables
 - Adding Apache Iceberg CREATE TABLE support
 - Adding a number of Apache Kudu compatibility improvements
 - Supporting Apache Parquet FIXED_LEN_BYTE_ARRAY DECIMAL
 - Supporting Apache Hadoop Ozone in "load data inpath"

The Impala community removed some or all support for the following in the
4.0
branch:
 - Dateless timestamps
 - Impala-lzo
 - Sentry
 - Hive 2

## Community Health:

The community is overall healthy. This quarter has a common amount of
variability in some previous metrics. It is not infrequent that this
variability has no plainly obvious cause.

 - Commits are down this quarter from 221 to 197.
 - Six community members authored their first patch.
 - JIRAs created is down to 315 from 360; JIRAs resolved are up
   to 357 from 243. A significant number of these are Later, WontFix,
   CannotReproduce, etc.
 - user@ traffic is up 50% to 30 emails; dev@ traffic is down 48% to 69
emails.

Re: New Impala Community Slack Workspace

[Jim Apple]

Should we just use the-asf.slack.com and create an #impala channel? It
looks like that space is already active with other projects like Beam and I
think the space is blessed by ASF infra.

On Wed, Mar 20, 2019 at 2:47 PM Lars Volker  wrote:

> Hi All,
>
> We have created a Slack community for the Impala project. It is public and
> we welcome everyone to join us. You can do so by clicking on this invite
> link
> <
> https://join.slack.com/t/apache-impala/shared_invite/enQtNTgzMzAyNzIyNTk0LTQwMzJjMDI0YzEwOWRmZDk2MzNlZTk5OWZkNTI4M2Y5MmU1MjQ1ZWIzYzQxMWQyMjUzNjNjNWU0NDQ1MTMyNWM
> >
> [1].
> We look forward to seeing you there!
>
> Cheers, Lars
>
> [1]
>
> https://join.slack.com/t/apache-impala/shared_invite/enQtNTgzMzAyNzIyNTk0LTQwMzJjMDI0YzEwOWRmZDk2MzNlZTk5OWZkNTI4M2Y5MmU1MjQ1ZWIzYzQxMWQyMjUzNjNjNWU0NDQ1MTMyNWM
>

Re: Impala - 5314

[Jim Apple]

In my experience, a failure in the master branch during data load indicated
being out of disk space. This sometimes has manifested even when df -h
shows just enough space, so this was successful for me most often when I
had more than enough space.

On Fri, Jul 17, 2020 at 7:06 AM Aayush Bhan  wrote:

> Hi Impala Dev Team,
>
> I was assigned the ticket - 5314 on jira. As a first time contributor, I
> was going through the cwiki links on how to get started. So I was setting
> up my development environment in a docker container. I went ahead with all
> the steps that were given. So every time I try to run the
> bootstrap_development script, it leads to some build errors which i'm not
> able to ascertain how to fix.
>
> The log i'm being directed to after running the script tells me the
> following -
> Error in /home/impdev/Impala/testdata/bin/create-load-data.sh at line 85:
> fi
>
> I got the following output while running the bootstrap_development script -
> https://gist.github.com/aayushbhan/dee15ba212225e780196e2964d279681
>
> I would really appreciate it if anyone could tell me any resources or
> anything of the sort to help me fix this issue and get my development
> environment set up.
>
> Thanks,
> Aayush
>

Re: Is it possible to compile Impala on Apache hadoop/hive/hbase/sentry/ranger?

[Jim Apple]

Hi LiFu! I'm Jim, and I started contributing to the Impala project in 2015.

When we were working on learning about Apache and ASF standards back when
Impala was incubating, this question came up. The analogy that we learned
was that Apache projects are not, themselves, owned by anyone other than
the Foundation, they sometimes rely on software projects, such as Microsoft
Windows, that are owned by another corporation.

While that's the organizational rationale, I think the community would
welcome a build and deploy process that used only officially released
snapshots from the other Apache projects that Impala depends upon. I think
this is a question of someone taking the time to put in the work, similarly
to how Impala has supported only x86-64 for years, but there are new
efforts to make it work on AArch64.

On Wed, Jul 8, 2020 at 6:38 AM LiFu He  wrote:

> At the same time, I'm curious why an open source project relies on a
> business product, though I understand that there are some historical
> reasons. Could anybody share more info?
>
>
> On 2020/07/07 12:46:44, LiFu He  wrote:
> > Hi everybody,
> >
> > I have a question that why Impala relies on the CDH/CDP
> > distribution(hadoop/hive/hbase/sentry/ranger)? And is it possible to
> switch
> > to Apache version? Thanks in advance.
> >
> >
> > --helifu
> >
>

May ASF board report

[Jim Apple]

Hello, all! I have posted the following report for the ASF board. Please
LMK if there's anything you'd like to add or change.

## Description:
The mission of Apache Impala is the creation and maintenance of software
related to a high-performance distributed SQL engine

## Issues:
There are no issues requiring board attention.

## Membership Data:
Apache Impala was founded 2017-11-14 (2 years ago) There are currently 51
committers and 33 PMC members in this project. The Committer-to-PMC ratio is
roughly 3:2.

Community changes, past quarter:
- Csaba Ringhofer was added to the PMC on 2020-02-18
- Norbert Luksa was added as committer on 2020-04-09

## Project Activity:

 - Support for Apache Hudi tables
 - 3.4.0 release and move of HEAD to 4.0, allowing breaking changes to land
 - Fix numerous flaky tests caused by races, including many found using
   ThreadSanitizer.
 - Improvements to interoperability (or interoperability documentation) with
   many Apache projects, including Parquet, Kudu, Ranger, HDFS Ozone, and
ORC
 - Continued significant efforts towards aarch64 support
 - Improvements to zstd read support
 - Reduction in duplicate codegen work by sharing codegen models between
   fragment instances
 - Numerous improvements to Kerberos ergonomics
 - Significant performance improvements via query rewrites as well as work
   sharing of codegen and join builds
 - Support for CentOS 8.1 and Ubuntu 18.04

## Community Health:

Activity on most metrics increased last quarter: dev@ +86%, issues@
+56%, reviews@ +33%, commits +37%.

Re: Please add me as a contributor on the Apache JIRA bug tracker for the Impala project

[Jim Apple]

All set. Welcome!

On Fri, May 8, 2020 at 7:42 AM Qifan Chen  wrote:

> Sorry, forgot to include my JIRA username which is sql_forever.
>
> Thanks! —Qifan
>
> > On May 7, 2020, at 8:40 AM, Qifan Chen  wrote:
> >
> > Hi,
> >
> > Could you please add me as a contributor on the Apache JIRA bug tracker
> for the Impala project?  In this way, I can have the necessary permissions
> to assign/unassign/resolve the JIRA issues.
> >
> > Thank you!
> > —Qifan Chen
>
>

Re: Please add me as a contributor on the Apache JIRA bug tracker for the Impala project

[Jim Apple]

What is your Apache JIRA username? (There are three "Qifan Chen"s in the
Apache JIRA)

On Thu, May 7, 2020 at 8:08 AM Qifan Chen  wrote:

> Hi,
>
> Could you please add me as a contributor on the Apache JIRA bug tracker
> for the Impala project?  In this way, I can have the necessary permissions
> to assign/unassign/resolve the JIRA issues.
>
> Thank you!
> —Qifan Chen

Re: Impala 4.0 breaking changes

[Jim Apple]

I agree. I don’t know how far we are from having arm64 support, though, and
we might not get there for a 4.0 release, I’d guess. But that doesn’t mean
it couldn’t arrive by the time for 4.1 or 4.7 or 5.55 or whatever.

On Wed, Mar 18, 2020 at 6:32 PM Joe McDonnell 
wrote:

> Patches to add support for arm64 are definitely welcome in any release.
>
> Thanks,
> Joe
>
> On Mon, Mar 16, 2020 at 6:11 PM 赵 仁海  wrote:
>
> > Hi
> >
> > Could we  add support for arm64?
> >
> > Thanks
> > Zhao Renhai
> >
> > 
> > 发件人: Joe McDonnell 
> > 发送时间: 2020年3月17日 1:07
> > 收件人: dev@impala.apache.org 
> > 主题: Impala 4.0 breaking changes
> >
> > Now that Impala 3.4 is branched and master is Impala 4.0, we need to
> decide
> > what breaking changes will happen in Impala 4.0. I have provided a series
> > of proposals below. I welcome feedback on them. Other proposals are also
> > welcome.
> >
> > Thanks,
> > Joe
> >
> > Proposal 0: Hadoop component versions
> >
> > Switch to CDP versions of components by default. This means that Impala
> > will use Hive 3+ (which is already essentially Hive 4 and may change
> names
> > to being Hive 4).
> > Remove support for CDH versions of components.
> > This was already discussed in the original thread for Impala 4, so this
> is
> > not new.
> >
> > Proposal 1: OS support
> >
> > Drop support for Centos 6, Ubuntu 14, and Debian (all versions)
> > Retain support for Ubuntu 16, Ubuntu 18, Centos 7, and SLES 12
> > Centos 7 development will be focused on newer Centos 7 versions such as
> 7.6
> > and 7.7.
> > Add support for Centos 8
> > Move main development from Ubuntu 16 to Ubuntu 18 over time.
> >
> > Proposal 2: Python support
> >
> > Drop support for Python 2.6
> > Add support for Python 3 over time.
> >
> > Proposal 3: Impala-lzo
> >
> > Drop support for Impala-lzo/hadoop-lzo
> >
> > Proposal 4: Clients
> >
> > Deprecate beeswax protocol. This means that it can be removed in the next
> > major version number, but it would not be removed in Impala 4. Current
> > users of beeswax would need to start migrating to HS2.
> >
> > Proposal 5: Sentry
> >
> > Drop support for Sentry in favor of Ranger.
> >
> > Proposal 6: Metadata
> >
> > Metadata V2 will become the default. Metadata V1 will be deprecated.
> >
> > Thanks,
> > Joe
> >
>

Re: Impala Board Report due for February 2020 - Initial Reminder

[Jim Apple]

I've added a note about Ranger masking. I'm also going to add a note about
aarch64; I have just seen that there were ten active tickets on this topic
in this quarter.

There are still a few days until the board meeting; please continue to LMK
if there's something you think we should change.

On Wed, Feb 12, 2020 at 7:27 AM Jim Apple  wrote:

> I have published the below report. Please let me know if you would like me
> to revise it in some way.
>
> ## Description:
> The mission of Apache Impala is the creation and maintenance of software
> related to a high-performance distributed SQL engine
>
> ## Issues:
> There are no issues requiring board attention.
>
> ## Membership Data:
> Apache Impala was founded 2017-11-14 (2 years ago)
> There are currently 50 committers and 32 PMC members in this project.
> The Committer-to-PMC ratio is roughly 7:4.
>
> Community changes, past quarter:
> - No new PMC members. Last addition was Fredy Wijaya on 2019-07-27.
> - No new committers. Last addition was Laszlo Gaal on 2019-06-19.
>
> ## Project Activity:
>
>  - Discussions on a release of 3.4 have begun
>  - Planner and executor improvements for multi-threaded execution
>  - Improvements to tests on ACID tables
>  - Continued iterations on local catalog mode
>  - The enablement of primary/foreign key hints during table creation
>  - A number of improvements to test reproducability
>  - A correctness fix for negative zero
>  - Numerous improvements to ORC file handling
>
> ## Community Health:
>
> Activity on many metrics decreased last quarter. This is typical for
> the project, and it corresponds to the US holiday season.
>
> The most prominent decrease was in the number of commits, which was
> down to 164. The November-December-January quarter has, in years past,
> seen 238, 258, 310, 199, and 183 commits (reverse chronological
> order).
>
> -- Forwarded message -
> From: Craig L Russell 
> Date: Wed, Jan 29, 2020 at 4:01 PM
> Subject: Impala Board Report due for February 2020 - Initial Reminder
> To: Jim Apple 
> Cc: 
>
>
> This email was sent on behalf of the ASF Board.  It is an initial reminder
> to
> give you plenty of time to prepare the report.
>
> According to board records, you are listed as the chair of a committee
> that is
> due to submit a report this month. [1] [2]
>
> The meeting is scheduled for Wed, 19 Feb 2020 at 10:30 PST and the
> deadline for
> submitting your report is 1 full week prior to that (Wed Feb 12th)!
>
> Meeting times in other time zones:
>
>   https://timeanddate.com/s/41zj
>
> Please submit your report with sufficient time to allow the board members
> to review and digest. Again, the very latest you should submit your report
> is 1 full week (7days) prior to the board meeting (Wed Feb 12th).
>
> If you feel that an error has been made, please consult [1] and if there
> is still an issue then contact the board directly.
>
> As always, PMC chairs are welcome to attend the board meeting.
>
> Thanks,
> The ASF Board
>
> [1] -
> https://svn.apache.org/repos/private/committers/board/committee-info.txt
> [2] - https://svn.apache.org/repos/private/committers/board/calendar.txt
> [3] - https://svn.apache.org/repos/private/committers/board/templates
> [4] - https://reporter.apache.org/
>
>
> Submitting your Report
> --
>
> Full details about the process and schedule are in [1]. Details about what
> a
> report should look like can be found at:
>
>   https://www.apache.org/foundation/board/reporting
>
> Reports should be posted using the online agenda tool:
>
>   https://whimsy.apache.org/board/agenda/2020-02-19/Impala
>
> Cutting and pasting directly from a Wiki is not acceptable due to
> formatting
> issues. Line lengths should be limited to 77 characters.  If submitting
> using Whimsy please press Reflow before Submit.
>
> Chairs may use the Apache Reporter Service [4] to help them compile and
> submit a board report.
>
>
> Resolutions
> ---
>
> There are several templates for use for various Board resolutions.
> They can be found in [3] and you are encouraged to use them. It is
> strongly recommended that if you have a resolution before the board,
> you are encouraged to attend that board meeting.
>

Re: Self introduction & IMPALA-8547 fix

[Jim Apple]

Since the patch made it to Gerrit, you don't need to register any further.
Review will happen manually and then a person with commit rights will push
it to the repo if/when it gets approved.

Thanks for your interest and your contribution!

On Mon, Dec 16, 2019 at 10:45 AM Eugene Zimichev  wrote:

> Hello all,
>
> I submitted a supposed fix for IMPALA-8547 on gerrit (
> https://gerrit.cloudera.org/c/14905/ 
> ) but later realized that I skipped steps of the process of  registering
> myself as a contributor.
> Could you please add me and if the fix looks appropriate approve it.

Re: Ask help for build/test impala on aarch64 platform

[Jim Apple]

Impala does not currently support aarch64.

On Sun, Nov 3, 2019 at 11:50 PM Tianhua huang  wrote:
>
> Hi, all,
> Sorry to disturb you, I am trying to build Impala on aarch64 platform, but
> I did not find any useful guide/document, I wonder if impala supports
> aarch64 platform? Could anyone help me and any suggestions? Thank you all!

Re: Building Impala on ARM

[Jim Apple]

Impala only builds on x86-64, unfortunately.

On Wed, Sep 11, 2019 at 7:58 PM zs...@outlook.com  wrote:

> Hi ,
> Is there any guide for building impala on ARM ?
> Or, does Impala/kudu support ARM now ?
>

Re: [VOTE] 3.3.0 release candidate 1

[Jim Apple]

+1 (binding), based on
https://jenkins.impala.io/view/Utility/job/release-test-ub1604/25/

On Mon, Aug 19, 2019 at 1:01 PM Quanlong Huang 
wrote:

> Hi folks,
>
> This is a vote for Impala 3.3.0
>
> The artifacts for testing can be downloaded from:
> https://dist.apache.org/repos/dist/dev/impala/3.3.0/RC1/
> Git tag: 3.3.0-rc1
> Tree hash: 5ef67bca619d19402f8c7186b2ab6895bd0603ba
>
> Please vote +1 or -1. -1 votes should be accompanied by an explanation of
> the reason. Only PMC members have binding votes, but other
> community members are encouraged to cast non-binding votes. This vote will
> pass if there are 3 binding +1 votes and more binding +1 votes than -1
> votes.
>
> This wiki page describes how to check the release before you vote:
> https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release#
>
> Cheers,
> Quanlong
>

Re: Impala Board Report due for August 2019 - Initial Reminder

[Jim Apple]

The following is the board report I just posted. Comments welcome; the
report can still be edited before the board meeting on the 21st.

## Description:
Apache Impala is a high-performance distributed SQL engine.

## Issues:
There are no issues requiring board attention.

## Membership Data:
Apache Impala was founded 2017-11-14 (2 years ago)
There are currently 50 committers and 32 PMC members in this project.
The Committer-to-PMC ratio is roughly 7:4.

Community changes, past quarter:
- Bikramjeet Vig was added to the PMC on 2019-05-29
- Fredy Wijaya was added to the PMC on 2019-07-27
- Gabor Kaszab was added to the PMC on 2019-05-22
- Andrew Sherman was added as committer on 2019-06-07
- Laszlo Gaal was added as committer on 2019-06-19
- Sahil Takiar was added as committer on 2019-05-22
- Vihang Karajgaonkar was added as committer on 2019-05-14

## Project Activity:
Notable activity in the last quarter includes:

 - Numerous commits related to support for Hive's ACID table format
 - Improvements to the consideration of nodes as individual executors
   or coordinators, including:
 - Improvements to admission control and executor pool management when
   there is a dedicated coordinator
 - "Executor groups", a feature that allows users to run different
   queries in disjoint sets of executors
 - The addition of admission control parameters that scale with
   the number of executors
 - Improvements the developer experience on Docker
 - Increased compatibility with Apache projects, including Hive 3,
   erasure coding and S3Guard in HDFS, page skipping and Zstd and lz4
   compression in Parquet, and miscellaneous compatibility improvements
   for Ranger, Knox, Kudu, and Atlas
 - The addition of several built-in functions for the DATE type as
   well as the ability to read and write DATE in Parquet
 - Move closer to deprecating the Beeswax protocol by adding HS2
   support to the Impala shell
 - Numerous patches improving tracing, logging, and metrics
 - Multiple improvements to build times or isolation
 - The addition of a data cache for remote reads, improving TPC-DS
   performance on S3 by 30% in one scenario, which made S3 performance
   as good as HDFS-on-EBS

## Community Health:
By almost all metrics, Impala activity is down quarter-over-quarter
and year-over-year. That said, the project is still very active, with
each day featuring approximately:

 - Three commits
 - Two dev@ emails
 - 25 JIRA updates
 - 70 code reviews or patch updates

On Mon, Aug 5, 2019 at 8:46 AM Craig L Russell  wrote:

> This email was sent on behalf of the ASF Board.  It is an initial reminder
> to
> give you plenty of time to prepare the report.
>
> According to board records, you are listed as the chair of a committee
> that is
> due to submit a report this month. [1] [2]
>
> The meeting is scheduled for Wed, 21 Aug 2019 at 10:30 PDT and the
> deadline for
> submitting your report is 1 full week prior to that (Wed Aug 14th)!
>
> Meeting times in other time zones:
>
>   https://timeanddate.com/s/3xqa
>
> Please submit your report with sufficient time to allow the board members
> to review and digest. Again, the very latest you should submit your report
> is 1 full week (7days) prior to the board meeting (Wed Aug 14th).
>
> If you feel that an error has been made, please consult [1] and if there
> is still an issue then contact the board directly.
>
> As always, PMC chairs are welcome to attend the board meeting.
>
> Thanks,
> The ASF Board
>
> [1] -
> https://svn.apache.org/repos/private/committers/board/committee-info.txt
> [2] - https://svn.apache.org/repos/private/committers/board/calendar.txt
> [3] - https://svn.apache.org/repos/private/committers/board/templates
> [4] - https://reporter.apache.org/
>
>
> Submitting your Report
> --
>
> Full details about the process and schedule are in [1]. Details about what
> a
> report should look like can be found at:
>
>   https://www.apache.org/foundation/board/reporting
>
> Reports should be posted using the online agenda tool:
>
>   https://whimsy.apache.org/board/agenda/2019-08-21/Impala
>
> Cutting and pasting directly from a Wiki is not acceptable due to
> formatting
> issues. Line lengths should be limited to 77 characters.  If submitting
> using Whimsy please press Reflow before Submit.
>
> Chairs may use the Apache Reporter Service [4] to help them compile and
> submit a board report.
>
>
> Resolutions
> ---
>
> There are several templates for use for various Board resolutions.
> They can be found in [3] and you are encouraged to use them. It is
> strongly recommended that if you have a resolution before the board,
> you are encouraged to attend that board meeting.
>

Re: How to count aggregate function

[Jim Apple]

It sounds like you'll want to look at Impala 3.1 or later, where
https://issues.apache.org/jira/browse/IMPALA-110 is done.

On Fri, Jul 5, 2019 at 10:56 PM chenbix...@banggood.com <
chenbix...@banggood.com> wrote:

> Hi,
>I am a analyst from banggood.com,China.We have use impala to
> optimize hive or spark's work.But we found impala don't support aggregate
> function.There are some example below.While statement like 'set
> APPX_COUNT_DISTINCT=true; ' is not so sure for us to use.I would like to
> hear from you.Thank you!
>
> https://segmentfault.com/a/119007530586
>
>
> --复购率
> set mapreduce.job.queuename=os;
> select a.date1,
> count(distinct a.customers_id) users,
> count(distinct case when datediff(b.date1,a.date1) between 1 and 7 then
> b.customers_id end) rebuy_1,
> count(distinct case when datediff(b.date1,a.date1) between 8 and 15 then
> b.customers_id end) rebuy_2,
> count(distinct case when datediff(b.date1,a.date1) between 16 and 30 then
> b.customers_id end) rebuy_3,
> count(distinct case when datediff(b.date1,a.date1) between 31 and 60 then
> b.customers_id end) rebuy_4,
> count(distinct case when datediff(b.date1,a.date1) between 61 and 90 then
> b.customers_id end) rebuy_5,
> count(distinct case when datediff(b.date1,a.date1) between 91 and 120 then
> b.customers_id end) rebuy_6,
> count(distinct case when datediff(b.date1,a.date1) between 121 and 180
> then b.customers_id end) rebuy_7,
> count(distinct case when datediff(b.date1,a.date1) between 181 and 360
> then b.customers_id end) rebuy_8,
> count(distinct case when datediff(b.date1,a.date1) > 361 then
> b.customers_id end) rebuy_9
> from
> (select from_unixtime(add_time,'-MM-dd')
> date1,customers_id,row_number() over(partition by customers_id order by
> from_unixtime(add_time,'-MM-dd')) rown
> from datacube.dc_orders
> where from_unixtime(add_time,'-MM-dd') >= '2017-01-01'
> and domain = 1
> and order_type not in
> ('dropship','dropshipping','wholesale','gift','snatch')
> and customers_id <> 0
> and orders_status NOT IN (1 , 4, 6, 12, 17, 20, 21, 22, 23, 27)
> -- and site in ('pwa.yoins.com','android','ios','www.yoins.com','
> m.yoins.com')
> group by from_unixtime(add_time,'-MM-dd'),customers_id
> ) a
> left outer join
> (
> select from_unixtime(add_time,'-MM-dd') as
> date1,customers_id,row_number() over(partition by customers_id order by
> from_unixtime(add_time,'-MM-dd')) as rown
> from datacube.dc_orders
> where from_unixtime(add_time,'-MM-dd') >= '2017-01-01'
> and domain = 1
> and order_type not in
> ('dropship','dropshipping','wholesale','gift','snatch')
> and customers_id <> 0
> and orders_status NOT IN (1 , 4, 6, 12, 17, 20, 21, 22, 23, 27)
> -- and site in ('pwa.yoins.com','android','ios','www.yoins.com','
> m.yoins.com')
> group by from_unixtime(add_time,'-MM-dd'),customers_id
> ) b
> on (a.customers_id=b.customers_id and a.rown=b.rown-1)
> group by a.date1
> order by date1
> ;
>
>
> chenbix...@banggood.com
>

Re: Enabled backend tests for UBSAN

[Jim Apple]

The JDBC, FE, and custom cluster tests are now UBSAN-clean. Can you turn
those on, too?

Thank you!

On Mon, May 6, 2019 at 10:49 AM Tim Armstrong 
wrote:

> Hi All,
>   I turned on backend tests under UBSAN in our precommit tests - Jim Apple
> let me know that they were passing after he fixed the final set of issues
> there.
>
> - Tim
>

Re: Rebase past the fix for IMPALA-8700 before running tests

[Jim Apple]

Thanks for fixing this, Joe!

On Mon, Jun 24, 2019 at 4:38 PM Joe McDonnell 
wrote:

> Hi all,
>
> Impala tests (including precommit) broke this weekend due to IMPALA-8700
> (introduced by my fix for IMPALA-8561). The fix for IMPALA-8700 has been
> merged, but any test runs without this fix will fail. Please rebase past
> IMPALA-8700 before submitting tests to run.
> https://issues.apache.org/jira/browse/IMPALA-8700
>
> Thanks,
> Joe
>

Re: Ubuntu 18.04 in pre-merge tests?

[Jim Apple]

>
> Generally I think precommit running on something closer to the oldest
> supported OS is better than running on the newest, since it's more likely
> that new OSes are backward-compatible. Otherwise it's very easy to
> introduce code that uses features not available on el7, for example.
>

I find that argument compelling. Do you think we should switch the
pre-commit job to CentOS 7?

Re: Ubuntu 18.04 in pre-merge tests?

[Jim Apple]

Maybe now would be a good time to implement Everblue jobs that ping dev@
when they fail. Thoughts?

On Mon, May 20, 2019 at 9:09 AM Todd Lipcon  wrote:

> Adding a build-only job for 18.04 makes sense to me. A full test run on
> every precommit seems a bit expensive but doing one once a week or
> something like that might be a good idea to prevent runtime regressions.
>
> As for switching the precommit from 16.04 to 18.04, I'd lean towards
> keeping to 16.04 due to it being closer in terms of component versions to
> common enterprise distros like RHEL 7.
>
> -Todd
>
> On Sun, May 19, 2019 at 5:03 PM Jim Apple  wrote:
>
> > HEAD now passes on Ubuntu 18.04:
> >
> > https://jenkins.impala.io/job/ubuntu-18.04-from-scratch/
> >
> > Thanks to the community members who have made this happen!
> >
> > Should we add Ubuntu 18.04 to our pre-merge Jenkins job, replace 16.04
> with
> > 18.04 in our pre-merge Jenkins job, or neither?
> >
> > I propose adding 18.04 for now (ans so running both 16.04 and 18.04 on
> > merge) and removing 16.04 when it starts to become inconvenient.
> >
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

Ubuntu 18.04 in pre-merge tests?

[Jim Apple]

HEAD now passes on Ubuntu 18.04:

https://jenkins.impala.io/job/ubuntu-18.04-from-scratch/

Thanks to the community members who have made this happen!

Should we add Ubuntu 18.04 to our pre-merge Jenkins job, replace 16.04 with
18.04 in our pre-merge Jenkins job, or neither?

I propose adding 18.04 for now (ans so running both 16.04 and 18.04 on
merge) and removing 16.04 when it starts to become inconvenient.

May 2019 Impala board report

[Jim Apple]

I posted the following board report. Please let me know if you have any
suggested changes by Tuesday, May 7th.

## Description:

Impala is a high-performance distributed SQL engine.

## Issues:

There are no issues requiring board attention at this time.

## Activity and health report:

The previous three months had 237 patches to the master branch, while
this three-month period had 286. This is likely the recovery from the
usual seasonal dip around the end of each calendar year.

Prominent work in the last three months includes:

 - An admission controller debugging page

 - Thousands of lines of new planner tests

 - A number of improvements to the shell scripts used to build and to
   start the various daemons

 - A few patches that reduced the disk space needed for development by
   tens of gigabytes

 - Support for development on Ubuntu 18.04.

 - Support for Apache Ranger and decoupling Apache Sentry

 - Support for complex types in ORC files

 - Better hardware detection

 - Compatibility with Hive 3.x for data loading and the Metastore

 - Numerous improvements to metrics and counters

 - Continued work on supporting Docker in development environments and
   in production

 - Initial support for a timeless DATE type

Health is a subjective metric, but the increased compatibility with
other open source and Apache projects is a good sign, as is the nine
new patch authors.

## PMC changes:

 - Currently 29 PMC members.
 - Quanlong Huang was added to the PMC on Sun Mar 10 2019

## Committer base changes:

 - Currently 46 committers.
 - New committers:
- Pooja Nilangekar was added as a committer on Tue Apr 09 2019
- Paul Rogers was added as a committer on Mon Feb 04 2019

## Releases:

 - 3.2.0 was released on Wed Mar 27 2019

## JIRA activity:

 - 326 JIRA tickets created in the last 3 months
 - 296 JIRA tickets closed/resolved in the last 3 months

Re: Unplanned maintenance downtime for jenkins.impala.io

[Jim Apple]

Ah, I see. Thanks for cleaning everything up!

On Mon, Apr 29, 2019 at 6:13 AM Laszlo Gaal 
wrote:

> >
> > It should be possible to configure the pre-review job so at most N jobs
> run
> > at once. I think the rest will then properly queue up.
>
>
> That's actually what happened -- it was the unusually long build queue that
> caught my attention.
> It was pretty quick to find what caused the build requests, and to verify
> with the submitter
> that none of these submissions are worth checking.
> Jenkins was suspended to isolate the build requests that had to be killed,
> and to prevent
> new build requests from being created during the churn of the cleanup
> process.
>
> On Mon, Apr 29, 2019 at 2:30 PM Jim Apple  wrote:
>
> > It should be possible to configure the pre-review job so at most N jobs
> run
> > at once. I think the rest will then properly queue up.
> >
> > On Mon, Apr 29, 2019 at 3:29 AM Laszlo Gaal 
> > wrote:
> >
> > > There was an unintended, errant review pushed to Impala's public Gerrit
> > > instance, which resulted in about 30 review requests. This has also
> > kicked
> > > off a similar number of pre-review runs on jenkins.impala.io.
> > >
> > > I have personally verified the unintended nature of the problem with
> the
> > > owner.
> > >
> > > jenkins.impala.io is temporarily closed for builds until further
> notice
> > as
> > > the fallout from the review storm is cleaned up. Once that is done,
> I'll
> > > send out the all clear signal when the service is restored.
> > >
> > > Apologies for the disruption, and thanks you for your patience.
> > >
> > > - Laszlo
> > >
> >
>

Re: Unplanned maintenance downtime for jenkins.impala.io

[Jim Apple]

It should be possible to configure the pre-review job so at most N jobs run
at once. I think the rest will then properly queue up.

On Mon, Apr 29, 2019 at 3:29 AM Laszlo Gaal 
wrote:

> There was an unintended, errant review pushed to Impala's public Gerrit
> instance, which resulted in about 30 review requests. This has also kicked
> off a similar number of pre-review runs on jenkins.impala.io.
>
> I have personally verified the unintended nature of the problem with the
> owner.
>
> jenkins.impala.io is temporarily closed for builds until further notice as
> the fallout from the review storm is cleaned up. Once that is done, I'll
> send out the all clear signal when the service is restored.
>
> Apologies for the disruption, and thanks you for your patience.
>
> - Laszlo
>

ub1604 from scratch job major run time increase?

[Jim Apple]

The major from-scratch job in the pre-merge tests seems to frequently be
taking more than 5 hours. It used to frequently take less than 4, as of a
month or two ago.

Here's a job that calls that, to show how long the job has been taking
recently:
https://jenkins.impala.io/view/Utility/job/parallel-all-tests/buildTimeTrend

Was something big added that increased the run time? It's possible I missed
an email thread about this.

Re: Remote read testing in precommit

[Jim Apple]

I'm in favor. Given the importance of remote reads, I would even be in
favor of these if it DID extend the critical path.

On Fri, Apr 5, 2019 at 10:41 AM Tim Armstrong 
wrote:

> This is really about testing the dockerised minicluster, but gives us
> coverage of remote read code paths for free, and more people care about
> that right now.
>
> I got the core end-to-end tests passing locally as part of
> https://issues.apache.org/jira/browse/IMPALA-7995. That change is up for
> review here https://gerrit.cloudera.org/c/12639/. The next step is to get
> a
> Jenkins job running, which I've been working on.
>
> I'd like to run it regularly so we can catch any regressions. Initially
> I'll just have it email me when it fails, but after it's stable for a week
> or two I'd like to make it part of the regular set of jobs.
>
> My preference is to run it as part of the precommit jobs, in parallel to
> the Ubuntu 16.04 tests. It should not extend the critical path of precommit
> because it only runs the end-to-end tests. We could alternatively run it as
> a scheduled post-commit job, but that tends to create additional work when
> it breaks.
>
> What do people think?
>
> - Tim
>

Re: +2ability on gerrit

[Jim Apple]

Works now, thank you!

On Fri, Mar 29, 2019 at 10:43 AM Todd Lipcon  wrote:

> Ah, try now. Seems I added you to the wrong group
>
> On Fri, Mar 29, 2019 at 10:33 AM Jim Apple  wrote:
>
> > Hm, didn't seem to work.
> >
> > On Thu, Mar 28, 2019 at 9:28 AM Todd Lipcon  wrote:
> >
> > > I think you're all set. Give it a shot?
> > >
> > > -Todd
> > >
> > > On Tue, Mar 26, 2019 at 8:05 PM Jim Apple  wrote:
> > >
> > > > Hello! I am now using "jbapple", not "jbapple-cloudera", as my gerrit
> > > > handle. Can someone with an admin login give me the auths to +2
> > changes?
> > > >
> > > > Thanks in advance!
> > > >
> > >
> > >
> > > --
> > > Todd Lipcon
> > > Software Engineer, Cloudera
> > >
> >
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

Re: +2ability on gerrit

[Jim Apple]

Hm, didn't seem to work.

On Thu, Mar 28, 2019 at 9:28 AM Todd Lipcon  wrote:

> I think you're all set. Give it a shot?
>
> -Todd
>
> On Tue, Mar 26, 2019 at 8:05 PM Jim Apple  wrote:
>
> > Hello! I am now using "jbapple", not "jbapple-cloudera", as my gerrit
> > handle. Can someone with an admin login give me the auths to +2 changes?
> >
> > Thanks in advance!
> >
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

+2ability on gerrit

[Jim Apple]

Hello! I am now using "jbapple", not "jbapple-cloudera", as my gerrit
handle. Can someone with an admin login give me the auths to +2 changes?

Thanks in advance!

Re: Impala Jenkins server update and restart

[Jim Apple]

Thanks, Laszlo!

On Tue, Mar 26, 2019 at 5:09 AM Laszlo Gaal 
wrote:

> The service has been updated successfully, it is up and running again.
>
> On Tue, Mar 26, 2019 at 9:35 AM Laszlo Gaal 
> wrote:
>
> > The Impala Jenkins server at jenkins.impala.io will be restarted shortly
> > for security updates. The server is idle now, so I expect the outage to
> be
> > short; mail will be sent when it is up and running again.
> >
> > Thank you for your patience.
> >
>

New PMC member: Quanlong Huang

[Jim Apple]

The Project Management Committee (PMC) for Apache Impala has invited
Quanlong Huang to become a PMC member and we are pleased to announce
that they have accepted.

Congratulations and welcome, Quanlong!

Re: [DISCUSS] 3.2.0 release

[Jim Apple]

+1 to both a release and a habit of curated release notes as Quanlong suggests.

On Mon, Mar 4, 2019 at 6:54 AM Quanlong Huang  wrote:
>
> +1. Thanks for your volunteer!
>
> If we decide to release, could we manage a doc that everyone can write down
> his/her notable works in this version? Got this idea from the Kudu
> community and feel it very helpful:
> https://lists.apache.org/thread.html/8a10a4d244369f43b97c79513f13766c393a98252d3a2dade7611cc3@%3Cdev.kudu.apache.org%3E
>
> Thanks,
> Quanlong
>
> On Mon, Mar 4, 2019 at 10:18 PM Gabor Kaszab 
> wrote:
>
> > Hey,
> >
> > 3.1.0 has been released at the beginning of December and I feel that since
> > then there are a number of bug fixes and improvements checked in.
> > What do you think about releasing 3.2.0 soon? I volunteer as a release
> > manager in case no one is against.
> >
> > Cheers,
> > Gabor
> >

Re: Remove write support for unsupported formats in 2.x?

[Jim Apple]

Thanks for pointing to the original thread. I think because of the
language in the docs "An obsolete query option from early work on
support for file formats. Do
not use. Might be removed in the future", it's OK to remove it even in
the 2.x line.

On Tue, Feb 26, 2019 at 10:58 PM Quanlong Huang  wrote:
>
> Hi all,
>
> Branch 2.x is moving forward and now the next patch to pick is
> 30e82c63ecdd56ded10fed931d95ab6d994b9244 (
> http://gerrit.cloudera.org:8080/10823)
> It's about removing the query option ALLOW_UNSUPPORTED_FORMATS and some
> relative codes that were not well tested.
>
> We had a consensus to remove it in Impala-3.1 per the discussion:
> https://lists.apache.org/thread.html/749bef4914350ae0756bc88961db2dd39901a649a9cef6949eda5870@%3Cdev.impala.apache.org%3E
>
> However, since we haven't removed any query options in 2.x yet, should we
> consider keeping it for compatibility? Please share your thoughts.
>
> Thanks,
> Quanlong

February Apache Impala board report

[Jim Apple]

I posted the following report for the board. Let me know if you'd like any
changes to be made:

## Description:

Impala is a high-performance distributed SQL engine.

## Activity:

The previous three months had 330 patches to the master branch, while this
three-month period had 237. This is likely a seasonal dip.

Prominent work in the last three months includes:

 - The revival of the 2.x branch

 - Modernization of the documentation of the HBase integration

 - Significant changes to enable Impala to run better in containers

 - Improvements to the user experience of dealing with changing metadata

 - Multiple improvements to profile statistics

 - Numerous build process improvements for performance

 - Support for reading additional Parquet field types

## Health report:

The project remains healthy and metrics (number of commits, bugs filed, and
mailing list activity) remain healthy.

Three new contributors had patches committed.

## PMC changes:

 - Currently 28 PMC members.
 - Zoltán Borók-Nagy was added to the PMC on Thu Jan 03 2019

## Committer base changes:

 - Currently 45 committers.
 - Paul Rogers was added as a committer on Mon Feb 04 2019
 - Zoram Thanga was added as a committer on Fri Nov 16 2018

## Releases:

 - 3.1.0 was released on Wed Dec 05 2018

## Mailing list and JIRA activity:

Activity dropped, consistent with a seasonal dip during US holidays that
Impala sees every year: reviews@, issues@, dev@ traffic decreased by about
30%.

Re: You'll need to fetch your Impala-lzo...

[Jim Apple]

Do you want to change default branch to master, rather than cdh5-trunk?

https://help.github.com/articles/setting-the-default-branch/

On Fri, Feb 1, 2019 at 3:51 PM Philip Zeyliger  wrote:

> "cdh5-trunk" was that branch, but I also pushed it to 2.x, for a bit more
> consistency.
>
> -- Philip
>
> On Fri, Feb 1, 2019 at 3:44 PM Quanlong Huang 
> wrote:
>
> > Hi Philip,
> >
> > Thanks for your kindly reminder! Could you help to create a tag in the
> > cloudera/Impala-lzo project to point to the previous commit? Then
> > it's convenient for branch-2.x to point to a pined version.
> >
> > Thanks,
> > Quanlong
> >
> > On Sat, Feb 2, 2019 at 5:33 AM Philip Zeyliger 
> > wrote:
> >
> > > Hi Fredy,
> > >
> > > I think you need to rebase your change onto mine. I've just done so by
> > > clicking the button, and we'll see.
> > >
> > > -- Philip
> > >
> > > On Fri, Feb 1, 2019 at 12:24 PM Fredy Wijaya 
> > wrote:
> > >
> > > > gerrit-code-review-checks seems a bit unhappy with the change, e.g.
> > > >
> > > >
> > >
> >
> https://jenkins.impala.io/job/gerrit-code-review-checks/1963/artifact/https%3A%5E%5Ejenkins.impala.io%5Ejob%5Eclang-tidy-ub1604%5E5041%5E/*view*/
> > > >
> > > > + ./buildall.sh -skiptests -tidy -so -noclean
> > > > + echo 'tidy build failed; See /home/ubuntu/tidylog.txt for full tidy
> > > > build output. Guess:'
> > > > tidy build failed; See /home/ubuntu/tidylog.txt for full tidy build
> > > > output. Guess:
> > > > + grep ': error: ' /home/ubuntu/tidylog.txt
> > > > /home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc:140:17: error:
> > > > ‘class impala::HdfsScanNodeBase’ has no member named
> > > > ‘UpdateRemainingScanRangeSubmissions’
> > > > /home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc:168:14: error:
> > > > ‘class impala::HdfsScanNodeBase’ has no member named
> > > > ‘UpdateRemainingScanRangeSubmissions’
> > > > /home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc:169:59: error: no
> > > > matching function for call to
> > > >
> > > >
> > >
> >
> ‘impala::HdfsScanNodeBase::AddDiskIoRanges(std::vector&)’
> > > > /home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc:200:49: error: no
> > > > matching function for call to
> > > >
> > > >
> > >
> >
> ‘impala::HdfsScanNodeBase::AddDiskIoRanges(std::vector)’
> > > >
> > > >
> > > > On Fri, Feb 1, 2019 at 1:20 PM Philip Zeyliger 
> > > > wrote:
> > > >
> > > > > Hi folks,
> > > > >
> > > > > I just pushed
> > > > >
> > > > >
> > > >
> > >
> >
> https://gitbox.apache.org/repos/asf?p=impala.git;a=commit;h=a8e30506aafef14646d95a56fb87cf7c28d259d6
> > > > > and
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/cloudera/impala-lzo/commit/dccb1be88a5e237b06ae69cd99b048a38d9f024b
> > > > > to tackle IMPALA-7980. If you're working on master, and you have
> > > > > ../Impala-lzo checked out, you'll need to pull that repo. There's
> > also
> > > a
> > > > > divergence now for Impala-lzo between branches, so if you're still
> > > > working
> > > > > on 2.x (say), you'll need to figure out how to manage that.
> > > > >
> > > > > Please let me know if you run into issues; I'll be happy to help.
> > > > >
> > > > > -- Philip
> > > > >
> > > >
> > >
> >
>

Re: JIRA access for Impala project

[Jim Apple]

done

On Wed, Jan 30, 2019 at 8:25 PM Bharathkrishna Guruvayoor Murali <
bhar...@cloudera.com> wrote:

> Hi team,
>
> I am currently working on the Impala project from Cloudera.
> Please provide me contributor privileges on the Apache JIRA bug tracker for
> the Impala project.
> My Apache ID is bharos92. Thanks in advance.
>
> Regards,
> Bharath Krishna
>

Re: Timestamps with less than nano precision: rounding vs truncating

[Jim Apple]

You use the phrase "truncation towards negative infinity". Can you give a
concrete example where "truncation towards negative infinity" is different
from "truncation" and another where "truncation towards negative infinity"
is different from "rounding towards negative infinity"? As I mentioned
above, I am not aware of a rounding mode entitled "truncation towards
negative infinity".

On Thu, Jan 24, 2019 at 11:41 AM Csaba Ringhofer 
wrote:

> Thanks for the comments!
>
> > Can you give a concrete example of an query where you are proposing a
> change?
> create table tkudu (id int primary key, t timestamp) stored as kudu;
> insert into tkudu values
>   (1,"1970-01-01 00:00:00.111"), -- all sub-second parts are 7 digit
>   (2,"1970-01-01 23:59:59.999"),
>   (3,"1969-12-31 23:59:59.999");
> select * from tkudu;
>
> This currently returns:
> 1,1970-01-01 00:00:00.11000
> 2,1970-01-02 00:00:00
> 3,1970-01-01 00:00:00
>
> 1 was rounded down to microsec precision, while 2 and 3 were rounded up and
> also stepped to another way.
> With truncation towards negative infinity the query would return this:
> 1,1970-01-01 00:00:00.11000
> 2,1970-01-01 23:59:59.99000
> 3,1969-12-31 23:59:59.99000
> So 1 would be the same, and 2 and 3 would be truncated from 7 to 6 digits
> and would not step to a new day.
>
> My goal is not to change how Impala writes Kudu, but to get to a consensus
> before going forward with writing Parquet milli/micro timestamps. (
> https://gerrit.cloudera.org/#/c/12247/ )
>
> >That said, Oracle, Netezza, Vertica, and Postgres all round. Db2
> truncates.
> Thanks, good to know!
> So it looks like that rounding is more popular, but consistency with Hive
> may be more important in Impala's case.
>
> On Thu, Jan 24, 2019 at 7:59 PM Greg Rahn  wrote:
>
> > For things like this the ISO SQL spec states  "the choice of whether to
> > round or truncate is implementation-defined".  That said, Oracle,
> Netezza,
> > Vertica, and Postgres all round. Db2 truncates.
> >
> > On Wed, Jan 23, 2019 at 12:26 PM Csaba Ringhofer <
> csringho...@cloudera.com
> > >
> > wrote:
> >
> > > Timestamps are often represented as ticks since some epoch, e.g.
> > 1970.01.01
> > > 00:00:00, so negative timestamps make sense as times before the epoch
> - I
> > > meant rounding vs truncating towards 0 vs rounding towards negative
> > > infinite in this sense. Truncating towards negative infinity means that
> > > timestamps are always truncated to an earlier timestamp. Truncating
> > towards
> > > 0 would mean that before 1970, timestamps are truncated upwards, which
> > can
> > > lead to similar troubles as the I ones mentioned with rounding. On x86
> > c++,
> > > when a time_t is divided by an integer, the result is rounded towards
> 0,
> > so
> > > a naive implementation that uses time_t to represent timestamps can
> > > truncate towards 0, but In impala::TimestampValue, time_ should be the
> > > non-negative nanoseconds since midnight, so it can be simply divided
> with
> > > 1000 to convert from nanoseconds to microseconds.
> > >
> >
>

Re: Jira access for IMPALA project.

[Jim Apple]

CC the original post author. by default, even reply-all to dev@ only goes
back to dev@.

On Fri, Jan 18, 2019 at 3:43 PM Jim Apple  wrote:

> What exactly do you need to do? I think you should be able to file tickets
> without any extra authorities. Do you want to assign a JIRA to yourself?
>
> On Fri, Jan 18, 2019 at 3:10 PM Naveen Gangam 
> wrote:
>
>> Hello all,
>> I am requesting JIRA access for IMPALA project so I could contribute to
>> the
>> project. my upstream jira user name is "ngangam".
>>
>> Thanks in advance
>> Naveen
>>
>

Re: Timestamps with less than nano precision: rounding vs truncating

[Jim Apple]

>
> - Hive seems to truncate timestamps towards negative infinity when it has
> to reduce precision.
>
> I lean towards truncating - theoretically rounding introduces smaller
> error, but it can move the timestamp to a different day / DST rule / year


I'm familiar with rounding towards negative infinity and truncation, but my
understanding of those is different, in that -1.5 rounded to negative
infinity is -2, while -1.5 truncated is -1. Does this timestamp format
allow negative timestamps?

https://en.wikipedia.org/wiki/Floating-point_arithmetic#Rounding_modes

Re: Jira access for IMPALA project.

[Jim Apple]

What exactly do you need to do? I think you should be able to file tickets
without any extra authorities. Do you want to assign a JIRA to yourself?

On Fri, Jan 18, 2019 at 3:10 PM Naveen Gangam  wrote:

> Hello all,
> I am requesting JIRA access for IMPALA project so I could contribute to the
> project. my upstream jira user name is "ngangam".
>
> Thanks in advance
> Naveen
>

Re: [Patch Ready] Add support for reading ORC complex types

[Jim Apple]

Thanks, Lars!

On Tue, Jan 8, 2019 at 1:39 PM Lars Volker  wrote:

> I think it's an excellent idea to have a face-to-face video meeting with
> the larger community on this mailing list. It might not be convenient to
> have a regular schedule due to time zone constraints, but a one-off to say
> "hi", introduce each other, and have a discussion would be helpful. I can
> start a thread to see who's in which TZ and to find a good time.
>
> On Mon, Jan 7, 2019 at 5:53 PM Quanlong Huang 
> wrote:
>
> > Sorry for my words that may make you uncomfortable... Yes, I mean
> in-person
> > because it's more efficient. I thought you have many discussions and
> > meetings about the design, roadmap or planning. But different time zones
> > are really an obstacle. Having discussions face to face is not quite
> > realistic.
> >
> > Maybe we can start another thread to discuss the interaction of the
> Impala
> > community, e.g. regular meetings (via Slack or WeChat), roadmap and
> > planning discussion, etc.
> >
> > Thanks for all your efforts to make the community grow!
> >
> > On Tue, Jan 8, 2019 at 6:07 AM Jim Apple  wrote:
> >
> > > "As a community developer, I can't discuss with you Cloudera folks in
> > > real."
> > >
> > > Do you mean in-person? Because if you mean "real", we want to strive to
> > > have dev@impala.apache.org be very real. In particular, I and others I
> > > work
> > > with at Cloudera, often in the same building, have discussions (that we
> > > could have in person) here on dev@ instead in order to do community
> work
> > > in
> > > the open.
> > >
> > > Some other communities, like Parquet, will have regularly scheduled
> > > face-to-face meetings over videoconferencing. Time zones are hard, of
> > > course, but this is something we could try if someone wants to take
> > charge
> > > of organizing it, recruiting people to come, sending out notes, etc.
> > >
> > > In the meantime, let's all work to make dev@ and gerrit great places
> to
> > > have productive discussions.
> > >
> > > Thanks for making the google doc, BTW: I think that's really helpful
> for
> > > large and complex patches.
> > >
> > > On Mon, Jan 7, 2019 at 7:28 AM Quanlong Huang  >
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > I'm excited that we can support reading ORC complex types
> (IMPALA-6503)
> > > > now!
> > > >
> > > > I just finish the patch. As a community developer, I can't discuss
> with
> > > you
> > > > Cloudera folks in real. It may be hard to make codes clear via
> comments
> > > for
> > > > a big patch, so I wrote a doc about the implementation:
> > > >
> > > >
> > >
> >
> https://docs.google.com/document/d/10Gzcge57VUOQ0ZQWfWiIW9cUix8_FumMewl0Gja3ZA4
> > > > (I may need to add more details, please let me know)
> > > >
> > > > Patch link: https://gerrit.cloudera.org/#/c/12168/
> > > > Have passed pre-review-test:
> > > > https://jenkins.impala.io/job/pre-review-test/269/
> > > >
> > > > Hopes you can take some time to review it.
> > > >
> > > > Thanks!
> > > > Quanlong
> > > >
> > >
> >
>

Re: [Patch Ready] Add support for reading ORC complex types

[Jim Apple]

"As a community developer, I can't discuss with you Cloudera folks in real."

Do you mean in-person? Because if you mean "real", we want to strive to
have dev@impala.apache.org be very real. In particular, I and others I work
with at Cloudera, often in the same building, have discussions (that we
could have in person) here on dev@ instead in order to do community work in
the open.

Some other communities, like Parquet, will have regularly scheduled
face-to-face meetings over videoconferencing. Time zones are hard, of
course, but this is something we could try if someone wants to take charge
of organizing it, recruiting people to come, sending out notes, etc.

In the meantime, let's all work to make dev@ and gerrit great places to
have productive discussions.

Thanks for making the google doc, BTW: I think that's really helpful for
large and complex patches.

On Mon, Jan 7, 2019 at 7:28 AM Quanlong Huang 
wrote:

> Hi all,
>
> I'm excited that we can support reading ORC complex types (IMPALA-6503)
> now!
>
> I just finish the patch. As a community developer, I can't discuss with you
> Cloudera folks in real. It may be hard to make codes clear via comments for
> a big patch, so I wrote a doc about the implementation:
>
> https://docs.google.com/document/d/10Gzcge57VUOQ0ZQWfWiIW9cUix8_FumMewl0Gja3ZA4
> (I may need to add more details, please let me know)
>
> Patch link: https://gerrit.cloudera.org/#/c/12168/
> Have passed pre-review-test:
> https://jenkins.impala.io/job/pre-review-test/269/
>
> Hopes you can take some time to review it.
>
> Thanks!
> Quanlong
>

Re: [NOTICE] Mandatory migration of git repositories to gitbox.apache.org

[Jim Apple]

Filed https://issues.apache.org/jira/browse/INFRA-17572. Will update when I
know planned day of chaneover

On Thu, Jan 3, 2019 at 9:55 AM Jim Apple  wrote:

> Good idea. Assuming there are no objections on this thread, I'll ask the
> infra people to plan it for a specific time at least a few days out and
> then announce that time to this list
>
> On Thu, Jan 3, 2019 at 9:44 AM Tim Armstrong 
> wrote:
>
>> It would be good to give everyone at least a few days to plan their
>> migration and do it at a specific time for the convenience of others that
>> might have jobs or infrastructure using git-wip-us.apache.org
>>
>> On Thu, Jan 3, 2019 at 9:26 AM Jim Apple  wrote:
>>
>> > And some jenkins.impala.io jobs
>> >
>> > On Thu, Jan 3, 2019 at 9:25 AM Joe McDonnell > >
>> > wrote:
>> >
>> > > +1 to going ahead with the migration
>> > >
>> > > bin/push_to_asf.py and bin/bootstrap_system.sh require updates when we
>> > make
>> > > the migration.
>> > >
>> > > -Joe
>> > >
>> > > On Thu, Jan 3, 2019 at 8:51 AM Jim Apple 
>> wrote:
>> > >
>> > > > +1 to moving ahead of the mass migration.
>> > > >
>> > > > Any thoughts on this? I can do the paperwork.
>> > > >
>> > > >
>> > > >
>> > > > On Thu, Jan 3, 2019 at 5:19 AM Apache Infrastructure Team <
>> > > > infrastruct...@apache.org> wrote:
>> > > >
>> > > > > Hello, impala folks.
>> > > > > As stated earlier in 2018, all git repositories must be migrated
>> from
>> > > > > the git-wip-us.apache.org URL to gitbox.apache.org, as the old
>> > service
>> > > > > is being decommissioned. Your project is receiving this email
>> because
>> > > > > you still have repositories on git-wip-us that needs to be
>> migrated.
>> > > > >
>> > > > > The following repositories on git-wip-us belong to your project:
>> > > > >  - impala.git
>> > > > >
>> > > > >
>> > > > > We are now entering the mandated (coordinated) move stage of the
>> > > roadmap,
>> > > > > and you are asked to please coordinate migration with the Apache
>> > > > > Infrastructure Team before February 7th. All repositories not
>> > migrated
>> > > > > on February 7th will be mass migrated without warning, and we'd
>> > > > appreciate
>> > > > > it if we could work together to avoid a big mess that day :-).
>> > > > >
>> > > > > Moving to gitbox means you will get full write access on GitHub as
>> > > well,
>> > > > > and be able to close/merge pull requests and much more.
>> > > > >
>> > > > > To have your repositories moved, please follow these steps:
>> > > > >
>> > > > > - Ensure consensus on the move (a link to a lists.apache.org
>> thread
>> > > will
>> > > > >   suffice for us as evidence).
>> > > > > - Create a JIRA ticket at
>> > https://issues.apache.org/jira/browse/INFRA
>> > > > >
>> > > > > Your migration should only take a few minutes. If you wish to
>> migrate
>> > > > > at a specific time of day or date, please do let us know in the
>> > ticket.
>> > > > >
>> > > > > As always, we appreciate your understanding and patience as we
>> move
>> > > > > things around and work to provide better services and features for
>> > > > > the Apache Family.
>> > > > >
>> > > > > Should you wish to contact us with feedback or questions, please
>> do
>> > so
>> > > > > at: us...@infra.apache.org.
>> > > > >
>> > > > >
>> > > > > With regards,
>> > > > > Apache Infrastructure
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>

Re: Enroll

[Jim Apple]

>
> Use
> dev-subscr...@impala.apache.org
>
>
>

Re: [NOTICE] Mandatory migration of git repositories to gitbox.apache.org

[Jim Apple]

Good idea. Assuming there are no objections on this thread, I'll ask the
infra people to plan it for a specific time at least a few days out and
then announce that time to this list

On Thu, Jan 3, 2019 at 9:44 AM Tim Armstrong 
wrote:

> It would be good to give everyone at least a few days to plan their
> migration and do it at a specific time for the convenience of others that
> might have jobs or infrastructure using git-wip-us.apache.org
>
> On Thu, Jan 3, 2019 at 9:26 AM Jim Apple  wrote:
>
> > And some jenkins.impala.io jobs
> >
> > On Thu, Jan 3, 2019 at 9:25 AM Joe McDonnell 
> > wrote:
> >
> > > +1 to going ahead with the migration
> > >
> > > bin/push_to_asf.py and bin/bootstrap_system.sh require updates when we
> > make
> > > the migration.
> > >
> > > -Joe
> > >
> > > On Thu, Jan 3, 2019 at 8:51 AM Jim Apple  wrote:
> > >
> > > > +1 to moving ahead of the mass migration.
> > > >
> > > > Any thoughts on this? I can do the paperwork.
> > > >
> > > >
> > > >
> > > > On Thu, Jan 3, 2019 at 5:19 AM Apache Infrastructure Team <
> > > > infrastruct...@apache.org> wrote:
> > > >
> > > > > Hello, impala folks.
> > > > > As stated earlier in 2018, all git repositories must be migrated
> from
> > > > > the git-wip-us.apache.org URL to gitbox.apache.org, as the old
> > service
> > > > > is being decommissioned. Your project is receiving this email
> because
> > > > > you still have repositories on git-wip-us that needs to be
> migrated.
> > > > >
> > > > > The following repositories on git-wip-us belong to your project:
> > > > >  - impala.git
> > > > >
> > > > >
> > > > > We are now entering the mandated (coordinated) move stage of the
> > > roadmap,
> > > > > and you are asked to please coordinate migration with the Apache
> > > > > Infrastructure Team before February 7th. All repositories not
> > migrated
> > > > > on February 7th will be mass migrated without warning, and we'd
> > > > appreciate
> > > > > it if we could work together to avoid a big mess that day :-).
> > > > >
> > > > > Moving to gitbox means you will get full write access on GitHub as
> > > well,
> > > > > and be able to close/merge pull requests and much more.
> > > > >
> > > > > To have your repositories moved, please follow these steps:
> > > > >
> > > > > - Ensure consensus on the move (a link to a lists.apache.org
> thread
> > > will
> > > > >   suffice for us as evidence).
> > > > > - Create a JIRA ticket at
> > https://issues.apache.org/jira/browse/INFRA
> > > > >
> > > > > Your migration should only take a few minutes. If you wish to
> migrate
> > > > > at a specific time of day or date, please do let us know in the
> > ticket.
> > > > >
> > > > > As always, we appreciate your understanding and patience as we move
> > > > > things around and work to provide better services and features for
> > > > > the Apache Family.
> > > > >
> > > > > Should you wish to contact us with feedback or questions, please do
> > so
> > > > > at: us...@infra.apache.org.
> > > > >
> > > > >
> > > > > With regards,
> > > > > Apache Infrastructure
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: [NOTICE] Mandatory migration of git repositories to gitbox.apache.org

[Jim Apple]

+1 to moving ahead of the mass migration.

Any thoughts on this? I can do the paperwork.



On Thu, Jan 3, 2019 at 5:19 AM Apache Infrastructure Team <
infrastruct...@apache.org> wrote:

> Hello, impala folks.
> As stated earlier in 2018, all git repositories must be migrated from
> the git-wip-us.apache.org URL to gitbox.apache.org, as the old service
> is being decommissioned. Your project is receiving this email because
> you still have repositories on git-wip-us that needs to be migrated.
>
> The following repositories on git-wip-us belong to your project:
>  - impala.git
>
>
> We are now entering the mandated (coordinated) move stage of the roadmap,
> and you are asked to please coordinate migration with the Apache
> Infrastructure Team before February 7th. All repositories not migrated
> on February 7th will be mass migrated without warning, and we'd appreciate
> it if we could work together to avoid a big mess that day :-).
>
> Moving to gitbox means you will get full write access on GitHub as well,
> and be able to close/merge pull requests and much more.
>
> To have your repositories moved, please follow these steps:
>
> - Ensure consensus on the move (a link to a lists.apache.org thread will
>   suffice for us as evidence).
> - Create a JIRA ticket at https://issues.apache.org/jira/browse/INFRA
>
> Your migration should only take a few minutes. If you wish to migrate
> at a specific time of day or date, please do let us know in the ticket.
>
> As always, we appreciate your understanding and patience as we move
> things around and work to provide better services and features for
> the Apache Family.
>
> Should you wish to contact us with feedback or questions, please do so
> at: us...@infra.apache.org.
>
>
> With regards,
> Apache Infrastructure
>
>

Re: HiveServer2 client for C++ and Python

[Jim Apple]

Sounds great!

On Tue, Nov 27, 2018 at 6:35 AM Wes McKinney  wrote:

> hi folks,
>
> I noticed impyla and hs2client mentioned on
>
> https://cwiki.apache.org/confluence/display/IMPALA/Impala+Connectors
>
> It's my intent to "adopt" the hs2client C++ library into Apache Arrow
> and eventually ship this as part of the Python library pyarrow. I
> started working on this here:
>
> https://github.com/apache/arrow/tree/master/cpp/src/arrow/dbi/hiveserver2
>
> Hopefully this will eventually lead to decommissioning impyla
> altogether, which has not been maintained by Apache Impala and is both
> slow and buggy.
>
> It would be great to have a first-rate C++ and Python client for HS2,
> with excellent fetch performance (where Arrow will help with this) and
> good support for SSL and Kerberos. I hope to collaborate with the
> Impala community on this.
>
> best
> Wes
>

Re: [VOTE] 3.1.0 release candidate 1

[Jim Apple]

+1 (binding)

I looked over the output of
https://jenkins.impala.io/job/release-test-ub1604/16/console, which checks
all of the things I care about: signature, sha512, equality of tarball and
tree, and the usual pre-merge tests.

On Tue, Nov 27, 2018 at 9:40 AM Zoltan Borok-Nagy 
wrote:

> This is a vote to release Impala 3.1.0
>
> The artifacts for testing can be downloaded from:
> https://dist.apache.org/repos/dist/dev/impala/3.1.0/RC1/
> Git tag: 3.1.0-rc1
> Tree hash: 0fb7b90d5dad7aeedd48db28939b1999a7a3
>
> Please vote +1 or -1. -1 votes should be accompanied by an explanation of
> the reason. Only PMC members have binding votes, but other
> community members are encouraged to cast non-binding votes. This vote will
> pass if there are 3 binding +1 votes and more binding +1 votes than -1
> votes.
>
> This wiki page describes how to check the release before you vote:
> https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release#
>

PMC members: sign up for private@

[Jim Apple]

If you are a member of the PMC, please ensure you are signed up for the
priv...@impala.apache.org mailing list by mailing
private-subscr...@impala.apache.org

Re: jenkins.impala.io DNS expired

[Jim Apple]

Fixed.

On Thu, Nov 15, 2018 at 6:51 PM Jim Apple  wrote:

> Please use 35.164.73.121 in the meantime while we get this fixed.
>

New committer: Zoram Thanga

[Jim Apple]

The Project Management Committee (PMC) for Apache Impala has invited Zoram
Thanga to become a committer and we are pleased to announce that they have
accepted. Congratulations and welcome, Zoram!

jenkins.impala.io DNS expired

[Jim Apple]

Please use 35.164.73.121 in the meantime while we get this fixed.

Re: UBSAN in pre-merge testing

[Jim Apple]

My only concern with this is the unfinished nature of the UBSAN work. Here
are some things left to do:

1. Make UBSAN builds work with FE and JDBC tests. These aren't UBSAN
unclean -- they plain don't work.
2. Make e2e, custom cluster, and BE tests UBSAN clean. This is in progress.
3. Make e2e tests pass with -full_ubsan. This turns UBSAN on during
codegen, and it seems to break some e2e tests.
4. Give -full_ubsan a tolerable run time, probably by sharding the tests.
Right now the core suite takes 7 or 8 hours.

If you get centos6 up and working as part of pre-merge, I'm comfortable
using it as a platform to test UBSAN on as we build more UBSAN coverage.
I'm also happy to try and keep any sharding in #4 distribution-agnostic so
that if UBSAN goes up first, you can easily port it to centos6. With those
two in place, my feeling is that the two improvements we want to make
"commute", in that whichever one we do first, the work and result should be
similar.

What do you think?

On Mon, Nov 12, 2018 at 1:26 PM Philip Zeyliger  wrote:

> On Mon, Nov 12, 2018 at 1:21 PM Jim Apple  wrote:
>
> > I don't think I understand the "one stone" part - are you suggesting that
> > we do UBSAN testing within a centos6 container?
> >
>
> Exactly. If we're doing multiple builds, we may as well be mutating other
> variables to get coverage from them. (Here, you're mutating "build type",
> and I'm proposing we also mutate "base OS" to get additional coverage.)
>
> -- Philip
>
>
>
> >
> > On Mon, Nov 12, 2018 at 1:01 PM Philip Zeyliger 
> > wrote:
> >
> > > Seems useful to me.
> > >
> > > If you're interested, we could kill multiple birds with one stone.
> > > Specifically, I'm also interested in centos6/rh6 pre-merge testing.
> There
> > > are a variety of ways to do so, including running with test-with-docker
> > > stuff. I recognize it's more work, but happy to help if you want to try
> > it.
> > >
> > > -- Philip
> > >
> > > On Sat, Nov 10, 2018 at 11:10 PM Jim Apple 
> wrote:
> > >
> > > > C++ has some constructs that have undefined behavior. Shall we test
> for
> > > > this during pre-merge testing?
> > > >
> > > > When the behavior of C++ code is formally "undefined" by the
> standard,
> > > > compilers can behave erratically, like not taking either branch of a
> > > > if/else statement. This can be reproduced in the wild. The standard
> > > itself
> > > > notes:
> > > >
> > > > "Using a bool value in ways described by this International Standard
> as
> > > > 'undefined,' such as by examining the value of an uninitialized
> > automatic
> > > > object, might cause it to behave as if it is neither true nor false."
> > > >
> > > > Clang has a checker for this called UBSAN, and, after some effort,
> the
> > > data
> > > > loading part of our build is now UBSAN-clean. I'm suggesting we add
> > that
> > > > test to the pre-merge testing. I'm happy to handle the details.
> > > >
> > > > When it fails, the output will look something like this:
> > > >
> > > > https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/3573/console
> > > >
> > > > exprs/math-functions-ir.cc:405:13: runtime error: signed integer
> > > overflow:
> > > > 4738381338321616896 * 36 cannot be represented in type 'long'
> > > > runtime/decimal-value.inline.h:254:17: runtime error: signed integer
> > > > overflow: 0x4b3b4ca85a86c47a098a223f +
> > > > 0x4b3b4ca85a86c47a098a223f cannot be represented in type
> > > '__int128'
> > > > runtime/row-batch-serialize-test.cc:243:18: runtime error: variable
> > > length
> > > > array bound evaluates to non-positive value 0
> > > >
> > >
> >
>

Re: UBSAN in pre-merge testing

[Jim Apple]

I don't think I understand the "one stone" part - are you suggesting that
we do UBSAN testing within a centos6 container?

On Mon, Nov 12, 2018 at 1:01 PM Philip Zeyliger  wrote:

> Seems useful to me.
>
> If you're interested, we could kill multiple birds with one stone.
> Specifically, I'm also interested in centos6/rh6 pre-merge testing. There
> are a variety of ways to do so, including running with test-with-docker
> stuff. I recognize it's more work, but happy to help if you want to try it.
>
> -- Philip
>
> On Sat, Nov 10, 2018 at 11:10 PM Jim Apple  wrote:
>
> > C++ has some constructs that have undefined behavior. Shall we test for
> > this during pre-merge testing?
> >
> > When the behavior of C++ code is formally "undefined" by the standard,
> > compilers can behave erratically, like not taking either branch of a
> > if/else statement. This can be reproduced in the wild. The standard
> itself
> > notes:
> >
> > "Using a bool value in ways described by this International Standard as
> > 'undefined,' such as by examining the value of an uninitialized automatic
> > object, might cause it to behave as if it is neither true nor false."
> >
> > Clang has a checker for this called UBSAN, and, after some effort, the
> data
> > loading part of our build is now UBSAN-clean. I'm suggesting we add that
> > test to the pre-merge testing. I'm happy to handle the details.
> >
> > When it fails, the output will look something like this:
> >
> > https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/3573/console
> >
> > exprs/math-functions-ir.cc:405:13: runtime error: signed integer
> overflow:
> > 4738381338321616896 * 36 cannot be represented in type 'long'
> > runtime/decimal-value.inline.h:254:17: runtime error: signed integer
> > overflow: 0x4b3b4ca85a86c47a098a223f +
> > 0x4b3b4ca85a86c47a098a223f cannot be represented in type
> '__int128'
> > runtime/row-batch-serialize-test.cc:243:18: runtime error: variable
> length
> > array bound evaluates to non-positive value 0
> >
>

UBSAN in pre-merge testing

[Jim Apple]

C++ has some constructs that have undefined behavior. Shall we test for
this during pre-merge testing?

When the behavior of C++ code is formally "undefined" by the standard,
compilers can behave erratically, like not taking either branch of a
if/else statement. This can be reproduced in the wild. The standard itself
notes:

"Using a bool value in ways described by this International Standard as
'undefined,' such as by examining the value of an uninitialized automatic
object, might cause it to behave as if it is neither true nor false."

Clang has a checker for this called UBSAN, and, after some effort, the data
loading part of our build is now UBSAN-clean. I'm suggesting we add that
test to the pre-merge testing. I'm happy to handle the details.

When it fails, the output will look something like this:

https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/3573/console

exprs/math-functions-ir.cc:405:13: runtime error: signed integer overflow:
4738381338321616896 * 36 cannot be represented in type 'long'
runtime/decimal-value.inline.h:254:17: runtime error: signed integer
overflow: 0x4b3b4ca85a86c47a098a223f +
0x4b3b4ca85a86c47a098a223f cannot be represented in type '__int128'
runtime/row-batch-serialize-test.cc:243:18: runtime error: variable length
array bound evaluates to non-positive value 0

Re: [DISCUSS] 3.1.0 release

[Jim Apple]

>
> I just have a technical question about it. Should we
> a) select an early branching point then do a lot of cherry picks for the
> commits we want in and leave out the risky ones
> b) select a recent branching point then revert the risky commits on the
> release branch
>

I think (a) is easier for someone who is doing some git work on the branch,
but our branches tend to be used once for releases and then rarely touched
again, so it's not a disaster to do (b).

Re: [DISCUSS] 3.1.0 release

[Jim Apple]

+1. Thanks for volunteering, Zoltan! Do you concur with Tim about leaving
out some big recent changes, assuming authors request so?

On Mon, Nov 5, 2018 at 9:10 AM Tim Armstrong 
wrote:

> +1
>
> I know that I and some other people have been pushing to burn down the
> critical bugs remaining targeted for 3.1 and push out less critical work to
> 3.2 so we will be in a good position to do a release (it's been a while
> since 3.0!). I have this filter to track the open JIRAs targeted for 3.1:
> https://issues.apache.org/jira/issues/?filter=12345062
>
> We should figure out the best branching point. Most of the recent commits
> should go into the release but there are a few larger changes that we might
> want to take on a case-by-case basis. E.g. I'm not sure if Michael Ho wants
> to give some of the RPC changes more time to bake or not.
>
> Cheers,
> Tim
>
> On Mon, Nov 5, 2018 at 6:51 AM Zoltan Borok-Nagy 
> wrote:
>
> > Hi Folks,
> >
> > It's been a while since we last released a minor version of Impala.
> > 3.0.0 is out since May, and since then a couple of pretty cool features
> and
> > a good number of improvements are checked in.
> >
> > I propose that we release 3.1.0 soon and I volunteer to be its release
> > manager. Please speak up and let the community know if anyone has any
> > objections to this.
> >
> > Thanks,
> > Zoltan
> >
>

Fwd: Impala Board Report due for November 2018 - Initial Reminder

[Jim Apple]

I posted the report below. We can make changes for a couple of weeks, so
let me know any clarifications you have.

## Description:

Impala is a high-performance distributed SQL engine.

## Activity:

The previous three months had 350 patches to the master branch, while this
three-month period had 330.

Prominent work in the last three months includes:

 - Support for multiple DISTINCT

 - The first Apache two-dot release (3.0.1) was made; normally we only do
   x.y.0 releases. This was done to fix two security issues.

 - Official CentOS support for developers.

 - A number of changes to make the C++ code have a reduced number of
undefined
   behaviors.

 - Support for Hadoop's connector for Azure's new storage system,
   "Azure Data Lake Storage Gen2".

 - Multiple improvements in resource estimation and resource management.

 - Continued improvements in "local catalog" mode.

 - The addition of builtin JSON parsing functions.

 - Graceful node shutdown (with drain/quiesce).

## Health report:

The project remains healthy and metrics (number of commits, bugs filed, and
mailing list activity) remain healthy.

Four new contributors had patches committed.

## PMC changes:

 - Currently 27 PMC members.
 - Joe McDonnell was added to the PMC on Mon Aug 20 2018

## Committer base changes:

 - Currently 44 committers.
 - Quanlong Huang was added as a committer on Thu Aug 23 2018

## Releases:

 - 3.0.1 was released on Tue Oct 23 2018

## Mailing list activity:

Mailing lists metrics that held steady:

 - user@: 83 emails sent in the past 3 months, 87 in the previous cycle.

Mailing list metrics that changed more:

 - dev@: 205 emails sent in the past 3 months, 299 in the previous cycle.
   There is no obvious immediate cause and this is likely just statistical
   fluctuation.

 - reviews@: 7312 emails sent in the past 3 months, 5638 in the previous
   cycle.

## JIRA activity:

 - 409 JIRA tickets created in the last 3 months

 - 386 JIRA tickets closed/resolved in the last 3 months
send email


-- Forwarded message -
From: Phil Steitz 
Date: Wed, Oct 31, 2018 at 8:35 PM
Subject: Impala Board Report due for November 2018 - Initial Reminder
To: Jim Apple 
Cc: 


This email was sent on behalf of the ASF Board.  It is an initial reminder
to
give you plenty of time to prepare the report.

According to board records, you are listed as the chair of a committee that
is
due to submit a report this month. [1] [2]

The meeting is scheduled for Wed, 21 Nov 2018 at 10:30 PST and the deadline
for
submitting your report is 1 full week prior to that (Wed Nov 14th)!

Meeting times in other time zones:

  https://timeanddate.com/s/3pdg

Please submit your report with sufficient time to allow the board members
to review and digest. Again, the very latest you should submit your report
is 1 full week (7days) prior to the board meeting (Wed Nov 14th).

If you feel that an error has been made, please consult [1] and if there
is still an issue then contact the board directly.

As always, PMC chairs are welcome to attend the board meeting.

Thanks,
The ASF Board

[1] -
https://svn.apache.org/repos/private/committers/board/committee-info.txt
[2] - https://svn.apache.org/repos/private/committers/board/calendar.txt
[3] - https://svn.apache.org/repos/private/committers/board/templates
[4] - https://reporter.apache.org/


Submitting your Report
--

Full details about the process and schedule are in [1]. Details about what a
report should look like can be found at:

  https://www.apache.org/foundation/board/reporting

Reports should be posted using the online agenda tool:

  https://whimsy.apache.org/board/agenda/2018-11-21/Impala

Cutting and pasting directly from a Wiki is not acceptable due to formatting
issues. Line lengths should be limited to 77 characters.  If submitting
using Whimsy please press Reflow before Submit.

Chairs may use the Apache Reporter Service [4] to help them compile and
submit a board report.


Resolutions
---

There are several templates for use for various Board resolutions.
They can be found in [3] and you are encouraged to use them. It is
strongly recommended that if you have a resolution before the board,
you are encouraged to attend that board meeting.