[Impala-ASF-CR] IMPALA-2195: Improper handling of comments in queries

[Fredy Wijaya (Code Review)]

Fredy Wijaya has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9933 )

Change subject: IMPALA-2195: Improper handling of comments in queries
..


Patch Set 7:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9933/3/shell/impala_shell.py
File shell/impala_shell.py:

http://gerrit.cloudera.org:8080/#/c/9933/3/shell/impala_shell.py@1279
PS3, Line 1279: self._replace_history_delimiters('\n', 
ImpalaShell.HISTORY_FILE_QUERY_DELIM)
> Right. Currently, if you see the web UI of the running/completed queries, i
Sorry for the late reply. I was busy with some other tasks. Your concern is 
totally valid. I have an updated PR that is hopefully fixes the bug as well as 
addressing your concern.

Upon further investigation, apparently any non front-end commands, e.g. 
SUMMARY, HELP, etc. that start with a leading comment will result in a syntax 
error because it calls the default() method which sends the command to the 
front-end and front-end has no knowledge about those commands.



--
To view, visit http://gerrit.cloudera.org:8080/9933
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7ac7cb5a30e6dda73ebe761d9f0eb9ba038e14a7
Gerrit-Change-Number: 9933
Gerrit-PatchSet: 7
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Comment-Date: Wed, 18 Apr 2018 05:51:56 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-2195: Improper handling of comments in queries

[Fredy Wijaya (Code Review)]

Fredy Wijaya has uploaded a new patch set (#7). ( 
http://gerrit.cloudera.org:8080/9933 )

Change subject: IMPALA-2195: Improper handling of comments in queries
..

IMPALA-2195: Improper handling of comments in queries

This patch fixes an issue where parseline is unable to determine the
correct command when a statement has a leading comment.

Before:
> -- comment
> insert into table t values(100);
Fetched 1 row(s) in 0.01s

After:
> -- comment
> insert into table t values(100);
Modified 1 row(s) in 0.01s

Before (FE syntax error):
> /*comment*/ help;

After (show help correctly):
> /*comment*/ help;

Testing:
- Added shell tests
- Ran end-to-end shell tests

Change-Id: I7ac7cb5a30e6dda73ebe761d9f0eb9ba038e14a7
---
M shell/impala_shell.py
M tests/shell/test_shell_interactive.py
2 files changed, 87 insertions(+), 11 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/9933/7
--
To view, visit http://gerrit.cloudera.org:8080/9933
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7ac7cb5a30e6dda73ebe761d9f0eb9ba038e14a7
Gerrit-Change-Number: 9933
Gerrit-PatchSet: 7
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Taras Bobrovytsky 

[Impala-ASF-CR] IMPALA-6749: Implement ALTER DATABASE SET DBPROPERTIES

[Fredy Wijaya (Code Review)]

Fredy Wijaya has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10085 )

Change subject: IMPALA-6749: Implement ALTER DATABASE SET DBPROPERTIES
..


Patch Set 4:

I'll abandon this PR then.


--
To view, visit http://gerrit.cloudera.org:8080/10085
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I08bc4a9c6fa95dd135274fa1081bf24ebe14c99a
Gerrit-Change-Number: 10085
Gerrit-PatchSet: 4
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Wed, 18 Apr 2018 05:40:35 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6749: Implement ALTER DATABASE SET DBPROPERTIES

[Fredy Wijaya (Code Review)]

Fredy Wijaya has abandoned this change. ( http://gerrit.cloudera.org:8080/10085 
)

Change subject: IMPALA-6749: Implement ALTER DATABASE SET DBPROPERTIES
..


Abandoned
--
To view, visit http://gerrit.cloudera.org:8080/10085
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: I08bc4a9c6fa95dd135274fa1081bf24ebe14c99a
Gerrit-Change-Number: 10085
Gerrit-PatchSet: 4
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Philip Zeyliger 

[Impala-ASF-CR] IMPALA-6749: Implement ALTER DATABASE SET DBPROPERTIES

[Alex Behm (Code Review)]

Alex Behm has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10085 )

Change subject: IMPALA-6749: Implement ALTER DATABASE SET DBPROPERTIES
..


Patch Set 4:

I agree with some comments but not with others. I don't think the fear of abuse 
is rooted in any evidence. I have yet to see a single user that has seriously 
abused the TBLPROPERTIES mechanism. Really, we are the only ones abusing these 
properties, but we'll continue to do that regardless of whether we have this 
ALTER DATABASE or not.

That said, I think it's fair to say that there's no real use case for this 
feature today, so it might not be worth the added complexity.


--
To view, visit http://gerrit.cloudera.org:8080/10085
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I08bc4a9c6fa95dd135274fa1081bf24ebe14c99a
Gerrit-Change-Number: 10085
Gerrit-PatchSet: 4
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Wed, 18 Apr 2018 05:23:09 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6862: Privilege.java needs to support Sentry 1.5.1 and 2.0.0

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10082 )

Change subject: IMPALA-6862: Privilege.java needs to support Sentry 1.5.1 and 
2.0.0
..


Patch Set 5:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2327/


--
To view, visit http://gerrit.cloudera.org:8080/10082
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibbf4eeab3927ff93ed51066b164b8874d5f76e23
Gerrit-Change-Number: 10082
Gerrit-PatchSet: 5
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Wed, 18 Apr 2018 04:08:29 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6862: Privilege.java needs to support Sentry 1.5.1 and 2.0.0

[Alex Behm (Code Review)]

Alex Behm has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10082 )

Change subject: IMPALA-6862: Privilege.java needs to support Sentry 1.5.1 and 
2.0.0
..


Patch Set 5: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/10082
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibbf4eeab3927ff93ed51066b164b8874d5f76e23
Gerrit-Change-Number: 10082
Gerrit-PatchSet: 5
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Wed, 18 Apr 2018 04:07:46 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6862: Privilege.java needs to support Sentry 1.5.1 and 2.0.0

[Fredy Wijaya (Code Review)]

Fredy Wijaya has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10082 )

Change subject: IMPALA-6862: Privilege.java needs to support Sentry 1.5.1 and 
2.0.0
..


Patch Set 5:

GVO failed because of the PyPI redirect error. I just rebased the PR. Can 
anyone give a +2 and run the GVO again?


--
To view, visit http://gerrit.cloudera.org:8080/10082
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibbf4eeab3927ff93ed51066b164b8874d5f76e23
Gerrit-Change-Number: 10082
Gerrit-PatchSet: 5
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Wed, 18 Apr 2018 04:07:11 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6862: Privilege.java needs to support Sentry 1.5.1 and 2.0.0

[Fredy Wijaya (Code Review)]

Fredy Wijaya has uploaded a new patch set (#5). ( 
http://gerrit.cloudera.org:8080/10082 )

Change subject: IMPALA-6862: Privilege.java needs to support Sentry 1.5.1 and 
2.0.0
..

IMPALA-6862: Privilege.java needs to support Sentry 1.5.1 and 2.0.0

BitFieldAction is an asbract class in Sentry 1.5.1, but it is no longer
an abstract class in Sentry 2.0.0. This patch fixes the issue by making
Privilege.java support both versions of Sentry.

Testing:
- Ran all front-end tests with profile=2 and profile=3

Cherry-picks: not for 2.x

Change-Id: Ibbf4eeab3927ff93ed51066b164b8874d5f76e23
---
M fe/pom.xml
R 
fe/src/compat-minicluster-profile-3/test/java/org/apache/impala/authorization/ImpalaActionFactoryTest.java
M fe/src/main/java/org/apache/impala/authorization/Privilege.java
3 files changed, 22 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/10082/5
--
To view, visit http://gerrit.cloudera.org:8080/10082
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ibbf4eeab3927ff93ed51066b164b8874d5f76e23
Gerrit-Change-Number: 10082
Gerrit-PatchSet: 5
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 

[Impala-ASF-CR] IMPALA-6867: [DOCS] SHUFFLE DISTINCT EXPRS Query option added

[Alex Rodoni (Code Review)]

Alex Rodoni has uploaded a new patch set (#2). ( 
http://gerrit.cloudera.org:8080/10094 )

Change subject: IMPALA-6867: [DOCS] SHUFFLE_DISTINCT_EXPRS Query option added
..

IMPALA-6867: [DOCS] SHUFFLE_DISTINCT_EXPRS Query option added

Change-Id: I42cb22a13b26bdc3d22585e4fc23934dfc91722a
---
M docs/impala.ditamap
M docs/impala_keydefs.ditamap
A docs/topics/impala_shuffle_distinct_exprs.xml
3 files changed, 58 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/10094/2
--
To view, visit http://gerrit.cloudera.org:8080/10094
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I42cb22a13b26bdc3d22585e4fc23934dfc91722a
Gerrit-Change-Number: 10094
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni 

[Impala-ASF-CR](2.x) IMPALA-6822: Add a query option to control shuffling by distinct exprs

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10087 )

Change subject: IMPALA-6822: Add a query option to control shuffling by 
distinct exprs
..


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2326/


--
To view, visit http://gerrit.cloudera.org:8080/10087
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: comment
Gerrit-Change-Id: Icb4b4576fb29edd62cf4b4ba0719c0e0a2a5a8dc
Gerrit-Change-Number: 10087
Gerrit-PatchSet: 2
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Comment-Date: Wed, 18 Apr 2018 03:04:49 +
Gerrit-HasComments: No

[Impala-ASF-CR](2.x) IMPALA-6822: Add a query option to control shuffling by distinct exprs

[Joe McDonnell (Code Review)]

Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10087 )

Change subject: IMPALA-6822: Add a query option to control shuffling by 
distinct exprs
..


Patch Set 2: Code-Review+2

Rebased to get PyPi fix ( https://gerrit.cloudera.org/#/c/10093/ ). Carry +2.


--
To view, visit http://gerrit.cloudera.org:8080/10087
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: comment
Gerrit-Change-Id: Icb4b4576fb29edd62cf4b4ba0719c0e0a2a5a8dc
Gerrit-Change-Number: 10087
Gerrit-PatchSet: 2
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Comment-Date: Wed, 18 Apr 2018 03:03:54 +
Gerrit-HasComments: No

[Impala-ASF-CR](2.x) IMPALA-6863: Make pip download.py honor redirects.

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10093 )

Change subject: IMPALA-6863: Make pip_download.py honor redirects.
..


Patch Set 1: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/10093
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7551cec43a2d378df7e3cc7d521ace338b56ba2
Gerrit-Change-Number: 10093
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Wed, 18 Apr 2018 02:41:19 +
Gerrit-HasComments: No

[Impala-ASF-CR](2.x) IMPALA-6863: Make pip download.py honor redirects.

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/10093 )

Change subject: IMPALA-6863: Make pip_download.py honor redirects.
..

IMPALA-6863: Make pip_download.py honor redirects.

As part of our continuing woes with PyPi infrastructure, we've now seen
redirects. Following redirects seems like the right thing to do, so I've
changed the downloader code to follow them.

I checked that this is available in Python 2.6.

The build failure signature looks like:

   Downloading AllPairs-2.0.1.tar.gz from 
https://pypi.python.org/packages/04/44/cb85d029b33ddfa7b5a27ae523434c2230f11a89d1f0c73203e9007a6eb0/AllPairs-2.0.1.tar.gz
   ('http error', 302, 'Found', )
   Download failed after several attempts.
   Warning: Unable to download Python requirements.
   Warning: bootstrap_virtualenv or other Python-based tooling may fail.

Change-Id: Ic7551cec43a2d378df7e3cc7d521ace338b56ba2
Reviewed-on: http://gerrit.cloudera.org:8080/10083
Reviewed-by: Philip Zeyliger 
Tested-by: Philip Zeyliger 
Reviewed-by: Lars Volker 
(cherry picked from commit eaf66172df113dbf10cdb0a08a2bc51e4077ca38)
Reviewed-on: http://gerrit.cloudera.org:8080/10093
Tested-by: Impala Public Jenkins 
---
M infra/python/deps/pip_download.py
1 file changed, 2 insertions(+), 2 deletions(-)

Approvals:
  Philip Zeyliger: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/10093
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic7551cec43a2d378df7e3cc7d521ace338b56ba2
Gerrit-Change-Number: 10093
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Philip Zeyliger 

[Impala-ASF-CR] IMPALA-5690: Part 2: Upgrade thrift to 0.9.3-p3

[Tianyi Wang (Code Review)]

Tianyi Wang has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9300 )

Change subject: IMPALA-5690: Part 2: Upgrade thrift to 0.9.3-p3
..


Patch Set 11:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9300/11/be/src/rpc/thrift-server-test.cc
File be/src/rpc/thrift-server-test.cc:

http://gerrit.cloudera.org:8080/#/c/9300/11/be/src/rpc/thrift-server-test.cc@362
PS11, Line 362:   {TLSv1_1, {SSLTLS, TLSv1_1}},
> I realized I didn't think it through.
I think if we take option 1, the implementation should be:
When impala initializes, read ssl_minimum_version and disable versions lower 
than the minimum process-wide. Then always use SSLTLS to initialize ssl 
context. This way we don't need to patch thrift. And this test needs to be 
refactored a little.



--
To view, visit http://gerrit.cloudera.org:8080/9300
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I639227721502eaa10398d9490ff6ac63aa71b3a6
Gerrit-Change-Number: 9300
Gerrit-PatchSet: 11
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: anujphadke 
Gerrit-Comment-Date: Wed, 18 Apr 2018 02:25:22 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-5690: Part 2: Upgrade thrift to 0.9.3-p3

[Tianyi Wang (Code Review)]

Tianyi Wang has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9300 )

Change subject: IMPALA-5690: Part 2: Upgrade thrift to 0.9.3-p3
..


Patch Set 11:

(4 comments)

Sailesh, can you have a look at the TLS behavior in thrift-server-test.cc?

http://gerrit.cloudera.org:8080/#/c/9300/11/CMakeLists.txt
File CMakeLists.txt:

http://gerrit.cloudera.org:8080/#/c/9300/11/CMakeLists.txt@159
PS11, Line 159: 1.0.1
> Is the plan to also cherry-pick the change to 2.x branch ?
I think the current consensus is that it's not for 2.x.


http://gerrit.cloudera.org:8080/#/c/9300/11/be/src/common/init.cc
File be/src/common/init.cc:

http://gerrit.cloudera.org:8080/#/c/9300/11/be/src/common/init.cc@240
PS11, Line 240: signal(SIGPIPE, SIG_IGN);
> Why is this suddenly needed as part of the upgrade to thrift 0.9.3 ?
I hit this when running tests and didn't look deeper into it. I believe it's 
always needed. (As an example, kudu always ignores this signal.)


http://gerrit.cloudera.org:8080/#/c/9300/11/be/src/rpc/thrift-server-test.cc
File be/src/rpc/thrift-server-test.cc:

http://gerrit.cloudera.org:8080/#/c/9300/11/be/src/rpc/thrift-server-test.cc@362
PS11, Line 362:   {TLSv1_1, {SSLTLS, TLSv1_1}},
> Should this also include TLSv1_0 ?
I realized I didn't think it through.
The current behavior of this test in this patch doesn't make sense but I don't 
really know what behavior we want here. TLSv*_plus are removed in thrift 0.9.3. 
We have several options here:
1. Make TLSv* behave like TLSv*_plus in 0.9.0. In that case, any combination 
would work in this test, because the client and server can always agree on 
TLSv1_2.
2. Patch thrift and add Tlsv*_plus back.

The current behavior in this patch is that TLSv1_1 can only connect to TLSv1_1, 
while SSLTLS can connect to anything. This is different from impala's existing 
behavior and should be changed.
I think option 1 is OK. I think we want the configuration to be flexible but 
respects the tls minimum version constraint, and option 1 should deliver that.


http://gerrit.cloudera.org:8080/#/c/9300/11/common/thrift/CMakeLists.txt
File common/thrift/CMakeLists.txt:

http://gerrit.cloudera.org:8080/#/c/9300/11/common/thrift/CMakeLists.txt@68
PS11, Line 68: "ImpalaService.thrift"
> So, is the generated java file of ImpalaService.thrift not needed in the FE
That file defines the impalad RPC interface and is not used in FE.



--
To view, visit http://gerrit.cloudera.org:8080/9300
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I639227721502eaa10398d9490ff6ac63aa71b3a6
Gerrit-Change-Number: 9300
Gerrit-PatchSet: 11
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: anujphadke 
Gerrit-Comment-Date: Wed, 18 Apr 2018 01:59:04 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6862: Privilege.java needs to support Sentry 1.5.1 and 2.0.0

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10082 )

Change subject: IMPALA-6862: Privilege.java needs to support Sentry 1.5.1 and 
2.0.0
..


Patch Set 4: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2320/


--
To view, visit http://gerrit.cloudera.org:8080/10082
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibbf4eeab3927ff93ed51066b164b8874d5f76e23
Gerrit-Change-Number: 10082
Gerrit-PatchSet: 4
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Wed, 18 Apr 2018 01:28:56 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6847: work around high memory estimates for AC

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/10058 )

Change subject: IMPALA-6847: work around high memory estimates for AC
..

IMPALA-6847: work around high memory estimates for AC

Adds MAX_MEM_ESTIMATE_FOR_ADMISSION query option, which takes
effect if and only if
* Memory-based admission control is enabled for the pool
* No mem_limit is set (i.e. best practices are not being followed)

In that case min(MAX_MEM_ESTIMATE_FOR_ADMISSION, mem_estimate)
is used for admission control instead of mem_estimate.

This provides a way to override the planner's estimate if
it happens to be incorrect and are preventing the query from
running. Setting MEM_LIMIT is usually a better alternative
but sometimes it is not feasible to set MEM_LIMIT for each
individual query.

Testing:
Added an admission control test to verify that query option allows
queries with high estimates to run.

Also tested manually on a minicluster started with:

  start-impala-cluster.py --impalad_args='-vmodule admission-controller=3 \
  -default_pool_mem_limit 12884901888'

Change-Id: Ia5fc32a507ad0f00f564dfe4f954a829ac55d14e
Reviewed-on: http://gerrit.cloudera.org:8080/10058
Reviewed-by: Tim Armstrong 
Tested-by: Impala Public Jenkins 
---
M be/src/service/query-options-test.cc
M be/src/service/query-options.cc
M be/src/service/query-options.h
M common/thrift/ImpalaInternalService.thrift
M common/thrift/ImpalaService.thrift
M fe/src/main/java/org/apache/impala/service/Frontend.java
A 
testdata/workloads/functional-query/queries/QueryTest/admission-reject-mem-estimate.test
M tests/custom_cluster/test_admission_controller.py
8 files changed, 135 insertions(+), 11 deletions(-)

Approvals:
  Tim Armstrong: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/10058
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia5fc32a507ad0f00f564dfe4f954a829ac55d14e
Gerrit-Change-Number: 10058
Gerrit-PatchSet: 15
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 

[Impala-ASF-CR] IMPALA-6847: work around high memory estimates for AC

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10058 )

Change subject: IMPALA-6847: work around high memory estimates for AC
..


Patch Set 14: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/10058
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia5fc32a507ad0f00f564dfe4f954a829ac55d14e
Gerrit-Change-Number: 10058
Gerrit-PatchSet: 14
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 18 Apr 2018 01:18:19 +
Gerrit-HasComments: No

[Impala-ASF-CR] Add new queries to targeted-perf workload

[Mostafa Mokhtar (Code Review)]

Mostafa Mokhtar has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9979 )

Change subject: Add new queries to targeted-perf workload
..


Patch Set 3:

Please add a description to the change describing the high level goals behind 
the queries.


--
To view, visit http://gerrit.cloudera.org:8080/9979
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c415924d0bb6da1b1f5df6cb16b95a1d2eaa3ab
Gerrit-Change-Number: 9979
Gerrit-PatchSet: 3
Gerrit-Owner: Nithya Janarthanan 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Mostafa Mokhtar 
Gerrit-Comment-Date: Wed, 18 Apr 2018 00:55:54 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6790: Upgrade sqlparse to 0.1.19

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/10044 )

Change subject: IMPALA-6790: Upgrade sqlparse to 0.1.19
..

IMPALA-6790: Upgrade sqlparse to 0.1.19

Some remote cluster tests have failed to load data
due to sqlparse failing to split SQL statements
appropriately. The SQL file itself is identical
to our usual dataload, so it must be a unique
environment. The current version of sqlparse is
0.1.15.

This upgrades sqlparse to 0.1.19. When running on
the same environment with the newer version,
the problem does not occur. Note that this
is not the version used for the Impala shell.
Impala shell has sqlparse checked-in under
shell/ext-py.

Change-Id: Ic5289f86b78f1d77d91a8fa47d63b7a7eaa3af38
Reviewed-on: http://gerrit.cloudera.org:8080/10044
Reviewed-by: Joe McDonnell 
Tested-by: Impala Public Jenkins 
---
M infra/python/deps/requirements.txt
1 file changed, 4 insertions(+), 1 deletion(-)

Approvals:
  Joe McDonnell: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/10044
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic5289f86b78f1d77d91a8fa47d63b7a7eaa3af38
Gerrit-Change-Number: 10044
Gerrit-PatchSet: 4
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 

[Impala-ASF-CR] IMPALA-6790: Upgrade sqlparse to 0.1.19

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10044 )

Change subject: IMPALA-6790: Upgrade sqlparse to 0.1.19
..


Patch Set 3: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/10044
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic5289f86b78f1d77d91a8fa47d63b7a7eaa3af38
Gerrit-Change-Number: 10044
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 18 Apr 2018 00:38:26 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6679,IMPALA-6678: reduce scan reservation

[Dan Hecht (Code Review)]

Dan Hecht has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9757 )

Change subject: IMPALA-6679,IMPALA-6678: reduce scan reservation
..


Patch Set 6:

(17 comments)

Will send backend comments separately.

http://gerrit.cloudera.org:8080/#/c/9757/6//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/9757/6//COMMIT_MSG@37
PS6, Line 37: ideal reservation
above you say we no longer need a concept of ideal reservation -- so what is 
this referring to? I guess here you are talking about the ideal determined at 
runtime once we examine the footer?


http://gerrit.cloudera.org:8080/#/c/9757/6/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
File fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java:

http://gerrit.cloudera.org:8080/#/c/9757/6/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@150
PS6, Line 150: backend
IOMgr is the terminology we use below (and I think that's clearer).


http://gerrit.cloudera.org:8080/#/c/9757/6/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@157
PS6, Line 157: FOOTER_SIZE
maybe name both the same so that a grep will also find both.


http://gerrit.cloudera.org:8080/#/c/9757/6/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@1264
PS6, Line 1264: fileFormats_.contains(HdfsFileFormat.PARQUET)
> will estimates be biased towards parquet if there are multiple formats?
Do we have enough testing for the multiple format case when it comes to 
reservations?


http://gerrit.cloudera.org:8080/#/c/9757/6/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@1320
PS6, Line 1320: Bound the reservation
  :*  based on:
is the following list a lower bound, upper bound, or both?


http://gerrit.cloudera.org:8080/#/c/9757/6/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@1326
PS6, Line 1326: .
and is available. or .. try to increase ..


http://gerrit.cloudera.org:8080/#/c/9757/6/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@1337
PS6, Line 1337: columnReservations != null
since != and == have same precedence, let's parenthesize this for clarity.


http://gerrit.cloudera.org:8080/#/c/9757/6/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@1373
PS6, Line 1373:
memory
(in the future we could imagine other resource reservations)


http://gerrit.cloudera.org:8080/#/c/9757/6/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@1374
PS6, Line 1374:  Returns one value per column.
like you say below, these values aren't yet quantized the buffer sizes, right?


http://gerrit.cloudera.org:8080/#/c/9757/6/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@1378
PS6, Line 1378:* logic for non-shredded columnar formats if added.
generally, do we need to worry about orc for this patch?


http://gerrit.cloudera.org:8080/#/c/9757/6/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@1394
PS6, Line 1394: // Not a top-level column, e.g. a nested collection 
- no stats available.
what does this case mean? scalar but not top level but also different than the 
case at line 1400? is this the array/list case?


http://gerrit.cloudera.org:8080/#/c/9757/6/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@1417
PS6, Line 1417: compute
since this one appends (rather than returns like the other functions) maybe 
calll it addMinColumnReservation() or appendMinColumnReservation().


http://gerrit.cloudera.org:8080/#/c/9757/6/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@1440
PS6, Line 1440: column
scalar column?


http://gerrit.cloudera.org:8080/#/c/9757/6/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@1448
PS6, Line 1448: computeMinColumnReservation
should this be computeMinColumnReservationForScalar() or similar?


http://gerrit.cloudera.org:8080/#/c/9757/6/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@1469
PS6, Line 1469: return estimate;
have you verified how close this comes out to the actual size for various 
datatypes for some real column chunks? is it worth adding some counter to the 
profile for the actual size to debug cases where the reservation estimate was 
way off, or is that overkill?


http://gerrit.cloudera.org:8080/#/c/9757/6/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@1472
PS6, Line 1472:   private static long roundUpToIoBuffer(long bytes, long 
maxIoBufferSize) {
add a quick function comment.


http://gerrit.cloudera.org:8080/#/c/9757/6/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@1475
PS6, Line 1475: BitUtil.roundUpToPowerOf2Factor(bytes, maxIoBufferSize);
why is it the right thing to always have a multiple of maxIoBufferSize in this 
case? i think that's worth explaining briefly.



--
To view, visit http://gerrit.cloudera.org:8080/9757
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master

[Impala-ASF-CR](2.x) IMPALA-6863: Make pip download.py honor redirects.

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10093 )

Change subject: IMPALA-6863: Make pip_download.py honor redirects.
..


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2325/


--
To view, visit http://gerrit.cloudera.org:8080/10093
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7551cec43a2d378df7e3cc7d521ace338b56ba2
Gerrit-Change-Number: 10093
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Tue, 17 Apr 2018 22:43:55 +
Gerrit-HasComments: No

[Impala-ASF-CR](2.x) IMPALA-6863: Make pip download.py honor redirects.

[Philip Zeyliger (Code Review)]

Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10093 )

Change subject: IMPALA-6863: Make pip_download.py honor redirects.
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/10093
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7551cec43a2d378df7e3cc7d521ace338b56ba2
Gerrit-Change-Number: 10093
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Tue, 17 Apr 2018 22:41:40 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6793: Fix empty metadata after statestore restarts

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9921 )

Change subject: IMPALA-6793: Fix empty metadata after statestore restarts
..


Patch Set 9:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2324/


--
To view, visit http://gerrit.cloudera.org:8080/9921
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I332a60e172af84b93b3544373fe363cdced5e8d0
Gerrit-Change-Number: 9921
Gerrit-PatchSet: 9
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Dimitris Tsirogiannis 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Tue, 17 Apr 2018 22:40:16 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6793: Fix empty metadata after statestore restarts

[Michael Brown (Code Review)]

Michael Brown has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9921 )

Change subject: IMPALA-6793: Fix empty metadata after statestore restarts
..


Patch Set 9: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/9921
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I332a60e172af84b93b3544373fe363cdced5e8d0
Gerrit-Change-Number: 9921
Gerrit-PatchSet: 9
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Dimitris Tsirogiannis 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Tue, 17 Apr 2018 22:38:33 +
Gerrit-HasComments: No

[Impala-ASF-CR](2.x) IMPALA-6863: Make pip download.py honor redirects.

[Joe McDonnell (Code Review)]

Hello Lars Volker, Philip Zeyliger,

I'd like you to do a code review. Please visit

http://gerrit.cloudera.org:8080/10093

to review the following change.


Change subject: IMPALA-6863: Make pip_download.py honor redirects.
..

IMPALA-6863: Make pip_download.py honor redirects.

As part of our continuing woes with PyPi infrastructure, we've now seen
redirects. Following redirects seems like the right thing to do, so I've
changed the downloader code to follow them.

I checked that this is available in Python 2.6.

The build failure signature looks like:

   Downloading AllPairs-2.0.1.tar.gz from 
https://pypi.python.org/packages/04/44/cb85d029b33ddfa7b5a27ae523434c2230f11a89d1f0c73203e9007a6eb0/AllPairs-2.0.1.tar.gz
   ('http error', 302, 'Found', )
   Download failed after several attempts.
   Warning: Unable to download Python requirements.
   Warning: bootstrap_virtualenv or other Python-based tooling may fail.

Change-Id: Ic7551cec43a2d378df7e3cc7d521ace338b56ba2
Reviewed-on: http://gerrit.cloudera.org:8080/10083
Reviewed-by: Philip Zeyliger 
Tested-by: Philip Zeyliger 
Reviewed-by: Lars Volker 
(cherry picked from commit eaf66172df113dbf10cdb0a08a2bc51e4077ca38)
---
M infra/python/deps/pip_download.py
1 file changed, 2 insertions(+), 2 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/93/10093/1
--
To view, visit http://gerrit.cloudera.org:8080/10093
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic7551cec43a2d378df7e3cc7d521ace338b56ba2
Gerrit-Change-Number: 10093
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Philip Zeyliger 

[Impala-ASF-CR](2.x) IMPALA-6863: Make pip download.py honor redirects.

[Joe McDonnell (Code Review)]

Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10093 )

Change subject: IMPALA-6863: Make pip_download.py honor redirects.
..


Patch Set 1:

Clean cherry-pick. 2.x is hitting the same PyPi issue.


--
To view, visit http://gerrit.cloudera.org:8080/10093
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7551cec43a2d378df7e3cc7d521ace338b56ba2
Gerrit-Change-Number: 10093
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Tue, 17 Apr 2018 22:37:07 +
Gerrit-HasComments: No

[Impala-ASF-CR](2.x) IMPALA-6822: Add a query option to control shuffling by distinct exprs

[Joe McDonnell (Code Review)]

Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10087 )

Change subject: IMPALA-6822: Add a query option to control shuffling by 
distinct exprs
..


Patch Set 1:

Job failed on pypi stuff. Rebasing.


--
To view, visit http://gerrit.cloudera.org:8080/10087
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: comment
Gerrit-Change-Id: Icb4b4576fb29edd62cf4b4ba0719c0e0a2a5a8dc
Gerrit-Change-Number: 10087
Gerrit-PatchSet: 1
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Comment-Date: Tue, 17 Apr 2018 22:28:57 +
Gerrit-HasComments: No

[Impala-ASF-CR](2.x) IMPALA-6822: Add a query option to control shuffling by distinct exprs

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10087 )

Change subject: IMPALA-6822: Add a query option to control shuffling by 
distinct exprs
..


Patch Set 1: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2319/


--
To view, visit http://gerrit.cloudera.org:8080/10087
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: comment
Gerrit-Change-Id: Icb4b4576fb29edd62cf4b4ba0719c0e0a2a5a8dc
Gerrit-Change-Number: 10087
Gerrit-PatchSet: 1
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Comment-Date: Tue, 17 Apr 2018 22:28:15 +
Gerrit-HasComments: No

[Impala-ASF-CR] Add new queries to targeted-perf workload

[Nithya Janarthanan (Code Review)]

Nithya Janarthanan has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/9979


Change subject: Add new queries to targeted-perf workload
..

Add new queries to targeted-perf workload

Change-Id: I5c415924d0bb6da1b1f5df6cb16b95a1d2eaa3ab
---
A testdata/workloads/targeted-perf/queries/primitive_conjunct_ordering_1.test
A testdata/workloads/targeted-perf/queries/primitive_conjunct_ordering_2.test
A testdata/workloads/targeted-perf/queries/primitive_conjunct_ordering_3.test
A testdata/workloads/targeted-perf/queries/primitive_conjunct_ordering_4.test
A testdata/workloads/targeted-perf/queries/primitive_count_star.test
A testdata/workloads/targeted-perf/queries/primitive_decimal_arithmetic.test
A testdata/workloads/targeted-perf/queries/primitive_filter_bigint_in_list.test
A testdata/workloads/targeted-perf/queries/primitive_intrinsic_appx_median.test
A testdata/workloads/targeted-perf/queries/primitive_intrinsic_to_date.test
M testdata/workloads/targeted-perf/queries/primitive_long_predicate.test
A testdata/workloads/targeted-perf/queries/primitive_many_fragments.test
A 
testdata/workloads/targeted-perf/queries/primitive_many_independent_fragments.test
A 
testdata/workloads/targeted-perf/queries/primitive_orderby_bigint_expression.test
A testdata/workloads/targeted-perf/queries/primitive_shuffle_1mb_rows.test
14 files changed, 675 insertions(+), 171 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/79/9979/3
--
To view, visit http://gerrit.cloudera.org:8080/9979
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I5c415924d0bb6da1b1f5df6cb16b95a1d2eaa3ab
Gerrit-Change-Number: 9979
Gerrit-PatchSet: 3
Gerrit-Owner: Nithya Janarthanan 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Mostafa Mokhtar 

[Impala-ASF-CR] IMPALA-6733: [DOCS] Impala 3.0 Release Notes

[Alex Rodoni (Code Review)]

Alex Rodoni has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10090 )

Change subject: IMPALA-6733: [DOCS] Impala 3.0 Release Notes
..


Patch Set 3:

> (1 comment)

Done.
However, there is a separate review for just the version number.


--
To view, visit http://gerrit.cloudera.org:8080/10090
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I971323993aa6dced0791476e3e798d2f246fab5c
Gerrit-Change-Number: 10090
Gerrit-PatchSet: 3
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Tue, 17 Apr 2018 21:36:46 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6733: [DOCS] Impala 3.0 Release Notes

[Alex Rodoni (Code Review)]

Hello Sailesh Mukil, Impala Public Jenkins, Vuk Ercegovac,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10090

to look at the new patch set (#3).

Change subject: IMPALA-6733: [DOCS] Impala 3.0 Release Notes
..

IMPALA-6733: [DOCS] Impala 3.0 Release Notes

Release Version was updated in ImpalaVariables.xml.

Change-Id: I971323993aa6dced0791476e3e798d2f246fab5c
Cherry-picks: not for 2.x.
---
M docs/shared/ImpalaVariables.xml
M docs/topics/impala_fixed_issues.xml
M docs/topics/impala_incompatible_changes.xml
M docs/topics/impala_known_issues.xml
M docs/topics/impala_new_features.xml
5 files changed, 46 insertions(+), 1,268 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/10090/3
--
To view, visit http://gerrit.cloudera.org:8080/10090
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I971323993aa6dced0791476e3e798d2f246fab5c
Gerrit-Change-Number: 10090
Gerrit-PatchSet: 3
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Vuk Ercegovac 

[Impala-ASF-CR] IMPALA-6847: work around high memory estimates for AC

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10058 )

Change subject: IMPALA-6847: work around high memory estimates for AC
..


Patch Set 14:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2323/


--
To view, visit http://gerrit.cloudera.org:8080/10058
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia5fc32a507ad0f00f564dfe4f954a829ac55d14e
Gerrit-Change-Number: 10058
Gerrit-PatchSet: 14
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Tue, 17 Apr 2018 21:34:54 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6847: work around high memory estimates for AC

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10058 )

Change subject: IMPALA-6847: work around high memory estimates for AC
..


Patch Set 14: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/10058
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia5fc32a507ad0f00f564dfe4f954a829ac55d14e
Gerrit-Change-Number: 10058
Gerrit-PatchSet: 14
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Tue, 17 Apr 2018 21:34:45 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6847: work around high memory estimates for AC

[Bikramjeet Vig (Code Review)]

Bikramjeet Vig has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10058 )

Change subject: IMPALA-6847: work around high memory estimates for AC
..


Patch Set 13: Code-Review+2

Looks good to me. Carrying over +2


--
To view, visit http://gerrit.cloudera.org:8080/10058
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia5fc32a507ad0f00f564dfe4f954a829ac55d14e
Gerrit-Change-Number: 10058
Gerrit-PatchSet: 13
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Tue, 17 Apr 2018 21:33:29 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6733: [DOCS] Impala 3.0 Release Notes

[Alex Rodoni (Code Review)]

Hello Sailesh Mukil, Impala Public Jenkins, Vuk Ercegovac,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10090

to look at the new patch set (#2).

Change subject: IMPALA-6733: [DOCS] Impala 3.0 Release Notes
..

IMPALA-6733: [DOCS] Impala 3.0 Release Notes

Change-Id: I971323993aa6dced0791476e3e798d2f246fab5c
Cherry-picks: not for 2.x.
---
M docs/shared/ImpalaVariables.xml
M docs/topics/impala_fixed_issues.xml
M docs/topics/impala_incompatible_changes.xml
M docs/topics/impala_known_issues.xml
M docs/topics/impala_new_features.xml
5 files changed, 46 insertions(+), 1,268 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/10090/2
--
To view, visit http://gerrit.cloudera.org:8080/10090
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I971323993aa6dced0791476e3e798d2f246fab5c
Gerrit-Change-Number: 10090
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Vuk Ercegovac 

[Impala-ASF-CR] IMPALA-6733: [DOCS] Impala 3.0 Release Notes

[Sailesh Mukil (Code Review)]

Sailesh Mukil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10090 )

Change subject: IMPALA-6733: [DOCS] Impala 3.0 Release Notes
..


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/10090/1/docs/shared/ImpalaVariables.xml
File docs/shared/ImpalaVariables.xml:

http://gerrit.cloudera.org:8080/#/c/10090/1/docs/shared/ImpalaVariables.xml@a58
PS1, Line 58:
This looks like it's not been updated for a while and hence a bug. Can you 
mention that in the commit message?



--
To view, visit http://gerrit.cloudera.org:8080/10090
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I971323993aa6dced0791476e3e798d2f246fab5c
Gerrit-Change-Number: 10090
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Tue, 17 Apr 2018 21:26:47 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] Allow bootstrap system.sh to work on existing impala/ repo

[Sailesh Mukil (Code Review)]

Sailesh Mukil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10068 )

Change subject: Allow bootstrap_system.sh to work on existing impala/ repo
..


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/10068/2/bin/bootstrap_system.sh
File bin/bootstrap_system.sh:

http://gerrit.cloudera.org:8080/#/c/10068/2/bin/bootstrap_system.sh@100
PS2, Line 100: apt-get update
> But I don't want it to check out to ~/Impala. I explicitly set IMPALA_HOME
Yes, you're right. My assumption was more conservative in that if IMPALA_HOME 
is set, it must be set to a valid repo.

But I've uploaded a new patchset that takes care of all the cases Fredy 
mentioned, including the one you mentioned above.



--
To view, visit http://gerrit.cloudera.org:8080/10068
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If18a34ae54933d9ced2069602a8dca172826b18c
Gerrit-Change-Number: 10068
Gerrit-PatchSet: 4
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Tue, 17 Apr 2018 21:11:37 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] Allow bootstrap system.sh to work on existing impala/ repo

[Sailesh Mukil (Code Review)]

Hello Fredy Wijaya, Jim Apple, Philip Zeyliger,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10068

to look at the new patch set (#4).

Change subject: Allow bootstrap_system.sh to work on existing impala/ repo
..

Allow bootstrap_system.sh to work on existing impala/ repo

bootstrap_system.sh always assumes that Impala should be checked
out to the user's home directory, which does not need to be the case.
We end up checking out a second Impala repo if that assumption doesn't
hold true.

Also, since we need to check out the repo to access bin/bootstrap_*.sh,
it's a bit awkward to assume that this might be run as a standalone
script.

This change assumes that the script is being run from $IMPALA_HOME/bin/
and falls back to checking out the repo if that's not the case.

Testing: Ran bin/bootstrap_development.sh on a fresh machine and made
sure that it works as expected in the following cases:

- User has already set $IMPALA_HOME to a valid Impala repo:
  Uses that existing repo

- User has already set $IMPALA_HOME but to an invalid location:
  Checks out new repo to the location specified by IMPALA_HOME

- $IMPALA_HOME not specified and script run from a valid impala repo, i.e.
  /path/to/impala/bin/ : Uses existing repo that was detected

- $IMPALA_HOME not specified but script run in a standalone manner, i.e. not
  from a valid impala repo: Checks out new repo into ~/Impala

Change-Id: If18a34ae54933d9ced2069602a8dca172826b18c
---
M bin/bootstrap_system.sh
1 file changed, 26 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/10068/4
--
To view, visit http://gerrit.cloudera.org:8080/10068
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If18a34ae54933d9ced2069602a8dca172826b18c
Gerrit-Change-Number: 10068
Gerrit-PatchSet: 4
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 

[Impala-ASF-CR] IMPALA-6847: work around high memory estimates for AC

[Dan Hecht (Code Review)]

Dan Hecht has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10058 )

Change subject: IMPALA-6847: work around high memory estimates for AC
..


Patch Set 13: Code-Review+2

Please wait for Bikram's sign off as well.


--
To view, visit http://gerrit.cloudera.org:8080/10058
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia5fc32a507ad0f00f564dfe4f954a829ac55d14e
Gerrit-Change-Number: 10058
Gerrit-PatchSet: 13
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Tue, 17 Apr 2018 20:55:18 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6733: [DOCS] Impala 3.0 Release Notes

[Alex Rodoni (Code Review)]

Alex Rodoni has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/10090


Change subject: IMPALA-6733: [DOCS] Impala 3.0 Release Notes
..

IMPALA-6733: [DOCS] Impala 3.0 Release Notes

Change-Id: I971323993aa6dced0791476e3e798d2f246fab5c
Cherry-picks: not for 2.x.
---
M docs/shared/ImpalaVariables.xml
M docs/topics/impala_fixed_issues.xml
M docs/topics/impala_incompatible_changes.xml
M docs/topics/impala_new_features.xml
4 files changed, 32 insertions(+), 1 deletion(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/10090/1
--
To view, visit http://gerrit.cloudera.org:8080/10090
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I971323993aa6dced0791476e3e798d2f246fab5c
Gerrit-Change-Number: 10090
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 

[Impala-ASF-CR] IMPALA-6847: work around high memory estimates for AC

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10058 )

Change subject: IMPALA-6847: work around high memory estimates for AC
..


Patch Set 12:

(7 comments)

http://gerrit.cloudera.org:8080/#/c/10058/12//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/10058/12//COMMIT_MSG@9
PS12, Line 9: MAX_MEM_LIMIT_FOR_ADMISSION
> update
Done


http://gerrit.cloudera.org:8080/#/c/10058/12//COMMIT_MSG@14
PS12, Line 14: MAX_MEM_LIMIT_FOR_ADMISSION
> same
Done


http://gerrit.cloudera.org:8080/#/c/10058/12/common/thrift/ImpalaService.thrift
File common/thrift/ImpalaService.thrift:

http://gerrit.cloudera.org:8080/#/c/10058/12/common/thrift/ImpalaService.thrift@294
PS12, Line 294:   // See comment in ImpalaInternalService.thrift
> I like the idea of de-duping the comments, but perhaps it should go the oth
Done


http://gerrit.cloudera.org:8080/#/c/10058/12/fe/src/main/java/org/apache/impala/service/Frontend.java
File fe/src/main/java/org/apache/impala/service/Frontend.java:

http://gerrit.cloudera.org:8080/#/c/10058/12/fe/src/main/java/org/apache/impala/service/Frontend.java@1063
PS12, Line 1063: set
> typo: sent
Done


http://gerrit.cloudera.org:8080/#/c/10058/12/fe/src/main/java/org/apache/impala/service/Frontend.java@1065
PS12, Line 1065: // shown in the plan.
> given that, will it be hard to diagnose admission decisions? though I guess
Yeah, it also change the memory estimate shown in the profile, so you can see 
that the memory estimate in the plan embedded in the profile is different from 
the memory estimate in the profile counter.

It seemed useful to have the original memory estimate in the profile in some 
form too so we could see how much the override changed the behaviour.


http://gerrit.cloudera.org:8080/#/c/10058/12/fe/src/main/java/org/apache/impala/service/Frontend.java@1129
PS12, Line 1129: checkForMemEstimateOverride
> nit: how about checkAndOverrideMemEstimate
Done


http://gerrit.cloudera.org:8080/#/c/10058/12/testdata/workloads/functional-query/queries/QueryTest/admission-reject-mem-estimate.test
File 
testdata/workloads/functional-query/queries/QueryTest/admission-reject-mem-estimate.test:

http://gerrit.cloudera.org:8080/#/c/10058/12/testdata/workloads/functional-query/queries/QueryTest/admission-reject-mem-estimate.test@1
PS12, Line 1: 
> add case where mem_limit and max_mem_estimate_for_admission both are set, t
Added cases where mem_limit is higher and lower.



--
To view, visit http://gerrit.cloudera.org:8080/10058
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia5fc32a507ad0f00f564dfe4f954a829ac55d14e
Gerrit-Change-Number: 10058
Gerrit-PatchSet: 12
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Tue, 17 Apr 2018 20:46:53 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6790: Upgrade sqlparse to 0.1.19

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10044 )

Change subject: IMPALA-6790: Upgrade sqlparse to 0.1.19
..


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2322/


--
To view, visit http://gerrit.cloudera.org:8080/10044
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic5289f86b78f1d77d91a8fa47d63b7a7eaa3af38
Gerrit-Change-Number: 10044
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Tue, 17 Apr 2018 20:28:50 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6790: Upgrade sqlparse to 0.1.19

[Joe McDonnell (Code Review)]

Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10044 )

Change subject: IMPALA-6790: Upgrade sqlparse to 0.1.19
..


Patch Set 3: Code-Review+2

Rebased. Carrying +2.


--
To view, visit http://gerrit.cloudera.org:8080/10044
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic5289f86b78f1d77d91a8fa47d63b7a7eaa3af38
Gerrit-Change-Number: 10044
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Tue, 17 Apr 2018 20:28:10 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6651: [DOCS] Fine grained privileges

[Alex Rodoni (Code Review)]

Hello Fredy Wijaya, Alex Behm, Impala Public Jenkins, Adam Holley,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10079

to look at the new patch set (#2).

Change subject: IMPALA-6651: [DOCS] Fine grained privileges
..

IMPALA-6651: [DOCS] Fine grained privileges

Change-Id: I7b018bf847537ed461df6d9caee99f90b139f8ab
Cherry-picks: not for 2.x.
---
M docs/topics/impala_grant.xml
M docs/topics/impala_revoke.xml
2 files changed, 135 insertions(+), 27 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/79/10079/2
--
To view, visit http://gerrit.cloudera.org:8080/10079
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7b018bf847537ed461df6d9caee99f90b139f8ab
Gerrit-Change-Number: 10079
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 

[Impala-ASF-CR] Add new performance test workload - tpcds-unmodified

[Nithya Janarthanan (Code Review)]

Nithya Janarthanan has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/9973


Change subject: Add new performance test workload - tpcds-unmodified
..

Add new performance test workload - tpcds-unmodified

Description:
Impala versions prior to 2.5 didn't have Runtimefilters, which made TPC-DS 
queries run very slow,
so queries under tpcds have explicit partition filters to workaround the 
limitation.
Post Impala 2.5 adding tpcds-unmodified which has the unmodified version of the
workload to provide more coverage

Change-Id: I3957621d88b80fffc8fc89fd8104a58137b86e92
---
A testdata/workloads/tpcds-unmodified/queries/tpcds-q1.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q10.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q11.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q12.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q13.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q14.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q15.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q16.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q17.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q18.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q19.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q2.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q20.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q21.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q22.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q23.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q25.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q26.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q27.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q28.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q29.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q3.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q30.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q31.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q32.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q33.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q34.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q35.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q36.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q37.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q38.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q39.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q4.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q40.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q41.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q42.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q43.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q44.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q45.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q46.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q47.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q48.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q49.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q5.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q50.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q51.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q52.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q53.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q54.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q55.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q56.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q57.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q58.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q59.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q6.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q60.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q61.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q62.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q63.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q64.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q65.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q66.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q68.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q69.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q7.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q70.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q71.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q72.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q73.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q74.test
A testdata/workloads/tpcds-unmodified/queries/tpcds-q75.test
A 

[Impala-ASF-CR] IMPALA-6749: Implement ALTER DATABASE SET DBPROPERTIES

[Philip Zeyliger (Code Review)]

Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10085 )

Change subject: IMPALA-6749: Implement ALTER DATABASE SET DBPROPERTIES
..


Patch Set 4:

I'm happy to be overridden, but I think "Hive does it, so it must be a good 
idea" isn't a sufficiently good over-arching principle. Even though this is a 
minor feature in the grand scheme of things, it's more surface area for us to 
think about.


--
To view, visit http://gerrit.cloudera.org:8080/10085
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I08bc4a9c6fa95dd135274fa1081bf24ebe14c99a
Gerrit-Change-Number: 10085
Gerrit-PatchSet: 4
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Tue, 17 Apr 2018 20:16:47 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6748: [DOCS] Separators when casting STRING to TIMESTAMP

[Alex Rodoni (Code Review)]

Alex Rodoni has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10052 )

Change subject: IMPALA-6748: [DOCS] Separators when casting STRING to TIMESTAMP
..


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/10052/3/docs/shared/impala_common.xml
File docs/shared/impala_common.xml:

http://gerrit.cloudera.org:8080/#/c/10052/3/docs/shared/impala_common.xml@1411
PS3, Line 1411: month and date number
> and time components (hour, minute, second) too?
Done



--
To view, visit http://gerrit.cloudera.org:8080/10052
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib82884d5f56c520712c4391b53b799d518d6a54f
Gerrit-Change-Number: 10052
Gerrit-PatchSet: 3
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Vincent Tran 
Gerrit-Comment-Date: Tue, 17 Apr 2018 20:15:57 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6748: [DOCS] Separators when casting STRING to TIMESTAMP

[Alex Rodoni (Code Review)]

Hello Vincent Tran, Tim Armstrong, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10052

to look at the new patch set (#4).

Change subject: IMPALA-6748: [DOCS] Separators when casting STRING to TIMESTAMP
..

IMPALA-6748: [DOCS] Separators when casting STRING to TIMESTAMP

Change-Id: Ib82884d5f56c520712c4391b53b799d518d6a54f
---
M docs/shared/impala_common.xml
M docs/topics/impala_timestamp.xml
2 files changed, 182 insertions(+), 107 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/52/10052/4
--
To view, visit http://gerrit.cloudera.org:8080/10052
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib82884d5f56c520712c4391b53b799d518d6a54f
Gerrit-Change-Number: 10052
Gerrit-PatchSet: 4
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Vincent Tran 

[Impala-ASF-CR] IMPALA-6863: Make pip download.py honor redirects.

[Philip Zeyliger (Code Review)]

Philip Zeyliger has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/10083 )

Change subject: IMPALA-6863: Make pip_download.py honor redirects.
..

IMPALA-6863: Make pip_download.py honor redirects.

As part of our continuing woes with PyPi infrastructure, we've now seen
redirects. Following redirects seems like the right thing to do, so I've
changed the downloader code to follow them.

I checked that this is available in Python 2.6.

The build failure signature looks like:

   Downloading AllPairs-2.0.1.tar.gz from 
https://pypi.python.org/packages/04/44/cb85d029b33ddfa7b5a27ae523434c2230f11a89d1f0c73203e9007a6eb0/AllPairs-2.0.1.tar.gz
   ('http error', 302, 'Found', )
   Download failed after several attempts.
   Warning: Unable to download Python requirements.
   Warning: bootstrap_virtualenv or other Python-based tooling may fail.

Change-Id: Ic7551cec43a2d378df7e3cc7d521ace338b56ba2
Reviewed-on: http://gerrit.cloudera.org:8080/10083
Reviewed-by: Philip Zeyliger 
Tested-by: Philip Zeyliger 
Reviewed-by: Lars Volker 
---
M infra/python/deps/pip_download.py
1 file changed, 2 insertions(+), 2 deletions(-)

Approvals:
  Philip Zeyliger: Looks good to me, approved; Verified
  Lars Volker: Looks good to me, approved

--
To view, visit http://gerrit.cloudera.org:8080/10083
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic7551cec43a2d378df7e3cc7d521ace338b56ba2
Gerrit-Change-Number: 10083
Gerrit-PatchSet: 4
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Laszlo Gaal 
Gerrit-Reviewer: Philip Zeyliger 

[Impala-ASF-CR] IMPALA-5893: Remove old kinit code for Impala 3

[Dan Hecht (Code Review)]

Dan Hecht has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9941 )

Change subject: IMPALA-5893: Remove old kinit code for Impala 3
..


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9941/4/be/src/rpc/authentication.cc
File be/src/rpc/authentication.cc:

http://gerrit.cloudera.org:8080/#/c/9941/4/be/src/rpc/authentication.cc@a80
PS4, Line 80:
Maybe this needs to be moved to the "graveyard" rather than removed altogether, 
to avoid upgrade issues?



--
To view, visit http://gerrit.cloudera.org:8080/9941
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic78de10f3fb9ec36537de7a090916e4be123234b
Gerrit-Change-Number: 9941
Gerrit-PatchSet: 4
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Tue, 17 Apr 2018 19:48:27 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] Allow bootstrap system.sh to work on existing impala/ repo

[Fredy Wijaya (Code Review)]

Fredy Wijaya has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10068 )

Change subject: Allow bootstrap_system.sh to work on existing impala/ repo
..


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/10068/2/bin/bootstrap_system.sh
File bin/bootstrap_system.sh:

http://gerrit.cloudera.org:8080/#/c/10068/2/bin/bootstrap_system.sh@100
PS2, Line 100: apt-get update
> But I don't want it to check out to ~/Impala. I explicitly set IMPALA_HOME
In summary, with my change, the script will respect IMPALA_HOME it's set. If 
not, the default IMPALA_HOME will be ~/Impala. For example, if we download the 
bootstrap_system.sh script from GitHub.

/home/foo/tmp/Impala does not exist
IMPALA_HOME=/home/foo/tmp/Impala ./bootstrap_system.sh (will clone Impala in 
/home/foo/tmp/Impala and run a build in /home/foo/tmp/Impala)

~/Impala does not exist
./bootstrap_system (will clone Impala in the default IMPALA_HOME, which is 
~/Impala and run the build in ~/Impala)

/home/foo/tmp/Impala already exists
IMPALA_HOME=/home/foo/tmp/Impala ./bootstrap_system.sh (will skip the clone, 
but will run the build in /home/foo/tmp/Impala)

~/Impala exists
./bootstrap_system.sh (will skip the clone, but will run the build in the 
default IMPALA_HOME, which is ~/Impala)



--
To view, visit http://gerrit.cloudera.org:8080/10068
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If18a34ae54933d9ced2069602a8dca172826b18c
Gerrit-Change-Number: 10068
Gerrit-PatchSet: 3
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Tue, 17 Apr 2018 19:35:56 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6863: Make pip download.py honor redirects.

[Lars Volker (Code Review)]

Lars Volker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10083 )

Change subject: IMPALA-6863: Make pip_download.py honor redirects.
..


Patch Set 3: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/10083
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7551cec43a2d378df7e3cc7d521ace338b56ba2
Gerrit-Change-Number: 10083
Gerrit-PatchSet: 3
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Laszlo Gaal 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Tue, 17 Apr 2018 19:23:34 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6847: work around high memory estimates for AC

[Dan Hecht (Code Review)]

Dan Hecht has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10058 )

Change subject: IMPALA-6847: work around high memory estimates for AC
..


Patch Set 12:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/10058/12//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/10058/12//COMMIT_MSG@9
PS12, Line 9: MAX_MEM_LIMIT_FOR_ADMISSION
update


http://gerrit.cloudera.org:8080/#/c/10058/12//COMMIT_MSG@14
PS12, Line 14: MAX_MEM_LIMIT_FOR_ADMISSION
same


http://gerrit.cloudera.org:8080/#/c/10058/12/common/thrift/ImpalaService.thrift
File common/thrift/ImpalaService.thrift:

http://gerrit.cloudera.org:8080/#/c/10058/12/common/thrift/ImpalaService.thrift@294
PS12, Line 294:   // See comment in ImpalaInternalService.thrift
I like the idea of de-duping the comments, but perhaps it should go the other 
direction, given that this is the public impala service interface, and 
ImpalaInternalService is the internal impalad to impalad interface.


http://gerrit.cloudera.org:8080/#/c/10058/12/fe/src/main/java/org/apache/impala/service/Frontend.java
File fe/src/main/java/org/apache/impala/service/Frontend.java:

http://gerrit.cloudera.org:8080/#/c/10058/12/fe/src/main/java/org/apache/impala/service/Frontend.java@1065
PS12, Line 1065: // shown in the plan.
given that, will it be hard to diagnose admission decisions? though I guess we 
now print query options in the profile, so probably we're okay.



--
To view, visit http://gerrit.cloudera.org:8080/10058
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia5fc32a507ad0f00f564dfe4f954a829ac55d14e
Gerrit-Change-Number: 10058
Gerrit-PatchSet: 12
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Tue, 17 Apr 2018 19:20:08 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] Allow bootstrap system.sh to work on existing impala/ repo

[Philip Zeyliger (Code Review)]

Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10068 )

Change subject: Allow bootstrap_system.sh to work on existing impala/ repo
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/10068/2/bin/bootstrap_system.sh
File bin/bootstrap_system.sh:

http://gerrit.cloudera.org:8080/#/c/10068/2/bin/bootstrap_system.sh@100
PS2, Line 100: # Get the location of this script. We assume it's being run as 
part of an already
> No, that works too. If IMPALA_HOME=/foo (which is not a valid repo), then o
But I don't want it to check out to ~/Impala. I explicitly set IMPALA_HOME to 
check out at /foo. That's what Fredy's change did?

Let's sync up over some interactive channel?



--
To view, visit http://gerrit.cloudera.org:8080/10068
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If18a34ae54933d9ced2069602a8dca172826b18c
Gerrit-Change-Number: 10068
Gerrit-PatchSet: 2
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Tue, 17 Apr 2018 19:03:22 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6862: Privilege.java needs to support Sentry 1.5.1 and 2.0.0

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10082 )

Change subject: IMPALA-6862: Privilege.java needs to support Sentry 1.5.1 and 
2.0.0
..


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2320/


--
To view, visit http://gerrit.cloudera.org:8080/10082
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibbf4eeab3927ff93ed51066b164b8874d5f76e23
Gerrit-Change-Number: 10082
Gerrit-PatchSet: 4
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Tue, 17 Apr 2018 18:48:41 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6862: Privilege.java needs to support Sentry 1.5.1 and 2.0.0

[Alex Behm (Code Review)]

Alex Behm has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10082 )

Change subject: IMPALA-6862: Privilege.java needs to support Sentry 1.5.1 and 
2.0.0
..


Patch Set 4: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/10082
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibbf4eeab3927ff93ed51066b164b8874d5f76e23
Gerrit-Change-Number: 10082
Gerrit-PatchSet: 4
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Tue, 17 Apr 2018 18:47:53 +
Gerrit-HasComments: No

[Impala-ASF-CR] Allow bootstrap system.sh to work on existing impala/ repo

[Sailesh Mukil (Code Review)]

Sailesh Mukil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10068 )

Change subject: Allow bootstrap_system.sh to work on existing impala/ repo
..


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/10068/2/bin/bootstrap_system.sh
File bin/bootstrap_system.sh:

http://gerrit.cloudera.org:8080/#/c/10068/2/bin/bootstrap_system.sh@100
PS2, Line 100: apt-get update
> Ok, then the case where IMPALA_HOME=/foo but IMPALA_HOME is not a current g
No, that works too. If IMPALA_HOME=/foo (which is not a valid repo), then on 
L109, we would detect that it's not a valid repo, and go ahead and check out 
impala to '~/Impala', and also update IMPALA_HOME appropriately.

The only way this can fail is if '/foo' is a git repo of some other project. If 
we think that's a serious possibility, I can add code to handle that as well in 
L109.



--
To view, visit http://gerrit.cloudera.org:8080/10068
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If18a34ae54933d9ced2069602a8dca172826b18c
Gerrit-Change-Number: 10068
Gerrit-PatchSet: 3
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Tue, 17 Apr 2018 18:47:41 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] Allow bootstrap system.sh to work on existing impala/ repo

[Philip Zeyliger (Code Review)]

Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10068 )

Change subject: Allow bootstrap_system.sh to work on existing impala/ repo
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/10068/2/bin/bootstrap_system.sh
File bin/bootstrap_system.sh:

http://gerrit.cloudera.org:8080/#/c/10068/2/bin/bootstrap_system.sh@100
PS2, Line 100: # Get the location of this script. We assume it's being run as 
part of an already
> If IMPALA_HOME is not preset by the user, on L108, we will set IMPALA_HOME
Ok, then the case where IMPALA_HOME=/foo but IMPALA_HOME is not a current git 
checkout is broken.



--
To view, visit http://gerrit.cloudera.org:8080/10068
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If18a34ae54933d9ced2069602a8dca172826b18c
Gerrit-Change-Number: 10068
Gerrit-PatchSet: 2
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Tue, 17 Apr 2018 18:35:54 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6749: Implement ALTER DATABASE SET DBPROPERTIES

[Fredy Wijaya (Code Review)]

Fredy Wijaya has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10085 )

Change subject: IMPALA-6749: Implement ALTER DATABASE SET DBPROPERTIES
..


Patch Set 4:

> Patch Set 4:
>
> Why are we exposing this? I'm worried that we're exposing (more) untyped 
> key-value stores that will get abused.

We are already displaying the DB properties with DESCRIBE DATABASE 
EXTENDED/FORMATTED set externally, e.g. in Hive: 
https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/DescribeResultFactory.java#L154-L167

Hive also supports ALTER DATABASE SET DBPROPERTIES: 
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Create/Drop/Alter/UseDatabase

I thought it would be nice to have that feature in Impala.


--
To view, visit http://gerrit.cloudera.org:8080/10085
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I08bc4a9c6fa95dd135274fa1081bf24ebe14c99a
Gerrit-Change-Number: 10085
Gerrit-PatchSet: 4
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Tue, 17 Apr 2018 18:26:10 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6862: Privilege.java needs to support Sentry 1.5.1 and 2.0.0

[Sailesh Mukil (Code Review)]

Sailesh Mukil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10082 )

Change subject: IMPALA-6862: Privilege.java needs to support Sentry 1.5.1 and 
2.0.0
..


Patch Set 4: Code-Review+1

The core change LGTM, but I'd rather someone else sign off on this.


--
To view, visit http://gerrit.cloudera.org:8080/10082
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibbf4eeab3927ff93ed51066b164b8874d5f76e23
Gerrit-Change-Number: 10082
Gerrit-PatchSet: 4
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Tue, 17 Apr 2018 18:20:33 +
Gerrit-HasComments: No

[Impala-ASF-CR](2.x) IMPALA-6822: Add a query option to control shuffling by distinct exprs

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10087 )

Change subject: IMPALA-6822: Add a query option to control shuffling by 
distinct exprs
..


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2319/


--
To view, visit http://gerrit.cloudera.org:8080/10087
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: comment
Gerrit-Change-Id: Icb4b4576fb29edd62cf4b4ba0719c0e0a2a5a8dc
Gerrit-Change-Number: 10087
Gerrit-PatchSet: 1
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Comment-Date: Tue, 17 Apr 2018 18:19:02 +
Gerrit-HasComments: No

[Impala-ASF-CR](2.x) IMPALA-6822: Add a query option to control shuffling by distinct exprs

[Alex Behm (Code Review)]

Alex Behm has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10087 )

Change subject: IMPALA-6822: Add a query option to control shuffling by 
distinct exprs
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/10087
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: comment
Gerrit-Change-Id: Icb4b4576fb29edd62cf4b4ba0719c0e0a2a5a8dc
Gerrit-Change-Number: 10087
Gerrit-PatchSet: 1
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Comment-Date: Tue, 17 Apr 2018 18:17:43 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6749: Implement ALTER DATABASE SET DBPROPERTIES

[Philip Zeyliger (Code Review)]

Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10085 )

Change subject: IMPALA-6749: Implement ALTER DATABASE SET DBPROPERTIES
..


Patch Set 4:

Why are we exposing this? I'm worried that we're exposing (more) untyped 
key-value stores that will get abused.


--
To view, visit http://gerrit.cloudera.org:8080/10085
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I08bc4a9c6fa95dd135274fa1081bf24ebe14c99a
Gerrit-Change-Number: 10085
Gerrit-PatchSet: 4
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Tue, 17 Apr 2018 18:13:50 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6847: work around high memory estimates for AC

[Bikramjeet Vig (Code Review)]

Bikramjeet Vig has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10058 )

Change subject: IMPALA-6847: work around high memory estimates for AC
..


Patch Set 12:

> > (3 comments)
 >
 > > I also really like the ideas in patch 8 and looks like a good
 > > direction to steer our solution towards in IMPALA-6460. It was
 > > essentially decoupling the 2 functions performed by mem_limit (
 > 1.
 > > to enforce a runtime limit and 2. to be used in admission control
 > > for admitting and accounting of memory) and would have made
 > impala
 > > more stable, but the fact that a query option was used to
 > override
 > > mem_limit's functionality and had overlapping functionality with
 > > mem_estimate (acting as an upper limit for it) was making it
 > > confusing.
 >
 > Just for historical context, the current way we use MEM_LIMIT as
 > the "reservation" for admission control (i.e. #2) was put in place
 > as a placeholder until the backend actually implements real memory
 > reservations. Now that we have real memory reservations for the
 > most hungry operators, we want to leverage that (with additional
 > heuristics) rather than MEM_LIMIT for admission.  But without
 > memory reservations in the backend, MEM_LIMIT seems the most
 > conservative approach for stability.

Makes sense! Thanks for the context


--
To view, visit http://gerrit.cloudera.org:8080/10058
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia5fc32a507ad0f00f564dfe4f954a829ac55d14e
Gerrit-Change-Number: 10058
Gerrit-PatchSet: 12
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Tue, 17 Apr 2018 18:12:07 +
Gerrit-HasComments: No

[Impala-ASF-CR](2.x) IMPALA-6822: Add a query option to control shuffling by distinct exprs

[Tianyi Wang (Code Review)]

Hello Impala Public Jenkins,

I'd like you to do a code review. Please visit

http://gerrit.cloudera.org:8080/10087

to review the following change.


Change subject: IMPALA-6822: Add a query option to control shuffling by 
distinct exprs
..

IMPALA-6822: Add a query option to control shuffling by distinct exprs

IMPALA-4794 changed the distinct aggregation behavior to shuffling by
both grouping exprs and the distinct expr. It's slower in queries
where the NDVs of grouping exprs are high and data are uniformly
distributed among groups. This patch adds a query option controlling
this behavior, letting users switch to the old plan.

Change-Id: Icb4b4576fb29edd62cf4b4ba0719c0e0a2a5a8dc
Reviewed-on: http://gerrit.cloudera.org:8080/9949
Reviewed-by: Tianyi Wang 
Tested-by: Impala Public Jenkins 
(cherry picked from commit 9a751f00b8a399116c12a81e130a696b01eb1ba8)
---
M be/src/service/query-options.cc
M be/src/service/query-options.h
M common/thrift/ImpalaInternalService.thrift
M common/thrift/ImpalaService.thrift
M fe/src/main/java/org/apache/impala/planner/DistributedPlanner.java
M fe/src/test/java/org/apache/impala/planner/PlannerTest.java
A 
testdata/workloads/functional-planner/queries/PlannerTest/shuffle-by-distinct-exprs.test
M testdata/workloads/functional-query/queries/QueryTest/distinct.test
M tests/query_test/test_aggregation.py
9 files changed, 454 insertions(+), 24 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/87/10087/1
--
To view, visit http://gerrit.cloudera.org:8080/10087
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: newchange
Gerrit-Change-Id: Icb4b4576fb29edd62cf4b4ba0719c0e0a2a5a8dc
Gerrit-Change-Number: 10087
Gerrit-PatchSet: 1
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 

[Impala-ASF-CR] IMPALA-6863: Make pip download.py honor redirects.

[Philip Zeyliger (Code Review)]

Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10083 )

Change subject: IMPALA-6863: Make pip_download.py honor redirects.
..


Patch Set 3: Verified+1 Code-Review+2

Whoops; thanks.

Jenkins came back happy, so pushing ahead.


--
To view, visit http://gerrit.cloudera.org:8080/10083
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7551cec43a2d378df7e3cc7d521ace338b56ba2
Gerrit-Change-Number: 10083
Gerrit-PatchSet: 3
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Laszlo Gaal 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Tue, 17 Apr 2018 17:52:49 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6748: [DOCS] Separators when casting STRING to TIMESTAMP

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10052 )

Change subject: IMPALA-6748: [DOCS] Separators when casting STRING to TIMESTAMP
..


Patch Set 3:

(1 comment)

LGTM except for one minor thing.

http://gerrit.cloudera.org:8080/#/c/10052/3/docs/shared/impala_common.xml
File docs/shared/impala_common.xml:

http://gerrit.cloudera.org:8080/#/c/10052/3/docs/shared/impala_common.xml@1411
PS3, Line 1411: month and date number
and time components (hour, minute, second) too?



--
To view, visit http://gerrit.cloudera.org:8080/10052
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib82884d5f56c520712c4391b53b799d518d6a54f
Gerrit-Change-Number: 10052
Gerrit-PatchSet: 3
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Vincent Tran 
Gerrit-Comment-Date: Tue, 17 Apr 2018 17:49:52 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6863: Make pip download.py honor redirects.

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10083 )

Change subject: IMPALA-6863: Make pip_download.py honor redirects.
..


Patch Set 2: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/10083
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7551cec43a2d378df7e3cc7d521ace338b56ba2
Gerrit-Change-Number: 10083
Gerrit-PatchSet: 2
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Laszlo Gaal 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Tue, 17 Apr 2018 17:48:47 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6847: work around high memory estimates for AC

[Dan Hecht (Code Review)]

Dan Hecht has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10058 )

Change subject: IMPALA-6847: work around high memory estimates for AC
..


Patch Set 12:

> (3 comments)

 > I also really like the ideas in patch 8 and looks like a good
 > direction to steer our solution towards in IMPALA-6460. It was
 > essentially decoupling the 2 functions performed by mem_limit ( 1.
 > to enforce a runtime limit and 2. to be used in admission control
 > for admitting and accounting of memory) and would have made impala
 > more stable, but the fact that a query option was used to override
 > mem_limit's functionality and had overlapping functionality with
 > mem_estimate (acting as an upper limit for it) was making it
 > confusing.

Just for historical context, the current way we use MEM_LIMIT as the 
"reservation" for admission control (i.e. #2) was put in place as a placeholder 
until the backend actually implements real memory reservations. Now that we 
have real memory reservations for the most hungry operators, we want to 
leverage that (with additional heuristics) rather than MEM_LIMIT for admission. 
 But without memory reservations in the backend, MEM_LIMIT seems the most 
conservative approach for stability.


--
To view, visit http://gerrit.cloudera.org:8080/10058
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia5fc32a507ad0f00f564dfe4f954a829ac55d14e
Gerrit-Change-Number: 10058
Gerrit-PatchSet: 12
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Tue, 17 Apr 2018 16:58:14 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6790: Upgrade sqlparse to 0.1.19

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10044 )

Change subject: IMPALA-6790: Upgrade sqlparse to 0.1.19
..


Patch Set 2: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2317/


--
To view, visit http://gerrit.cloudera.org:8080/10044
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic5289f86b78f1d77d91a8fa47d63b7a7eaa3af38
Gerrit-Change-Number: 10044
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Tue, 17 Apr 2018 16:15:50 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6651: [DOCS] Fine grained privileges

[Fredy Wijaya (Code Review)]

Fredy Wijaya has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10079 )

Change subject: IMPALA-6651: [DOCS] Fine grained privileges
..


Patch Set 1:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/10079/1/docs/topics/impala_grant.xml
File docs/topics/impala_grant.xml:

http://gerrit.cloudera.org:8080/#/c/10079/1/docs/topics/impala_grant.xml@105
PS1, Line 105:   The REFRESH privilege is required to 
execute
It would be much nicer to have all this information presented in a table with 3 
columns (privilege, scope, SQL statements).


http://gerrit.cloudera.org:8080/#/c/10079/1/docs/topics/impala_revoke.xml
File docs/topics/impala_revoke.xml:

http://gerrit.cloudera.org:8080/#/c/10079/1/docs/topics/impala_revoke.xml@45
PS1, Line 45:   specified object from groups. The revocation has a 
cascading effect. For
I don't know where we want to add this info. Currently we are treating ALL as a 
distinct privilege. ALL is not a union of all other privileges. In other words, 
revoking SELECT, INSERT, CREATE, ALTER, etc. on ALL is a no-op.


http://gerrit.cloudera.org:8080/#/c/10079/1/docs/topics/impala_revoke.xml@58
PS1, Line 58:   privilege ::= ALTER | CREATE | SELECT | 
SELECT(column_name) | INSERT | ALL
Missing REFRESH privilege.



--
To view, visit http://gerrit.cloudera.org:8080/10079
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7b018bf847537ed461df6d9caee99f90b139f8ab
Gerrit-Change-Number: 10079
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Tue, 17 Apr 2018 16:14:38 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-5706: Parallelise read I/O in sorter

[Gabor Kaszab (Code Review)]

Hello Tim Armstrong,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9943

to look at the new patch set (#3).

Change subject: IMPALA-5706: Parallelise read I/O in sorter
..

IMPALA-5706: Parallelise read I/O in sorter

This patch covers multiple changes with the purpose of optimizing
spilling sort mechanism:
  - Use double-buffering when merging the sorted runs. As a result
while a run's page is being processed the next one can be loaded
from disk in the background.
  - Remove the hard-coded maximum limit of buffers that can be used
for merging the sorted runs. Instead this number is calculated
based on the available memeory through buffer pool.
  - The already sorted runs are distributed evenly between the last
intermediate merge and the final merge to avoid that a heavy
intermediate merge is followed by a light final merge.
  - Right before starting the merging phase Sorter tries to allocate
additional memory through the buffer pool.
  - An output run is not allocated anymore for the final merge.

Performance measurements were made during manual testing to verify
that this is in fact an optimization:
  - In case doing a sort on top of a join when working with restricted
amount of memory then additional memory is successfully allocated
for merging than was available during the initial sort runs. This
results in shallower merging trees (more runs grabbed for a
merge).
  - Manual tests showed that when a single final merge is performed
this change slighlty decrease the execution time for sorting.

Further testing should be done to cover double-buffering scenarios as
my manual testing so far didn't show any performance gain when
intermediate merges were performed. Most probably because due to
double-buffering the number of runs in a single merge decreases and I
have to hit an I/O heavy scenario to overcome this.

Change-Id: I74857c1694802e81f1cfc765d2b4e8bc644387f9
---
M be/src/runtime/sorter.cc
M be/src/runtime/sorter.h
M fe/src/main/java/org/apache/impala/planner/SortNode.java
M 
testdata/workloads/functional-planner/queries/PlannerTest/constant-folding.test
M testdata/workloads/functional-planner/queries/PlannerTest/max-row-size.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/sort-expr-materialization.test
M testdata/workloads/functional-query/queries/QueryTest/analytic-fns.test
M 
testdata/workloads/functional-query/queries/QueryTest/spilling-naaj-no-deny-reservation.test
M 
testdata/workloads/functional-query/queries/QueryTest/spilling-sorts-exhaustive.test
M testdata/workloads/tpch/queries/sort-reservation-usage.test
M tests/custom_cluster/test_mem_reservations.py
M tests/query_test/test_sort.py
14 files changed, 319 insertions(+), 245 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/9943/3
--
To view, visit http://gerrit.cloudera.org:8080/9943
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I74857c1694802e81f1cfc765d2b4e8bc644387f9
Gerrit-Change-Number: 9943
Gerrit-PatchSet: 3
Gerrit-Owner: Gabor Kaszab 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Tim Armstrong 

[Impala-ASF-CR] IMPALA-6863: Make pip download.py honor redirects.

[Lars Volker (Code Review)]

Lars Volker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10083 )

Change subject: IMPALA-6863: Make pip_download.py honor redirects.
..


Patch Set 3: Code-Review-2

Updated the commit message, thanks for catching this Laszlo. I'm marking this 
as -2 to make sure that Jenkins doesn't merge PS2 (it shouldn't anyways). Then 
we can submit it manually once the job reports success.


--
To view, visit http://gerrit.cloudera.org:8080/10083
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7551cec43a2d378df7e3cc7d521ace338b56ba2
Gerrit-Change-Number: 10083
Gerrit-PatchSet: 3
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Laszlo Gaal 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Tue, 17 Apr 2018 15:48:48 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6863: Make pip download.py honor redirects.

[Lars Volker (Code Review)]

Lars Volker has uploaded a new patch set (#3) to the change originally created 
by Philip Zeyliger. ( http://gerrit.cloudera.org:8080/10083 )

Change subject: IMPALA-6863: Make pip_download.py honor redirects.
..

IMPALA-6863: Make pip_download.py honor redirects.

As part of our continuing woes with PyPi infrastructure, we've now seen
redirects. Following redirects seems like the right thing to do, so I've
changed the downloader code to follow them.

I checked that this is available in Python 2.6.

The build failure signature looks like:

   Downloading AllPairs-2.0.1.tar.gz from 
https://pypi.python.org/packages/04/44/cb85d029b33ddfa7b5a27ae523434c2230f11a89d1f0c73203e9007a6eb0/AllPairs-2.0.1.tar.gz
   ('http error', 302, 'Found', )
   Download failed after several attempts.
   Warning: Unable to download Python requirements.
   Warning: bootstrap_virtualenv or other Python-based tooling may fail.

Change-Id: Ic7551cec43a2d378df7e3cc7d521ace338b56ba2
---
M infra/python/deps/pip_download.py
1 file changed, 2 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/10083/3
--
To view, visit http://gerrit.cloudera.org:8080/10083
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic7551cec43a2d378df7e3cc7d521ace338b56ba2
Gerrit-Change-Number: 10083
Gerrit-PatchSet: 3
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Laszlo Gaal 
Gerrit-Reviewer: Philip Zeyliger 

[Impala-ASF-CR] IMPALA-6651: [DOCS] Fine grained privileges

[Adam Holley (Code Review)]

Adam Holley has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10079 )

Change subject: IMPALA-6651: [DOCS] Fine grained privileges
..


Patch Set 1:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/10079/1/docs/topics/impala_grant.xml
File docs/topics/impala_grant.xml:

http://gerrit.cloudera.org:8080/#/c/10079/1/docs/topics/impala_grant.xml@44
PS1, Line 44: statement grants roles or privileges on specified
:   objects to groups.
Should we change the wording of this?  We can grant privileges to a role, and 
we can grant roles to a group, but we cannot grant privileges to a group.  This 
statement seems to indicate we can.


http://gerrit.cloudera.org:8080/#/c/10079/1/docs/topics/impala_grant.xml@118
PS1, Line 118: The DROP privilege is required to drop a 
database
 :   or a table at the SERVER and
 : DATABASE levels.
This is slightly different than create.  DROP at the server level allows you to 
drop databases and tables.  DROP at the database level allows you to drop that 
specific database and all tables underneath.  DROP at the table level allows 
you to drop that specific table.



--
To view, visit http://gerrit.cloudera.org:8080/10079
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7b018bf847537ed461df6d9caee99f90b139f8ab
Gerrit-Change-Number: 10079
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Tue, 17 Apr 2018 15:06:20 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-686: Make pip download.py honor redirects.

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10083 )

Change subject: IMPALA-686: Make pip_download.py honor redirects.
..


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2316/


--
To view, visit http://gerrit.cloudera.org:8080/10083
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7551cec43a2d378df7e3cc7d521ace338b56ba2
Gerrit-Change-Number: 10083
Gerrit-PatchSet: 2
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Tue, 17 Apr 2018 13:52:16 +
Gerrit-HasComments: No