date:20180815

Vuk Ercegovac has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11230 )

Change subject: IMPALA-7140 fix: fix NDV calculation for partition columns in 
LocalFsTable
..


Patch Set 2: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/11230
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0a86750d52dcd744ace030a3c1ec70510d054acf
Gerrit-Change-Number: 11230
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Thu, 16 Aug 2018 05:21:55 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7439. CREATE DATABASE creates a catalog entry with missing location

Vuk Ercegovac has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11229 )

Change subject: IMPALA-7439. CREATE DATABASE creates a catalog entry with 
missing location
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/11229
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I13df31cee1e5768b073e0e35c4c16ebf1892be23
Gerrit-Change-Number: 11229
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Thu, 16 Aug 2018 05:20:27 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7399: Remove third-party dependencies from junit xml script

David Knupp has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11235 )

Change subject: IMPALA-7399: Remove third-party dependencies from junit xml 
script
..


Patch Set 5:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/11235/4/lib/python/impala_py_lib/jenkins/generate_junitxml.py
File lib/python/impala_py_lib/jenkins/generate_junitxml.py:

http://gerrit.cloudera.org:8080/#/c/11235/4/lib/python/impala_py_lib/jenkins/generate_junitxml.py@38
PS4, Line 38:I'll wait for your reply as to whether or not to do away with the class.
Done


http://gerrit.cloudera.org:8080/#/c/11235/4/lib/python/impala_py_lib/jenkins/generate_junitxml.py@69
PS4, Line 69:   content for the report.
> I think this would be clearer if you just created this in init().
Done


http://gerrit.cloudera.org:8080/#/c/11235/4/lib/python/impala_py_lib/jenkins/generate_junitxml.py@79
PS4, Line 79:   elif os.path.exists(file_or_string):
: with open(file_or_string, 'r') as
> This is just:
Done


http://gerrit.cloudera.org:8080/#/c/11235/4/lib/python/impala_py_lib/jenkins/generate_junitxml.py@102
PS4, Line 102:   help="If known, the elapsed time in 
seconds for this step.")
 :   parser.add_argument("--stdout",
 :   help=textwrap.dedent(
> Same here: you don't re-use base_attributes ever, so I think it makes sense
Done



--
To view, visit http://gerrit.cloudera.org:8080/11235
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I958ee0d8420b6a4197aaf0a7e0538a566332ea97
Gerrit-Change-Number: 11235
Gerrit-PatchSet: 5
Gerrit-Owner: David Knupp 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Nithya Janarthanan 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Thu, 16 Aug 2018 05:18:18 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7345: Add the OWNER privilege

2018-08-15 Thread Fredy Wijaya (Code Review)

Fredy Wijaya has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11245 )

Change subject: IMPALA-7345: Add the OWNER privilege
..


Patch Set 2: Code-Review+1


--
To view, visit http://gerrit.cloudera.org:8080/11245
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If63c2faa6daea6deb6d771503fe50943ae070705
Gerrit-Change-Number: 11245
Gerrit-PatchSet: 2
Gerrit-Owner: Adam Holley 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 16 Aug 2018 05:12:13 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7345: Add the OWNER privilege

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11245 )

Change subject: IMPALA-7345: Add the OWNER privilege
..


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3022/ 
DRY_RUN=true


--
To view, visit http://gerrit.cloudera.org:8080/11245
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If63c2faa6daea6deb6d771503fe50943ae070705
Gerrit-Change-Number: 11245
Gerrit-PatchSet: 2
Gerrit-Owner: Adam Holley 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 16 Aug 2018 05:11:30 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7406. Avoid memory overhead of FlatBuffer object wrappers for FileDescriptor

Vuk Ercegovac has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11232 )

Change subject: IMPALA-7406. Avoid memory overhead of FlatBuffer object 
wrappers for FileDescriptor
..


Patch Set 2: Code-Review+1


--
To view, visit http://gerrit.cloudera.org:8080/11232
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I986dd5c05ac8904898d01a6ef3487bd7225e0500
Gerrit-Change-Number: 11232
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Thu, 16 Aug 2018 05:04:05 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7345: Add the OWNER privilege

2018-08-15 Thread Adam Holley (Code Review)

Adam Holley has uploaded a new patch set (#2). ( 
http://gerrit.cloudera.org:8080/11245 )

Change subject: IMPALA-7345: Add the OWNER privilege
..

IMPALA-7345: Add the OWNER privilege

This patch adds the OWNER privilege to the set of privileges that can
exist for a role/user. The privilege is equivalent to ALL, but cannot
be granted or revoked. It is granted/revoked by Sentry, if configured,
during CREATE, DROP, or ALTER DATABASE/TABLE SET OWNER statements.

Testing:
- Updated authorization tests
- Ran core tests

Change-Id: If63c2faa6daea6deb6d771503fe50943ae070705
---
M common/thrift/CatalogObjects.thrift
M fe/src/main/java/org/apache/impala/authorization/Privilege.java
M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java
M fe/src/test/java/org/apache/impala/authorization/ImpalaActionFactoryTest.java
4 files changed, 569 insertions(+), 241 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/11245/2
--
To view, visit http://gerrit.cloudera.org:8080/11245
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If63c2faa6daea6deb6d771503fe50943ae070705
Gerrit-Change-Number: 11245
Gerrit-PatchSet: 2
Gerrit-Owner: Adam Holley 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins

[Impala-ASF-CR] IMPALA-7345: Add the OWNER privilege

2018-08-15 Thread Adam Holley (Code Review)

Adam Holley has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11245 )

Change subject: IMPALA-7345: Add the OWNER privilege
..


Patch Set 1:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/11245/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/11245/1//COMMIT_MSG@10
PS1, Line 10: role
> for a role/user?
Done


http://gerrit.cloudera.org:8080/#/c/11245/1//COMMIT_MSG@10
PS1, Line 10:
> nit: extra whitespace
Done


http://gerrit.cloudera.org:8080/#/c/11245/1//COMMIT_MSG@11
PS1, Line 11:
> nit: extra whitespace
Done


http://gerrit.cloudera.org:8080/#/c/11245/1/fe/src/main/java/org/apache/impala/authorization/Privilege.java
File fe/src/main/java/org/apache/impala/authorization/Privilege.java:

http://gerrit.cloudera.org:8080/#/c/11245/1/fe/src/main/java/org/apache/impala/authorization/Privilege.java@68
PS1, Line 68: OWNER("owner",
> You should use the code from ALL, i.e.
Done



--
To view, visit http://gerrit.cloudera.org:8080/11245
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If63c2faa6daea6deb6d771503fe50943ae070705
Gerrit-Change-Number: 11245
Gerrit-PatchSet: 1
Gerrit-Owner: Adam Holley 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 16 Aug 2018 05:04:31 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7345: Add the OWNER privilege

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11245 )

Change subject: IMPALA-7345: Add the OWNER privilege
..


Patch Set 1:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/367/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11245
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If63c2faa6daea6deb6d771503fe50943ae070705
Gerrit-Change-Number: 11245
Gerrit-PatchSet: 1
Gerrit-Owner: Adam Holley 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 16 Aug 2018 05:00:08 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7452: fix file check for test disable catalog data ops test

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/11243 )

Change subject: IMPALA-7452: fix file check for test_disable_catalog_data_ops 
test
..

IMPALA-7452: fix file check for test_disable_catalog_data_ops test

The test's regex did not handle s3 paths. The fix loosens the check
to allow for these paths.

Change-Id: I7408bc233714620f502af3b8c19400aa3bb1666c
Reviewed-on: http://gerrit.cloudera.org:8080/11243
Reviewed-by: Impala Public Jenkins 
Tested-by: Impala Public Jenkins 
---
M tests/custom_cluster/test_disable_catalog_data_ops.py
1 file changed, 2 insertions(+), 4 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

--
To view, visit http://gerrit.cloudera.org:8080/11243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I7408bc233714620f502af3b8c19400aa3bb1666c
Gerrit-Change-Number: 11243
Gerrit-PatchSet: 3
Gerrit-Owner: Vuk Ercegovac 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong

[Impala-ASF-CR] IMPALA-7452: fix file check for test disable catalog data ops test

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11243 )

Change subject: IMPALA-7452: fix file check for test_disable_catalog_data_ops 
test
..


Patch Set 2: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/11243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7408bc233714620f502af3b8c19400aa3bb1666c
Gerrit-Change-Number: 11243
Gerrit-PatchSet: 2
Gerrit-Owner: Vuk Ercegovac 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 16 Aug 2018 05:00:24 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7436: initial fetch-from-catalogd implementation

Vuk Ercegovac has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11182 )

Change subject: IMPALA-7436: initial fetch-from-catalogd implementation
..


Patch Set 6: Code-Review+2

(2 comments)

http://gerrit.cloudera.org:8080/#/c/11182/5/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
File fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java:

http://gerrit.cloudera.org:8080/#/c/11182/5/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java@100
PS5, Line 100:  * index 'dstIndex' instead of the original index 
'origIndex'.
> there's a bit comment at the call site. Is that sufficient? Didn't want to
yeah, just saw it. that's fine, no need to duplicate.


http://gerrit.cloudera.org:8080/#/c/11182/5/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java
File fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java:

http://gerrit.cloudera.org:8080/#/c/11182/5/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java@218
PS5, Line 218: return ret;
 :
nit: extra ws



--
To view, visit http://gerrit.cloudera.org:8080/11182
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If49207fc592b1cc552fbcc7199568b6833f86901
Gerrit-Change-Number: 11182
Gerrit-PatchSet: 6
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Thu, 16 Aug 2018 04:54:51 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7345: Add the OWNER privilege

2018-08-15 Thread Fredy Wijaya (Code Review)

Fredy Wijaya has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11245 )

Change subject: IMPALA-7345: Add the OWNER privilege
..


Patch Set 1:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/11245/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/11245/1//COMMIT_MSG@10
PS1, Line 10:
nit: extra whitespace


http://gerrit.cloudera.org:8080/#/c/11245/1//COMMIT_MSG@10
PS1, Line 10: role
for a role/user?


http://gerrit.cloudera.org:8080/#/c/11245/1//COMMIT_MSG@11
PS1, Line 11:
nit: extra whitespace


http://gerrit.cloudera.org:8080/#/c/11245/1/fe/src/main/java/org/apache/impala/authorization/Privilege.java
File fe/src/main/java/org/apache/impala/authorization/Privilege.java:

http://gerrit.cloudera.org:8080/#/c/11245/1/fe/src/main/java/org/apache/impala/authorization/Privilege.java@68
PS1, Line 68: OWNER("owner",
You should use the code from ALL, i.e.

OWNER("owner", ALL.getCode())



--
To view, visit http://gerrit.cloudera.org:8080/11245
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If63c2faa6daea6deb6d771503fe50943ae070705
Gerrit-Change-Number: 11245
Gerrit-PatchSet: 1
Gerrit-Owner: Adam Holley 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 16 Aug 2018 04:39:51 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7421. Static methods use wrong JNI call function

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11181 )

Change subject: IMPALA-7421. Static methods use wrong JNI call function
..


Patch Set 3: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/11181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If7cde6ca91613b63afe5307f4d819fb24cb17fd6
Gerrit-Change-Number: 11181
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Thu, 16 Aug 2018 04:37:20 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7399: Remove third-party dependencies from junit xml script

David Knupp has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11235 )

Change subject: IMPALA-7399: Remove third-party dependencies from junit xml 
script
..


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11235/4/lib/python/impala_py_lib/jenkins/generate_junitxml.py
File lib/python/impala_py_lib/jenkins/generate_junitxml.py:

http://gerrit.cloudera.org:8080/#/c/11235/4/lib/python/impala_py_lib/jenkins/generate_junitxml.py@64
PS4, Line 64:   """
:   Derive additional conte
> This is too fancy for my tastes. Perhaps just create the structures at the
One last point. There's an additional (again, probably minor) benefit to 
retaining the @property approach. If one uses:

  init:
self.make_root()
self.attach_testsuite()
self.attach_testcase()

...then the order of those lines is significant. If the file gets edited in the 
future and those somehow get shuffled, then the code would break. (It'll be 
obvious, but it's still slightly more brittle.)



--
To view, visit http://gerrit.cloudera.org:8080/11235
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I958ee0d8420b6a4197aaf0a7e0538a566332ea97
Gerrit-Change-Number: 11235
Gerrit-PatchSet: 5
Gerrit-Owner: David Knupp 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Nithya Janarthanan 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Thu, 16 Aug 2018 04:27:49 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7345: Add the OWNER privilege

2018-08-15 Thread Adam Holley (Code Review)

Adam Holley has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/11245


Change subject: IMPALA-7345: Add the OWNER privilege
..

IMPALA-7345: Add the OWNER privilege

This patch adds the OWNER privilege to the set of privileges that can
exist for a role.  The privilege is equivalent to ALL, but cannot be
granted or revoked.  It is granted/revoked by Sentry, if configured,
during CREATE, DROP, or ALTER DATABASE/TABLE SET OWNER statements.

Testing:
- Updated authorization tests
- Ran core tests

Change-Id: If63c2faa6daea6deb6d771503fe50943ae070705
---
M common/thrift/CatalogObjects.thrift
M fe/src/main/java/org/apache/impala/authorization/Privilege.java
M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java
M fe/src/test/java/org/apache/impala/authorization/ImpalaActionFactoryTest.java
4 files changed, 574 insertions(+), 240 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/11245/1
--
To view, visit http://gerrit.cloudera.org:8080/11245
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: If63c2faa6daea6deb6d771503fe50943ae070705
Gerrit-Change-Number: 11245
Gerrit-PatchSet: 1
Gerrit-Owner: Adam Holley

[Impala-ASF-CR] IMPALA-7399: Remove third-party dependencies from junit xml script

David Knupp has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11235 )

Change subject: IMPALA-7399: Remove third-party dependencies from junit xml 
script
..


Patch Set 5:

Hmm. I might bat this aside. :-)

FWIW, my starting point was to look at junit xml output from various Evergreen 
test runs, and match it. They don't all follow the same template. E.g., if 
there's an error, the testcase node also contains a child node like

  

Otherwise, it's simply not there. Same is true for stdout and stderr. In a 
passing test with no meaningful output, the testcase becomes the last element, 
with nothing more than:

  

Patchset 4 produces the same variety of XML formats. Perhaps I was too slavish 
in my attempt to reproduce the same exact structure, but hopefully subtle XML 
parsing bugs won't be an issue as a result.

Admittedly, it's not a perfect replica. Testsuite nodes usually contain 
multiple testcase child nodes -- which isn't true for us, since each invocation 
of the script can only produce a single file, with a single test case. (I did 
test the effect of this. Luckily, Jenkins seems to renders multiple tests 
belonging to a single testsuite correctly, even if the suite is spread out over 
different individual files.)

A last minor point -- I think the current approach more readily lends itself to 
reusability (with minor refactoring), as unlikely as that may be. But we 
thought it unlikely with the mirror-jira.py file as well, until Bikram wound up 
having to redo a lot of the same steps in his dev productivity project because 
the mirror-jira script was not written in such a way that he could reuse it 
easily.


--
To view, visit http://gerrit.cloudera.org:8080/11235
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I958ee0d8420b6a4197aaf0a7e0538a566332ea97
Gerrit-Change-Number: 11235
Gerrit-PatchSet: 5
Gerrit-Owner: David Knupp 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Nithya Janarthanan 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Thu, 16 Aug 2018 04:06:45 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7449: Fix network throughput calculation of DataStreamSender

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11241 )

Change subject: IMPALA-7449: Fix network throughput calculation of 
DataStreamSender
..


Patch Set 2: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/11241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I344ac76c0a1a49b4da3d37d2c547f3d5051ebe24
Gerrit-Change-Number: 11241
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Ho 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 16 Aug 2018 04:01:42 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7406. Avoid memory overhead of FlatBuffer object wrappers for FileDescriptor

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11232 )

Change subject: IMPALA-7406. Avoid memory overhead of FlatBuffer object 
wrappers for FileDescriptor
..


Patch Set 2: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/11232
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I986dd5c05ac8904898d01a6ef3487bd7225e0500
Gerrit-Change-Number: 11232
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Thu, 16 Aug 2018 03:05:23 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7452: fix file check for test disable catalog data ops test

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11243 )

Change subject: IMPALA-7452: fix file check for test_disable_catalog_data_ops 
test
..


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3021/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/11243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7408bc233714620f502af3b8c19400aa3bb1666c
Gerrit-Change-Number: 11243
Gerrit-PatchSet: 2
Gerrit-Owner: Vuk Ercegovac 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 16 Aug 2018 01:44:00 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7452: fix file check for test disable catalog data ops test

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11243 )

Change subject: IMPALA-7452: fix file check for test_disable_catalog_data_ops 
test
..


Patch Set 2: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/11243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7408bc233714620f502af3b8c19400aa3bb1666c
Gerrit-Change-Number: 11243
Gerrit-PatchSet: 2
Gerrit-Owner: Vuk Ercegovac 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 16 Aug 2018 01:43:59 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7452: fix file check for test disable catalog data ops test

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11243 )

Change subject: IMPALA-7452: fix file check for test_disable_catalog_data_ops 
test
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/11243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7408bc233714620f502af3b8c19400aa3bb1666c
Gerrit-Change-Number: 11243
Gerrit-PatchSet: 1
Gerrit-Owner: Vuk Ercegovac 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 16 Aug 2018 01:39:28 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7452: fix file check for test disable catalog data ops test

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11243 )

Change subject: IMPALA-7452: fix file check for test_disable_catalog_data_ops 
test
..


Patch Set 1:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/366/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7408bc233714620f502af3b8c19400aa3bb1666c
Gerrit-Change-Number: 11243
Gerrit-PatchSet: 1
Gerrit-Owner: Vuk Ercegovac 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 16 Aug 2018 01:36:09 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7421. Static methods use wrong JNI call function

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11181 )

Change subject: IMPALA-7421. Static methods use wrong JNI call function
..


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3020/ 
DRY_RUN=true


--
To view, visit http://gerrit.cloudera.org:8080/11181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If7cde6ca91613b63afe5307f4d819fb24cb17fd6
Gerrit-Change-Number: 11181
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Thu, 16 Aug 2018 01:20:53 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7140 fix: fix NDV calculation for partition columns in LocalFsTable

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11230 )

Change subject: IMPALA-7140 fix: fix NDV calculation for partition columns in 
LocalFsTable
..


Patch Set 1: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/3015/


--
To view, visit http://gerrit.cloudera.org:8080/11230
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0a86750d52dcd744ace030a3c1ec70510d054acf
Gerrit-Change-Number: 11230
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Thu, 16 Aug 2018 01:20:51 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7452: fix file check for test disable catalog data ops test

Vuk Ercegovac has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/11243


Change subject: IMPALA-7452: fix file check for test_disable_catalog_data_ops 
test
..

IMPALA-7452: fix file check for test_disable_catalog_data_ops test

The test's regex did not handle s3 paths. The fix loosens the check
to allow for these paths.

Change-Id: I7408bc233714620f502af3b8c19400aa3bb1666c
---
M tests/custom_cluster/test_disable_catalog_data_ops.py
1 file changed, 2 insertions(+), 4 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/11243/1
--
To view, visit http://gerrit.cloudera.org:8080/11243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7408bc233714620f502af3b8c19400aa3bb1666c
Gerrit-Change-Number: 11243
Gerrit-PatchSet: 1
Gerrit-Owner: Vuk Ercegovac

[Impala-ASF-CR] [DOCS] A typo fixed in titles

Alex Rodoni has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/11242 )

Change subject: [DOCS] A typo fixed in titles
..

[DOCS] A typo fixed in titles

Change-Id: Iefcec53a6272a9221800678b9ed1ef52a898ed42
Reviewed-on: http://gerrit.cloudera.org:8080/11242
Reviewed-by: Alex Rodoni 
Tested-by: Impala Public Jenkins 
---
M docs/topics/impala_thread_reservation_aggregate_limit.xml
M docs/topics/impala_thread_reservation_limit.xml
2 files changed, 3 insertions(+), 2 deletions(-)

Approvals:
  Alex Rodoni: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/11242
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Iefcec53a6272a9221800678b9ed1ef52a898ed42
Gerrit-Change-Number: 11242
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins

[Impala-ASF-CR] [DOCS] A typo fixed in titles

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11242 )

Change subject: [DOCS] A typo fixed in titles
..


Patch Set 1: Verified+1

Build Successful

https://jenkins.impala.io/job/gerrit-docs-auto-test/50/ : Doc tests passed.


--
To view, visit http://gerrit.cloudera.org:8080/11242
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefcec53a6272a9221800678b9ed1ef52a898ed42
Gerrit-Change-Number: 11242
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 16 Aug 2018 01:09:42 +
Gerrit-HasComments: No

[Impala-ASF-CR] [DOCS] A typo fixed in titles

Alex Rodoni has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11242 )

Change subject: [DOCS] A typo fixed in titles
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/11242
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefcec53a6272a9221800678b9ed1ef52a898ed42
Gerrit-Change-Number: 11242
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 16 Aug 2018 01:00:12 +
Gerrit-HasComments: No

[Impala-ASF-CR] [DOCS] A typo fixed in titles

Alex Rodoni has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/11242


Change subject: [DOCS] A typo fixed in titles
..

[DOCS] A typo fixed in titles

Change-Id: Iefcec53a6272a9221800678b9ed1ef52a898ed42
---
M docs/topics/impala_thread_reservation_aggregate_limit.xml
M docs/topics/impala_thread_reservation_limit.xml
2 files changed, 3 insertions(+), 2 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/11242/1
--
To view, visit http://gerrit.cloudera.org:8080/11242
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iefcec53a6272a9221800678b9ed1ef52a898ed42
Gerrit-Change-Number: 11242
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni

[Impala-ASF-CR] [DOCS] A typo fixed in titles

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11242 )

Change subject: [DOCS] A typo fixed in titles
..


Patch Set 1:

Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/50/

Testing docs change - this change appears to modify docs/ and no code. This is 
experimental - please report any issues to tarmstr...@cloudera.com or on this 
JIRA: IMPALA-7317


--
To view, visit http://gerrit.cloudera.org:8080/11242
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefcec53a6272a9221800678b9ed1ef52a898ed42
Gerrit-Change-Number: 11242
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 16 Aug 2018 00:59:16 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7449: Fix network throughput calculation of DataStreamSender

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11241 )

Change subject: IMPALA-7449: Fix network throughput calculation of 
DataStreamSender
..


Patch Set 1:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/365/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/11241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I344ac76c0a1a49b4da3d37d2c547f3d5051ebe24
Gerrit-Change-Number: 11241
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Ho 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 16 Aug 2018 00:50:19 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7399: Remove third-party dependencies from junit xml script

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11235 )

Change subject: IMPALA-7399: Remove third-party dependencies from junit xml 
script
..


Patch Set 5:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/364/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11235
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I958ee0d8420b6a4197aaf0a7e0538a566332ea97
Gerrit-Change-Number: 11235
Gerrit-PatchSet: 5
Gerrit-Owner: David Knupp 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Nithya Janarthanan 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Thu, 16 Aug 2018 00:47:00 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7449: Fix network throughput calculation of DataStreamSender

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11241 )

Change subject: IMPALA-7449: Fix network throughput calculation of 
DataStreamSender
..


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3019/ 
DRY_RUN=true


--
To view, visit http://gerrit.cloudera.org:8080/11241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I344ac76c0a1a49b4da3d37d2c547f3d5051ebe24
Gerrit-Change-Number: 11241
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Ho 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 16 Aug 2018 00:44:29 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7421. Static methods use wrong JNI call function

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11181 )

Change subject: IMPALA-7421. Static methods use wrong JNI call function
..


Patch Set 3:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/363/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If7cde6ca91613b63afe5307f4d819fb24cb17fd6
Gerrit-Change-Number: 11181
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Thu, 16 Aug 2018 00:42:54 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7449: Fix network throughput calculation of DataStreamSender

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11241 )

Change subject: IMPALA-7449: Fix network throughput calculation of 
DataStreamSender
..


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3018/ 
DRY_RUN=true


--
To view, visit http://gerrit.cloudera.org:8080/11241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I344ac76c0a1a49b4da3d37d2c547f3d5051ebe24
Gerrit-Change-Number: 11241
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Ho 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 16 Aug 2018 00:34:04 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7436: initial fetch-from-catalogd implementation

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11182 )

Change subject: IMPALA-7436: initial fetch-from-catalogd implementation
..


Patch Set 6:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/360/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11182
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If49207fc592b1cc552fbcc7199568b6833f86901
Gerrit-Change-Number: 11182
Gerrit-PatchSet: 6
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Thu, 16 Aug 2018 00:28:05 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7447. Evict LocalCatalog cache entries based on size

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11231 )

Change subject: IMPALA-7447. Evict LocalCatalog cache entries based on size
..


Patch Set 2:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/362/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11231
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia96af49b35c17e505b7b6785e78d140939085d91
Gerrit-Change-Number: 11231
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Thu, 16 Aug 2018 00:26:56 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7399: Remove third-party dependencies from junit xml script

Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11235 )

Change subject: IMPALA-7399: Remove third-party dependencies from junit xml 
script
..


Patch Set 5:

I don't know if it's bad form, but I figured I'd just push a proposal here via 
PS5 instead of figuring out how to use gist or cutting and pasting an entire 
file of python. For comparison, doing string interpolation is about the same 
size as the original (160 lines), and beats generating the XML via code. It's 
about the same as the original. I didn't test it with Jenkins (or diff), so I'm 
sure it'll take another few iterations to figure that out.

I'm definitely not tied to it; I just wanted to see if I buy my own arguments 
here.

As a reader of the PS4 code, I got confused by the functions. I think if we go 
that route, it would be clearer to do:

init:
  self.make_root()
  self.attach_testsuite()
  self.attach_testcase()

i.e., make it clear that we're constituting this thing.

I do worry that I'm bike-shedding here. Feel free to bat me away, and we can 
move on :)


--
To view, visit http://gerrit.cloudera.org:8080/11235
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I958ee0d8420b6a4197aaf0a7e0538a566332ea97
Gerrit-Change-Number: 11235
Gerrit-PatchSet: 5
Gerrit-Owner: David Knupp 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Nithya Janarthanan 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Thu, 16 Aug 2018 00:26:31 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7442: reduce mem requirement of semi-joins-exhaustive

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11223 )

Change subject: IMPALA-7442: reduce mem requirement of semi-joins-exhaustive
..


Patch Set 3: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/11223
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9d9480cad6bf8222abe990e7046498a0531e2849
Gerrit-Change-Number: 11223
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 16 Aug 2018 00:21:53 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7442: reduce mem requirement of semi-joins-exhaustive

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/11223 )

Change subject: IMPALA-7442: reduce mem requirement of semi-joins-exhaustive
..

IMPALA-7442: reduce mem requirement of semi-joins-exhaustive

The test started running into IMPALA-7446, maybe because of
a timing change. This appears to have always been possible.

The fix is to reduce the memory requirement of the test.
IMPALA-2256 is no longer really possible because the
BufferedTupleStream code was simplified to avoid the 32-bit
row index limitation, so we're not losing important coverage
on the current code with this change.

Testing:
Ran test in a loop to confirm it did not OOM.

Change-Id: I9d9480cad6bf8222abe990e7046498a0531e2849
Reviewed-on: http://gerrit.cloudera.org:8080/11223
Reviewed-by: Impala Public Jenkins 
Tested-by: Impala Public Jenkins 
---
M 
testdata/workloads/functional-query/queries/QueryTest/semi-joins-exhaustive.test
1 file changed, 7 insertions(+), 8 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

--
To view, visit http://gerrit.cloudera.org:8080/11223
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I9d9480cad6bf8222abe990e7046498a0531e2849
Gerrit-Change-Number: 11223
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong

[Impala-ASF-CR] IMPALA-7399: Remove third-party dependencies from junit xml script

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11235 )

Change subject: IMPALA-7399: Remove third-party dependencies from junit xml 
script
..


Patch Set 5:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/11235/5/lib/python/impala_py_lib/jenkins/generate_junitxml.py
File lib/python/impala_py_lib/jenkins/generate_junitxml.py:

http://gerrit.cloudera.org:8080/#/c/11235/5/lib/python/impala_py_lib/jenkins/generate_junitxml.py@39
PS5, Line 39: e
flake8: E501 line too long (93 > 90 characters)


http://gerrit.cloudera.org:8080/#/c/11235/5/lib/python/impala_py_lib/jenkins/generate_junitxml.py@48
PS5, Line 48: def xml(phase, step, error_msg, stdout, stderr, elapsed_time):
flake8: E302 expected 2 blank lines, found 1


http://gerrit.cloudera.org:8080/#/c/11235/5/lib/python/impala_py_lib/jenkins/generate_junitxml.py@63
PS5, Line 63: def get_content(file_or_string=None):
flake8: E302 expected 2 blank lines, found 1


http://gerrit.cloudera.org:8080/#/c/11235/5/lib/python/impala_py_lib/jenkins/generate_junitxml.py@153
PS5, Line 153: '
flake8: E501 line too long (93 > 90 characters)



--
To view, visit http://gerrit.cloudera.org:8080/11235
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I958ee0d8420b6a4197aaf0a7e0538a566332ea97
Gerrit-Change-Number: 11235
Gerrit-PatchSet: 5
Gerrit-Owner: David Knupp 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Nithya Janarthanan 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Thu, 16 Aug 2018 00:20:13 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7399: Remove third-party dependencies from junit xml script

Philip Zeyliger has uploaded a new patch set (#5) to the change originally 
created by David Knupp. ( http://gerrit.cloudera.org:8080/11235 )

Change subject: IMPALA-7399: Remove third-party dependencies from junit xml 
script
..

IMPALA-7399: Remove third-party dependencies from junit xml script

The original patch for this Jira relied on a third party python lib
for generating Junit XML output, That proved to be limiting because
setting up the necessary virtualenv across a variety of dev and test
scenarios (private dev environment, jenkins.impala.io, and others)
proved to be confusing and messy.

This update to the script maintains the same functionality and the
same interface, but uses only the python standard library. A symlink
has also been added to Impala/bin for convenience.

Change-Id: I958ee0d8420b6a4197aaf0a7e0538a566332ea97
---
A bin/generate_junitxml.py
M bin/rat_exclude_files.txt
A lib/python/README.md
M lib/python/impala_py_lib/jenkins/generate_junitxml.py
M lib/python/requirements.txt
5 files changed, 67 insertions(+), 58 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/11235/5
--
To view, visit http://gerrit.cloudera.org:8080/11235
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I958ee0d8420b6a4197aaf0a7e0538a566332ea97
Gerrit-Change-Number: 11235
Gerrit-PatchSet: 5
Gerrit-Owner: David Knupp 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Nithya Janarthanan 
Gerrit-Reviewer: Philip Zeyliger

[Impala-ASF-CR] IMPALA-7096: restore scanner thread memory heuristics

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11103 )

Change subject: IMPALA-7096: restore scanner thread memory heuristics
..


Patch Set 11: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/3016/


--
To view, visit http://gerrit.cloudera.org:8080/11103
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib9907fa8c4d2b0b85f67f4f160899c1c258ad82b
Gerrit-Change-Number: 11103
Gerrit-PatchSet: 11
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 16 Aug 2018 00:19:21 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7421. Static methods use wrong JNI call function

Hello Bharath Vissapragada, Philip Zeyliger, Csaba Ringhofer, Impala Public 
Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/11181

to look at the new patch set (#3).

Change subject: IMPALA-7421. Static methods use wrong JNI call function
..

IMPALA-7421. Static methods use wrong JNI call function

Various places throughout the code were using the JNI CallObjectMethod
function when in fact they should have been using
CallStaticObjectMethod. It appears that this doesn't cause crashes in
release builds, but if running with -Xcheck:jni, it causes an assertion
failure:

   guarantee(method->size_of_parameters() == size_of_parameters())
  failed: wrong no. of arguments pushed

This patch cleans up JniUtil a bit to add a "fluent" style utility for
calling JNI functions. This calling style should make the above mistake
more obvious because it forces the caller to write 'on_class(...)' for
static methods and 'on_instance(...)' for instance methods.

Change-Id: If7cde6ca91613b63afe5307f4d819fb24cb17fd6
---
M be/src/common/init.cc
M be/src/common/status.h
M be/src/service/fe-support.h
M be/src/service/frontend.cc
M be/src/util/backend-gflag-util.h
M be/src/util/jni-util.cc
M be/src/util/jni-util.h
M be/src/util/logging-support.cc
M be/src/util/logging-support.h
M be/src/util/zip-util.cc
M be/src/util/zip-util.h
11 files changed, 184 insertions(+), 94 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/11181/3
--
To view, visit http://gerrit.cloudera.org:8080/11181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If7cde6ca91613b63afe5307f4d819fb24cb17fd6
Gerrit-Change-Number: 11181
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Todd Lipcon

[Impala-ASF-CR] IMPALA-7421. Static methods use wrong JNI call function

Todd Lipcon has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11181 )

Change subject: IMPALA-7421. Static methods use wrong JNI call function
..


Patch Set 2:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/11181/2/be/src/service/frontend.cc
File be/src/service/frontend.cc:

http://gerrit.cloudera.org:8080/#/c/11181/2/be/src/service/frontend.cc@229
PS2, Line 229: if (!err.empty()) {
 : return Status(err);
> single line.
Done


http://gerrit.cloudera.org:8080/#/c/11181/2/be/src/util/jni-util.h
File be/src/util/jni-util.h:

http://gerrit.cloudera.org:8080/#/c/11181/2/be/src/util/jni-util.h@273
PS2, Line 273: ObjectToResult
> Digging into this, more information in IMPALA-2615 and
Done


http://gerrit.cloudera.org:8080/#/c/11181/2/be/src/util/jni-util.h@451
PS2, Line 451: ||
> Make it more strict with ^?
Done


http://gerrit.cloudera.org:8080/#/c/11181/2/be/src/util/jni-util.h@472
PS2, Line 472: jstring jstr = static_cast(obj);
 :   const char *str = env_->GetStringUTFChars(jstr, &is_copy);
 :   RETURN_ERROR_IF_EXC(env_);
 :   *result = str;
 :   env_->ReleaseStringUTFChars(jstr, str);
 :   RETURN_ERROR_IF_EXC(env_);
> Use JniUtfCharGuard?
Done



--
To view, visit http://gerrit.cloudera.org:8080/11181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If7cde6ca91613b63afe5307f4d819fb24cb17fd6
Gerrit-Change-Number: 11181
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Thu, 16 Aug 2018 00:05:26 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7437. LRU caching of partitions in impalad

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11208 )

Change subject: IMPALA-7437. LRU caching of partitions in impalad
..


Patch Set 4:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/361/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11208
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a57521ad851da605604a1e7c48d3d6627da5df5
Gerrit-Change-Number: 11208
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Thu, 16 Aug 2018 00:00:44 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7399: Remove third-party dependencies from junit xml script

David Knupp has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11235 )

Change subject: IMPALA-7399: Remove third-party dependencies from junit xml 
script
..


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11235/4/lib/python/impala_py_lib/jenkins/generate_junitxml.py
File lib/python/impala_py_lib/jenkins/generate_junitxml.py:

http://gerrit.cloudera.org:8080/#/c/11235/4/lib/python/impala_py_lib/jenkins/generate_junitxml.py@38
PS4, Line 38:   """A Junit XML style report parseable by Jenkins for reporting 
build status.
> add a comment here about how to use this?
I'll wait for your reply as to whether or not to do away with the class.



--
To view, visit http://gerrit.cloudera.org:8080/11235
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I958ee0d8420b6a4197aaf0a7e0538a566332ea97
Gerrit-Change-Number: 11235
Gerrit-PatchSet: 4
Gerrit-Owner: David Knupp 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Nithya Janarthanan 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Wed, 15 Aug 2018 23:56:12 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7399: Remove third-party dependencies from junit xml script

David Knupp has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11235 )

Change subject: IMPALA-7399: Remove third-party dependencies from junit xml 
script
..


Patch Set 4:

(1 comment)

> I don't feel all that strongly about this stuff, but let me know what you 
> think.

Sure, happy to discuss.

To me one, long function seems like it would be annoying to read. I actually 
started to write this as a series of functions, but quickly switched to a class 
for a couple of (probably minor) reasons.

- Multiple functions took the same params, just with subtle variation, e.g., 
the immediate parent of the child that was being created, phase, step, and 
sometimes time or timestamp. It made main() and the function signatures 
repetitive and messy. The class seemed like a cleaner approach, and it felt 
like the amount of code decreased. I thought it was more readable.

- I tend to write my classes in a similar style. Member variables that are just 
scalars get assigned immediately, and anything that requires some computation 
gets a @property, though I agree there's not a lot to be saved in terms of lazy 
loading here. But it feels like enough of a language convention that I don't 
really think about it.

- When I find myself calling the same function four or five times in immediate 
succession, I usually default to a loop, instead of writing root.set(a, b), 
root.set(c, d), root.set(x, y), etc. I don't mind having essentially a temp 
dictionary.

As you said, it's a matter of style. I can change it.

http://gerrit.cloudera.org:8080/#/c/11235/4/bin/rat_exclude_files.txt
File bin/rat_exclude_files.txt:

http://gerrit.cloudera.org:8080/#/c/11235/4/bin/rat_exclude_files.txt@13
PS4, Line 13: bin/generate_junitxml.py
> Does rat not ignore symlinks by default?
Sadly, no.

https://jenkins.impala.io/job/gerrit-code-review-checks/353/

  19:39:45 ] UNAPPROVED: bin/generate_junitxml.py



--
To view, visit http://gerrit.cloudera.org:8080/11235
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I958ee0d8420b6a4197aaf0a7e0538a566332ea97
Gerrit-Change-Number: 11235
Gerrit-PatchSet: 4
Gerrit-Owner: David Knupp 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Nithya Janarthanan 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Wed, 15 Aug 2018 23:52:28 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7450. Set thread name during refresh/load operations

Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11228 )

Change subject: IMPALA-7450. Set thread name during refresh/load operations
..


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11228/1/fe/src/main/java/org/apache/impala/util/ThreadNameAnnotator.java
File fe/src/main/java/org/apache/impala/util/ThreadNameAnnotator.java:

http://gerrit.cloudera.org:8080/#/c/11228/1/fe/src/main/java/org/apache/impala/util/ThreadNameAnnotator.java@48
PS1, Line 48:   public void close() {
> maybe I'll evolve this class into something a bit more broad like:
This seems nice. Can be done later, obviously.



--
To view, visit http://gerrit.cloudera.org:8080/11228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7c850d6bb2eedc375ee567c19eb17add335f60c
Gerrit-Change-Number: 11228
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Wed, 15 Aug 2018 23:51:09 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7450. Set thread name during refresh/load operations

Todd Lipcon has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11228 )

Change subject: IMPALA-7450. Set thread name during refresh/load operations
..


Patch Set 1:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/11228/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/11228/1//COMMIT_MSG@25
PS1, Line 25: This patch is tricky to automate tests for, but I verified it 
manually
> What do you think of the following test:
k, I'll add a simple test. I think just doing it single-threaded and asserting 
on the expected thread name from the same thread is probably easiest instead of 
futzing with semaphores.


http://gerrit.cloudera.org:8080/#/c/11228/1/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java:

http://gerrit.cloudera.org:8080/#/c/11228/1/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@1005
PS1, Line 1005:   String annotation = String.format("invalidating 
metadata - %s/%s dbs complete",
> In an ideal world, the RPC from the impalad would be carrying a thread id,
yea, distributed tracing is also on my list, but wanted to start with this 
simple incremental improvement before I open the can of worms of which 
framework to use (eg opentracing vs opencensus)


http://gerrit.cloudera.org:8080/#/c/11228/1/fe/src/main/java/org/apache/impala/util/ThreadNameAnnotator.java
File fe/src/main/java/org/apache/impala/util/ThreadNameAnnotator.java:

http://gerrit.cloudera.org:8080/#/c/11228/1/fe/src/main/java/org/apache/impala/util/ThreadNameAnnotator.java@48
PS1, Line 48:   public void close() {
> Do we want this to also double as something that logs long requests? i.e.,
maybe I'll evolve this class into something a bit more broad like:

try (InstrumentedScope s = new InstrumentedScope("blah blah")
  .annotateThreadName()
  .logIfSlow(1, TimeUnit.SECONDS)
  .incrementMetric(METRIC_LOAD_TIME)
  .addToProfile("FooTime"))

etc. What do you think?



--
To view, visit http://gerrit.cloudera.org:8080/11228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7c850d6bb2eedc375ee567c19eb17add335f60c
Gerrit-Change-Number: 11228
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Wed, 15 Aug 2018 23:46:50 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7437. LRU caching of partitions in impalad

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11208 )

Change subject: IMPALA-7437. LRU caching of partitions in impalad
..


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11208/4/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java
File fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java:

http://gerrit.cloudera.org:8080/#/c/11208/4/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java@292
PS4, Line 292: checkResponse(p.isSetId(), req, "response 
missing partition IDs for partition %s",
line too long (98 > 90)



--
To view, visit http://gerrit.cloudera.org:8080/11208
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a57521ad851da605604a1e7c48d3d6627da5df5
Gerrit-Change-Number: 11208
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Wed, 15 Aug 2018 23:41:47 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7406. Avoid memory overhead of FlatBuffer object wrappers for FileDescriptor

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11232 )

Change subject: IMPALA-7406. Avoid memory overhead of FlatBuffer object 
wrappers for FileDescriptor
..


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3017/ 
DRY_RUN=true


--
To view, visit http://gerrit.cloudera.org:8080/11232
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I986dd5c05ac8904898d01a6ef3487bd7225e0500
Gerrit-Change-Number: 11232
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Wed, 15 Aug 2018 23:40:32 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7436: initial fetch-from-catalogd implementation

Hello Bharath Vissapragada, Tianyi Wang, Impala Public Jenkins, Vuk Ercegovac,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/11182

to look at the new patch set (#6).

Change subject: IMPALA-7436: initial fetch-from-catalogd implementation
..

IMPALA-7436: initial fetch-from-catalogd implementation

This patch adds a new RPC to the catalogd which allows a client to fetch
a partial view of table or database metadata. Various subsets of
information can be specified and are sent back in fairly "raw" format.

A new MetaProvider implementation is added which uses this API to
support granular fetching of metadata into the impalad. The interface
had to be reworked in a few ways to support this:

- This API uses partition IDs instead of names to specify them. So, the
  listPartitions API now returns opaque PartitionRefs which are passed
  back to the MetaProvider when loading more partition details. The new
  implementation stores the IDs in these refs while the direct-to-HMS
  implementation just uses names.

- The fetching of file descriptors was merged into the loading of other
  partition metadata. I couldn't think of any cases where we needed to
  list partition details without also fetching the file descriptors so
  it simplified things a bit to merge the two. This was a lot easier to
  implement for CatalogdMetaProvider since the file metadata is stored
  by partition rather than looked up by a directory as in the previous
  API.

  This necessitated moving some of the logic out of LocalFsTable into
  DirectMetaProvider, so LocalFsTable no longer deals directly with HDFS
  APIs like FileStatus.

- The handling of "default partition" for an unpartitioned table moved
  into the MetaProvider implementations itself instead of LocalFsTable.
  This is because the CatalogdProvider sees the "default partition" as a
  partition that actually has an identifier on the catalogd, whereas the
  DirectMetaProvider does not. So, now both providers export the
  "default partition" as a partition like all the others.

This patch also starts to address one of the potential semantic risks of
partial caching on the impalad. If one query fetches some subset of
partitions, then a DDL occurs to change the table metadata, and another
query is submitted, we want to ensure that the metadata for the latter
query still reads a consistent snapshot. In other words, we need to
ensure that the metadata like partition list and table schema come from
the same snapshot as the finer-grained metadata like partition contents.

In order to implement this, the MetadataProvider API now requires that
callers use a 'TableRef' object to specify the table to be read, instead
of the dbName/tableName. In the DirectMetaProvider we don't have any
convenient version numbers for a table, so the TableRef just
encapsulates the naming. In the CatalogdMetaProvider, we additionally
store the version number of the table, and then all subsequent requests
verify that the version number has not changed. If it detects a
concurrent modification, an exception is thrown. The frontend catches
this exception and triggers a "re-plan".

I tested this "replan" manually for now by running queries against a
table in a loop from my shell while issuing concurrent 'refresh' queries
to that same table. I verified that the warning log indicated the query
had been replanned but the query itself was not disrupted. I filed
IMPALA-7438 to add an automated version of this test.

Change-Id: If49207fc592b1cc552fbcc7199568b6833f86901
---
M be/src/catalog/catalog-server.cc
M be/src/catalog/catalog-service-client-wrapper.h
M be/src/catalog/catalog.cc
M be/src/catalog/catalog.h
M be/src/exec/catalog-op-executor.cc
M be/src/exec/catalog-op-executor.h
M be/src/service/fe-support.cc
M common/fbs/CMakeLists.txt
M common/thrift/CatalogService.thrift
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/ColumnStats.java
M fe/src/main/java/org/apache/impala/catalog/Db.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/IncompleteTable.java
M fe/src/main/java/org/apache/impala/catalog/Table.java
A fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java
M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java
A 
fe/src/main/java/org/apache/impala/catalog/local/InconsistentMetadataFetchException.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalHbaseTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/i

[Impala-ASF-CR] IMPALA-7437. LRU caching of partitions in impalad

Hello Bharath Vissapragada, Tianyi Wang, Impala Public Jenkins, Vuk Ercegovac,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/11208

to look at the new patch set (#4).

Change subject: IMPALA-7437. LRU caching of partitions in impalad
..

IMPALA-7437. LRU caching of partitions in impalad

This changes the CatalogdMetaProvider to use a Guava-based LRU cache.
The eviction strategy is currently time-based (1 hour), and it only
performs caching of partition information and table column statistics.
It does not cache the table entries themselves, which means that we
don't need to do any invalidation propagation via the statestore quite
yet. Instead, every query will do an initial fetch of the table metadata
in order to know the current version number. That version number is then
used as part of the cache key for all further metadata, so when the
version number changes, all of the prior cache entries become
"unreachable" and effectively evicted.

Initially, I attempted to implement this by adding a new MetaProvider
implementation that would transparently wrap another MetaProvider
implementation (either catalogd-based or direct-from-source). However, I
found that I wanted to use catalogd-based implementation details like
the version number in the cache key, and trying to abstract this behind
an interface wasn't very clear. So, I elected to just embed the caching
logic into the CatalogdMetaProvider itself.

This includes some basic unit testing and I also verified that some
query tests like TPCH pass.

Change-Id: I9a57521ad851da605604a1e7c48d3d6627da5df5
---
M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
A fe/src/test/java/org/apache/impala/catalog/local/CatalogdMetaProviderTest.java
4 files changed, 436 insertions(+), 37 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/08/11208/4
--
To view, visit http://gerrit.cloudera.org:8080/11208
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9a57521ad851da605604a1e7c48d3d6627da5df5
Gerrit-Change-Number: 11208
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Vuk Ercegovac

[Impala-ASF-CR] IMPALA-7447. Evict LocalCatalog cache entries based on size

Hello Bharath Vissapragada, Tianyi Wang, Impala Public Jenkins, Vuk Ercegovac,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/11231

to look at the new patch set (#2).

Change subject: IMPALA-7447. Evict LocalCatalog cache entries based on size
..

IMPALA-7447. Evict LocalCatalog cache entries based on size

This pulls in the 'sizeof' library from ehcache (Apache-licensed) and
uses it to implement size-based eviction of cache entries in
LocalCatalog.

This is difficult to test without being quite fragile to small changes
in the cached structures. However, I added a simple unit test as a
general sanity-check that it is computing some reasonable result.

Change-Id: Ia96af49b35c17e505b7b6785e78d140939085d91
---
M be/src/runtime/exec-env.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/pom.xml
M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java
M fe/src/test/java/org/apache/impala/catalog/local/CatalogdMetaProviderTest.java
7 files changed, 109 insertions(+), 11 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/11231/2
--
To view, visit http://gerrit.cloudera.org:8080/11231
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia96af49b35c17e505b7b6785e78d140939085d91
Gerrit-Change-Number: 11231
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Vuk Ercegovac

[Impala-ASF-CR] IMPALA-7436: initial fetch-from-catalogd implementation

Todd Lipcon has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11182 )

Change subject: IMPALA-7436: initial fetch-from-catalogd implementation
..


Patch Set 5:

(10 comments)

http://gerrit.cloudera.org:8080/#/c/11182/5/be/src/service/fe-support.cc
File be/src/service/fe-support.cc:

http://gerrit.cloudera.org:8080/#/c/11182/5/be/src/service/fe-support.cc@536
PS5, Line 536: NULL
> nit: use nullptr for these to be consistent within the function. ideally we
Done


http://gerrit.cloudera.org:8080/#/c/11182/5/common/thrift/CatalogService.thrift
File common/thrift/CatalogService.thrift:

http://gerrit.cloudera.org:8080/#/c/11182/5/common/thrift/CatalogService.thrift@283
PS5, Line 283: partitions
> is this guaranteed to have info for each partition that was requested? if n
yep, guaranteed. It would have thrown an NPE if you request a missing one. 
Added a check on the server side for this and will document in the thrift file. 
I also rejiggered the API for requesting partition info a bit so that 
"partition list" is just the same as "ask for the names of all partitions" 
instead of being handled differently.


http://gerrit.cloudera.org:8080/#/c/11182/5/common/thrift/CatalogService.thrift@311
PS5, Line 311:   2: optional TCatalogServiceRequestHeader header
> heh, I omitted these for the incremental stats pull and nothing complained.
yea, this seems unused, i'll just remove it from this RPC.


http://gerrit.cloudera.org:8080/#/c/11182/5/common/thrift/CatalogService.thrift@324
PS5, Line 324: // The status of the operation, OK if the operation was 
successful.
> nit: indentation off
Done


http://gerrit.cloudera.org:8080/#/c/11182/5/common/thrift/CatalogService.thrift@325
PS5, Line 325: optional
> make it required if it'll always be set? I think I went with not setting it
I fiddled with this a bit but it ends up being a little messy since we have 
several places we construct these response objects on the Java side. I think a 
convention of "no status means OK" is fairly normal. I'll add a comment 
indicating it.


http://gerrit.cloudera.org:8080/#/c/11182/5/common/thrift/CatalogService.thrift@417
PS5, Line 417:   TGetPartialCatalogObjectResponse GetPartialCatalogObject(
> pls add a comment
Done


http://gerrit.cloudera.org:8080/#/c/11182/5/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java:

http://gerrit.cloudera.org:8080/#/c/11182/5/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@1861
PS5, Line 1861: table
> I had a bit of trouble with this one for pulling incremental stats. For exa
yea I was a bit confused by that comment as well. From what I can tell, it can 
return an IncompleteTable with a bad "cause". I'll try to add a test case that 
provokes this.


http://gerrit.cloudera.org:8080/#/c/11182/5/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
File fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java:

http://gerrit.cloudera.org:8080/#/c/11182/5/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java@100
PS5, Line 100:  * index 'dstIndex' instead of the original index 
'origIndex'.
> perhaps add why this is needed.
there's a bit comment at the call site. Is that sufficient? Didn't want to 
duplicate the explanation.


http://gerrit.cloudera.org:8080/#/c/11182/5/fe/src/main/java/org/apache/impala/catalog/Table.java
File fe/src/main/java/org/apache/impala/catalog/Table.java:

http://gerrit.cloudera.org:8080/#/c/11182/5/fe/src/main/java/org/apache/impala/catalog/Table.java@381
PS5, Line 381:   public TGetPartialCatalogObjectResponse getPartialInfo(
> pls add a comment for this one stating that its only called on the catalog,
Done


http://gerrit.cloudera.org:8080/#/c/11182/5/fe/src/main/java/org/apache/impala/service/Frontend.java
File fe/src/main/java/org/apache/impala/service/Frontend.java:

http://gerrit.cloudera.org:8080/#/c/11182/5/fe/src/main/java/org/apache/impala/service/Frontend.java@1051
PS5, Line 1051: new EventSequence("Query Compilation");
> agreed. there's enough going on here so a todo works for me.
k, removed from this patch.



--
To view, visit http://gerrit.cloudera.org:8080/11182
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If49207fc592b1cc552fbcc7199568b6833f86901
Gerrit-Change-Number: 11182
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Wed, 15 Aug 2018 23:22:08 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7096: restore scanner thread memory heuristics

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11103 )

Change subject: IMPALA-7096: restore scanner thread memory heuristics
..


Patch Set 10:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/359/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11103
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib9907fa8c4d2b0b85f67f4f160899c1c258ad82b
Gerrit-Change-Number: 11103
Gerrit-PatchSet: 10
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 15 Aug 2018 23:12:24 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7392: [DOCS] SCAN BYTES LIMIT query option documented

Alex Rodoni has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/11240 )

Change subject: IMPALA-7392: [DOCS] SCAN_BYTES_LIMIT query option documented
..

IMPALA-7392: [DOCS] SCAN_BYTES_LIMIT query option documented

Change-Id: I6430e06cabe21b8080239f3225d3bfdd5cc502cb
Reviewed-on: http://gerrit.cloudera.org:8080/11240
Tested-by: Impala Public Jenkins 
Reviewed-by: Tim Armstrong 
---
M docs/impala.ditamap
A docs/topics/impala_scan_bytes_limit.xml
2 files changed, 132 insertions(+), 0 deletions(-)

Approvals:
  Impala Public Jenkins: Verified
  Tim Armstrong: Looks good to me, approved

--
To view, visit http://gerrit.cloudera.org:8080/11240
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I6430e06cabe21b8080239f3225d3bfdd5cc502cb
Gerrit-Change-Number: 11240
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong

[Impala-ASF-CR] IMPALA-7392: [DOCS] SCAN BYTES LIMIT query option documented

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11240 )

Change subject: IMPALA-7392: [DOCS] SCAN_BYTES_LIMIT query option documented
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/11240
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6430e06cabe21b8080239f3225d3bfdd5cc502cb
Gerrit-Change-Number: 11240
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 15 Aug 2018 22:54:39 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7096: restore scanner thread memory heuristics

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11103 )

Change subject: IMPALA-7096: restore scanner thread memory heuristics
..


Patch Set 11:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3016/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/11103
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib9907fa8c4d2b0b85f67f4f160899c1c258ad82b
Gerrit-Change-Number: 11103
Gerrit-PatchSet: 11
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 15 Aug 2018 22:40:11 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7096: restore scanner thread memory heuristics

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11103 )

Change subject: IMPALA-7096: restore scanner thread memory heuristics
..


Patch Set 11: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/11103
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib9907fa8c4d2b0b85f67f4f160899c1c258ad82b
Gerrit-Change-Number: 11103
Gerrit-PatchSet: 11
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 15 Aug 2018 22:40:10 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7096: restore scanner thread memory heuristics

Hello Bikramjeet Vig, Impala Public Jenkins, Dan Hecht,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/11103

to look at the new patch set (#10).

Change subject: IMPALA-7096: restore scanner thread memory heuristics
..

IMPALA-7096: restore scanner thread memory heuristics

This restores some of the heuristics removed in IMPALA-4835 that can
help scans from hitting OOM conditions. The heuristics are implemented
at the query level rather than in each scan node in isolation.

Introduce a ScannerMemLimiter class that belongs to the QueryState that
tracks the amount of memory estimated to be consumed for all scanner
threads running for the query on the current backend.

Also check soft memory limits to see if scanner threads should be
started or the current scanner thread should stop.

The long-term plan is to switch to the MT scan node implementations.
When that happens this code can be removed. In the meantime this
code is imperfect but will help avoid OOM in many scenarios.

Testing:
Added regression tests for HDFS and Kudu where we previously could
run out of memory with a low mem_limit.

Manual testing:
* Ran query tests with --thread_creation_fault_injection=true for a
bit, confirmed no crashes.
* ran single-node stress test for Kudu and Parquet for 10-20 min each.

Change-Id: Ib9907fa8c4d2b0b85f67f4f160899c1c258ad82b
---
M be/src/exec/hdfs-scan-node.cc
M be/src/exec/hdfs-scan-node.h
M be/src/exec/kudu-scan-node-base.h
M be/src/exec/kudu-scan-node.cc
M be/src/exec/kudu-scan-node.h
M be/src/exec/scan-node.cc
M be/src/exec/scan-node.h
M be/src/runtime/CMakeLists.txt
M be/src/runtime/query-state.cc
M be/src/runtime/query-state.h
A be/src/runtime/scanner-mem-limiter.cc
A be/src/runtime/scanner-mem-limiter.h
A
testdata/workloads/functional-query/queries/QueryTest/hdfs-scanner-thread-mem-scaling.test
A testdata/workloads/functional-query/queries/QueryTest/kudu-scan-mem-usage.test
M tests/query_test/test_mem_usage_scaling.py
15 files changed, 444 insertions(+), 49 deletions(-)

git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/03/11103/10
--
To view, visit http://gerrit.cloudera.org:8080/11103
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib9907fa8c4d2b0b85f67f4f160899c1c258ad82b
Gerrit-Change-Number: 11103
Gerrit-PatchSet: 10
Gerrit-Owner: Tim Armstrong
Gerrit-Reviewer: Bikramjeet Vig
Gerrit-Reviewer: Dan Hecht
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong

[Impala-ASF-CR] IMPALA-7096: restore scanner thread memory heuristics

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11103 )

Change subject: IMPALA-7096: restore scanner thread memory heuristics
..


Patch Set 10: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/11103
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib9907fa8c4d2b0b85f67f4f160899c1c258ad82b
Gerrit-Change-Number: 11103
Gerrit-PatchSet: 10
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 15 Aug 2018 22:39:56 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7096: restore scanner thread memory heuristics

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11103 )

Change subject: IMPALA-7096: restore scanner thread memory heuristics
..


Patch Set 8:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11103/8/be/src/runtime/scanner-mem-limiter.cc
File be/src/runtime/scanner-mem-limiter.cc:

http://gerrit.cloudera.org:8080/#/c/11103/8/be/src/runtime/scanner-mem-limiter.cc@32
PS8, Line 32:   ScanNode* const node;
> it might be clearer to remove that now (so there's no question as to whethe
Done



--
To view, visit http://gerrit.cloudera.org:8080/11103
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib9907fa8c4d2b0b85f67f4f160899c1c258ad82b
Gerrit-Change-Number: 11103
Gerrit-PatchSet: 8
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 15 Aug 2018 22:39:51 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7364: Bump RapidJSON version to 1.1.0

2018-08-15 Thread Quanlong Huang (Code Review)

Quanlong Huang has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11124 )

Change subject: IMPALA-7364: Bump RapidJSON version to 1.1.0
..


Patch Set 8:

Thank you, Tim! You help me to find why I can't receive new emails these two 
days... I have another two emails: huangquanl...@gmail.com and 
quanlong.hu...@hulu.com. You can use my gmail since 126-email sometimes have 
troubles receiving emails outside of China...

Also, I just found your email sent at last Friday burning in some junk 
emails... Have replied it. Thanks!


--
To view, visit http://gerrit.cloudera.org:8080/11124
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I21353b0d769f81c13f506737e41fbac17655245c
Gerrit-Change-Number: 11124
Gerrit-PatchSet: 8
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Wed, 15 Aug 2018 22:26:43 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7449: Fix network throughput calculation of DataStreamSender

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11241 )

Change subject: IMPALA-7449: Fix network throughput calculation of 
DataStreamSender
..


Patch Set 1:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/358/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/11241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I344ac76c0a1a49b4da3d37d2c547f3d5051ebe24
Gerrit-Change-Number: 11241
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Ho 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 15 Aug 2018 22:23:56 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7140 fix: fix NDV calculation for partition columns in LocalFsTable

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11230 )

Change subject: IMPALA-7140 fix: fix NDV calculation for partition columns in 
LocalFsTable
..


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3015/ 
DRY_RUN=true


--
To view, visit http://gerrit.cloudera.org:8080/11230
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0a86750d52dcd744ace030a3c1ec70510d054acf
Gerrit-Change-Number: 11230
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Wed, 15 Aug 2018 22:07:30 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7392: [DOCS] SCAN BYTES LIMIT query option documented

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11240 )

Change subject: IMPALA-7392: [DOCS] SCAN_BYTES_LIMIT query option documented
..


Patch Set 1: Verified+1

Build Successful

https://jenkins.impala.io/job/gerrit-docs-auto-test/49/ : Doc tests passed.


--
To view, visit http://gerrit.cloudera.org:8080/11240
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6430e06cabe21b8080239f3225d3bfdd5cc502cb
Gerrit-Change-Number: 11240
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 15 Aug 2018 22:02:44 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7096: restore scanner thread memory heuristics

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11103 )

Change subject: IMPALA-7096: restore scanner thread memory heuristics
..


Patch Set 9: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/3014/


--
To view, visit http://gerrit.cloudera.org:8080/11103
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib9907fa8c4d2b0b85f67f4f160899c1c258ad82b
Gerrit-Change-Number: 11103
Gerrit-PatchSet: 9
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 15 Aug 2018 22:02:26 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7342: Add initial support for user-level permissions

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11039 )

Change subject: IMPALA-7342: Add initial support for user-level permissions
..


Patch Set 26: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/11039
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I07e0d46d2e50d35bd64ee573b5aa4b779eb9e62f
Gerrit-Change-Number: 11039
Gerrit-PatchSet: 26
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Wed, 15 Aug 2018 22:02:45 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7342: Add initial support for user-level permissions

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/11039 )

Change subject: IMPALA-7342: Add initial support for user-level permissions
..

IMPALA-7342: Add initial support for user-level permissions

This patch refactors the authorization code in preparation to add initial
support for for user-level permissions (IMPALA-6794) and object ownership
(IMPALA-7075). It introduces the notion of Principal that can be either
Role or User. The authorization tests are updated to run the tests with
user and role permissions.

Testing:
- Update authorization tests
- Ran core tests

Change-Id: I07e0d46d2e50d35bd64ee573b5aa4b779eb9e62f
Reviewed-on: http://gerrit.cloudera.org:8080/11039
Reviewed-by: Impala Public Jenkins 
Tested-by: Impala Public Jenkins 
---
M be/src/catalog/catalog-util.cc
M common/thrift/CatalogObjects.thrift
M fe/src/main/java/org/apache/impala/analysis/GrantRevokePrivStmt.java
M fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java
M fe/src/main/java/org/apache/impala/analysis/ShowGrantRoleStmt.java
M fe/src/main/java/org/apache/impala/catalog/AuthorizationPolicy.java
M fe/src/main/java/org/apache/impala/catalog/Catalog.java
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java
A fe/src/main/java/org/apache/impala/catalog/Principal.java
R fe/src/main/java/org/apache/impala/catalog/PrincipalPrivilege.java
M fe/src/main/java/org/apache/impala/catalog/Role.java
A fe/src/main/java/org/apache/impala/catalog/User.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M fe/src/main/java/org/apache/impala/util/SentryPolicyService.java
M fe/src/main/java/org/apache/impala/util/SentryProxy.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java
M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java
M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java
M fe/src/test/java/org/apache/impala/testutil/ImpaladTestCatalog.java
21 files changed, 1,010 insertions(+), 479 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

--
To view, visit http://gerrit.cloudera.org:8080/11039
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I07e0d46d2e50d35bd64ee573b5aa4b779eb9e62f
Gerrit-Change-Number: 11039
Gerrit-PatchSet: 27
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Vuk Ercegovac

[Impala-ASF-CR] IMPALA-6481: [DOCS] Documented WIDTH BUCKET function

Alex Rodoni has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/11170 )

Change subject: IMPALA-6481: [DOCS] Documented WIDTH_BUCKET function
..

IMPALA-6481: [DOCS] Documented WIDTH_BUCKET function

Change-Id: Ife9577a65fe342fde160c7cb5fa666e407d5b093
Reviewed-on: http://gerrit.cloudera.org:8080/11170
Tested-by: Impala Public Jenkins 
Reviewed-by: Zoltan Borok-Nagy 
---
M docs/topics/impala_math_functions.xml
1 file changed, 73 insertions(+), 0 deletions(-)

Approvals:
  Impala Public Jenkins: Verified
  Zoltan Borok-Nagy: Looks good to me, approved

--
To view, visit http://gerrit.cloudera.org:8080/11170
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ife9577a65fe342fde160c7cb5fa666e407d5b093
Gerrit-Change-Number: 11170
Gerrit-PatchSet: 6
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Reviewer: anujphadke

[Impala-ASF-CR] IMPALA-7449: Fix network throughput calculation of DataStreamSender

2018-08-15 Thread Michael Ho (Code Review)

Michael Ho has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/11241


Change subject: IMPALA-7449: Fix network throughput calculation of 
DataStreamSender
..

IMPALA-7449: Fix network throughput calculation of DataStreamSender

Currently, the network throughput presented in the query profile
for DataStreamSender is computed by dividing the total bytes sent
by the total network time which is the sum of observed network time
of all individual RPCs. This is wrong in general and may only make
sense if the network throughput is fixed. In addition, RPCs are
asynchronous and they overlap with each other. So, dividing the
total byte sent by network throughput may result in time which exceeds
the wall clock time, making it impossible to interpret.

This change fixes the problem by measuring the network throughput
of each individual RPC and uses a summary counter to track avg/min/max
of network throughputs instead.

Change-Id: I344ac76c0a1a49b4da3d37d2c547f3d5051ebe24
---
M be/src/runtime/krpc-data-stream-sender.cc
M be/src/runtime/krpc-data-stream-sender.h
2 files changed, 11 insertions(+), 14 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/11241/1
--
To view, visit http://gerrit.cloudera.org:8080/11241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I344ac76c0a1a49b4da3d37d2c547f3d5051ebe24
Gerrit-Change-Number: 11241
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Ho

[Impala-ASF-CR] IMPALA-7206: [DOCS] THREAD RESERVATION LIMIT & THREAD RESERVATION AGGREGATE LIMIT

Alex Rodoni has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/11237 )

Change subject: IMPALA-7206: [DOCS] THREAD_RESERVATION_LIMIT & 
THREAD_RESERVATION_AGGREGATE_LIMIT
..

IMPALA-7206: [DOCS] THREAD_RESERVATION_LIMIT & 
THREAD_RESERVATION_AGGREGATE_LIMIT

Change-Id: I94b560f6098711a6458ac5a7c8584f6fe3e3fdb8
Reviewed-on: http://gerrit.cloudera.org:8080/11237
Reviewed-by: Tim Armstrong 
Tested-by: Impala Public Jenkins 
---
M docs/impala.ditamap
A docs/topics/impala_thread_reservation_aggregate_limit.xml
A docs/topics/impala_thread_reservation_limit.xml
3 files changed, 186 insertions(+), 0 deletions(-)

Approvals:
  Tim Armstrong: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/11237
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I94b560f6098711a6458ac5a7c8584f6fe3e3fdb8
Gerrit-Change-Number: 11237
Gerrit-PatchSet: 3
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong

[Impala-ASF-CR] IMPALA-7392: [DOCS] SCAN BYTES LIMIT query option documented

Alex Rodoni has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/11240


Change subject: IMPALA-7392: [DOCS] SCAN_BYTES_LIMIT query option documented
..

IMPALA-7392: [DOCS] SCAN_BYTES_LIMIT query option documented

Change-Id: I6430e06cabe21b8080239f3225d3bfdd5cc502cb
---
M docs/impala.ditamap
A docs/topics/impala_scan_bytes_limit.xml
2 files changed, 132 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/40/11240/1
--
To view, visit http://gerrit.cloudera.org:8080/11240
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I6430e06cabe21b8080239f3225d3bfdd5cc502cb
Gerrit-Change-Number: 11240
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni

[Impala-ASF-CR] IMPALA-7392: [DOCS] SCAN BYTES LIMIT query option documented

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11240 )

Change subject: IMPALA-7392: [DOCS] SCAN_BYTES_LIMIT query option documented
..


Patch Set 1:

Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/49/

Testing docs change - this change appears to modify docs/ and no code. This is 
experimental - please report any issues to tarmstr...@cloudera.com or on this 
JIRA: IMPALA-7317


--
To view, visit http://gerrit.cloudera.org:8080/11240
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6430e06cabe21b8080239f3225d3bfdd5cc502cb
Gerrit-Change-Number: 11240
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 15 Aug 2018 21:52:14 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7096: restore scanner thread memory heuristics

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11103 )

Change subject: IMPALA-7096: restore scanner thread memory heuristics
..


Patch Set 8:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/357/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/11103
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib9907fa8c4d2b0b85f67f4f160899c1c258ad82b
Gerrit-Change-Number: 11103
Gerrit-PatchSet: 8
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 15 Aug 2018 21:49:24 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7206: [DOCS] THREAD RESERVATION LIMIT & THREAD RESERVATION AGGREGATE LIMIT

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11237 )

Change subject: IMPALA-7206: [DOCS] THREAD_RESERVATION_LIMIT & 
THREAD_RESERVATION_AGGREGATE_LIMIT
..


Patch Set 2: Verified+1

Build Successful

https://jenkins.impala.io/job/gerrit-docs-auto-test/48/ : Doc tests passed.


--
To view, visit http://gerrit.cloudera.org:8080/11237
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I94b560f6098711a6458ac5a7c8584f6fe3e3fdb8
Gerrit-Change-Number: 11237
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 15 Aug 2018 21:41:39 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7442: reduce mem requirement of semi-joins-exhaustive

2018-08-15 Thread Bikramjeet Vig (Code Review)

Bikramjeet Vig has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11223 )

Change subject: IMPALA-7442: reduce mem requirement of semi-joins-exhaustive
..


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11223/2/testdata/workloads/functional-query/queries/QueryTest/semi-joins-exhaustive.test
File 
testdata/workloads/functional-query/queries/QueryTest/semi-joins-exhaustive.test:

http://gerrit.cloudera.org:8080/#/c/11223/2/testdata/workloads/functional-query/queries/QueryTest/semi-joins-exhaustive.test@6
PS2, Line 6: ~12M
> Yeah I thought it was worth keeping around since it's still interesting to
makes sense.



--
To view, visit http://gerrit.cloudera.org:8080/11223
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9d9480cad6bf8222abe990e7046498a0531e2849
Gerrit-Change-Number: 11223
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 15 Aug 2018 21:38:23 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7140 fix: fix NDV calculation for partition columns in LocalFsTable

Vuk Ercegovac has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11230 )

Change subject: IMPALA-7140 fix: fix NDV calculation for partition columns in 
LocalFsTable
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/11230
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0a86750d52dcd744ace030a3c1ec70510d054acf
Gerrit-Change-Number: 11230
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Wed, 15 Aug 2018 21:36:14 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7399: Remove third-party dependencies from junit xml script

Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11235 )

Change subject: IMPALA-7399: Remove third-party dependencies from junit xml 
script
..


Patch Set 4:

(6 comments)

Thanks. The general idea makes sense to me.

It's a matter of style, and this is pretty functionally contained, but it seems 
like the interface here is:

junit_report_string(name, elapsed_time, etc.)

which is to say that saving this to a file should be the caller's job. 
Moreover, I don't think having the class for the JunitReport here adds much, 
since the only reasonable thing to do is to_file().

Again, it's still a matter of style, but I don't like the @property 
annotations. You're generating an annoying XML structure, and I think that's 
best done imperatively, in one boring function. We don't need the lazy-loading 
or the @property stuff. It obscures the simplicity here.

I don't feel all that strongly about this stuff, but let me know what you think.

http://gerrit.cloudera.org:8080/#/c/11235/4/bin/rat_exclude_files.txt
File bin/rat_exclude_files.txt:

http://gerrit.cloudera.org:8080/#/c/11235/4/bin/rat_exclude_files.txt@13
PS4, Line 13: bin/generate_junitxml.py
Does rat not ignore symlinks by default?


http://gerrit.cloudera.org:8080/#/c/11235/4/lib/python/impala_py_lib/jenkins/generate_junitxml.py
File lib/python/impala_py_lib/jenkins/generate_junitxml.py:

http://gerrit.cloudera.org:8080/#/c/11235/4/lib/python/impala_py_lib/jenkins/generate_junitxml.py@38
PS4, Line 38:   """A Junit XML style report parseable by Jenkins for reporting 
build status.
add a comment here about how to use this?


http://gerrit.cloudera.org:8080/#/c/11235/4/lib/python/impala_py_lib/jenkins/generate_junitxml.py@64
PS4, Line 64: # Create the testcase element (if it hasn't already been 
created)
: self.testcase_element
This is too fancy for my tastes. Perhaps just create the structures at the 
beginning here and be done with it?


http://gerrit.cloudera.org:8080/#/c/11235/4/lib/python/impala_py_lib/jenkins/generate_junitxml.py@69
PS4, Line 69: """Return the testsuites root element, creating it if it 
doesn't exist."""
I think this would be clearer if you just created this in init().


http://gerrit.cloudera.org:8080/#/c/11235/4/lib/python/impala_py_lib/jenkins/generate_junitxml.py@79
PS4, Line 79:   for k, v in base_attributes.iteritems():
:   self._root_element.set(k, v)
This is just:

self._root_element("time", elapsed_time)
self._root_element("tests", 1)
and so on.

It's 4 lines, and you have the four lines on lines 72-75 anyway. I think this 
would be clearer without the extra loop and just setting these directly.


http://gerrit.cloudera.org:8080/#/c/11235/4/lib/python/impala_py_lib/jenkins/generate_junitxml.py@102
PS4, Line 102:   self._testsuite_element = ET.SubElement(self.root_element, 
"testsuite")
 :   for k, v in base_attributes.iteritems():
 :   self._testsuite_element.set(k, v)
Same here: you don't re-use base_attributes ever, so I think it makes sense to 
just inline this.



--
To view, visit http://gerrit.cloudera.org:8080/11235
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I958ee0d8420b6a4197aaf0a7e0538a566332ea97
Gerrit-Change-Number: 11235
Gerrit-PatchSet: 4
Gerrit-Owner: David Knupp 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Nithya Janarthanan 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Wed, 15 Aug 2018 21:35:16 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7096: restore scanner thread memory heuristics

2018-08-15 Thread Dan Hecht (Code Review)

Dan Hecht has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11103 )

Change subject: IMPALA-7096: restore scanner thread memory heuristics
..


Patch Set 8:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11103/8/be/src/runtime/scanner-mem-limiter.cc
File be/src/runtime/scanner-mem-limiter.cc:

http://gerrit.cloudera.org:8080/#/c/11103/8/be/src/runtime/scanner-mem-limiter.cc@32
PS8, Line 32:   ScanNode* const node;
it might be clearer to remove that now (so there's no question as to whether 
this equals the map key or not).



--
To view, visit http://gerrit.cloudera.org:8080/11103
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib9907fa8c4d2b0b85f67f4f160899c1c258ad82b
Gerrit-Change-Number: 11103
Gerrit-PatchSet: 8
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 15 Aug 2018 21:29:24 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7096: restore scanner thread memory heuristics

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11103 )

Change subject: IMPALA-7096: restore scanner thread memory heuristics
..


Patch Set 9:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3014/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/11103
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib9907fa8c4d2b0b85f67f4f160899c1c258ad82b
Gerrit-Change-Number: 11103
Gerrit-PatchSet: 9
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 15 Aug 2018 21:21:13 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7096: restore scanner thread memory heuristics

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11103 )

Change subject: IMPALA-7096: restore scanner thread memory heuristics
..


Patch Set 9: Code-Review+2

(6 comments)

Carry +2

http://gerrit.cloudera.org:8080/#/c/11103/7/be/src/runtime/scanner-mem-limiter.h
File be/src/runtime/scanner-mem-limiter.h:

http://gerrit.cloudera.org:8080/#/c/11103/7/be/src/runtime/scanner-mem-limiter.h@32
PS7, Line 32: limiting the aggregate memory consumpt
> is it to limit the number of scanner threads, or the aggregate memory consu
Done


http://gerrit.cloudera.org:8080/#/c/11103/7/be/src/runtime/scanner-mem-limiter.h@42
PS7, Line 42: as this objec
> garbled
Done


http://gerrit.cloudera.org:8080/#/c/11103/7/be/src/runtime/scanner-mem-limiter.h@43
PS7, Line 43: (i.e. as long as the below methods
> is that requirement because the instance of this class happens to also be a
Done


http://gerrit.cloudera.org:8080/#/c/11103/7/be/src/runtime/scanner-mem-limiter.h@71
PS7, Line 71:   /// ClaimMemoryForScannerThread() will not be called.
> do we need that? why not just iterate over the map? do we want ordering but
I started off with the vector then kept it as an optimisation to allow more 
efficient iteration in ClaimMemoryForScannerThread. Now that I look at it again 
I doubt there's a significant enough difference between this and unordered_map 
to justify the complexity.


http://gerrit.cloudera.org:8080/#/c/11103/7/be/src/runtime/scanner-mem-limiter.cc
File be/src/runtime/scanner-mem-limiter.cc:

http://gerrit.cloudera.org:8080/#/c/11103/7/be/src/runtime/scanner-mem-limiter.cc@75
PS7, Line 75: e a crude heuristic of guessing that the scan
:   // will
> what code? before this change or before the change that removed the origina
Added a more concrete description of which commits added/removed the code.


http://gerrit.cloudera.org:8080/#/c/11103/7/be/src/runtime/scanner-mem-limiter.cc@81
PS7, Line 81: addtl_consumption += static_cast((consumption * 
1.5) / num_threads);
:   }
> I don't really understand that. Why would adding this thread increase consu
Yeah exactly. Tried to improve the comment.



--
To view, visit http://gerrit.cloudera.org:8080/11103
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib9907fa8c4d2b0b85f67f4f160899c1c258ad82b
Gerrit-Change-Number: 11103
Gerrit-PatchSet: 9
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 15 Aug 2018 21:20:58 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7096: restore scanner thread memory heuristics

Hello Bikramjeet Vig, Impala Public Jenkins, Dan Hecht,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/11103

to look at the new patch set (#8).

Change subject: IMPALA-7096: restore scanner thread memory heuristics
..

IMPALA-7096: restore scanner thread memory heuristics

Introduce a ScannerMemLimiter class that belongs to the QueryState that
tracks the amount of memory estimated to be consumed for all scanner
threads running for the query on the current backend.

Also check soft memory limits to see if scanner threads should be
started or the current scanner thread should stop.

The long-term plan is to switch to the MT scan node implementations.
When that happens this code can be removed. In the meantime this
code is imperfect but will help avoid OOM in many scenarios.

Testing:
Added regression tests for HDFS and Kudu where we previously could
run out of memory with a low mem_limit.

Manual testing:
* Ran query tests with --thread_creation_fault_injection=true for a
bit, confirmed no crashes.
* ran single-node stress test for Kudu and Parquet for 10-20 min each.

git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/03/11103/8
--
To view, visit http://gerrit.cloudera.org:8080/11103
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib9907fa8c4d2b0b85f67f4f160899c1c258ad82b
Gerrit-Change-Number: 11103
Gerrit-PatchSet: 8
Gerrit-Owner: Tim Armstrong
Gerrit-Reviewer: Bikramjeet Vig
Gerrit-Reviewer: Dan Hecht
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong

[Impala-ASF-CR] IMPALA-7206: [DOCS] THREAD RESERVATION LIMIT & THREAD RESERVATION AGGREGATE LIMIT

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11237 )

Change subject: IMPALA-7206: [DOCS] THREAD_RESERVATION_LIMIT & 
THREAD_RESERVATION_AGGREGATE_LIMIT
..


Patch Set 1: Verified+1

Build Successful

https://jenkins.impala.io/job/gerrit-docs-auto-test/47/ : Doc tests passed.


--
To view, visit http://gerrit.cloudera.org:8080/11237
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I94b560f6098711a6458ac5a7c8584f6fe3e3fdb8
Gerrit-Change-Number: 11237
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 15 Aug 2018 21:19:19 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7206: [DOCS] THREAD RESERVATION LIMIT & THREAD RESERVATION AGGREGATE LIMIT

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11237 )

Change subject: IMPALA-7206: [DOCS] THREAD_RESERVATION_LIMIT & 
THREAD_RESERVATION_AGGREGATE_LIMIT
..


Patch Set 2: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/11237
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I94b560f6098711a6458ac5a7c8584f6fe3e3fdb8
Gerrit-Change-Number: 11237
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 15 Aug 2018 21:16:48 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7442: reduce mem requirement of semi-joins-exhaustive

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11223 )

Change subject: IMPALA-7442: reduce mem requirement of semi-joins-exhaustive
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11223/2/testdata/workloads/functional-query/queries/QueryTest/semi-joins-exhaustive.test
File 
testdata/workloads/functional-query/queries/QueryTest/semi-joins-exhaustive.test:

http://gerrit.cloudera.org:8080/#/c/11223/2/testdata/workloads/functional-query/queries/QueryTest/semi-joins-exhaustive.test@6
PS2, Line 6: ~12M
> looks like the original regression test required at least 16M rows to check
Yeah I thought it was worth keeping around since it's still interesting to have 
a test case with many "empty" tuples on the right side of a join. With the BTS 
rework the 32-bit overflow can't happen so that threshold is no longer so 
important.

So this is a compromise between maintaining test coverage and test suite 
stability.



--
To view, visit http://gerrit.cloudera.org:8080/11223
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9d9480cad6bf8222abe990e7046498a0531e2849
Gerrit-Change-Number: 11223
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 15 Aug 2018 21:14:25 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7206: [DOCS] THREAD RESERVATION LIMIT & THREAD RESERVATION AGGREGATE LIMIT

2018-08-15 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11237 )

Change subject: IMPALA-7206: [DOCS] THREAD_RESERVATION_LIMIT & 
THREAD_RESERVATION_AGGREGATE_LIMIT
..


Patch Set 2:

Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/48/

Testing docs change - this change appears to modify docs/ and no code. This is 
experimental - please report any issues to tarmstr...@cloudera.com or on this 
JIRA: IMPALA-7317


--
To view, visit http://gerrit.cloudera.org:8080/11237
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I94b560f6098711a6458ac5a7c8584f6fe3e3fdb8
Gerrit-Change-Number: 11237
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 15 Aug 2018 21:11:59 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7206: [DOCS] THREAD RESERVATION LIMIT & THREAD RESERVATION AGGREGATE LIMIT

Hello Tim Armstrong, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/11237

to look at the new patch set (#2).

Change subject: IMPALA-7206: [DOCS] THREAD_RESERVATION_LIMIT & 
THREAD_RESERVATION_AGGREGATE_LIMIT
..

IMPALA-7206: [DOCS] THREAD_RESERVATION_LIMIT & 
THREAD_RESERVATION_AGGREGATE_LIMIT

Change-Id: I94b560f6098711a6458ac5a7c8584f6fe3e3fdb8
---
M docs/impala.ditamap
A docs/topics/impala_thread_reservation_aggregate_limit.xml
A docs/topics/impala_thread_reservation_limit.xml
3 files changed, 186 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/37/11237/2
--
To view, visit http://gerrit.cloudera.org:8080/11237
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I94b560f6098711a6458ac5a7c8584f6fe3e3fdb8
Gerrit-Change-Number: 11237
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong

[Impala-ASF-CR] IMPALA-7206: [DOCS] THREAD RESERVATION LIMIT query option documented