[Impala-ASF-CR] IMPALA-7315: fix test update with clear entries flag race
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10986 ) Change subject: IMPALA-7315: fix test_update_with_clear_entries_flag race .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10986 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2be2b61b6deb0228fbc5a242e43076beb8871454 Gerrit-Change-Number: 10986 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Jul 2018 06:35:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7315: fix test update with clear entries flag race
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10986 ) Change subject: IMPALA-7315: fix test_update_with_clear_entries_flag race .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2841/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/10986 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2be2b61b6deb0228fbc5a242e43076beb8871454 Gerrit-Change-Number: 10986 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Jul 2018 06:35:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10980 ) Change subject: IMPALA-7298: Stop passing IP address as hostname in Kerberos principal .. IMPALA-7298: Stop passing IP address as hostname in Kerberos principal Previously, we pass the resolved IP address of a KRPC destination host as the hostname when creating a proxy for making KRPC calls. This may lead to connection negotiation failure in KRPC when Kerberos is enabled. In particular, if reversed DNS isn't enabled in Kerberos, KDC may fail to look up the principal of the destination host if the principal includes the hostname instead of resolved IP address. This change fixes the problem above by passing the actual hostname of the destination host when calling RpcMgr::GetProxy(). rpc-mgr-kerberized-test.cc is also updated to use hostname instead of the resolved IP address as Kerberos principal. Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e Reviewed-on: http://gerrit.cloudera.org:8080/10980 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/rpc/rpc-mgr-kerberized-test.cc M be/src/rpc/rpc-mgr-test-base.h M be/src/rpc/rpc-mgr-test.cc M be/src/rpc/rpc-mgr.h M be/src/rpc/rpc-mgr.inline.h M be/src/runtime/data-stream-sender.cc M be/src/runtime/data-stream-test.cc M be/src/runtime/krpc-data-stream-sender.cc M be/src/scheduling/scheduler.cc M common/thrift/ImpalaInternalService.thrift 10 files changed, 33 insertions(+), 27 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/10980 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e Gerrit-Change-Number: 10980 Gerrit-PatchSet: 3 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10980 ) Change subject: IMPALA-7298: Stop passing IP address as hostname in Kerberos principal .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10980 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e Gerrit-Change-Number: 10980 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Thu, 19 Jul 2018 05:00:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10965 ) Change subject: IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10965 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I41fd855fbf0e9ec58845ac0d2eb96a87b0172152 Gerrit-Change-Number: 10965 Gerrit-PatchSet: 3 Gerrit-Owner: Tianyi Wang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 19 Jul 2018 04:34:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10965 ) Change subject: IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog .. IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog This patch bumps glog version to 0.3.4-p3 to include the patch limiting fadvise calls. Change-Id: I41fd855fbf0e9ec58845ac0d2eb96a87b0172152 Reviewed-on: http://gerrit.cloudera.org:8080/10965 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M bin/impala-config.sh 1 file changed, 2 insertions(+), 2 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/10965 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I41fd855fbf0e9ec58845ac0d2eb96a87b0172152 Gerrit-Change-Number: 10965 Gerrit-PatchSet: 4 Gerrit-Owner: Tianyi Wang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7256: Aggregator mem usage isn't reflected in summary
Thomas Marshall has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10989 Change subject: IMPALA-7256: Aggregator mem usage isn't reflected in summary .. IMPALA-7256: Aggregator mem usage isn't reflected in summary This patch fixes a bug where memory used by an Aggregator wasn't being reflected in the exec summary for the corresponding aggregation node by ensuring that the Aggregator's MemTracker is a child of the node's MemTracker. Testing: - Manually ran a query and checked that all memory used by the Aggregator is accounted for in the exec summary. Change-Id: Iba6ef207bed47810fc742aec3481db5f313cf97f --- M be/src/exec/aggregator.cc M be/src/exec/aggregator.h M be/src/exec/exec-node.cc M be/src/exec/exec-node.h M be/src/exec/grouping-aggregator.cc M be/src/runtime/reservation-manager.cc M be/src/runtime/reservation-manager.h 7 files changed, 25 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/10989/1 -- To view, visit http://gerrit.cloudera.org:8080/10989 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Iba6ef207bed47810fc742aec3481db5f313cf97f Gerrit-Change-Number: 10989 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall
[Impala-ASF-CR] IMPALA-7307 (part 2). Support TABLESAMPLE in LocalCatalog
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/10972 ) Change subject: IMPALA-7307 (part 2). Support TABLESAMPLE in LocalCatalog .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/10972/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/10972/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@2160 PS1, Line 2160:*/ > ah, yes... I think I went back and forth on which place to put this static I'm fine with the move to utils. its consistent with the other changes and its a little odd to see a static method here operating on the super-type for tables. -- To view, visit http://gerrit.cloudera.org:8080/10972 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2f7baf05f16c6389ed900e0459708005ab44491e Gerrit-Change-Number: 10972 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Jul 2018 02:12:25 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7307 (part 2). Support TABLESAMPLE in LocalCatalog
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/10972 ) Change subject: IMPALA-7307 (part 2). Support TABLESAMPLE in LocalCatalog .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/10972/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/10972/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@2160 PS1, Line 2160:*/ > should this method be removed in favor of the one in the catalog util? ah, yes... I think I went back and forth on which place to put this static method and somehow ended up with both. Do you prefer here or FeCatalogUtil? The advantage of here is that it makes the patch show fewer changed lines vs moving it. -- To view, visit http://gerrit.cloudera.org:8080/10972 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2f7baf05f16c6389ed900e0459708005ab44491e Gerrit-Change-Number: 10972 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Jul 2018 02:08:37 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7257. Support Kudu tables in LocalCatalog
Todd Lipcon has posted comments on this change. (
http://gerrit.cloudera.org:8080/10912 )
Change subject: IMPALA-7257. Support Kudu tables in LocalCatalog
..
Patch Set 3:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/10912/3/fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java
File fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java:
http://gerrit.cloudera.org:8080/#/c/10912/3/fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java@369
PS3, Line 369: for (KuduPartitionParam partitionParam:
table.getPartitionBy()) {
> Why putting them here instead of KuduTable?
I've been somewhat inconsistent with what goes into KuduTable or HdfsTable vs
FeCatalogUtils. I'll move this there.
http://gerrit.cloudera.org:8080/#/c/10912/3/fe/src/main/java/org/apache/impala/catalog/KuduTable.java
File fe/src/main/java/org/apache/impala/catalog/KuduTable.java:
http://gerrit.cloudera.org:8080/#/c/10912/3/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@258
PS3, Line 258: public static List loadPartitionByParams(
> What's the future plan for those static methods shared with local table?
I think eventually they could become default methods in the FeKuduTable
interface. However right now I wanted to preserve the ability to backport this
back to the 2.x branch so I've been avoiding Java8-only features and making
static methods either here or in FeCatalogUtils. I was inconsistent in this
patch, so I'll move that FeCatalogUtils static method back here.
http://gerrit.cloudera.org:8080/#/c/10912/3/fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
File fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java:
http://gerrit.cloudera.org:8080/#/c/10912/3/fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java@140
PS3, Line 140: kuduTableName_ instead of name below
> Why can't we do the same here?
I think that was a bug in the old implementation. The name that shows up in the
descriptor should be the name of the table as seen by Impala, rather than the
name as seen by Kudu, don't you think? I think the only issue it would cause is
potentially wrong messages in the plan or summary output.
http://gerrit.cloudera.org:8080/#/c/10912/3/fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java@147
PS3, Line 147: // TODO(todd)
> please explain
oops, this was a bug. It was a TODO to setKey_columns, but then I did the TODO
and forgot to remove this stub.
Interesting that the code still works, though, despite not setting the key
columns in the descriptor.
--
To view, visit http://gerrit.cloudera.org:8080/10912
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5b6a317ee895e43e00ade953e814867b56b4e6dd
Gerrit-Change-Number: 10912
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon
Gerrit-Reviewer: Tianyi Wang
Gerrit-Reviewer: Todd Lipcon
Gerrit-Reviewer: Vuk Ercegovac
Gerrit-Comment-Date: Thu, 19 Jul 2018 02:07:36 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7315: fix test update with clear entries flag race
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10986 ) Change subject: IMPALA-7315: fix test_update_with_clear_entries_flag race .. Patch Set 3: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2837/ -- To view, visit http://gerrit.cloudera.org:8080/10986 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2be2b61b6deb0228fbc5a242e43076beb8871454 Gerrit-Change-Number: 10986 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Jul 2018 02:00:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7259: Improve Impala shell performance
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10939 ) Change subject: IMPALA-7259: Improve Impala shell performance .. Patch Set 11: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2836/ -- To view, visit http://gerrit.cloudera.org:8080/10939 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idac9f3caed7c44846a8c922dbe5ca3bf3b095b81 Gerrit-Change-Number: 10939 Gerrit-PatchSet: 11 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 19 Jul 2018 01:59:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10980 ) Change subject: IMPALA-7298: Stop passing IP address as hostname in Kerberos principal .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2839/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/10980 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e Gerrit-Change-Number: 10980 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Thu, 19 Jul 2018 01:57:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10980 ) Change subject: IMPALA-7298: Stop passing IP address as hostname in Kerberos principal .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10980 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e Gerrit-Change-Number: 10980 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Thu, 19 Jul 2018 01:57:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6034: Add scanned bytes limits per query
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/10415 ) Change subject: IMPALA-6034: Add scanned bytes limits per query .. Patch Set 12: Code-Review+1 Looks good -- To view, visit http://gerrit.cloudera.org:8080/10415 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4c6015e21da684bb9f33e236d71309dd4c178a20 Gerrit-Change-Number: 10415 Gerrit-PatchSet: 12 Gerrit-Owner: Mostafa Mokhtar Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 19 Jul 2018 01:43:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10965 ) Change subject: IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2838/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/10965 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I41fd855fbf0e9ec58845ac0d2eb96a87b0172152 Gerrit-Change-Number: 10965 Gerrit-PatchSet: 3 Gerrit-Owner: Tianyi Wang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 19 Jul 2018 01:24:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10965 ) Change subject: IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10965 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I41fd855fbf0e9ec58845ac0d2eb96a87b0172152 Gerrit-Change-Number: 10965 Gerrit-PatchSet: 3 Gerrit-Owner: Tianyi Wang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 19 Jul 2018 01:24:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog
Tianyi Wang has posted comments on this change. ( http://gerrit.cloudera.org:8080/10965 ) Change subject: IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog .. Patch Set 2: hit IMPALA-7316 -- To view, visit http://gerrit.cloudera.org:8080/10965 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I41fd855fbf0e9ec58845ac0d2eb96a87b0172152 Gerrit-Change-Number: 10965 Gerrit-PatchSet: 2 Gerrit-Owner: Tianyi Wang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 19 Jul 2018 01:23:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7257. Support Kudu tables in LocalCatalog
Tianyi Wang has posted comments on this change. (
http://gerrit.cloudera.org:8080/10912 )
Change subject: IMPALA-7257. Support Kudu tables in LocalCatalog
..
Patch Set 3:
(6 comments)
http://gerrit.cloudera.org:8080/#/c/10912/3/fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java
File fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java:
http://gerrit.cloudera.org:8080/#/c/10912/3/fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java@369
PS3, Line 369: for (KuduPartitionParam partitionParam:
table.getPartitionBy()) {
Why putting them here instead of KuduTable?
http://gerrit.cloudera.org:8080/#/c/10912/3/fe/src/main/java/org/apache/impala/catalog/KuduTable.java
File fe/src/main/java/org/apache/impala/catalog/KuduTable.java:
http://gerrit.cloudera.org:8080/#/c/10912/3/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@258
PS3, Line 258: public static List loadPartitionByParams(
What's the future plan for those static methods shared with local table?
http://gerrit.cloudera.org:8080/#/c/10912/3/fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
File fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java:
http://gerrit.cloudera.org:8080/#/c/10912/3/fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java@59
PS3, Line 59: public static LocalTable loadFromKudu(LocalDb db, Table
msTable) {
package-private? It's for the local package only.
http://gerrit.cloudera.org:8080/#/c/10912/3/fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java@139
PS3, Line 139: {
long line
http://gerrit.cloudera.org:8080/#/c/10912/3/fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java@140
PS3, Line 140: kuduTableName_ instead of name below
Why can't we do the same here?
http://gerrit.cloudera.org:8080/#/c/10912/3/fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java@147
PS3, Line 147: // TODO(todd)
please explain
--
To view, visit http://gerrit.cloudera.org:8080/10912
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5b6a317ee895e43e00ade953e814867b56b4e6dd
Gerrit-Change-Number: 10912
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon
Gerrit-Reviewer: Tianyi Wang
Gerrit-Reviewer: Vuk Ercegovac
Gerrit-Comment-Date: Thu, 19 Jul 2018 01:20:21 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7276. Support CREATE TABLE AS SELECT with LocalCatalog
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/10913 ) Change subject: IMPALA-7276. Support CREATE TABLE AS SELECT with LocalCatalog .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/10913/3/fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java File fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java: http://gerrit.cloudera.org:8080/#/c/10913/3/fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java@114 PS3, Line 114: load them? > nope, this would only show up temporarily in the analyzer during analysis o ok, so just for future proofing. -- To view, visit http://gerrit.cloudera.org:8080/10913 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I93937aed9b76ef6a62b1c588c59c34d3d6831a46 Gerrit-Change-Number: 10913 Gerrit-PatchSet: 3 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Jul 2018 01:10:56 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7307 (part 2). Support TABLESAMPLE in LocalCatalog
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/10972 ) Change subject: IMPALA-7307 (part 2). Support TABLESAMPLE in LocalCatalog .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/10972/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/10972/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@2160 PS1, Line 2160:*/ should this method be removed in favor of the one in the catalog util? -- To view, visit http://gerrit.cloudera.org:8080/10972 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2f7baf05f16c6389ed900e0459708005ab44491e Gerrit-Change-Number: 10972 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Jul 2018 01:03:29 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7307 (part 1). Support stats extrapolation in LocalCatalog
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/10971 ) Change subject: IMPALA-7307 (part 1). Support stats extrapolation in LocalCatalog .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10971 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I479b7f517091dd558768601e1e0704a1902b78a5 Gerrit-Change-Number: 10971 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Jul 2018 00:58:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Sailesh Mukil has posted comments on this change. ( http://gerrit.cloudera.org:8080/10980 ) Change subject: IMPALA-7298: Stop passing IP address as hostname in Kerberos principal .. Patch Set 1: Code-Review+2 Forgot to add +2 -- To view, visit http://gerrit.cloudera.org:8080/10980 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e Gerrit-Change-Number: 10980 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Thu, 19 Jul 2018 00:54:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Sailesh Mukil has posted comments on this change. ( http://gerrit.cloudera.org:8080/10980 ) Change subject: IMPALA-7298: Stop passing IP address as hostname in Kerberos principal .. Patch Set 1: (1 comment) Patch LGTM. Thanks for doing this. http://gerrit.cloudera.org:8080/#/c/10980/1/be/src/runtime/krpc-data-stream-sender.cc File be/src/runtime/krpc-data-stream-sender.cc: http://gerrit.cloudera.org:8080/#/c/10980/1/be/src/runtime/krpc-data-stream-sender.cc@598 PS1, Line 598: destinations[i].thrift_backend.hostname, > destinations[i].krpc_backend.hostname is actually set to the resolved IP ad Ok thanks for the explanation. -- To view, visit http://gerrit.cloudera.org:8080/10980 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e Gerrit-Change-Number: 10980 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Thu, 19 Jul 2018 00:53:59 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/10980 ) Change subject: IMPALA-7298: Stop passing IP address as hostname in Kerberos principal .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/10980/1/be/src/runtime/krpc-data-stream-sender.cc File be/src/runtime/krpc-data-stream-sender.cc: http://gerrit.cloudera.org:8080/#/c/10980/1/be/src/runtime/krpc-data-stream-sender.cc@598 PS1, Line 598: destinations[i].thrift_backend.hostname, > Instead of adding this new parameter, would it be possible to ensure that " destinations[i].krpc_backend.hostname is actually set to the resolved IP address of hostname so we have to pull it from thrift_backend.hosname. As we remove Thrift from communications between Impala backends in the future, we need to move thrift_backend.hostname into a separate field in TPlanFragmentDestination. -- To view, visit http://gerrit.cloudera.org:8080/10980 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e Gerrit-Change-Number: 10980 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Thu, 19 Jul 2018 00:47:50 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Sailesh Mukil has posted comments on this change. ( http://gerrit.cloudera.org:8080/10980 ) Change subject: IMPALA-7298: Stop passing IP address as hostname in Kerberos principal .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/10980/1/be/src/runtime/krpc-data-stream-sender.cc File be/src/runtime/krpc-data-stream-sender.cc: http://gerrit.cloudera.org:8080/#/c/10980/1/be/src/runtime/krpc-data-stream-sender.cc@598 PS1, Line 598: destinations[i].thrift_backend.hostname, Instead of adding this new parameter, would it be possible to ensure that "destinations[i].krpc_backend.hostname" is set? That way, we can avoid a lot of code changes. -- To view, visit http://gerrit.cloudera.org:8080/10980 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e Gerrit-Change-Number: 10980 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Thu, 19 Jul 2018 00:36:31 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6299: Modify IRBuilder to use LLVM's CPU features
Tim Armstrong has posted comments on this change. (
http://gerrit.cloudera.org:8080/10979 )
Change subject: IMPALA-6299: Modify IRBuilder to use LLVM's CPU features
..
Patch Set 2:
(6 comments)
http://gerrit.cloudera.org:8080/#/c/10979/2/be/src/codegen/llvm-codegen-test.cc
File be/src/codegen/llvm-codegen-test.cc:
http://gerrit.cloudera.org:8080/#/c/10979/2/be/src/codegen/llvm-codegen-test.cc@467
PS2, Line 467: CpuInfo::EnableFeature(CpuInfo::SSE4_2, false);
Do we still need to toggle CpuInfo?
http://gerrit.cloudera.org:8080/#/c/10979/2/be/src/codegen/llvm-codegen.h
File be/src/codegen/llvm-codegen.h:
http://gerrit.cloudera.org:8080/#/c/10979/2/be/src/codegen/llvm-codegen.h@737
PS2, Line 737: current_cpu_attrs_
Maybe enabled_cpu_attrs_?
I'm also wondering if we need this additional map - can we just modify
cpu_attrs_ for testing purposes?
On principle I'd prefer to keep the product code as simple as possible even if
it makes the test code a little more complex.
http://gerrit.cloudera.org:8080/#/c/10979/2/be/src/codegen/llvm-codegen.h@745
PS2, Line 745: /// Mapping between CpuInfo flags and the corresponding
strings.
Can you briefly document the key? Since it seems like you're doing something
non-trivial with bitwise negation.
http://gerrit.cloudera.org:8080/#/c/10979/2/be/src/codegen/llvm-codegen.cc
File be/src/codegen/llvm-codegen.cc:
http://gerrit.cloudera.org:8080/#/c/10979/2/be/src/codegen/llvm-codegen.cc@127
PS2, Line 127: const std::map
LlvmCodeGen::cpu_flag_mappings_{
nit: we generally avoid std:: prefixes in .cc files. common/names.h actually
imports a lot of these standard classes into the namespace.
http://gerrit.cloudera.org:8080/#/c/10979/2/be/src/util/bit-util-test.cc
File be/src/util/bit-util-test.cc:
http://gerrit.cloudera.org:8080/#/c/10979/2/be/src/util/bit-util-test.cc@325
PS2, Line 325: const int64_t CPU_INFO_FLAG = CpuInfo::SSSE3;
Either const or constexpr here seems fine - why the change?
http://gerrit.cloudera.org:8080/#/c/10979/2/be/src/util/cpu-info.cc
File be/src/util/cpu-info.cc:
http://gerrit.cloudera.org:8080/#/c/10979/2/be/src/util/cpu-info.cc@71
PS2, Line 71: const int64_t CpuInfo::SSSE3 = (1 << 1);
We still want to have the constant values visible in the header file so they
can be inlined in other modules. You can do that and just define them here
without the value, i.e.
const int64_t CpuInfo::SSE4_1;
Normally the benefit of inlining is minimal, but the CpuInfo checks are done on
some very frequently-executed code paths.
--
To view, visit http://gerrit.cloudera.org:8080/10979
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifece8949c143146d2a1b38d72d21c2d733bed90f
Gerrit-Change-Number: 10979
Gerrit-PatchSet: 2
Gerrit-Owner: Pooja Nilangekar
Gerrit-Reviewer: Bikramjeet Vig
Gerrit-Reviewer: Pooja Nilangekar
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Thu, 19 Jul 2018 00:31:11 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7276. Support CREATE TABLE AS SELECT with LocalCatalog
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/10913 ) Change subject: IMPALA-7276. Support CREATE TABLE AS SELECT with LocalCatalog .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/10913/3/fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java File fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java: http://gerrit.cloudera.org:8080/#/c/10913/3/fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java@114 PS3, Line 114: load them? > makes sense... I'd prefer the flag to guard. nope, this would only show up temporarily in the analyzer during analysis of a CREATE TABLE AS SELECT statement -- it never gets inserted into any Db and never sticks around outside the scope of a single CTAS -- To view, visit http://gerrit.cloudera.org:8080/10913 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I93937aed9b76ef6a62b1c588c59c34d3d6831a46 Gerrit-Change-Number: 10913 Gerrit-PatchSet: 3 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Jul 2018 00:21:01 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7276. Support CREATE TABLE AS SELECT with LocalCatalog
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/10913 ) Change subject: IMPALA-7276. Support CREATE TABLE AS SELECT with LocalCatalog .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/10913/3/fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java File fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java: http://gerrit.cloudera.org:8080/#/c/10913/3/fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java@114 PS3, Line 114: load them? > yea, the target table is sort of an odd temporary catalog entry which doesn makes sense... I'd prefer the flag to guard. so this can end up in a "show tables" and from there, such a load could be requested? if that's the case, seems like a hidden table scheme for tmp could make sense. -- To view, visit http://gerrit.cloudera.org:8080/10913 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I93937aed9b76ef6a62b1c588c59c34d3d6831a46 Gerrit-Change-Number: 10913 Gerrit-PatchSet: 3 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Jul 2018 00:19:34 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7276. Support CREATE TABLE AS SELECT with LocalCatalog
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/10913 ) Change subject: IMPALA-7276. Support CREATE TABLE AS SELECT with LocalCatalog .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/10913/3/fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java File fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java: http://gerrit.cloudera.org:8080/#/c/10913/3/fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java@114 PS3, Line 114: load them? > what would be loaded... this is a new table if I've understood it correctly yea, the target table is sort of an odd temporary catalog entry which doesn't actually have an entry in the HMS. I was afraid that, if someone someone uses this method to create an instance, and then ends up calling getPartitions(), we'll end up making an HMS API call to get partitions for a table and throw an error like TableDoesNotExistException. Was thinking of either a new subclass or adding some kind of boolean isCtasTarget_ to assert against to make sure we don't do something like that. I'll elaborate. -- To view, visit http://gerrit.cloudera.org:8080/10913 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I93937aed9b76ef6a62b1c588c59c34d3d6831a46 Gerrit-Change-Number: 10913 Gerrit-PatchSet: 3 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Jul 2018 00:11:31 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7140 (part 9): add support for SHOW FILES in LocalFsTable
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/10973 ) Change subject: IMPALA-7140 (part 9): add support for SHOW FILES in LocalFsTable .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/10973/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/10973/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@2111 PS1, Line 2111: new ArrayList<> > nit: existing code uses the google lists pkg. any reason not to use it in t no particular reason, figured that since Java 7 it's easier to just use the <> trick unless using one of the fancier methods like newArrayListWithCapacity(). I'll change. -- To view, visit http://gerrit.cloudera.org:8080/10973 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I143bea9f72b6a25ae2545663e06f4849b58533ba Gerrit-Change-Number: 10973 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 19 Jul 2018 00:09:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7140 (part 9): add support for SHOW FILES in LocalFsTable
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/10973 ) Change subject: IMPALA-7140 (part 9): add support for SHOW FILES in LocalFsTable .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/10973/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/10973/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@2111 PS1, Line 2111: new ArrayList<> nit: existing code uses the google lists pkg. any reason not to use it in this case? more of case for consistency to use it at the moment. -- To view, visit http://gerrit.cloudera.org:8080/10973 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I143bea9f72b6a25ae2545663e06f4849b58533ba Gerrit-Change-Number: 10973 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 18 Jul 2018 23:59:44 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7276. Support CREATE TABLE AS SELECT with LocalCatalog
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/10913 ) Change subject: IMPALA-7276. Support CREATE TABLE AS SELECT with LocalCatalog .. Patch Set 3: Code-Review+1 (2 comments) lgtm. I'll let Csaba have another look. http://gerrit.cloudera.org:8080/#/c/10913/3/fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java File fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java: http://gerrit.cloudera.org:8080/#/c/10913/3/fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java@188 PS3, Line 188: the nit: remove second 'the' http://gerrit.cloudera.org:8080/#/c/10913/3/fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java File fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java: http://gerrit.cloudera.org:8080/#/c/10913/3/fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java@114 PS3, Line 114: load them? what would be loaded... this is a new table if I've understood it correctly? -- To view, visit http://gerrit.cloudera.org:8080/10913 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I93937aed9b76ef6a62b1c588c59c34d3d6831a46 Gerrit-Change-Number: 10913 Gerrit-PatchSet: 3 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 18 Jul 2018 23:54:50 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6923: Update scripts in benchmark folder to store workload and few minor updates
Michael Brown has posted comments on this change. ( http://gerrit.cloudera.org:8080/10100 ) Change subject: IMPALA-6923: Update scripts in benchmark folder to store workload and few minor updates .. Patch Set 12: (5 comments) I think the changes are OK for the most part, but they could stand to use better Python conventions. One of my inline comments explains how to do that. http://gerrit.cloudera.org:8080/#/c/10100/12/tests/benchmark/perf_result_datastore.py File tests/benchmark/perf_result_datastore.py: http://gerrit.cloudera.org:8080/#/c/10100/12/tests/benchmark/perf_result_datastore.py@23 PS12, Line 23: from subprocess import * "*" is an antipattern for imports. It is better to be explicit about imported symbols. http://gerrit.cloudera.org:8080/#/c/10100/12/tests/benchmark/perf_result_datastore.py@63 PS12, Line 63: def insert_workload_summary(self, run_info_id): : self._insert_workload_summary(run_info_id) This doesn't seem to do anything useful, so just rename _insert_workload_summary to insert_workload_summary and remove this. http://gerrit.cloudera.org:8080/#/c/10100/12/tests/benchmark/perf_result_datastore.py@85 PS12, Line 85: dont_save_profiles): Negatives are weird. It would be better to call this save_profiles and flip conditions. The reason being, readability for stuff like: if not dont_save_profiles gets confusing. http://gerrit.cloudera.org:8080/#/c/10100/12/tests/benchmark/perf_result_datastore.py@88 PS12, Line 88: temp_profile_folder = os.path.join(home_dir,"workspace","impala-workload-runner","profiles") This is badly formated. There should be spaces between parameters. impala-flake8 perf_result_datastore.py should help find problems like this. Please run it against this file. You don't have to fix all the existing flake8 errors. But for your own submissions, you should format them better. You can do that via "git diff HEAD^ | impala-flake8 --diff" from the root Impala directory. http://gerrit.cloudera.org:8080/#/c/10100/12/tests/benchmark/perf_result_datastore.py@155 PS12, Line 155: self._cache: Dictionary of with run_info/user as key/value This isn't quite right. self._cache appears to be a hash in which a tuple of (run_info, user) is the key. -- To view, visit http://gerrit.cloudera.org:8080/10100 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ica7c00ad59963d466bae9e607a4692af0138962c Gerrit-Change-Number: 10100 Gerrit-PatchSet: 12 Gerrit-Owner: Nithya Janarthanan Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Jinchul Kim Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Nithya Janarthanan Gerrit-Comment-Date: Wed, 18 Jul 2018 23:38:37 + Gerrit-HasComments: Yes
[Impala-ASF-CR] [experimental] Clang Tidy Diff trial balloon
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/9751 ) Change subject: [experimental] Clang Tidy Diff trial balloon .. Patch Set 2: Filed https://issues.apache.org/jira/browse/IMPALA-7319 to track this. -- To view, visit http://gerrit.cloudera.org:8080/9751 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie2fb6a13400367fd3d12a4738bbb2dfc944466a7 Gerrit-Change-Number: 9751 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 23:34:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7314: Doc generation should fail on error
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10976 ) Change subject: IMPALA-7314: Doc generation should fail on error .. IMPALA-7314: Doc generation should fail on error This patch updates the doc generation to fail when there is an error. dita does not exit with non-zero exit code when there is an error. The patch checks for [ERROR] in the dita output and fails if it encounters one. Testing: - Manually tested by injecting failures Change-Id: Ic452aa282a3f2a761e3b04a7460e0d86bc51d721 Reviewed-on: http://gerrit.cloudera.org:8080/10976 Reviewed-by: Alex Rodoni Reviewed-by: Michael Brown Tested-by: Impala Public Jenkins --- M docs/.gitignore M docs/Makefile A docs/build-doc.sh 3 files changed, 39 insertions(+), 2 deletions(-) Approvals: Alex Rodoni: Looks good to me, but someone else must approve Michael Brown: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/10976 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ic452aa282a3f2a761e3b04a7460e0d86bc51d721 Gerrit-Change-Number: 10976 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7314: Doc generation should fail on error
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10976 ) Change subject: IMPALA-7314: Doc generation should fail on error .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10976 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic452aa282a3f2a761e3b04a7460e0d86bc51d721 Gerrit-Change-Number: 10976 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 23:32:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7314: Doc generation should fail on error
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10976 ) Change subject: IMPALA-7314: Doc generation should fail on error .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-docs-submit/351/ -- To view, visit http://gerrit.cloudera.org:8080/10976 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic452aa282a3f2a761e3b04a7460e0d86bc51d721 Gerrit-Change-Number: 10976 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 23:26:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7314: Doc generation should fail on error
Michael Brown has posted comments on this change. ( http://gerrit.cloudera.org:8080/10976 ) Change subject: IMPALA-7314: Doc generation should fail on error .. Patch Set 2: Code-Review+2 I'm happy to promote my +1. For docs, use https://jenkins.impala.io/job/gerrit-docs-submit/build -- To view, visit http://gerrit.cloudera.org:8080/10976 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic452aa282a3f2a761e3b04a7460e0d86bc51d721 Gerrit-Change-Number: 10976 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 23:25:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7296: bytes limit for row batch queue
Hello Michael Ho, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10977 to look at the new patch set (#4). Change subject: IMPALA-7296: bytes limit for row batch queue .. IMPALA-7296: bytes limit for row batch queue https://goo.gl/N9LgQt summarises the memory problems I'm trying to solve here. Limit the number of enqueued row batches to a number of bytes, instead of limiting the total number of batches. This helps avoid pathologically high memory consumption for wide rows where the # batches limit does not effectively limit the memory consumption. The bytes limit only lowers the effective capacity of the queue for wider rows, typically 150 bytes or wider. These are the cases when we want to reduce the queue's capacity. E.g. on a system with 10 disks, the previous sizing gave a queue of 100 batches. If we assume rows with 10x16 byte columns, then 100 batches is ~16MB of data. Remove RowBatchQueueCapacity counter that is less relevant now and was not correctly initialised. Testing: Added some basic unit tests. Add regression test that fails reliably before this change. Ran exhaustive build. Change-Id: Iaa06d1d8da2a6d101efda08f620c0bf84a71e681 --- M be/src/exec/scan-node.cc M be/src/exec/scan-node.h M be/src/runtime/row-batch-queue.cc M be/src/runtime/row-batch-queue.h M be/src/util/blocking-queue-test.cc M be/src/util/blocking-queue.h M tests/common/test_dimensions.py M tests/query_test/test_mem_usage_scaling.py 8 files changed, 204 insertions(+), 27 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/77/10977/4 -- To view, visit http://gerrit.cloudera.org:8080/10977 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iaa06d1d8da2a6d101efda08f620c0bf84a71e681 Gerrit-Change-Number: 10977 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7315: fix test update with clear entries flag race
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10986 ) Change subject: IMPALA-7315: fix test_update_with_clear_entries_flag race .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2837/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/10986 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2be2b61b6deb0228fbc5a242e43076beb8871454 Gerrit-Change-Number: 10986 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 18 Jul 2018 22:51:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7315: fix test update with clear entries flag race
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10986 ) Change subject: IMPALA-7315: fix test_update_with_clear_entries_flag race .. Patch Set 2: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2835/ -- To view, visit http://gerrit.cloudera.org:8080/10986 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2be2b61b6deb0228fbc5a242e43076beb8871454 Gerrit-Change-Number: 10986 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 18 Jul 2018 22:51:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7315: fix test update with clear entries flag race
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10986 ) Change subject: IMPALA-7315: fix test_update_with_clear_entries_flag race .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10986 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2be2b61b6deb0228fbc5a242e43076beb8871454 Gerrit-Change-Number: 10986 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 18 Jul 2018 22:51:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6034: Add scanned bytes limits per query
Tim Armstrong has posted comments on this change. (
http://gerrit.cloudera.org:8080/10415 )
Change subject: IMPALA-6034: Add scanned bytes limits per query
..
Patch Set 11:
(2 comments)
Moved CPU_LIMIT_S to DEVELOPMENT
http://gerrit.cloudera.org:8080/#/c/10415/9//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/10415/9//COMMIT_MSG@14
PS9, Line 14:
> can you include an example too?
Ooops sorry, intended to come back to it.
http://gerrit.cloudera.org:8080/#/c/10415/9/be/src/runtime/coordinator-backend-state.cc
File be/src/runtime/coordinator-backend-state.cc:
http://gerrit.cloudera.org:8080/#/c/10415/9/be/src/runtime/coordinator-backend-state.cc@600
PS9, Line 600: avg_profile_->AddInfoString("num instances",
lexical_cast(num_instances_));
: }
> can you address this too?
Sorry, I intended to come back to this but lost track of it.
--
To view, visit http://gerrit.cloudera.org:8080/10415
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4c6015e21da684bb9f33e236d71309dd4c178a20
Gerrit-Change-Number: 10415
Gerrit-PatchSet: 11
Gerrit-Owner: Mostafa Mokhtar
Gerrit-Reviewer: Bikramjeet Vig
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Mostafa Mokhtar
Gerrit-Reviewer: Sailesh Mukil
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Wed, 18 Jul 2018 22:50:28 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6034: Add scanned bytes limits per query
Tim Armstrong has uploaded a new patch set (#12) to the change originally created by Mostafa Mokhtar. ( http://gerrit.cloudera.org:8080/10415 ) Change subject: IMPALA-6034: Add scanned bytes limits per query .. IMPALA-6034: Add scanned bytes limits per query This adds support for aggregate resource limits at runtime, specified via query options. If a query exceeds a limit it is terminated. The checks are periodic so the query may go somewhat over the limits. SCAN_BYTES_LIMIT is exposed as an advanced query option. CPU_LIMIT_S is hidden as a development query option because it is flawed - the CPU user/sys time is only updated upon thread completion, so in many cases the limit will not take effect until well after the resources have been used. IMPALA-7318 tracks enabling this. Query profile is updated to include query wide and per backend metrics for CPU and scanned bytes. Example from "select count(*) from tpch_parquet.lineitem": Per Node Peak Memory Usage: tarmstrong-box:22000(289.50 KB) tarmstrong-box:22001(249.50 KB) tarmstrong-box:22002(249.50 KB) Per Node Bytes Read: tarmstrong-box:22000(100.00 KB) tarmstrong-box:22001(100.00 KB) tarmstrong-box:22002(100.00 KB) Per Node User Time: tarmstrong-box:22000(40.000ms) tarmstrong-box:22001(32.000ms) tarmstrong-box:22002(24.000ms) Per Node System Time: tarmstrong-box:22000(0.000ns) tarmstrong-box:22001(0.000ns) tarmstrong-box:22002(0.000ns) - FiltersReceived: 0 (0) - FinalizationTimer: 0.000ns - NumBackends: 3 (3) - NumFragmentInstances: 4 (4) - NumFragments: 2 (2) - TotalBytesRead: 300.00 KB (307200) - TotalCpuTime: 96.000ms Testing: Added tests for various permutations for CPU_LIMIT_S and SCAN_BYTES_LIMIT Change-Id: I4c6015e21da684bb9f33e236d71309dd4c178a20 --- M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator-backend-state.h M be/src/runtime/coordinator.cc M be/src/runtime/coordinator.h M be/src/service/impala-server.cc M be/src/service/impala-server.h M be/src/service/query-options-test.cc M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift A testdata/workloads/functional-query/queries/QueryTest/query-resource-limits.test M tests/query_test/test_cancellation.py M tests/query_test/test_resource_limits.py M www/query_backends.tmpl 15 files changed, 365 insertions(+), 52 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/15/10415/12 -- To view, visit http://gerrit.cloudera.org:8080/10415 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4c6015e21da684bb9f33e236d71309dd4c178a20 Gerrit-Change-Number: 10415 Gerrit-PatchSet: 12 Gerrit-Owner: Mostafa Mokhtar Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7259: Improve Impala shell performance
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10939 ) Change subject: IMPALA-7259: Improve Impala shell performance .. Patch Set 11: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10939 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idac9f3caed7c44846a8c922dbe5ca3bf3b095b81 Gerrit-Change-Number: 10939 Gerrit-PatchSet: 11 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 22:45:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7259: Improve Impala shell performance
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10939 ) Change subject: IMPALA-7259: Improve Impala shell performance .. Patch Set 11: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2836/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/10939 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idac9f3caed7c44846a8c922dbe5ca3bf3b095b81 Gerrit-Change-Number: 10939 Gerrit-PatchSet: 11 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 22:45:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7316: Fix broken build due to Hadoop JAR mismatch
Fredy Wijaya has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10981 ) Change subject: IMPALA-7316: Fix broken build due to Hadoop JAR mismatch .. IMPALA-7316: Fix broken build due to Hadoop JAR mismatch Update the CDH_BUILD_NUMBER to 479815 which contains commit: https://github.com/apache/hadoop/commit/7c9cdad6d04c98db5a83e2108219bf6e6c903daf Testing: - Ran all core tests Change-Id: I363abb91146ccdefc073e21477c847f4efcd8877 Reviewed-on: http://gerrit.cloudera.org:8080/10981 Tested-by: Impala Public Jenkins Reviewed-by: Philip Zeyliger --- M bin/impala-config.sh 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Impala Public Jenkins: Verified Philip Zeyliger: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/10981 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I363abb91146ccdefc073e21477c847f4efcd8877 Gerrit-Change-Number: 10981 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7316: Fix broken build due to Hadoop JAR mismatch
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/10981 ) Change subject: IMPALA-7316: Fix broken build due to Hadoop JAR mismatch .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10981 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I363abb91146ccdefc073e21477c847f4efcd8877 Gerrit-Change-Number: 10981 Gerrit-PatchSet: 1 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 22:37:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7315: fix test update with clear entries flag race
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10986 ) Change subject: IMPALA-7315: fix test_update_with_clear_entries_flag race .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10986 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2be2b61b6deb0228fbc5a242e43076beb8871454 Gerrit-Change-Number: 10986 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 18 Jul 2018 22:36:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7315: fix test update with clear entries flag race
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10986 ) Change subject: IMPALA-7315: fix test_update_with_clear_entries_flag race .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2835/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/10986 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2be2b61b6deb0228fbc5a242e43076beb8871454 Gerrit-Change-Number: 10986 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 18 Jul 2018 22:36:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7316: Fix broken build due to Hadoop JAR mismatch
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/10981 ) Change subject: IMPALA-7316: Fix broken build due to Hadoop JAR mismatch .. Patch Set 1: Jenkins build passed. Anyone wants to review this? -- To view, visit http://gerrit.cloudera.org:8080/10981 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I363abb91146ccdefc073e21477c847f4efcd8877 Gerrit-Change-Number: 10981 Gerrit-PatchSet: 1 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 22:26:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7316: Fix broken build due to Hadoop JAR mismatch
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10981 ) Change subject: IMPALA-7316: Fix broken build due to Hadoop JAR mismatch .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10981 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I363abb91146ccdefc073e21477c847f4efcd8877 Gerrit-Change-Number: 10981 Gerrit-PatchSet: 1 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Wed, 18 Jul 2018 22:25:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7314: Doc generation should fail on error
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/10976 ) Change subject: IMPALA-7314: Doc generation should fail on error .. Patch Set 2: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/10976 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic452aa282a3f2a761e3b04a7460e0d86bc51d721 Gerrit-Change-Number: 10976 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 22:24:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7315: fix test update with clear entries flag race
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/10986 ) Change subject: IMPALA-7315: fix test_update_with_clear_entries_flag race .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10986 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2be2b61b6deb0228fbc5a242e43076beb8871454 Gerrit-Change-Number: 10986 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 18 Jul 2018 22:23:24 + Gerrit-HasComments: No
[Impala-ASF-CR] [experimental] Clang Tidy Diff trial balloon
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/9751 ) Change subject: [experimental] Clang Tidy Diff trial balloon .. Patch Set 2: Here are my thoughts on this CR: 1. I'm going to file a JIRA about clang tidy diff. Even if we don't pursue this, there should be a record for later. I will incorporate that JIRA on this CR. 2. Clang tidy diff is dramatically faster than run_clang_tidy.sh, but it is also not 100% correct. There are code changes that can introduce clang tidy issues in code that hasn't changed. Unless we are comfortable with that, clang tidy diff won't replace run_clang_tidy.sh. 3. For automated jobs (like clang-tidy-ub1604 and whatever happens for IMPALA-7317), speed only matters so much. 4. I think some improvements to run_clang_tidy.sh to be able to run clang tidy on a list of files would cut down on the cost to run locally. -- To view, visit http://gerrit.cloudera.org:8080/9751 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie2fb6a13400367fd3d12a4738bbb2dfc944466a7 Gerrit-Change-Number: 9751 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 22:19:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6034: Add CPU and scanned bytes limits per query
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/10415 ) Change subject: IMPALA-6034: Add CPU and scanned bytes limits per query .. Patch Set 11: > I was talking to Pooja offline and realised that the CPU limit > implementation is deeply flawed because we don't update the CPU > usage at any regular cadence - e.g. scanner threads only update it > when they exit. > > I'm thinking about making cpu_limit_s a "DEVELOPMENT" query option > and leaving it undocumented until we can address that issue. > > The bytes read implementation works as expected. Sounds good, can you also create a JIRA for tracking that issue -- To view, visit http://gerrit.cloudera.org:8080/10415 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4c6015e21da684bb9f33e236d71309dd4c178a20 Gerrit-Change-Number: 10415 Gerrit-PatchSet: 11 Gerrit-Owner: Mostafa Mokhtar Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 22:17:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7314: Doc generation should fail on error
Michael Brown has posted comments on this change. ( http://gerrit.cloudera.org:8080/10976 ) Change subject: IMPALA-7314: Doc generation should fail on error .. Patch Set 2: Code-Review+1 Please make sure Alex understands and agrees with the workflow changes. -- To view, visit http://gerrit.cloudera.org:8080/10976 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic452aa282a3f2a761e3b04a7460e0d86bc51d721 Gerrit-Change-Number: 10976 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 22:16:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10965 ) Change subject: IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog .. Patch Set 2: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2834/ -- To view, visit http://gerrit.cloudera.org:8080/10965 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I41fd855fbf0e9ec58845ac0d2eb96a87b0172152 Gerrit-Change-Number: 10965 Gerrit-PatchSet: 2 Gerrit-Owner: Tianyi Wang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 22:07:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7314: Doc generation should fail on error
Fredy Wijaya has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/10976 ) Change subject: IMPALA-7314: Doc generation should fail on error .. IMPALA-7314: Doc generation should fail on error This patch updates the doc generation to fail when there is an error. dita does not exit with non-zero exit code when there is an error. The patch checks for [ERROR] in the dita output and fails if it encounters one. Testing: - Manually tested by injecting failures Change-Id: Ic452aa282a3f2a761e3b04a7460e0d86bc51d721 --- M docs/.gitignore M docs/Makefile A docs/build-doc.sh 3 files changed, 39 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/76/10976/2 -- To view, visit http://gerrit.cloudera.org:8080/10976 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic452aa282a3f2a761e3b04a7460e0d86bc51d721 Gerrit-Change-Number: 10976 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya
[Impala-ASF-CR] [DOCS] Fixed an invalid link in impala set.xml
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10985 ) Change subject: [DOCS] Fixed an invalid link in impala_set.xml .. [DOCS] Fixed an invalid link in impala_set.xml Change-Id: I25200f832fe6c3e4666ca391434558b1db570c59 Reviewed-on: http://gerrit.cloudera.org:8080/10985 Reviewed-by: Alex Rodoni Tested-by: Impala Public Jenkins --- M docs/topics/impala_set.xml 1 file changed, 2 insertions(+), 2 deletions(-) Approvals: Alex Rodoni: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/10985 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I25200f832fe6c3e4666ca391434558b1db570c59 Gerrit-Change-Number: 10985 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [DOCS] Fixed an invalid link in impala set.xml
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10985 ) Change subject: [DOCS] Fixed an invalid link in impala_set.xml .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10985 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I25200f832fe6c3e4666ca391434558b1db570c59 Gerrit-Change-Number: 10985 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 18 Jul 2018 21:52:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7315: fix test update with clear entries flag race
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10986 Change subject: IMPALA-7315: fix test_update_with_clear_entries_flag race .. IMPALA-7315: fix test_update_with_clear_entries_flag race We need to wait for the subscriber to process the second update in order to guarantee that the first update for that subscriber has been applied. Otherwise there is a race window where the second subscriber may see the older version of the statestore topic. Change-Id: I2be2b61b6deb0228fbc5a242e43076beb8871454 --- M tests/statestore/test_statestore.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/10986/1 -- To view, visit http://gerrit.cloudera.org:8080/10986 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I2be2b61b6deb0228fbc5a242e43076beb8871454 Gerrit-Change-Number: 10986 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong
[Impala-ASF-CR] [DOCS] Fixed an invalid link in impala set.xml
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10985 ) Change subject: [DOCS] Fixed an invalid link in impala_set.xml .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-docs-submit/350/ -- To view, visit http://gerrit.cloudera.org:8080/10985 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I25200f832fe6c3e4666ca391434558b1db570c59 Gerrit-Change-Number: 10985 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 18 Jul 2018 21:43:23 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Fixed an invalid link in impala set.xml
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/10985 ) Change subject: [DOCS] Fixed an invalid link in impala_set.xml .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10985 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I25200f832fe6c3e4666ca391434558b1db570c59 Gerrit-Change-Number: 10985 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Comment-Date: Wed, 18 Jul 2018 21:43:02 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Fixed an invalid link in impala set.xml
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10985 Change subject: [DOCS] Fixed an invalid link in impala_set.xml .. [DOCS] Fixed an invalid link in impala_set.xml Change-Id: I25200f832fe6c3e4666ca391434558b1db570c59 --- M docs/topics/impala_set.xml 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/85/10985/1 -- To view, visit http://gerrit.cloudera.org:8080/10985 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I25200f832fe6c3e4666ca391434558b1db570c59 Gerrit-Change-Number: 10985 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] [DOCS] Fixed an incorrect link in impala perf stats.xml
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10984 ) Change subject: [DOCS] Fixed an incorrect link in impala_perf_stats.xml .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10984 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8a2fa82d5c778fa05a94f84bf1a213e89a6231f1 Gerrit-Change-Number: 10984 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 18 Jul 2018 21:30:59 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Fixed an incorrect link in impala perf stats.xml
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10984 ) Change subject: [DOCS] Fixed an incorrect link in impala_perf_stats.xml .. [DOCS] Fixed an incorrect link in impala_perf_stats.xml Change-Id: I8a2fa82d5c778fa05a94f84bf1a213e89a6231f1 Reviewed-on: http://gerrit.cloudera.org:8080/10984 Reviewed-by: Alex Rodoni Tested-by: Impala Public Jenkins --- M docs/topics/impala_perf_stats.xml 1 file changed, 3 insertions(+), 7 deletions(-) Approvals: Alex Rodoni: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/10984 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I8a2fa82d5c778fa05a94f84bf1a213e89a6231f1 Gerrit-Change-Number: 10984 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [DOCS] Fix UPDATE/UPSERT/DELETE authorization doc
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10975 ) Change subject: [DOCS] Fix UPDATE/UPSERT/DELETE authorization doc .. [DOCS] Fix UPDATE/UPSERT/DELETE authorization doc The patch also fixes broken links in the authorization doc. Change-Id: I9bf6109636e44ca514cfe74fb565f7c506ec0708 Reviewed-on: http://gerrit.cloudera.org:8080/10975 Reviewed-by: Fredy Wijaya Tested-by: Impala Public Jenkins --- M docs/shared/impala_common.xml M docs/topics/impala_authorization.xml 2 files changed, 339 insertions(+), 1,076 deletions(-) Approvals: Fredy Wijaya: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/10975 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I9bf6109636e44ca514cfe74fb565f7c506ec0708 Gerrit-Change-Number: 10975 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [DOCS] Fix UPDATE/UPSERT/DELETE authorization doc
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10975 ) Change subject: [DOCS] Fix UPDATE/UPSERT/DELETE authorization doc .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10975 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9bf6109636e44ca514cfe74fb565f7c506ec0708 Gerrit-Change-Number: 10975 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 18 Jul 2018 21:30:31 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Fixed an incorrect link in impala perf stats.xml
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10984 ) Change subject: [DOCS] Fixed an incorrect link in impala_perf_stats.xml .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-docs-submit/349/ -- To view, visit http://gerrit.cloudera.org:8080/10984 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8a2fa82d5c778fa05a94f84bf1a213e89a6231f1 Gerrit-Change-Number: 10984 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 18 Jul 2018 21:23:12 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Fixed an incorrect link in impala perf stats.xml
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/10984 ) Change subject: [DOCS] Fixed an incorrect link in impala_perf_stats.xml .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10984 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8a2fa82d5c778fa05a94f84bf1a213e89a6231f1 Gerrit-Change-Number: 10984 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Comment-Date: Wed, 18 Jul 2018 21:22:49 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Fixed an incorrect link in impala perf stats.xml
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10984 Change subject: [DOCS] Fixed an incorrect link in impala_perf_stats.xml .. [DOCS] Fixed an incorrect link in impala_perf_stats.xml Change-Id: I8a2fa82d5c778fa05a94f84bf1a213e89a6231f1 --- M docs/topics/impala_perf_stats.xml 1 file changed, 3 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/10984/1 -- To view, visit http://gerrit.cloudera.org:8080/10984 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I8a2fa82d5c778fa05a94f84bf1a213e89a6231f1 Gerrit-Change-Number: 10984 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] [DOCS] Fix UPDATE/UPSERT/DELETE authorization doc
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10975 ) Change subject: [DOCS] Fix UPDATE/UPSERT/DELETE authorization doc .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-docs-submit/348/ -- To view, visit http://gerrit.cloudera.org:8080/10975 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9bf6109636e44ca514cfe74fb565f7c506ec0708 Gerrit-Change-Number: 10975 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 18 Jul 2018 21:20:02 + Gerrit-HasComments: No
[Impala-ASF-CR] [experimental] Clang Tidy Diff trial balloon
Tim Armstrong has abandoned this change. ( http://gerrit.cloudera.org:8080/9751 ) Change subject: [experimental] Clang Tidy Diff trial balloon .. Abandoned I think this is now covered by IMPALA-7317 so I'll abandon for now. Restore if you disagree. -- To view, visit http://gerrit.cloudera.org:8080/9751 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: Ie2fb6a13400367fd3d12a4738bbb2dfc944466a7 Gerrit-Change-Number: 9751 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-6034: Add CPU and scanned bytes limits per query
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10415 ) Change subject: IMPALA-6034: Add CPU and scanned bytes limits per query .. Patch Set 11: I was talking to Pooja offline and realised that the CPU limit implementation is deeply flawed because we don't update the CPU usage at any regular cadence - e.g. scanner threads only update it when they exit. I'm thinking about making cpu_limit_s a "DEVELOPMENT" query option and leaving it undocumented until we can address that issue. The bytes read implementation works as expected. -- To view, visit http://gerrit.cloudera.org:8080/10415 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4c6015e21da684bb9f33e236d71309dd4c178a20 Gerrit-Change-Number: 10415 Gerrit-PatchSet: 11 Gerrit-Owner: Mostafa Mokhtar Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 21:00:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5031: Fix undefined behavior: memset NULL
Tim Armstrong has posted comments on this change. (
http://gerrit.cloudera.org:8080/10948 )
Change subject: IMPALA-5031: Fix undefined behavior: memset NULL
..
Patch Set 3: Code-Review+1
(2 comments)
I'm fine with moving forward with this as-is, Todd's suggestion makes sense but
I'll leave it up to you two
http://gerrit.cloudera.org:8080/#/c/10948/3/be/src/util/ubsan.h
File be/src/util/ubsan.h:
http://gerrit.cloudera.org:8080/#/c/10948/3/be/src/util/ubsan.h@26
PS3, Line 26: class Ubsan {
> I think in those cases I'd probably put each of the methods into wherever t
Todd's suggestion makes sense to me. I don't think this way of organising
utilities is going to cause any significant issues though.
http://gerrit.cloudera.org:8080/#/c/10948/2/be/src/util/ubsan.h
File be/src/util/ubsan.h:
http://gerrit.cloudera.org:8080/#/c/10948/2/be/src/util/ubsan.h@29
PS2, Line 29: if (s == nullptr) {
> I picked the first one, because in a release build, the calling MemSet(NULL
In principle I don't agree with writing code to defensively handle invariant
violations. In this case it doesn't complicate the code but I feel like maybe
this is part of the reason we had trouble getting the ubsan changes through.
IMO, we should add the appropriate DCHECKs and tests and then streamline the
code as much as possible. It's impossible to test handling of invariant
violations by definition and once you start doing it there's an endless number
of opportunities to complicate code by being unnecessarily defensive.
--
To view, visit http://gerrit.cloudera.org:8080/10948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I18fa02dc887a42a94c6f81e4923d17568f2184f2
Gerrit-Change-Number: 10948
Gerrit-PatchSet: 3
Gerrit-Owner: Jim Apple
Gerrit-Reviewer: Jim Apple
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: Todd Lipcon
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Wed, 18 Jul 2018 20:55:10 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10965 ) Change subject: IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2834/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/10965 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I41fd855fbf0e9ec58845ac0d2eb96a87b0172152 Gerrit-Change-Number: 10965 Gerrit-PatchSet: 2 Gerrit-Owner: Tianyi Wang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 20:35:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10965 ) Change subject: IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10965 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I41fd855fbf0e9ec58845ac0d2eb96a87b0172152 Gerrit-Change-Number: 10965 Gerrit-PatchSet: 2 Gerrit-Owner: Tianyi Wang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 20:35:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7316: Fix broken build due to Hadoop JAR mismatch
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/10981 ) Change subject: IMPALA-7316: Fix broken build due to Hadoop JAR mismatch .. Patch Set 1: Running a dry-run to make sure everything is good. -- To view, visit http://gerrit.cloudera.org:8080/10981 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I363abb91146ccdefc073e21477c847f4efcd8877 Gerrit-Change-Number: 10981 Gerrit-PatchSet: 1 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 18 Jul 2018 19:12:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7316: Fix broken build due to Hadoop JAR mismatch
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10981 ) Change subject: IMPALA-7316: Fix broken build due to Hadoop JAR mismatch .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2833/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/10981 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I363abb91146ccdefc073e21477c847f4efcd8877 Gerrit-Change-Number: 10981 Gerrit-PatchSet: 1 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 18 Jul 2018 19:11:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7316: Fix broken build due to Hadoop JAR mismatch
Fredy Wijaya has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10981 Change subject: IMPALA-7316: Fix broken build due to Hadoop JAR mismatch .. IMPALA-7316: Fix broken build due to Hadoop JAR mismatch Update the CDH_BUILD_NUMBER to 479815 which contains commit: https://github.com/apache/hadoop/commit/7c9cdad6d04c98db5a83e2108219bf6e6c903daf Testing: - Ran all core tests Change-Id: I363abb91146ccdefc073e21477c847f4efcd8877 --- M bin/impala-config.sh 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/10981/1 -- To view, visit http://gerrit.cloudera.org:8080/10981 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I363abb91146ccdefc073e21477c847f4efcd8877 Gerrit-Change-Number: 10981 Gerrit-PatchSet: 1 Gerrit-Owner: Fredy Wijaya
[Impala-ASF-CR] [DOCS] Fix UPDATE/UPSERT/DELETE authorization doc
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/10975 ) Change subject: [DOCS] Fix UPDATE/UPSERT/DELETE authorization doc .. Patch Set 3: Code-Review+2 > Patch Set 3: > > Fixed the previous merges that went very wrong. Thanks for the fix, it looks good now. I ran the doc build and there were only few broken link errors, which we should fix in a different CR. -- To view, visit http://gerrit.cloudera.org:8080/10975 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9bf6109636e44ca514cfe74fb565f7c506ec0708 Gerrit-Change-Number: 10975 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Comment-Date: Wed, 18 Jul 2018 18:54:23 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Fix UPDATE/UPSERT/DELETE authorization doc
Alex Rodoni has uploaded a new patch set (#3) to the change originally created by Fredy Wijaya. ( http://gerrit.cloudera.org:8080/10975 ) Change subject: [DOCS] Fix UPDATE/UPSERT/DELETE authorization doc .. [DOCS] Fix UPDATE/UPSERT/DELETE authorization doc The patch also fixes broken links in the authorization doc. Change-Id: I9bf6109636e44ca514cfe74fb565f7c506ec0708 --- M docs/shared/impala_common.xml M docs/topics/impala_authorization.xml 2 files changed, 339 insertions(+), 1,076 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/75/10975/3 -- To view, visit http://gerrit.cloudera.org:8080/10975 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9bf6109636e44ca514cfe74fb565f7c506ec0708 Gerrit-Change-Number: 10975 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya
[Impala-ASF-CR] [DOCS] Fix UPDATE/UPSERT/DELETE authorization doc
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/10975 ) Change subject: [DOCS] Fix UPDATE/UPSERT/DELETE authorization doc .. Patch Set 3: Fixed the previous merges that went very wrong. -- To view, visit http://gerrit.cloudera.org:8080/10975 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9bf6109636e44ca514cfe74fb565f7c506ec0708 Gerrit-Change-Number: 10975 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya Gerrit-Comment-Date: Wed, 18 Jul 2018 18:46:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7212: Removes --use krpc flag and remove old DataStream services
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/10835 ) Change subject: IMPALA-7212: Removes --use_krpc flag and remove old DataStream services .. Patch Set 4: Code-Review+1 This looks good to me. -- To view, visit http://gerrit.cloudera.org:8080/10835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icfed200751508478a3d728a917448f2dabfc67c3 Gerrit-Change-Number: 10835 Gerrit-PatchSet: 4 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Wed, 18 Jul 2018 18:39:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10965 ) Change subject: IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10965 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I41fd855fbf0e9ec58845ac0d2eb96a87b0172152 Gerrit-Change-Number: 10965 Gerrit-PatchSet: 1 Gerrit-Owner: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 18:37:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog
Tianyi Wang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10965 Change subject: IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog .. IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog This patch bumps glog version to 0.3.4-p3 to include the patch limiting fadvise calls. Change-Id: I41fd855fbf0e9ec58845ac0d2eb96a87b0172152 --- M bin/impala-config.sh 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/65/10965/1 -- To view, visit http://gerrit.cloudera.org:8080/10965 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I41fd855fbf0e9ec58845ac0d2eb96a87b0172152 Gerrit-Change-Number: 10965 Gerrit-PatchSet: 1 Gerrit-Owner: Tianyi Wang
[Impala-ASF-CR] IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Michael Ho has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10980 Change subject: IMPALA-7298: Stop passing IP address as hostname in Kerberos principal .. IMPALA-7298: Stop passing IP address as hostname in Kerberos principal Previously, we pass the resolved IP address of a KRPC destination host as the hostname when creating a proxy for making KRPC calls. This may lead to connection negotiation failure in KRPC when Kerberos is enabled. In particular, if reversed DNS isn't enabled in Kerberos, KDC may fail to look up the principal of the destination host if the principal includes the hostname instead of resolved IP address. This change fixes the problem above by passing the actual hostname of the destination host when calling RpcMgr::GetProxy(). rpc-mgr-kerberized-test.cc is also updated to use hostname instead of the resolved IP address as Kerberos principal. Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e --- M be/src/rpc/rpc-mgr-kerberized-test.cc M be/src/rpc/rpc-mgr-test-base.h M be/src/rpc/rpc-mgr-test.cc M be/src/rpc/rpc-mgr.h M be/src/rpc/rpc-mgr.inline.h M be/src/runtime/data-stream-sender.cc M be/src/runtime/data-stream-test.cc M be/src/runtime/krpc-data-stream-sender.cc M be/src/scheduling/scheduler.cc M common/thrift/ImpalaInternalService.thrift 10 files changed, 33 insertions(+), 27 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/10980/1 -- To view, visit http://gerrit.cloudera.org:8080/10980 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e Gerrit-Change-Number: 10980 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Ho
[Impala-ASF-CR] IMPALA-6299: Modify IRBuilder to use LLVM's CPU features
Pooja Nilangekar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10979 Change subject: IMPALA-6299: Modify IRBuilder to use LLVM's CPU features .. IMPALA-6299: Modify IRBuilder to use LLVM's CPU features Previously, the IRBuilder of the LlvmCodeGen class used CpuInfo's list of enabled features while determine the validity of certian instructions. It did not consider the whitelist which passed while initilaizing the LlvmCodeGen class. Now, the IRBuilder inspects its own cpu attributes before emitting instruction. This change also adds functionality to modify the cpu attributes of the LlvmCodeGen class for testing. Testing: Verified that the current tests which use and modify CpuInfo produce expected results. Change-Id: Ifece8949c143146d2a1b38d72d21c2d733bed90f --- M be/src/benchmarks/bloom-filter-benchmark.cc M be/src/codegen/llvm-codegen-test.cc M be/src/codegen/llvm-codegen.cc M be/src/codegen/llvm-codegen.h M be/src/util/bit-util-test.cc M be/src/util/cpu-info.cc M be/src/util/cpu-info.h 7 files changed, 110 insertions(+), 30 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/79/10979/2 -- To view, visit http://gerrit.cloudera.org:8080/10979 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ifece8949c143146d2a1b38d72d21c2d733bed90f Gerrit-Change-Number: 10979 Gerrit-PatchSet: 2 Gerrit-Owner: Pooja Nilangekar Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] [DOCS] Fix UPDATE/UPSERT/DELETE authorization doc
Fredy Wijaya has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/10975 ) Change subject: [DOCS] Fix UPDATE/UPSERT/DELETE authorization doc .. [DOCS] Fix UPDATE/UPSERT/DELETE authorization doc The patch also fixes broken links in the authorization doc. Change-Id: I9bf6109636e44ca514cfe74fb565f7c506ec0708 --- M docs/shared/impala_common.xml M docs/topics/impala_authorization.xml 2 files changed, 8 insertions(+), 249 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/75/10975/2 -- To view, visit http://gerrit.cloudera.org:8080/10975 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9bf6109636e44ca514cfe74fb565f7c506ec0708 Gerrit-Change-Number: 10975 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Fredy Wijaya
[Impala-ASF-CR] IMPALA-7296: bytes limit for row batch queue
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10977 ) Change subject: IMPALA-7296: bytes limit for row batch queue .. Patch Set 3: kwho, would be nice for you to have a look when you have time - I think you have a lot of context on this problem and the queue code. -- To view, visit http://gerrit.cloudera.org:8080/10977 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iaa06d1d8da2a6d101efda08f620c0bf84a71e681 Gerrit-Change-Number: 10977 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 17:15:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7296: bytes limit for row batch queue
Tim Armstrong has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/10977 ) Change subject: IMPALA-7296: bytes limit for row batch queue .. IMPALA-7296: bytes limit for row batch queue https://goo.gl/N9LgQt summarises the memory problems I'm trying to solve here. Limit the number of enqueued row batches to a number of bytes, instead of limiting the total number of batches. This helps avoid pathologically high memory consumption for wide rows where the # batches limit does not effectively limit the memory consumption. The bytes limit only lowers the effective capacity of the queue for wider rows, typically 150 bytes or wider. These are the cases when we want to reduce the queue's capacity. E.g. on a system with 10 disks, the previous sizing gave a queue of 100 batches. If we assume rows with 10x16 byte columns, then 100 batches is ~16MB of data. Remove RowBatchQueueCapacity counter that is less relevant now and was not correctly initialised. Testing: Added some basic unit tests. Add regression test that fails reliably before this change. Ran exhaustive build. Change-Id: Iaa06d1d8da2a6d101efda08f620c0bf84a71e681 --- M be/src/exec/scan-node.cc M be/src/exec/scan-node.h M be/src/runtime/row-batch-queue.cc M be/src/runtime/row-batch-queue.h M be/src/util/blocking-queue-test.cc M be/src/util/blocking-queue.h M tests/common/test_dimensions.py M tests/query_test/test_mem_usage_scaling.py 8 files changed, 204 insertions(+), 27 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/77/10977/2 -- To view, visit http://gerrit.cloudera.org:8080/10977 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iaa06d1d8da2a6d101efda08f620c0bf84a71e681 Gerrit-Change-Number: 10977 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong
[Impala-ASF-CR] IMPALA-7259: Improve Impala shell performance
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10939 ) Change subject: IMPALA-7259: Improve Impala shell performance .. Patch Set 10: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2832/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/10939 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idac9f3caed7c44846a8c922dbe5ca3bf3b095b81 Gerrit-Change-Number: 10939 Gerrit-PatchSet: 10 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 16:30:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7259: Improve Impala shell performance
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10939 ) Change subject: IMPALA-7259: Improve Impala shell performance .. Patch Set 10: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10939 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idac9f3caed7c44846a8c922dbe5ca3bf3b095b81 Gerrit-Change-Number: 10939 Gerrit-PatchSet: 10 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 16:30:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7259: Improve Impala shell performance
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/10939 ) Change subject: IMPALA-7259: Improve Impala shell performance .. Patch Set 9: Code-Review+2 > Patch Set 9: Code-Review+1 > > I'll give Nghia a chance to look too. I spoke to Nghia and he said the CR looks good. I'm going to give it a +2. -- To view, visit http://gerrit.cloudera.org:8080/10939 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idac9f3caed7c44846a8c922dbe5ca3bf3b095b81 Gerrit-Change-Number: 10939 Gerrit-PatchSet: 9 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 18 Jul 2018 16:29:49 + Gerrit-HasComments: No
[Impala-ASF-CR](2.x) IMPALA-7304: Don't write floating column index until PARQUET-1222 is resolved.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/10960 ) Change subject: IMPALA-7304: Don't write floating column index until PARQUET-1222 is resolved. .. Patch Set 1: Verified+1 It was a trivial cherry-pick, GVO failed for other reasons, verified +1 -- To view, visit http://gerrit.cloudera.org:8080/10960 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: 2.x Gerrit-MessageType: comment Gerrit-Change-Id: I50aa2e6607de6a8943eb068b8162b0506763078b Gerrit-Change-Number: 10960 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Wed, 18 Jul 2018 10:31:35 + Gerrit-HasComments: No
[Impala-ASF-CR](2.x) IMPALA-7304: Don't write floating column index until PARQUET-1222 is resolved.
Zoltan Borok-Nagy has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10960 ) Change subject: IMPALA-7304: Don't write floating column index until PARQUET-1222 is resolved. .. IMPALA-7304: Don't write floating column index until PARQUET-1222 is resolved. Impala master branch can already write the Parquet page index. However, we still don't have a well-defined ordering for floating-point numbers in Parquet, see PARQUET-1222 Currently impala writes the page index with fmax()/fmin() semantics, but it might contradicts the future semantics that will be defined once PARQUET-1222 is resolved. >From this patch Impala won't write the column index for floating-point columns until PARQUET-1222 is resolved and implemented. I updated the python test accordingly. Change-Id: I50aa2e6607de6a8943eb068b8162b0506763078b Reviewed-on: http://gerrit.cloudera.org:8080/10951 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins (cherry picked from commit 041197444d2a73bc3e3da4c6dbfdf1d63c236fbf) Reviewed-on: http://gerrit.cloudera.org:8080/10960 Reviewed-by: Zoltan Borok-Nagy Tested-by: Zoltan Borok-Nagy --- M be/src/exec/hdfs-parquet-table-writer.cc M tests/query_test/test_parquet_page_index.py 2 files changed, 11 insertions(+), 0 deletions(-) Approvals: Zoltan Borok-Nagy: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/10960 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: 2.x Gerrit-MessageType: merged Gerrit-Change-Id: I50aa2e6607de6a8943eb068b8162b0506763078b Gerrit-Change-Number: 10960 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR](2.x) IMPALA-7304: Don't write floating column index until PARQUET-1222 is resolved.
Zoltan Borok-Nagy has removed a vote on this change. Change subject: IMPALA-7304: Don't write floating column index until PARQUET-1222 is resolved. .. Removed Verified-1 by Impala Public Jenkins -- To view, visit http://gerrit.cloudera.org:8080/10960 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: 2.x Gerrit-MessageType: deleteVote Gerrit-Change-Id: I50aa2e6607de6a8943eb068b8162b0506763078b Gerrit-Change-Number: 10960 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
