[Impala-ASF-CR] IMPALA-4578: Pick up bound predicates for Kudu scan nodes.
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4578: Pick up bound predicates for Kudu scan nodes. .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5365 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I19a38d6ea8cc0d2b0ddc3808d1f9ffef5ce306a8 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4578: Pick up bound predicates for Kudu scan nodes.
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4578: Pick up bound predicates for Kudu scan nodes. .. IMPALA-4578: Pick up bound predicates for Kudu scan nodes. The bug was a simple oversight. In KuduScanNiode.init() we forgot to call Analyzer.getBoundPredicates(). Change-Id: I19a38d6ea8cc0d2b0ddc3808d1f9ffef5ce306a8 Reviewed-on: http://gerrit.cloudera.org:8080/5365 Reviewed-by: Alex Behm Tested-by: Internal Jenkins --- M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java M testdata/workloads/functional-planner/queries/PlannerTest/kudu.test 2 files changed, 42 insertions(+), 1 deletion(-) Approvals: Internal Jenkins: Verified Alex Behm: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5365 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I19a38d6ea8cc0d2b0ddc3808d1f9ffef5ce306a8 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker
[Impala-ASF-CR] IMPALA-4584: Make alter table operations on Kudu tables synchronous
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4584: Make alter table operations on Kudu tables synchronous .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5364 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I385bce66691ae9040e72f97557e1bba31009e36b Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Dimitris Tsirogiannis Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4584: Make alter table operations on Kudu tables synchronous
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4584: Make alter table operations on Kudu tables synchronous .. IMPALA-4584: Make alter table operations on Kudu tables synchronous This commit changes the behavior of alter table operations on Kudu tables from asynchronous to synchronous. With this change, alter table operations return when either the operations complete successfully or a timeout is reached. Change-Id: I385bce66691ae9040e72f97557e1bba31009e36b Reviewed-on: http://gerrit.cloudera.org:8080/5364 Reviewed-by: Dimitris Tsirogiannis Tested-by: Internal Jenkins --- M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test 2 files changed, 36 insertions(+), 28 deletions(-) Approvals: Internal Jenkins: Verified Dimitris Tsirogiannis: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5364 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I385bce66691ae9040e72f97557e1bba31009e36b Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Dimitris Tsirogiannis Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins
[Impala-ASF-CR] IMPALA-4561: Replace DISTRIBUTE BY with PARTITION BY in CREATE TABLE
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4561: Replace DISTRIBUTE BY with PARTITION BY in CREATE TABLE .. Patch Set 4: Verified-1 Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/600/ -- To view, visit http://gerrit.cloudera.org:8080/5317 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I0e07c41eabb4c8cb95754cf04293cbd9e03d6ab2 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Dimitris Tsirogiannis Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4574: Do not treat UUID() like a constant expr.
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4574: Do not treat UUID() like a constant expr. .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5324 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: If2499f5f6ecdcb098623202c8e6dc2d02727194a Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4571: Push IN predicates to Kudu
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4571: Push IN predicates to Kudu .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5316 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I8988d4819d20d467b48e286917e347ca00f60cf0 Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Matthew Jacobs Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Matthew Jacobs Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4571: Push IN predicates to Kudu
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4571: Push IN predicates to Kudu .. IMPALA-4571: Push IN predicates to Kudu Fixes the KuduScanNode to convert InPredicates to KuduPredicates and push them to the Kudu scan if possible. An InPredicate can be pushed to the scan if expression is of the exact form: IN (, , ...) That means the InPredicate has the following properties: 1) It has a list of literal values (i.e. not a subquery); All values are LiteralExprs (not SlotRefs). 2) Not negative, i.e. only 'IN' supported, not 'NOT IN' 3) The SlotRef is not wrapped in any casts 4) The types of all values match the type of the SlotRef exactly. A planner test was added exercising all supported types as well as exprs where the values would not be supported. TODO: perf testing TODO: consider a limit on the number of list values before keeping the predicate on the Impala scan node (determine from testing) Change-Id: I8988d4819d20d467b48e286917e347ca00f60cf0 Reviewed-on: http://gerrit.cloudera.org:8080/5316 Reviewed-by: Matthew Jacobs Tested-by: Internal Jenkins --- M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java M testdata/workloads/functional-planner/queries/PlannerTest/kudu-selectivity.test M testdata/workloads/functional-planner/queries/PlannerTest/tpch-kudu.test 3 files changed, 99 insertions(+), 7 deletions(-) Approvals: Matthew Jacobs: Looks good to me, approved Internal Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/5316 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I8988d4819d20d467b48e286917e347ca00f60cf0 Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Matthew Jacobs Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Matthew Jacobs
[Impala-ASF-CR] IMPALA-4574: Do not treat UUID() like a constant expr.
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4574: Do not treat UUID() like a constant expr. .. Patch Set 3: Verified-1 Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/599/ -- To view, visit http://gerrit.cloudera.org:8080/5324 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: If2499f5f6ecdcb098623202c8e6dc2d02727194a Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4561: Replace DISTRIBUTE BY with PARTITION BY in CREATE TABLE
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4561: Replace DISTRIBUTE BY with PARTITION BY in CREATE TABLE .. Patch Set 3: Verified-1 Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/598/ -- To view, visit http://gerrit.cloudera.org:8080/5317 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I0e07c41eabb4c8cb95754cf04293cbd9e03d6ab2 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Dimitris Tsirogiannis Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3125: Fix assignment of equality predicates from an outer-join On-clause.
Internal Jenkins has posted comments on this change. Change subject: IMPALA-3125: Fix assignment of equality predicates from an outer-join On-clause. .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/4986 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I719e0eeacccad070b1f9509d80aaf761b572add0 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3125: Fix assignment of equality predicates from an outer-join On-clause.
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-3125: Fix assignment of equality predicates from an outer-join On-clause. .. IMPALA-3125: Fix assignment of equality predicates from an outer-join On-clause. Impala used to incorrectly assign On-clause equality predicates from an outer join if those predicates referenced multiple tables, but only one side of the outer join. The fix is to add an additional check in Analyzer.getEqJoinConjuncts() to prevent that incorrect assignment. Change-Id: I719e0eeacccad070b1f9509d80aaf761b572add0 Reviewed-on: http://gerrit.cloudera.org:8080/4986 Reviewed-by: Alex Behm Tested-by: Internal Jenkins --- M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M testdata/workloads/functional-planner/queries/PlannerTest/outer-joins.test 2 files changed, 44 insertions(+), 3 deletions(-) Approvals: Internal Jenkins: Verified Alex Behm: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/4986 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I719e0eeacccad070b1f9509d80aaf761b572add0 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker
[Impala-ASF-CR] IMPALA-3125: Fix assignment of equality predicates from an outer-join On-clause.
Internal Jenkins has posted comments on this change. Change subject: IMPALA-3125: Fix assignment of equality predicates from an outer-join On-clause. .. Patch Set 2: Verified-1 Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/596/ -- To view, visit http://gerrit.cloudera.org:8080/4986 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I719e0eeacccad070b1f9509d80aaf761b572add0 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4303: Do not reset() qualifier of union operands.
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4303: Do not reset() qualifier of union operands. .. IMPALA-4303: Do not reset() qualifier of union operands. The bug: We used to reset() the qualifier of union operands to their original value obtained during parsing. This leads to problems when union operands are unnested and we need to rewrite Subqueries. In particular, the first union operand of a nested union was reset() to a null qualifier, but that operand could be somewhere in the middle of the list of unnested operands in the parent. At that point, we've lost information about the qualifier of the unnested operand. The fix: The simplest solution is to not reset() the qualifier. The other alternative is be to reset() the qualifier, but also undo any unnesting. That seems unnecessary and wasteful. Change-Id: I157bb0f08c4a94fd779487d7c23edd64a537a1f6 Reviewed-on: http://gerrit.cloudera.org:8080/4963 Reviewed-by: Alex Behm Tested-by: Internal Jenkins --- M fe/src/main/java/org/apache/impala/analysis/UnionStmt.java M testdata/workloads/functional-planner/queries/PlannerTest/subquery-rewrite.test 2 files changed, 74 insertions(+), 12 deletions(-) Approvals: Internal Jenkins: Verified Alex Behm: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/4963 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I157bb0f08c4a94fd779487d7c23edd64a537a1f6 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker
[Impala-ASF-CR] IMPALA-4303: Do not reset() qualifier of union operands.
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4303: Do not reset() qualifier of union operands. .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/4963 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I157bb0f08c4a94fd779487d7c23edd64a537a1f6 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4579: SHOW CREATE VIEW fails for view containing a subquery
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4579: SHOW CREATE VIEW fails for view containing a subquery .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5333 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4a89e46a022f0ccec198b6e3e2b30230103831ce Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Dimitris Tsirogiannis Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4579: SHOW CREATE VIEW fails for view containing a subquery
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4579: SHOW CREATE VIEW fails for view containing a subquery .. IMPALA-4579: SHOW CREATE VIEW fails for view containing a subquery This commit fixes an issue where a SHOW CREATE VIEW statement throws an analysis error if the view contains a subquery. Change-Id: I4a89e46a022f0ccec198b6e3e2b30230103831ce Reviewed-on: http://gerrit.cloudera.org:8080/5333 Reviewed-by: Dimitris Tsirogiannis Tested-by: Internal Jenkins --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M testdata/workloads/functional-query/queries/QueryTest/show-create-table.test 2 files changed, 10 insertions(+), 1 deletion(-) Approvals: Internal Jenkins: Verified Dimitris Tsirogiannis: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5333 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I4a89e46a022f0ccec198b6e3e2b30230103831ce Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Dimitris Tsirogiannis Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins
[Impala-ASF-CR] IMPALA-3875: Thrift threaded server hang in some cases
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-3875: Thrift threaded server hang in some cases .. IMPALA-3875: Thrift threaded server hang in some cases We use socket timeouts for our backend connections. We set these timeouts only after we've open()'d the connection, which ideally should be fine. However, our TSaslTransport stack does read()'s and write()'s over the network on an open(), which means that on a secure cluster we send and recieve non-TCP-handshake packets on open(). This is because the current code tries to establish a SASL handshake during open(). If for any reason the peer server does not respond to the read()'s during the open() call (after connect() is successful), the client will wait on read() indefinitely. This patch sets the socket timeout before we call open(), so that the read()'s and write()'s during the open() are subject to the timeout as well. We should also consider making a larger change where this SASL handshake does not take place during an open(), but instead after the open() call is completed, so as to have the open() semantics be the same for both secure and insecure clusters. Change-Id: I6c8f91a88f723e0e58e81bb385c5a8f190021868 Reviewed-on: http://gerrit.cloudera.org:8080/5263 Reviewed-by: Sailesh Mukil Tested-by: Internal Jenkins --- M be/src/rpc/thrift-client.h M be/src/runtime/client-cache.cc M be/src/statestore/statestore-test.cc 3 files changed, 14 insertions(+), 9 deletions(-) Approvals: Internal Jenkins: Verified Sailesh Mukil: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5263 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I6c8f91a88f723e0e58e81bb385c5a8f190021868 Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Sailesh Mukil Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] IMPALA-3875: Thrift threaded server hang in some cases
Internal Jenkins has posted comments on this change. Change subject: IMPALA-3875: Thrift threaded server hang in some cases .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5263 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I6c8f91a88f723e0e58e81bb385c5a8f190021868 Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Sailesh Mukil Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Sailesh Mukil Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4572: Run COMPUTE STATS on Parquet tables with MT DOP=4.
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4572: Run COMPUTE STATS on Parquet tables with MT_DOP=4. .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5315 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I2be3c7c9f3004e9a759224a2e5756eb6e4efa359 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4572: Run COMPUTE STATS on Parquet tables with MT DOP=4.
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4572: Run COMPUTE STATS on Parquet tables with MT_DOP=4. .. IMPALA-4572: Run COMPUTE STATS on Parquet tables with MT_DOP=4. COMPUTE STATS on Parquet tables is run with MT_DOP=4 by default. COMPUTE STATS on non-Parquet tables will run without MT_DOP. Users can always override the behavior by setting MT_DOP manually. Setting MT_DOP to 0 means a statement will be run in the conventional execution mode (without intra-node paralellism based on multiple fragment instances). Users can set a higher MT_DOP even for Parquet tables. Testing: Added a new test that checks the effective MT_DOP. Locally ran test_mt_dop.py, test_scanners.py, test_nested_types.py, test_compute_stats.py, and test_cancellation.py. Change-Id: I2be3c7c9f3004e9a759224a2e5756eb6e4efa359 Reviewed-on: http://gerrit.cloudera.org:8080/5315 Reviewed-by: Alex Behm Tested-by: Internal Jenkins --- M common/thrift/ImpalaInternalService.thrift M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java M fe/src/test/java/org/apache/impala/service/FrontendTest.java M tests/query_test/test_mt_dop.py 9 files changed, 107 insertions(+), 11 deletions(-) Approvals: Internal Jenkins: Verified Alex Behm: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5315 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I2be3c7c9f3004e9a759224a2e5756eb6e4efa359 Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-4172/IMPALA-3653: Improvements to block metadata loading
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4172/IMPALA-3653: Improvements to block metadata loading .. IMPALA-4172/IMPALA-3653: Improvements to block metadata loading This patch improves the block metadata loading (locations and disk storage IDs) for partitioned and un-partitioned tables in the Catalog server. Without this patch: -- We loop through each and every file in the table/partition directories and call getFileBlockLocations() on it to obtain the block metadata. This results in large number of RPC calls to the Namenode, especially for tables with large no. of files/partitions. With this patch: --- We move the block metadata querying to use listStatus() call which accepts a directory as input and fetches the 'BlockLocation' objects for every file recursively in that directory. This improves the metadata loading in the following ways. - For non-partitioned tables, we query all the BlockLocations in a single RPC call in the base table directory and load the corresponding disk IDs. - For partitioned tables, we query the BlockLocations for all the partitions residing under the base table directories in a single RPC and then load every partition with non-default partition directory separately. - REFRESH on a table reloads the block metadata from scratch for every data file every time. So it can be used as a replacement for invalidate in situations like HDFS block rebalancing which needs block metadata update. Also, this patch does away with VolumeIds returned by the HDFS NN and uses the new StorageIDs returned by the BlockLocation class. These StorageIDs are UUID strings and hence are mapped to a per-node 0-based index as expected by the backend. In the upcoming versions of Hadoop APIs, getFileBlockStorageLocations() is deprecated and instead the listStatus() returns BlockLocations with storage IDs embedded. This patch makes use of this improvement to reduce an additional RPC to the NN to fetch the storage locations. Change-Id: Ie127658172e6e70dae441374530674a4ac9d5d26 Reviewed-on: http://gerrit.cloudera.org:8080/5148 Reviewed-by: Bharath Vissapragada Tested-by: Internal Jenkins --- M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java A fe/src/main/java/org/apache/impala/catalog/DiskIdMapper.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java 6 files changed, 367 insertions(+), 341 deletions(-) Approvals: Bharath Vissapragada: Looks good to me, approved Internal Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/5148 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ie127658172e6e70dae441374530674a4ac9d5d26 Gerrit-PatchSet: 20 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] IMPALA-4172/IMPALA-3653: Improvements to block metadata loading
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4172/IMPALA-3653: Improvements to block metadata loading .. Patch Set 19: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5148 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ie127658172e6e70dae441374530674a4ac9d5d26 Gerrit-PatchSet: 19 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Sailesh Mukil Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4580: Fix crash with FETCH FIRST when #rows < result cache size
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4580: Fix crash with FETCH_FIRST when #rows < result cache size .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5335 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I454cd8a6cf438bdd0c49fd27c2725d8f6c43bb1d Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Henry Robinson Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Internal Jenkins Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4580: Fix crash with FETCH FIRST when #rows < result cache size
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4580: Fix crash with FETCH_FIRST when #rows < result cache size .. IMPALA-4580: Fix crash with FETCH_FIRST when #rows < result cache size The following sequence can lead to a crash: 1. Client sets result cache size to N 2. Client issues query with #results < N 3. Client fetches all results, triggering eos and tearing down Coordinator::root_sink_. 4. Client restarts query with FETCH_FIRST. 5. Client reads all results again. After cache is exhausted, Coordinator::GetNext() is called to detect eos condition again. 6. GetNext() hits DCHECK(root_sink_ != nullptr). This patch makes GetNext() a no-op if called after it sets *eos, avoiding the crash.. Testing: Regression test that triggered the bug before this fix. Change-Id: I454cd8a6cf438bdd0c49fd27c2725d8f6c43bb1d Reviewed-on: http://gerrit.cloudera.org:8080/5335 Reviewed-by: Henry Robinson Reviewed-by: Dan Hecht Tested-by: Internal Jenkins --- M be/src/runtime/coordinator.cc M be/src/runtime/coordinator.h M tests/hs2/test_fetch_first.py 3 files changed, 39 insertions(+), 2 deletions(-) Approvals: Henry Robinson: Looks good to me, approved Internal Jenkins: Verified Dan Hecht: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5335 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I454cd8a6cf438bdd0c49fd27c2725d8f6c43bb1d Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Henry Robinson Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Internal Jenkins
[Impala-ASF-CR] IMPALA-4572: Run COMPUTE STATS on Parquet tables with MT DOP=4.
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4572: Run COMPUTE STATS on Parquet tables with MT_DOP=4. .. Patch Set 3: Verified-1 Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/589/ -- To view, visit http://gerrit.cloudera.org:8080/5315 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I2be3c7c9f3004e9a759224a2e5756eb6e4efa359 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4577: Adjust maximum size of row batch queue with MT DOP.
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4577: Adjust maximum size of row batch queue with MT_DOP. .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5330 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic2aa260f9265ec21173fb703c41934964ece6485 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4577: Adjust maximum size of row batch queue with MT DOP.
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4577: Adjust maximum size of row batch queue with MT_DOP. .. IMPALA-4577: Adjust maximum size of row batch queue with MT_DOP. When MT_DOP is set, non-Parquet scans are run with the old scan node with a fixed NUM_SCANNER_THREADS=1. This patch adjust the maximum size of the row batch queue based on MT_DOP for each such scan instance to avoid a significant increase in the memory consumption of such scans. The max queued batches per scan-node instance is at least 2 to always allow for some parallelism between the producer/consumer. Decreases the maximum allowed value for MT_DOP from 128 to 64. Change-Id: Ic2aa260f9265ec21173fb703c41934964ece6485 Reviewed-on: http://gerrit.cloudera.org:8080/5330 Reviewed-by: Alex Behm Tested-by: Internal Jenkins --- M be/src/exec/hdfs-scan-node.cc M be/src/exec/hdfs-scan-node.h M be/src/service/query-options.cc 3 files changed, 24 insertions(+), 11 deletions(-) Approvals: Internal Jenkins: Verified Alex Behm: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5330 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ic2aa260f9265ec21173fb703c41934964ece6485 Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-4363: Add Parquet timestamp validation
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4363: Add Parquet timestamp validation .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5343 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ieaab5d33e6f0df831d0e67e1d318e5416ffb90ac Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Taras Bobrovytsky Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Taras Bobrovytsky Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4363: Add Parquet timestamp validation
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4363: Add Parquet timestamp validation .. IMPALA-4363: Add Parquet timestamp validation Before this patch, we would simply read the INT96 Parquet timestamp representation and assume that it's valid. However, not all bit permutations represent a valid timestamp. One of the boost functions raised an exception (that we didn't catch) when passed an invalid boost date object, which resulted in a crash. This patch fixes problem by validating that the date falls into 1400.. year range as we are scanning Parquet. Change-Id: Ieaab5d33e6f0df831d0e67e1d318e5416ffb90ac Reviewed-on: http://gerrit.cloudera.org:8080/5343 Reviewed-by: Taras Bobrovytsky Tested-by: Internal Jenkins --- M be/src/exec/parquet-column-readers.cc M be/src/exec/parquet-column-readers.h M be/src/runtime/timestamp-value.h M common/thrift/generate_error_codes.py M testdata/data/README A testdata/data/out_of_range_timestamp.parquet M testdata/workloads/functional-query/queries/QueryTest/grant_revoke.test A testdata/workloads/functional-query/queries/QueryTest/out-of-range-timestamp-abort-on-error.test A testdata/workloads/functional-query/queries/QueryTest/out-of-range-timestamp-continue-on-error.test M testdata/workloads/functional-query/queries/QueryTest/parquet-resolution-by-name.test M testdata/workloads/functional-query/queries/QueryTest/parquet.test M tests/common/impala_test_suite.py M tests/query_test/test_scanners.py 13 files changed, 141 insertions(+), 36 deletions(-) Approvals: Taras Bobrovytsky: Looks good to me, approved Internal Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/5343 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ieaab5d33e6f0df831d0e67e1d318e5416ffb90ac Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Taras Bobrovytsky Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Taras Bobrovytsky Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-4363: Add Parquet timestamp validation
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4363: Add Parquet timestamp validation .. Patch Set 1: Verified-1 Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/586/ -- To view, visit http://gerrit.cloudera.org:8080/5343 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ieaab5d33e6f0df831d0e67e1d318e5416ffb90ac Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Taras Bobrovytsky Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Taras Bobrovytsky Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3875: Thrift threaded server hang in some cases
Internal Jenkins has posted comments on this change. Change subject: IMPALA-3875: Thrift threaded server hang in some cases .. Patch Set 4: Verified-1 Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/585/ -- To view, visit http://gerrit.cloudera.org:8080/5263 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I6c8f91a88f723e0e58e81bb385c5a8f190021868 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Sailesh Mukil Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Sailesh Mukil Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4357: Fix DROP TABLE to pass analysis if the table fails to load
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4357: Fix DROP TABLE to pass analysis if the table fails to load .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5144 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I6b41fc3c0e95508ab67f1d420b033b02ec75a5da Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Matthew Jacobs Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4357: Fix DROP TABLE to pass analysis if the table fails to load
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4357: Fix DROP TABLE to pass analysis if the table fails to load .. IMPALA-4357: Fix DROP TABLE to pass analysis if the table fails to load If a table fails to load, eg. because it was deleted externally from Kudu, we should still allow 'DROP TABLE' to pass analysis. Otherwise, you may be unable to drop tables that are in a bad state. Testing: - Updates existing Kudu tests to reflect the new behavior, and fixes a couple of problems with those tests that were causing them to pass spuriously (as well as fixing the same problem with another test in the file while I'm here). Change-Id: I6b41fc3c0e95508ab67f1d420b033b02ec75a5da Reviewed-on: http://gerrit.cloudera.org:8080/5144 Reviewed-by: Alex Behm Tested-by: Internal Jenkins --- M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/DropTableOrViewStmt.java M fe/src/main/java/org/apache/impala/analysis/PartitionSpecBase.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java M fe/src/test/java/org/apache/impala/analysis/AuditingTest.java M tests/query_test/test_kudu.py 7 files changed, 49 insertions(+), 19 deletions(-) Approvals: Internal Jenkins: Verified Alex Behm: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5144 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I6b41fc3c0e95508ab67f1d420b033b02ec75a5da Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Matthew Jacobs Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-4567: Fix test kudu alter table exhaustive failures
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4567: Fix test_kudu_alter_table exhaustive failures .. IMPALA-4567: Fix test_kudu_alter_table exhaustive failures The issue is that we set the Kudu table name explicitly via tblproperty so it doesn't have the unique db name in the underlying Kudu name. Meanwhile, the tests are run concurrently in exhaustive so this test may end up running the multiple times (w/ different parameters, e.g. disable_codegen) concurrently. This test needs to be run serially. Change-Id: Ibca64d5567c24240606e454b052d130fcd0c3968 Reviewed-on: http://gerrit.cloudera.org:8080/5312 Reviewed-by: David Knupp Reviewed-by: Dimitris Tsirogiannis Tested-by: Internal Jenkins --- M tests/query_test/test_kudu.py 1 file changed, 1 insertion(+), 0 deletions(-) Approvals: David Knupp: Looks good to me, but someone else must approve Internal Jenkins: Verified Dimitris Tsirogiannis: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5312 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ibca64d5567c24240606e454b052d130fcd0c3968 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Matthew Jacobs Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Matthew Jacobs Gerrit-Reviewer: Michael Brown
[Impala-ASF-CR] IMPALA-4567: Fix test kudu alter table exhaustive failures
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4567: Fix test_kudu_alter_table exhaustive failures .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5312 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ibca64d5567c24240606e454b052d130fcd0c3968 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Matthew Jacobs Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Matthew Jacobs Gerrit-Reviewer: Michael Brown Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4527: Columns in Kudu tables created from Impala default to "NULL"
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4527: Columns in Kudu tables created from Impala default to "NULL" .. IMPALA-4527: Columns in Kudu tables created from Impala default to "NULL" This commit reverts the behavior introduced by IMPALA-3719 which used the Kudu default behavior for column nullability if none was specified in the CREATE TABLE statement. With this commit, non-key columns of Kudu tables that are created from Impala are by default nullable unless specified otherwise. Change-Id: I950d9a9c64e3851e11a641573617790b340ece94 Reviewed-on: http://gerrit.cloudera.org:8080/5259 Reviewed-by: Alex Behm Tested-by: Internal Jenkins --- M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java M testdata/workloads/functional-query/queries/QueryTest/kudu_describe.test M testdata/workloads/functional-query/queries/QueryTest/kudu_stats.test 3 files changed, 27 insertions(+), 20 deletions(-) Approvals: Internal Jenkins: Verified Alex Behm: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5259 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I950d9a9c64e3851e11a641573617790b340ece94 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Dimitris Tsirogiannis Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-4527: Columns in Kudu tables created from Impala default to "NULL"
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4527: Columns in Kudu tables created from Impala default to "NULL" .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5259 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I950d9a9c64e3851e11a641573617790b340ece94 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Dimitris Tsirogiannis Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3875: Thrift threaded server hang in some cases
Internal Jenkins has posted comments on this change. Change subject: IMPALA-3875: Thrift threaded server hang in some cases .. Patch Set 2: Verified-1 Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/580/ -- To view, visit http://gerrit.cloudera.org:8080/5263 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I6c8f91a88f723e0e58e81bb385c5a8f190021868 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Sailesh Mukil Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Sailesh Mukil Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4504: fix races in PlanFragmentExecutor regarding status reporting
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4504: fix races in PlanFragmentExecutor regarding status reporting .. IMPALA-4504: fix races in PlanFragmentExecutor regarding status reporting The PlanFragmentExecutor has some state shared between the main execution thread and the periodic reporting thread that isn't synchronized properly. IMPALA-4504 describes one such problem, and that bug was introduced in an attempt to fix another similar race. Let's just simplify all of this and remove this shared state. Instead, the profile thread will always be responsible for sending periodic '!done' messages, and the main execution thread will always be responsible for sending the final 'done' message (after joining the periodic thread). This will allow for even more simplification, in particular the interaction between FragementExecState and PlanFragementExecutor, but I'm not doing that now as to avoid more conflicts with the MT work. Change-Id: I052b7b4fabb341ad27ad294cd5b0a53728d87d0e Reviewed-on: http://gerrit.cloudera.org:8080/5250 Reviewed-by: Dan Hecht Tested-by: Internal Jenkins --- M be/src/runtime/plan-fragment-executor.cc M be/src/runtime/plan-fragment-executor.h M be/src/service/fragment-exec-state.cc M be/src/service/fragment-exec-state.h 4 files changed, 67 insertions(+), 121 deletions(-) Approvals: Internal Jenkins: Verified Dan Hecht: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5250 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I052b7b4fabb341ad27ad294cd5b0a53728d87d0e Gerrit-PatchSet: 13 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Dan Hecht Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] IMPALA-4504: fix races in PlanFragmentExecutor regarding status reporting
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4504: fix races in PlanFragmentExecutor regarding status reporting .. Patch Set 12: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5250 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I052b7b4fabb341ad27ad294cd5b0a53728d87d0e Gerrit-PatchSet: 12 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Dan Hecht Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Sailesh Mukil Gerrit-HasComments: No
[Impala-ASF-CR] Fix E2E test infrastructure to handle missing exceptions correctly
Internal Jenkins has submitted this change and it was merged. Change subject: Fix E2E test infrastructure to handle missing exceptions correctly .. Fix E2E test infrastructure to handle missing exceptions correctly This change fixes a bug in the E2E infrastructure that handles the case when an expected exception wasn't thrown. The code was expecting that test_section['CATCH'] to be a string but in reality it's a list of strings. It also clarifies the error message about the missing exception. This change also enforces that the CATCH subsection in tests cannot be empty. Change-Id: I7d83c5db59e8a239e4e70694a1e625af6f21419c Reviewed-on: http://gerrit.cloudera.org:8080/5260 Reviewed-by: Michael Ho Tested-by: Internal Jenkins --- M tests/common/impala_test_suite.py M tests/util/test_file_parser.py 2 files changed, 7 insertions(+), 1 deletion(-) Approvals: Michael Ho: Looks good to me, approved Internal Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/5260 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I7d83c5db59e8a239e4e70694a1e625af6f21419c Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Michael Ho Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] Fix E2E test infrastructure to handle missing exceptions correctly
Internal Jenkins has posted comments on this change. Change subject: Fix E2E test infrastructure to handle missing exceptions correctly .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5260 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7d83c5db59e8a239e4e70694a1e625af6f21419c Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Michael Ho Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] Bump Kudu python version to 1.1
Internal Jenkins has submitted this change and it was merged. Change subject: Bump Kudu python version to 1.1 .. Bump Kudu python version to 1.1 Change-Id: I5834b3aa4eeae363eae938f61e473c52a0fe5596 Reviewed-on: http://gerrit.cloudera.org:8080/5307 Reviewed-by: Tim Armstrong Tested-by: Internal Jenkins --- M infra/python/deps/download_requirements M infra/python/deps/requirements.txt 2 files changed, 2 insertions(+), 2 deletions(-) Approvals: Internal Jenkins: Verified Tim Armstrong: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5307 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I5834b3aa4eeae363eae938f61e473c52a0fe5596 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Matthew Jacobs Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] Bump Kudu python version to 1.1
Internal Jenkins has posted comments on this change. Change subject: Bump Kudu python version to 1.1 .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5307 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I5834b3aa4eeae363eae938f61e473c52a0fe5596 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Matthew Jacobs Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4527: Columns in Kudu tables created from Impala default to "NULL"
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4527: Columns in Kudu tables created from Impala default to "NULL" .. Patch Set 3: Verified-1 Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/581/ -- To view, visit http://gerrit.cloudera.org:8080/5259 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I950d9a9c64e3851e11a641573617790b340ece94 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Dimitris Tsirogiannis Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4562: Fix for crash on kerberized clusters w/o Kudu support
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4562: Fix for crash on kerberized clusters w/o Kudu support .. IMPALA-4562: Fix for crash on kerberized clusters w/o Kudu support commit de88f0c4af3a07ae6bd6b8c94edcb8748468f522 for "IMPALA-4497: Fix Kudu client crash w/ SASL initialization" causes a crash on secure clusters where kudu is not supported. kudu::client::DisableSaslInitialization() from libkudu_client.so.0 impala::InitAuth(std::string const&) () impala::InitCommonRuntime() () ImpaladMain(int, char**) () main () This ensures Impala does not call the Kudu client to handle SASL init on systems where libkudu_client.so is a stub. Change-Id: Ib517d17ab12e215fe87f35bc5d03cdda736ff672 Reviewed-on: http://gerrit.cloudera.org:8080/5295 Reviewed-by: Henry Robinson Tested-by: Internal Jenkins --- M be/src/rpc/authentication.cc 1 file changed, 4 insertions(+), 2 deletions(-) Approvals: Henry Robinson: Looks good to me, approved Internal Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/5295 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ib517d17ab12e215fe87f35bc5d03cdda736ff672 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Matthew Jacobs Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Internal Jenkins
[Impala-ASF-CR] IMPALA-4562: Fix for crash on kerberized clusters w/o Kudu support
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4562: Fix for crash on kerberized clusters w/o Kudu support .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5295 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib517d17ab12e215fe87f35bc5d03cdda736ff672 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Matthew Jacobs Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Internal Jenkins Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4557: Fix flakiness with FLAGS stress free pool alloc
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4557: Fix flakiness with FLAGS_stress_free_pool_alloc .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5281 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I373035d209a0f3477650336ee6da458fa289dbe0 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Michael Ho Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4557: Fix flakiness with FLAGS stress free pool alloc
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4557: Fix flakiness with FLAGS_stress_free_pool_alloc .. IMPALA-4557: Fix flakiness with FLAGS_stress_free_pool_alloc FLAGS_stress_free_pool_alloc is a gflag for simulating malloc failure in debug builds. If set, FreePool::Allocate() and FreePool::Reallocate() will return NULL on every FLAGS_stress_free_pool_alloc allocations. The problem is that the counter for tracking number of allocations is updated racily by multiple threads, leading to non-determinism and flaky tests. This change fixes the problem by making the counter an AtomicInt32. Change-Id: I373035d209a0f3477650336ee6da458fa289dbe0 Reviewed-on: http://gerrit.cloudera.org:8080/5281 Reviewed-by: Jim Apple Reviewed-by: Dan Hecht Tested-by: Internal Jenkins --- M be/src/common/atomic.h M be/src/runtime/CMakeLists.txt A be/src/runtime/free-pool.cc M be/src/runtime/free-pool.h 4 files changed, 41 insertions(+), 6 deletions(-) Approvals: Jim Apple: Looks good to me, but someone else must approve Internal Jenkins: Verified Dan Hecht: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5281 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I373035d209a0f3477650336ee6da458fa289dbe0 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Michael Ho Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] Bracketing Java logging output with log level checks part 2.
Internal Jenkins has posted comments on this change. Change subject: Bracketing Java logging output with log level checks part 2. .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5297 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib94b3a20d439d854f579d9086755eb19699fcb68 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker Gerrit-HasComments: No
[Impala-ASF-CR] Bracketing Java logging output with log level checks part 2.
Internal Jenkins has submitted this change and it was merged. Change subject: Bracketing Java logging output with log level checks part 2. .. Bracketing Java logging output with log level checks part 2. This reduces creation of intermediate objects and improves performance. Change-Id: Ib94b3a20d439d854f579d9086755eb19699fcb68 Reviewed-on: http://gerrit.cloudera.org:8080/5297 Reviewed-by: Marcel Kornacker Tested-by: Internal Jenkins --- M fe/src/main/java/org/apache/impala/analysis/AggregateInfo.java M fe/src/main/java/org/apache/impala/analysis/AggregateInfoBase.java M fe/src/main/java/org/apache/impala/analysis/AlterTableStmt.java M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/AnalyticInfo.java M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateDropRoleStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateOrAlterViewStmtBase.java M fe/src/main/java/org/apache/impala/analysis/CreateUdfStmt.java M fe/src/main/java/org/apache/impala/analysis/DropFunctionStmt.java M fe/src/main/java/org/apache/impala/analysis/ExprSubstitutionMap.java M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java M fe/src/main/java/org/apache/impala/analysis/LiteralExpr.java M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java M fe/src/main/java/org/apache/impala/analysis/ModifyStmt.java M fe/src/main/java/org/apache/impala/analysis/PartitionSpec.java M fe/src/main/java/org/apache/impala/analysis/ResetMetadataStmt.java M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowFilesStmt.java M fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java M fe/src/main/java/org/apache/impala/analysis/TruncateStmt.java M fe/src/main/java/org/apache/impala/analysis/UnionStmt.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/AuthorizeableDb.java M fe/src/main/java/org/apache/impala/authorization/AuthorizeableUri.java M fe/src/main/java/org/apache/impala/authorization/ImpalaInternalAdminUser.java M fe/src/main/java/org/apache/impala/authorization/Privilege.java M fe/src/main/java/org/apache/impala/authorization/PrivilegeRequest.java M fe/src/main/java/org/apache/impala/authorization/User.java 30 files changed, 148 insertions(+), 135 deletions(-) Approvals: Marcel Kornacker: Looks good to me, approved Internal Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/5297 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ib94b3a20d439d854f579d9086755eb19699fcb68 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker
[Impala-ASF-CR] IMPALA-4458: Fix resource cleanup of cancelled mt scan nodes.
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4458: Fix resource cleanup of cancelled mt scan nodes. .. IMPALA-4458: Fix resource cleanup of cancelled mt scan nodes. The bug was that HdfsScanNodeMt::Close() did not properly clean up all in-flight resources when called through the query cancellation path. The main change is to clean up all resources when passing a NULL batch into HdfsparquetScanner::Close() which also needed similar changes in the scanner context. Testing: Ran test_cancellation.py, test_scanners.py and test_nested_types.py with MT_DOP=3. Added a test query with a limit that was failing before. A regular private hdfs/core test run succeeded. Change-Id: Ib32f87b3289ed9e8fc2db0885675845e11207438 Reviewed-on: http://gerrit.cloudera.org:8080/5274 Reviewed-by: Alex Behm Tested-by: Internal Jenkins --- M be/src/exec/hdfs-parquet-scanner.cc M be/src/exec/hdfs-scan-node-mt.cc M be/src/exec/parquet-column-readers.h M be/src/exec/scanner-context.cc M be/src/exec/scanner-context.h M testdata/workloads/functional-query/queries/QueryTest/mt-dop-parquet.test 6 files changed, 53 insertions(+), 35 deletions(-) Approvals: Internal Jenkins: Verified Alex Behm: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5274 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ib32f87b3289ed9e8fc2db0885675845e11207438 Gerrit-PatchSet: 7 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-4458: Fix resource cleanup of cancelled mt scan nodes.
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4458: Fix resource cleanup of cancelled mt scan nodes. .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5274 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib32f87b3289ed9e8fc2db0885675845e11207438 Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] Bracketing Java logging output with log level checks part 2.
Internal Jenkins has posted comments on this change. Change subject: Bracketing Java logging output with log level checks part 2. .. Patch Set 1: Verified-1 Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/572/ -- To view, visit http://gerrit.cloudera.org:8080/5297 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib94b3a20d439d854f579d9086755eb19699fcb68 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4562: Fix for crash on kerberized clusters w/o Kudu support
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4562: Fix for crash on kerberized clusters w/o Kudu support .. Patch Set 1: Verified-1 Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/571/ -- To view, visit http://gerrit.cloudera.org:8080/5295 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib517d17ab12e215fe87f35bc5d03cdda736ff672 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Matthew Jacobs Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Internal Jenkins Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4458: Fix resource cleanup of cancelled mt scan nodes.
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4458: Fix resource cleanup of cancelled mt scan nodes. .. Patch Set 5: Verified-1 Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/569/ -- To view, visit http://gerrit.cloudera.org:8080/5274 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib32f87b3289ed9e8fc2db0885675845e11207438 Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] Bracketing Java logging output with log level checks.
Internal Jenkins has submitted this change and it was merged. Change subject: Bracketing Java logging output with log level checks. .. Bracketing Java logging output with log level checks. This reduces creation of intermediate objects and improves performance. Change-Id: Ie0f5123dbf2caf3b03183c76820599920baa9785 Reviewed-on: http://gerrit.cloudera.org:8080/5284 Reviewed-by: Marcel Kornacker Tested-by: Internal Jenkins --- M fe/src/main/java/org/apache/impala/catalog/AuthorizationPolicy.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/Column.java M fe/src/main/java/org/apache/impala/catalog/DataSourceTable.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java M fe/src/main/java/org/apache/impala/catalog/MetaStoreClientPool.java M fe/src/main/java/org/apache/impala/catalog/PartitionStatsUtil.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/catalog/TableLoader.java M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java M fe/src/main/java/org/apache/impala/extdatasource/ExternalDataSourceExecutor.java M fe/src/main/java/org/apache/impala/planner/AggregationNode.java M fe/src/main/java/org/apache/impala/planner/AnalyticEvalNode.java M fe/src/main/java/org/apache/impala/planner/AnalyticPlanner.java M fe/src/main/java/org/apache/impala/planner/DataSourceScanNode.java M fe/src/main/java/org/apache/impala/planner/DistributedPlanner.java M fe/src/main/java/org/apache/impala/planner/HBaseScanNode.java M fe/src/main/java/org/apache/impala/planner/HdfsPartitionFilter.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/JoinNode.java M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java M fe/src/main/java/org/apache/impala/planner/ParallelPlanner.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java M fe/src/main/java/org/apache/impala/planner/SelectNode.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/main/java/org/apache/impala/planner/SortNode.java M fe/src/main/java/org/apache/impala/planner/UnionNode.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/MetadataOp.java M fe/src/main/java/org/apache/impala/util/FsPermissionChecker.java M fe/src/main/java/org/apache/impala/util/HdfsCachingUtil.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java M fe/src/main/java/org/apache/impala/util/RequestPoolService.java M fe/src/main/java/org/apache/impala/util/SentryPolicyService.java 39 files changed, 388 insertions(+), 193 deletions(-) Approvals: Marcel Kornacker: Looks good to me, approved Internal Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/5284 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ie0f5123dbf2caf3b03183c76820599920baa9785 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Marcel Kornacker Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker
[Impala-ASF-CR] Bracketing Java logging output with log level checks.
Internal Jenkins has posted comments on this change. Change subject: Bracketing Java logging output with log level checks. .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5284 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ie0f5123dbf2caf3b03183c76820599920baa9785 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Marcel Kornacker Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4482: Use ALTER TABLE / RECOVER PARTITIONS when loading tpcds.store sales
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4482: Use ALTER TABLE / RECOVER PARTITIONS when loading tpcds.store_sales .. Patch Set 4: Verified-1 Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/570/ -- To view, visit http://gerrit.cloudera.org:8080/5177 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Iaae97d1d44201aeeacacdd39adbae35753512950 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: David Knupp Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Harrison Sheinblatt Gerrit-Reviewer: Internal Jenkins Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4554: fix projection of nested collections with mt dop > 0
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4554: fix projection of nested collections with mt_dop > 0 .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5270 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I42e72eae8dfa78f7d53708eb8f2f0da8b780692d Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4550: Fix CastExpr analysis for substituted slots
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4550: Fix CastExpr analysis for substituted slots .. IMPALA-4550: Fix CastExpr analysis for substituted slots During slot substitution, the type of the child of a CastExpr can change. If the previous child type matched the CastExpr, then the cast was flagged as noOp_. During substitution and subsequent re-analysis the noOp_ flag was not revisited so that no cast was performed, even after it had become necessary. The fix is to always set noOp_ to the correct value in CastExpr.analyze(). Change-Id: I7f29cdc359558fad6df455b8eec0e0eaed00e996 Reviewed-on: http://gerrit.cloudera.org:8080/5267 Reviewed-by: Alex Behm Tested-by: Internal Jenkins --- M fe/src/main/java/org/apache/impala/analysis/CastExpr.java M testdata/workloads/functional-query/queries/QueryTest/exprs.test 2 files changed, 16 insertions(+), 4 deletions(-) Approvals: Internal Jenkins: Verified Alex Behm: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5267 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I7f29cdc359558fad6df455b8eec0e0eaed00e996 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars Volker Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Lars Volker
[Impala-ASF-CR] IMPALA-4550: Fix CastExpr analysis for substituted slots
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4550: Fix CastExpr analysis for substituted slots .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5267 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7f29cdc359558fad6df455b8eec0e0eaed00e996 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars Volker Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Lars Volker Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-2890: Support ALTER TABLE statements for Kudu tables
Internal Jenkins has submitted this change and it was merged.
Change subject: IMPALA-2890: Support ALTER TABLE statements for Kudu tables
..
IMPALA-2890: Support ALTER TABLE statements for Kudu tables
With this commit, we add support for additional ALTER TABLE statements
against Kudu tables. The new supported ALTER TABLE operations for Kudu are:
- ADD/DROP range partitions. Syntax:
ALTER TABLE ADD [IF NOT EXISTS] RANGE
ALTER TABLE DROP [IF EXISTS] RANGE
- ADD/DROP/RENAME column. Syntax:
ALTER TABLE ADD COLUMNS (col_spec, [col_spec, ...])
ALTER TABLE DROP COLUMN
ALTER TABLE CHANGE COLUMN
- Rename Kudu table using the 'kudu.table_name' table property. Example:
ALTER TABLE SET TBLPROPERTY ('kudu.tbl_name'=''),
will change the underlying Kudu table name to .
- Renaming the HMS/Catalog table entry of a Kudu table is supported using the
existing ALTER TABLE RENAME TO syntax.
Not supported:
- ALTER TABLE REPLACE COLUMNS
Change-Id: I04bc87e04e05da5cc03edec79d13cedfd2012896
Reviewed-on: http://gerrit.cloudera.org:8080/5136
Reviewed-by: Dimitris Tsirogiannis
Tested-by: Internal Jenkins
---
M common/thrift/JniCatalog.thrift
M fe/src/main/cup/sql-parser.cup
A
fe/src/main/java/org/apache/impala/analysis/AlterTableAddDropRangePartitionStmt.java
M fe/src/main/java/org/apache/impala/analysis/AlterTableAddPartitionStmt.java
M fe/src/main/java/org/apache/impala/analysis/AlterTableAddReplaceColsStmt.java
M fe/src/main/java/org/apache/impala/analysis/AlterTableChangeColStmt.java
M fe/src/main/java/org/apache/impala/analysis/AlterTableDropPartitionStmt.java
M fe/src/main/java/org/apache/impala/analysis/AlterTableSetFileFormatStmt.java
M fe/src/main/java/org/apache/impala/analysis/AlterTableSetLocationStmt.java
M fe/src/main/java/org/apache/impala/analysis/AlterTableStmt.java
M fe/src/main/java/org/apache/impala/analysis/ColumnDef.java
M fe/src/main/java/org/apache/impala/analysis/DistributeParam.java
M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java
M fe/src/main/java/org/apache/impala/catalog/KuduTable.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test
M tests/query_test/test_kudu.py
20 files changed, 964 insertions(+), 58 deletions(-)
Approvals:
Internal Jenkins: Verified
Dimitris Tsirogiannis: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/5136
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I04bc87e04e05da5cc03edec79d13cedfd2012896
Gerrit-PatchSet: 11
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Dimitris Tsirogiannis
Gerrit-Reviewer: Alex Behm
Gerrit-Reviewer: Dimitris Tsirogiannis
Gerrit-Reviewer: Internal Jenkins
[Impala-ASF-CR] IMPALA-2890: Support ALTER TABLE statements for Kudu tables
Internal Jenkins has posted comments on this change. Change subject: IMPALA-2890: Support ALTER TABLE statements for Kudu tables .. Patch Set 10: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5136 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I04bc87e04e05da5cc03edec79d13cedfd2012896 Gerrit-PatchSet: 10 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Dimitris Tsirogiannis Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4000: Restricted Sentry authorization for Kudu Tables
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4000: Restricted Sentry authorization for Kudu Tables .. Patch Set 7: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5047 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I183f08ad8ce80deee011a6b90ad67b9cefc0452c Gerrit-PatchSet: 7 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Taras Bobrovytsky Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Taras Bobrovytsky Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4000: Restricted Sentry authorization for Kudu Tables
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4000: Restricted Sentry authorization for Kudu Tables .. IMPALA-4000: Restricted Sentry authorization for Kudu Tables At this time, there is no comprehensive way of enforcing a Sentry authorization policy against tables stored in Kudu. The following behavior was implemented in this patch: - Only the ALL privilege level can be granted to Kudu tables. Finer-grained levels such as only SELECT or only INSERT are not supported. - Column level permissions on Kudu tables are not supported. - Only users with ALL privileges on SERVER may create external Kudu tables. Change-Id: I183f08ad8ce80deee011a6b90ad67b9cefc0452c Reviewed-on: http://gerrit.cloudera.org:8080/5047 Reviewed-by: Taras Bobrovytsky Tested-by: Internal Jenkins --- M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java M fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java M fe/src/main/java/org/apache/impala/authorization/PrivilegeRequestBuilder.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationTest.java 6 files changed, 58 insertions(+), 2 deletions(-) Approvals: Taras Bobrovytsky: Looks good to me, approved Internal Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/5047 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I183f08ad8ce80deee011a6b90ad67b9cefc0452c Gerrit-PatchSet: 8 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Taras Bobrovytsky Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Taras Bobrovytsky
[Impala-ASF-CR] IMPALA-4542: Fix use-after-free in some BE tests
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4542: Fix use-after-free in some BE tests .. IMPALA-4542: Fix use-after-free in some BE tests Change RuntimeState::ReleaseResources() to use cached ExecEnv pointer, rather than singleton. Change-Id: I5302d665756183289edf227588da1c094e197584 Testing: Ran affected BE tests under ASAN. Reviewed-on: http://gerrit.cloudera.org:8080/5243 Reviewed-by: Tim Armstrong Tested-by: Internal Jenkins --- M be/src/runtime/runtime-state.cc 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Internal Jenkins: Verified Tim Armstrong: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5243 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I5302d665756183289edf227588da1c094e197584 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Henry Robinson Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-4542: Fix use-after-free in some BE tests
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4542: Fix use-after-free in some BE tests .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5243 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I5302d665756183289edf227588da1c094e197584 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Henry Robinson Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4532: Fix use-after-free in ProcessBuildInputAsync()
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4532: Fix use-after-free in ProcessBuildInputAsync() .. IMPALA-4532: Fix use-after-free in ProcessBuildInputAsync() Once the Promise (of Status) is set in ProcessBuildInputAsync(), the main thread in ProcessBuildInputAndOpenProbe() may proceed to finish up the query and free RuntimeState. So, it's unsafe to access the RuntimeState afterwards. Commit bb1c633 broke that assumption (which is arguably fragile). This change fixes the problem by adding a scope for the that counter to avoid the use-after-free problem. Change-Id: I6bfd094e2e9500f1b7843486f3f745cb921764d4 Reviewed-on: http://gerrit.cloudera.org:8080/5246 Reviewed-by: Dan Hecht Reviewed-by: Tim Armstrong Tested-by: Internal Jenkins --- M be/src/exec/blocking-join-node.cc 1 file changed, 15 insertions(+), 11 deletions(-) Approvals: Internal Jenkins: Verified Tim Armstrong: Looks good to me, but someone else must approve Dan Hecht: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5246 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I6bfd094e2e9500f1b7843486f3f745cb921764d4 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Michael Ho Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-4532: Fix use-after-free in ProcessBuildInputAsync()
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4532: Fix use-after-free in ProcessBuildInputAsync() .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5246 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I6bfd094e2e9500f1b7843486f3f745cb921764d4 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Michael Ho Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4542: Fix use-after-free in some BE tests
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4542: Fix use-after-free in some BE tests .. Patch Set 1: Verified-1 Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/562/ -- To view, visit http://gerrit.cloudera.org:8080/5243 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I5302d665756183289edf227588da1c094e197584 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Henry Robinson Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4541: fix test dimensions for test codegen mem limit
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4541: fix test dimensions for test_codegen_mem_limit .. IMPALA-4541: fix test dimensions for test_codegen_mem_limit The test should only be run with codegen enabled. Change-Id: Iac460d2a1b69de638c557d7c8aa318a73ad0507b Reviewed-on: http://gerrit.cloudera.org:8080/5221 Reviewed-by: Alex Behm Tested-by: Internal Jenkins --- M tests/query_test/test_query_mem_limit.py 1 file changed, 2 insertions(+), 1 deletion(-) Approvals: Internal Jenkins: Verified Alex Behm: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5221 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Iac460d2a1b69de638c557d7c8aa318a73ad0507b Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins
[Impala-ASF-CR] IMPALA-4541: fix test dimensions for test codegen mem limit
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4541: fix test dimensions for test_codegen_mem_limit .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5221 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Iac460d2a1b69de638c557d7c8aa318a73ad0507b Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Michael Ho Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4540: Function call in DCHECK crashes scheduler
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4540: Function call in DCHECK crashes scheduler .. IMPALA-4540: Function call in DCHECK crashes scheduler The fix for IMPALA-4494 wrapped the call to backend_config_.LookUpBackendIp in simple-scheduler.cc:922 in a DCHECK, effectively disabling it in release builds. This leads to a crash when backend_ip is empty. This change makes sure that the function is always called. Change-Id: Iefbf719e86a59c09c45a5418f6ec0aa5cc4d8c44 Reviewed-on: http://gerrit.cloudera.org:8080/5218 Reviewed-by: Michael Ho Tested-by: Internal Jenkins --- M be/src/scheduling/simple-scheduler.cc 1 file changed, 2 insertions(+), 1 deletion(-) Approvals: Michael Ho: Looks good to me, approved Internal Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/5218 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Iefbf719e86a59c09c45a5418f6ec0aa5cc4d8c44 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars Volker Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho
[Impala-ASF-CR] IMPALA-4540: Function call in DCHECK crashes scheduler
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4540: Function call in DCHECK crashes scheduler .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5218 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Iefbf719e86a59c09c45a5418f6ec0aa5cc4d8c44 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars Volker Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4523: Correct max VARCHAR size to 65535 (2^16 - 1).
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4523: Correct max VARCHAR size to 65535 (2^16 - 1). .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5204 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: If76eb45b01692ed360fad8fa1722d56fa06c6c00 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Michael Ho Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4523: Correct max VARCHAR size to 65535 (2^16 - 1).
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4523: Correct max VARCHAR size to 65535 (2^16 - 1). .. IMPALA-4523: Correct max VARCHAR size to 65535 (2^16 - 1). Change-Id: If76eb45b01692ed360fad8fa1722d56fa06c6c00 Reviewed-on: http://gerrit.cloudera.org:8080/5204 Reviewed-by: Alex Behm Tested-by: Internal Jenkins --- M be/src/exprs/expr-test.cc M be/src/runtime/types.h M fe/src/main/java/org/apache/impala/catalog/ScalarType.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java 5 files changed, 14 insertions(+), 13 deletions(-) Approvals: Internal Jenkins: Verified Alex Behm: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5204 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: If76eb45b01692ed360fad8fa1722d56fa06c6c00 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Michael Ho
[Impala-ASF-CR] IMPALA-4525: fix crash when codegen mem limit exceeded
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4525: fix crash when codegen mem limit exceeded .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5211 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib38d0a44b54c47617cad1b971244f477d344d505 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4525: fix crash when codegen mem limit exceeded
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4525: fix crash when codegen mem limit exceeded .. IMPALA-4525: fix crash when codegen mem limit exceeded The error path in OptimizeLlvmModule() has not worked correctly for a long time because various places in the code assume that codegen'd function pointers will be filled in (e.g. ScalarFnCall) . Since the recent change "IMPALA-4397,IMPALA-3259: reduce codegen time and memory" it is more likely to go down this path. The cases when errors occur on this path: memory limit exceeded, internal codegen bugs, and corrupt IR UDFs, are all cases when it is not correct or safe to continue executing the query, so we should just fail the query. Testing: Add a test where codegen reliably fails with memory limit exceeded. Change-Id: Ib38d0a44b54c47617cad1b971244f477d344d505 Reviewed-on: http://gerrit.cloudera.org:8080/5211 Reviewed-by: Michael Ho Reviewed-by: Alex Behm Tested-by: Internal Jenkins --- M be/src/runtime/plan-fragment-executor.cc M be/src/runtime/plan-fragment-executor.h M be/src/service/fe-support.cc A testdata/workloads/functional-query/queries/QueryTest/codegen-mem-limit.test M tests/query_test/test_query_mem_limit.py 5 files changed, 41 insertions(+), 11 deletions(-) Approvals: Michael Ho: Looks good to me, approved Internal Jenkins: Verified Alex Behm: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5211 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ib38d0a44b54c47617cad1b971244f477d344d505 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-4529: speed up parsing of identifiers
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4529: speed up parsing of identifiers .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5210 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Iebef43a6a2f7923ca0e9c158d83f5c06f26da0cd Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4529: speed up parsing of identifiers
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4529: speed up parsing of identifiers .. IMPALA-4529: speed up parsing of identifiers Instead of using substring(), parseInt() and a try/catch, directly check the character. Change-Id: Iebef43a6a2f7923ca0e9c158d83f5c06f26da0cd Reviewed-on: http://gerrit.cloudera.org:8080/5210 Reviewed-by: Alex Behm Tested-by: Internal Jenkins --- M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java 1 file changed, 1 insertion(+), 6 deletions(-) Approvals: Internal Jenkins: Verified Alex Behm: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5210 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Iebef43a6a2f7923ca0e9c158d83f5c06f26da0cd Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins
[Impala-ASF-CR] IMPALA-2890: Support ALTER TABLE statements for Kudu tables
Internal Jenkins has posted comments on this change. Change subject: IMPALA-2890: Support ALTER TABLE statements for Kudu tables .. Patch Set 8: Verified-1 Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/552/ -- To view, visit http://gerrit.cloudera.org:8080/5136 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I04bc87e04e05da5cc03edec79d13cedfd2012896 Gerrit-PatchSet: 8 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Dimitris Tsirogiannis Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4535: Remove 'auto' from parameter list
Internal Jenkins has submitted this change and it was merged.
Change subject: IMPALA-4535: Remove 'auto' from parameter list
..
IMPALA-4535: Remove 'auto' from parameter list
Although GCC supports void f(auto arg) { }, this isn't part of C++14.
Testing: Local ASAN build passed.
Change-Id: I838aa2c1081f7ed21cc2faa209a9eaa4fd0512a4
Reviewed-on: http://gerrit.cloudera.org:8080/5214
Reviewed-by: Tim Armstrong
Tested-by: Internal Jenkins
---
M be/src/util/scope-exit-trigger.h
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
Internal Jenkins: Verified
Tim Armstrong: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/5214
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I838aa2c1081f7ed21cc2faa209a9eaa4fd0512a4
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-4535: Remove 'auto' from parameter list
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4535: Remove 'auto' from parameter list .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5214 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I838aa2c1081f7ed21cc2faa209a9eaa4fd0512a4 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Henry Robinson Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4488: HS2 GetOperationStatus() should keep session alive
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4488: HS2 GetOperationStatus() should keep session alive .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5213 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ibb3f66188209563b4b74b2ca96480f16ace0f190 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Henry Robinson Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Internal Jenkins Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4488: HS2 GetOperationStatus() should keep session alive
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4488: HS2 GetOperationStatus() should keep session alive .. IMPALA-4488: HS2 GetOperationStatus() should keep session alive GetOperationStatus() is used by *DBC drivers to check if a query is ready for its results to be fetched. However, it did not keep the associated session alive, so queries would time out if they took longer than the timeout to materialize their first rows to be fetched. * Add withSession() to GetOperationStatus() * Add a test that failed before this patch, and succeeds after. Change-Id: Ibb3f66188209563b4b74b2ca96480f16ace0f190 Reviewed-on: http://gerrit.cloudera.org:8080/5213 Reviewed-by: Alex Behm Tested-by: Internal Jenkins --- M be/src/service/impala-hs2-server.cc M tests/hs2/test_hs2.py 2 files changed, 34 insertions(+), 8 deletions(-) Approvals: Internal Jenkins: Verified Alex Behm: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5213 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ibb3f66188209563b4b74b2ca96480f16ace0f190 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Henry Robinson Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Internal Jenkins
[Impala-ASF-CR] IMPALA-4494: Fix crash in SimpleScheduler
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4494: Fix crash in SimpleScheduler .. IMPALA-4494: Fix crash in SimpleScheduler The scheduler maintains a local list of active backends, which is updated through messages from the statestore. Even the local backend enters this list by registering with the statestore and being included in a statestore update message. Thus, during restarts it can happen that a query gets scheduled with exec_at_coord set to true, while the local backend has not been registered with the scheduler. In this case the IP address lookup in the internal BackendConfig fails and an empty IP address is returned, leading to a nullptr dereference down the line. This change adds an additional check when handling updates from the statestore to make sure that the backend config always contains the local backend. It also changes scheduling when exec_at_coord is true to always use the local backend, irrespective of whether it is present in the backend config. Change-Id: I6e1196a2fa47e5954c4a190aa326c135d039a77f Reviewed-on: http://gerrit.cloudera.org:8080/5127 Reviewed-by: Bharath Vissapragada Tested-by: Internal Jenkins --- M be/src/scheduling/simple-scheduler-test.cc M be/src/scheduling/simple-scheduler.cc M be/src/scheduling/simple-scheduler.h 3 files changed, 157 insertions(+), 96 deletions(-) Approvals: Bharath Vissapragada: Looks good to me, approved Internal Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/5127 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I6e1196a2fa47e5954c4a190aa326c135d039a77f Gerrit-PatchSet: 11 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars Volker Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Marcel Kornacker Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-4494: Fix crash in SimpleScheduler
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4494: Fix crash in SimpleScheduler .. Patch Set 10: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5127 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I6e1196a2fa47e5954c4a190aa326c135d039a77f Gerrit-PatchSet: 10 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars Volker Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Marcel Kornacker Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4525: fix crash when codegen mem limit exceeded
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4525: fix crash when codegen mem limit exceeded .. Patch Set 2: Verified-1 Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/547/ -- To view, visit http://gerrit.cloudera.org:8080/5211 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib38d0a44b54c47617cad1b971244f477d344d505 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4529: speed up parsing of identifiers
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4529: speed up parsing of identifiers .. Patch Set 1: Verified-1 Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/548/ -- To view, visit http://gerrit.cloudera.org:8080/5210 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Iebef43a6a2f7923ca0e9c158d83f5c06f26da0cd Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4522: Bound Kudu client threads to avoid stress crash
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4522: Bound Kudu client threads to avoid stress crash .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5205 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I3940df776eaa5ad22e1bbb572559afcc8990bf1d Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Matthew Jacobs Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Matthew Jacobs Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4522: Bound Kudu client threads to avoid stress crash
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4522: Bound Kudu client threads to avoid stress crash .. IMPALA-4522: Bound Kudu client threads to avoid stress crash In stress testing on physical boxes (80 cores, 200gb ram) we discovered that the Kudu Java client creates a huge number of threads (2x the #cores) per Kudu client, and this was causing the impalad to crash when the JVM couldn't create more threads. This addresses the issue by setting the number of Kudu client worker threads rather than letting the Kudu client pick the default (2 * #cores). The number set here was suggested by the Kudu team as being sufficient for Impala's FE usage and this has been tested for 8+ hours on the stress cluster where the crash was previously observed quickly. In the future, Impala should probably be sharing a single Kudu client (it is multithreaded), but additional support from Kudu may be needed to ensure this usage is correct (e.g. client metadata may need invalidation after some operations). Change-Id: I3940df776eaa5ad22e1bbb572559afcc8990bf1d Reviewed-on: http://gerrit.cloudera.org:8080/5205 Reviewed-by: Alex Behm Tested-by: Internal Jenkins --- M fe/src/main/java/org/apache/impala/util/KuduUtil.java 1 file changed, 17 insertions(+), 11 deletions(-) Approvals: Internal Jenkins: Verified Alex Behm: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5205 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I3940df776eaa5ad22e1bbb572559afcc8990bf1d Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Matthew Jacobs Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Matthew Jacobs
[Impala-ASF-CR] IMPALA-4283: Ensure Kudu-specific lineage and audit behavior
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4283: Ensure Kudu-specific lineage and audit behavior .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5151 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Idc4ca1cd63bcfa4370c240a5c4a4126ed6704f4d Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Dimitris Tsirogiannis Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4283: Ensure Kudu-specific lineage and audit behavior
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4283: Ensure Kudu-specific lineage and audit behavior .. IMPALA-4283: Ensure Kudu-specific lineage and audit behavior With this commit we add support for auditing all Kudu-specific operations and we enable column lineage for INSERT and UPSERT statements on Kudu tables. No lineage output is generated for DELETE and UPDATE statements. Change-Id: Idc4ca1cd63bcfa4370c240a5c4a4126ed6704f4d Reviewed-on: http://gerrit.cloudera.org:8080/5151 Reviewed-by: Dimitris Tsirogiannis Tested-by: Internal Jenkins --- M fe/src/main/java/org/apache/impala/analysis/ColumnLineageGraph.java M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/main/java/org/apache/impala/analysis/ModifyStmt.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/PlannerContext.java M fe/src/test/java/org/apache/impala/analysis/AuditingTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java M testdata/workloads/functional-planner/queries/PlannerTest/lineage.test 8 files changed, 614 insertions(+), 39 deletions(-) Approvals: Internal Jenkins: Verified Dimitris Tsirogiannis: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5151 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Idc4ca1cd63bcfa4370c240a5c4a4126ed6704f4d Gerrit-PatchSet: 7 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Dimitris Tsirogiannis Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins
[Impala-ASF-CR] IMPALA-4523: Correct max VARCHAR size to 65535 (2^16 - 1).
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4523: Correct max VARCHAR size to 65535 (2^16 - 1). .. Patch Set 2: Verified-1 Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/544/ -- To view, visit http://gerrit.cloudera.org:8080/5204 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: If76eb45b01692ed360fad8fa1722d56fa06c6c00 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Michael Ho Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4518: CopyStringVal() doesn't copy null string
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4518: CopyStringVal() doesn't copy null string .. IMPALA-4518: CopyStringVal() doesn't copy null string Previously, CopyStringVal() mistakenly copies a null StringVal as an empty string (i.e. a non-null string with zero length). This change fixes the problem by distinguishing between these two cases in CopyStringVal() and handles them properly. Also added a test case for it. This problem only started showing up recently due to commit 51268c053ffe41dc1aa9f1b250878113d4225258 which calls CopyStringVal() in OffsetFnInit(). All other pre-existing callers of CopyStringVal() before that commit checks if 'src' is null before calling it so the problem never showed up. In that sense, this is a latent bug exposed by the aforementioned commit. Change-Id: I3a5b9349dd08556eba5cfedc8c0063cc59f5be03 Reviewed-on: http://gerrit.cloudera.org:8080/5198 Reviewed-by: Michael Ho Tested-by: Internal Jenkins --- M be/src/exprs/aggregate-functions-ir.cc M testdata/workloads/functional-query/queries/QueryTest/analytic-fns.test 2 files changed, 20 insertions(+), 6 deletions(-) Approvals: Michael Ho: Looks good to me, approved Internal Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/5198 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I3a5b9349dd08556eba5cfedc8c0063cc59f5be03 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Michael Ho Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-4518: CopyStringVal() doesn't copy null string
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4518: CopyStringVal() doesn't copy null string .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/5198 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I3a5b9349dd08556eba5cfedc8c0063cc59f5be03 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Michael Ho Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4410: Safer tear-down of RuntimeState
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4410: Safer tear-down of RuntimeState .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/4893 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ie416e4d57240142bf685385299b749c3a6792c45 Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Henry Robinson Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-2890: Support ALTER TABLE statements for Kudu tables
Internal Jenkins has posted comments on this change. Change subject: IMPALA-2890: Support ALTER TABLE statements for Kudu tables .. Patch Set 7: Verified-1 Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/540/ -- To view, visit http://gerrit.cloudera.org:8080/5136 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I04bc87e04e05da5cc03edec79d13cedfd2012896 Gerrit-PatchSet: 7 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Dimitris Tsirogiannis Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Internal Jenkins Gerrit-HasComments: No
