[Impala-ASF-CR] IMPALA-7604: part 2: fixes for AggregationNode cardinality
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14132 ) Change subject: IMPALA-7604: part 2: fixes for AggregationNode cardinality .. Patch Set 10: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14132 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieed41d60c0e0dfeca64035e919cb8c28a054a9ab Gerrit-Change-Number: 14132 Gerrit-PatchSet: 10 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 28 Aug 2019 09:00:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
Hello Attila Jeges, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13722 to look at the new patch set (#15). Change subject: IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1 .. IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1 This enhancement introduces FORMAT clause for CAST() operator that is applicable for casts between string types and timestamp types. Instead of accepting SimpleDateFormat patterns the FORMAT clause supports datetime patterns following the ISO:SQL:2016 standard. Note, the CAST() operator without the FORMAT clause still uses Impala's implementation of SimpleDateFormat handling. Similarly, the existing conversion functions such as to_timestamp(), from_timestamp() etc. remain unchanged and use SimpleDateFormat. Contrary to how these functions work the FORMAT clause must specify a string literal and cannot be used with any other kind of a string expression. Milestone 1 contains all the format tokens covered by the SQL standard. Further milestones will add more functionality on top of this list to cover functionality provided by other RDBMS systems. List of tokens implemented by this change: - , YYY, YY, Y: Year tokens - , RR: Round year tokens - MM: Month (1-12) - DD: Day (1-31) - DDD: Day of year (1-366) - HH, HH12: Hour of day (1-12) - HH24: Hour of day (0-23) - MI: Minute (0-59) - SS: Second (0-59) - S: Second of day (0-86399) - FF, FF1, ..., FF9: Fractional second - AM, PM, A.M., P.M.: Meridiem indicators - TZH: Timezone hour (-99-+99) - TZM: Timezone minute (0-99) - Separators: - . / , ' ; : space - ISO8601 date indicators (T, Z) Some notes about the matching algorithm: - The parsing algorithm uses these tokens in a case insensitive manner. - The separators are interchangeable with each other. For example a '-' separator in the format will match with a '.' character in the input. - The length of the separator sequences is handled flexibly meaning that a single separator character in the format for instance would match with a multi-separator sequence in the input. - In a string type to timestamp conversion the timezone offset tokens are parsed, expected to match with the input but they don't adjust the result as the input is already expected to be in UTC format. Usage example: SELECT CAST('01-02-2019' AS TIMESTAMP FORMAT 'MM-DD-'); SELECT CAST('2019.10.10 13:30:40.123456 +01:30' AS TIMESTAMP FORMAT '-MM-DD HH24:MI:SS.FF9 TZH:TZM'); SELECT CAST(timestamp_column as STRING FORMAT " MM HH12 YY") from some_table; Change-Id: I19d8d097a45ae6f103b6cd1b2d81aad38dfd9e23 --- M be/src/benchmarks/convert-timestamp-benchmark.cc M be/src/benchmarks/parse-timestamp-benchmark.cc M be/src/common/init.cc M be/src/exec/text-converter.inline.h M be/src/exprs/CMakeLists.txt A be/src/exprs/cast-format-expr.cc A be/src/exprs/cast-format-expr.h M be/src/exprs/cast-functions-ir.cc M be/src/exprs/date-functions-ir.cc M be/src/exprs/expr-test.cc M be/src/exprs/scalar-expr-evaluator.h M be/src/exprs/scalar-expr.cc M be/src/exprs/scalar-expr.h M be/src/exprs/timestamp-functions-ir.cc M be/src/exprs/timestamp-functions.cc M be/src/exprs/timestamp-functions.h M be/src/runtime/CMakeLists.txt M be/src/runtime/date-parse-util.cc M be/src/runtime/date-parse-util.h M be/src/runtime/date-test.cc M be/src/runtime/date-value.cc M be/src/runtime/date-value.h A be/src/runtime/datetime-iso-sql-format-parser.cc A be/src/runtime/datetime-iso-sql-format-parser.h A be/src/runtime/datetime-iso-sql-format-tokenizer.cc A be/src/runtime/datetime-iso-sql-format-tokenizer.h D be/src/runtime/datetime-parse-util.h A be/src/runtime/datetime-parser-common.cc A be/src/runtime/datetime-parser-common.h R be/src/runtime/datetime-simple-date-format-parser.cc A be/src/runtime/datetime-simple-date-format-parser.h M be/src/runtime/runtime-state.cc M be/src/runtime/timestamp-parse-util.cc M be/src/runtime/timestamp-parse-util.h M be/src/runtime/timestamp-test.cc M be/src/runtime/timestamp-value.cc M be/src/runtime/timestamp-value.h M be/src/service/impala-server.cc M be/src/service/query-options.cc M be/src/service/query-options.h M be/src/testutil/random-vector-generators.h M be/src/util/dict-test.cc M be/src/util/min-max-filter-test.cc M be/src/util/string-parser.h M common/thrift/Exprs.thrift M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/CastExpr.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java A testdata/workloads/functional-query/queries/QueryTest/cast_format_from_table.test M testdata/workloads/functional-query/queries/QueryTest/date.test A tests/query_test/test_cast_with_format.py 54 files changed, 3,446 insertions(+), 865 deletions(-) git p
[Impala-ASF-CR] IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/13722 ) Change subject: IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1 .. Patch Set 15: (4 comments) http://gerrit.cloudera.org:8080/#/c/13722/13//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13722/13//COMMIT_MSG@24 PS13, Line 24: List of tokens implemented by this change: : - , YYY, YY, Y: Year tokens : - , RR: Round year tokens : - MM: Month (1-12) : - DD: Day (1-31) : - DDD: Day of year (1-366) : - HH, HH12: Hour of day (1-12) : - HH24: Hour of day (0-23) : - MI: Minute (0-59) : - SS: Second (0-59) : - S: Second of day (0-86399) : - FF, FF1, ..., FF9: Fractional second : - AM, PM, A.M., P.M.: Meridiem indicators : - TZH: Timezone hour (-99-+99) : - TZM: Timezone minute (0-99) : - Separators: - . / , ' ; : space : - ISO8601 date indicators (T, Z) > Please specify the allowed range of values for every token. Done except for year and round year tokens as the boundaries depend on the type we cast to/from. Also omitted to add range for fractional seconds as they depend of which precision we talk about. http://gerrit.cloudera.org:8080/#/c/13722/13/be/src/runtime/datetime-iso-sql-format-parser.cc File be/src/runtime/datetime-iso-sql-format-parser.cc: http://gerrit.cloudera.org:8080/#/c/13722/13/be/src/runtime/datetime-iso-sql-format-parser.cc@142 PS13, Line 142: // Deliberately ignore the timezone offsets. : int du > Does this mean that we accept any token as a timezone hour/min? Shouldn't w Indeed. Added checks for this. http://gerrit.cloudera.org:8080/#/c/13722/13/be/src/runtime/datetime-iso-sql-format-parser.cc@196 PS13, Line 196: if (*current_pos >= end_pos || *current_tok_idx >= dt_ctx.toks.size()) { : return (*current_pos >= end_pos && *current_tok_idx >= dt_ctx.toks.size()); : } : > How about TIMEZONE_MIN? Is it possible to specify a negative TIMEZONE_MIN w In case you want to specify negative TZM then you have to do provide zero TZH like "-00:30". This is explicitly mentioned in the design doc. http://gerrit.cloudera.org:8080/#/c/13722/13/be/src/runtime/datetime-iso-sql-format-parser.cc@219 PS13, Line 219:return ParseMeridiemIndicatorFromInput(input_str, input_len); : } : : int max_tok_len = min(input_len, tok.len); : c > Should we do the same for TIMEZONE_MIN as well? See above: TZM can only be positive. -- To view, visit http://gerrit.cloudera.org:8080/13722 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I19d8d097a45ae6f103b6cd1b2d81aad38dfd9e23 Gerrit-Change-Number: 13722 Gerrit-PatchSet: 15 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 28 Aug 2019 13:06:36 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13722 ) Change subject: IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1 .. Patch Set 15: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4414/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13722 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I19d8d097a45ae6f103b6cd1b2d81aad38dfd9e23 Gerrit-Change-Number: 13722 Gerrit-PatchSet: 15 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 28 Aug 2019 13:47:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes
Hello Michael Ho, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14129 to look at the new patch set (#5). Change subject: IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes .. IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes Adds support for non-default fetch sizes when result spooling is enabled (the default is to return BATCH_SIZE rows for each fetch request). When result spooling is disabled, Impala can only return up to BATCH_SIZE rows because it only buffers a single RowBatch at a time. When result spooling is enabled, each fetch request returns exactly the number of rows requested assuming there are that many rows left in the result set. There is also an upper limit on the fetch size to prevent the resulting QueryResultSet from getting too big. Unlike the behavior when result spooling is disabled, fetches do not break on RowBatch boundaries. For example, when result spooling is disabled, if the fetch size is 10 and the batch size is 15, the second fetch will return 5 rows. However, when result spooling is enabled the second fetch will return 10 rows (assuming there is another RowBatch to read). Testing: * Ran core tests * Added new tests to test_result_spooling.py * Added new tests to buffered-tuple-stream-test to validate writing to a BufferedTupleStream before releasing row batches with 'attach_on_read' set to true. Change-Id: I8dd4b397ab6457a4f85e635f239b2c67130fcce4 --- M be/src/exec/buffered-plan-root-sink.cc M be/src/exec/buffered-plan-root-sink.h M be/src/runtime/buffered-tuple-stream-test.cc M be/src/runtime/query-state.h M tests/query_test/test_result_spooling.py 5 files changed, 312 insertions(+), 56 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/14129/5 -- To view, visit http://gerrit.cloudera.org:8080/14129 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8dd4b397ab6457a4f85e635f239b2c67130fcce4 Gerrit-Change-Number: 14129 Gerrit-PatchSet: 5 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14129 ) Change subject: IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes .. Patch Set 5: Re-based patch. -- To view, visit http://gerrit.cloudera.org:8080/14129 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8dd4b397ab6457a4f85e635f239b2c67130fcce4 Gerrit-Change-Number: 14129 Gerrit-PatchSet: 5 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 28 Aug 2019 15:50:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8815: fix ranger startup after set-classpath.sh
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14152 Change subject: IMPALA-8815: fix ranger startup after set-classpath.sh .. IMPALA-8815: fix ranger startup after set-classpath.sh Having non-existent or incompatible jars on the classpath can cause Ranger startup to fail. Update run-ranger-server.sh to clean the classpath so that it works after sourcing set-classpath.sh. Also remove a couple of legacy jars from 2013. Those jars no longer exist in Hive. Testing: In my development environment. $ . bin/set-classpath.sh $ ./testdata/bin/run-ranger-server.sh Change-Id: Ie7036f9a07e5c9b8d46bb7f459d0b9d1e7e9d0a7 --- M bin/set-classpath.sh M testdata/bin/run-ranger-server.sh 2 files changed, 7 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/52/14152/1 -- To view, visit http://gerrit.cloudera.org:8080/14152 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ie7036f9a07e5c9b8d46bb7f459d0b9d1e7e9d0a7 Gerrit-Change-Number: 14152 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8803: Coordinator should release admitted memory per-backend
Hello Michael Ho, Tim Armstrong, Bikramjeet Vig, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14104 to look at the new patch set (#6). Change subject: IMPALA-8803: Coordinator should release admitted memory per-backend .. IMPALA-8803: Coordinator should release admitted memory per-backend Changes the Coordinator to release admitted memory when each Backend completes, rather than waiting for the entire query to complete before releasing admitted memory. When the Coordinator detects that a Backend has completed (via ControlService::ReportExecStatus) it updates the state of the Backend in Coordinator::BackendResourceState. BackendResourceState tracks the state of the admitted resources for each Backend and decides when the resources for a group of Backend states should be released. BackendResourceState defines a state machine to help coordinate the state of the admitted memory for each Backend. It guarantees that by the time the query is shutdown, all Backends release their admitted memory. BackendResourceState implements three rules to control the rate at which the Coordinator releases admitted memory from the AdmissionController: * Resources are released at most once every 1 second, this prevents short lived queries from causing high load on the admission controller * Resources are released at most O(log(num_backends)) times; the BackendResourceStates can release multiple BackendStates from the AdmissionController at a time * All pending resources are released if the only remaining Backend is the Coordinator Backend; this is useful for result spooling where all Backends may complete, except for the Coordinator Backend Exposes the following hidden startup flags to help tune the heuristics above: --batched_release_decay_factor * Defaults to 2 * Controls the base value for the O(log(num_backends)) bound when batching the release of Backends. --release_backend_states_delay_ms * Defaults to 1000 milliseconds * Controls how often Backends can release their resources. Testing: * Ran core tests * Added new tests to test_result_spooling.py and test_admission_controller.py Change-Id: I88bb11e0ede7574568020e0277dd8ac8d2586dc9 --- M be/src/runtime/CMakeLists.txt A be/src/runtime/coordinator-backend-state-test.cc M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator-backend-state.h M be/src/runtime/coordinator.cc M be/src/runtime/coordinator.h M be/src/scheduling/admission-controller-test.cc M be/src/scheduling/admission-controller.cc M be/src/scheduling/admission-controller.h M tests/custom_cluster/test_admission_controller.py M tests/custom_cluster/test_auto_scaling.py M tests/custom_cluster/test_executor_groups.py A tests/custom_cluster/test_result_spooling.py M tests/util/auto_scaler.py A tests/util/web_pages_util.py 15 files changed, 1,004 insertions(+), 94 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/14104/6 -- To view, visit http://gerrit.cloudera.org:8080/14104 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I88bb11e0ede7574568020e0277dd8ac8d2586dc9 Gerrit-Change-Number: 14104 Gerrit-PatchSet: 6 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8803: Coordinator should release admitted memory per-backend
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14104 ) Change subject: IMPALA-8803: Coordinator should release admitted memory per-backend .. Patch Set 6: Newest patch is mostly cosmetic changes. Core tests pass. -- To view, visit http://gerrit.cloudera.org:8080/14104 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I88bb11e0ede7574568020e0277dd8ac8d2586dc9 Gerrit-Change-Number: 14104 Gerrit-PatchSet: 6 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 28 Aug 2019 16:43:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8902: Bump timeout in test result spooling:::test spilling
Sahil Takiar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14155 Change subject: IMPALA-8902: Bump timeout in test_result_spooling:::test_spilling .. IMPALA-8902: Bump timeout in test_result_spooling:::test_spilling De-flake test_result_spooling.py::TestResultSpooling::test_spilling. Bump up the timeout that controls how long the test waits for result spooling to start spilling to disk (detected by the presence of PeakUnpinnedBytes in the PLAN_ROOT_SINK section of the runtime profile). Testing: * Looped TestResultSpooling::test_spilling on an ASAN build for an hour without any failure. Change-Id: Iabac8d7273735079dca48a1e0ecd4f341ea690a0 --- M tests/query_test/test_result_spooling.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/55/14155/1 -- To view, visit http://gerrit.cloudera.org:8080/14155 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Iabac8d7273735079dca48a1e0ecd4f341ea690a0 Gerrit-Change-Number: 14155 Gerrit-PatchSet: 1 Gerrit-Owner: Sahil Takiar
[Impala-ASF-CR] IMPALA-8815: fix ranger startup after set-classpath.sh
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14152 ) Change subject: IMPALA-8815: fix ranger startup after set-classpath.sh .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4416/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14152 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie7036f9a07e5c9b8d46bb7f459d0b9d1e7e9d0a7 Gerrit-Change-Number: 14152 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Comment-Date: Wed, 28 Aug 2019 16:58:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14129 ) Change subject: IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4415/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14129 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8dd4b397ab6457a4f85e635f239b2c67130fcce4 Gerrit-Change-Number: 14129 Gerrit-PatchSet: 5 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 28 Aug 2019 16:59:06 + Gerrit-HasComments: No
[Impala-ASF-CR](asf-site) IMPALA-8886: Delete old releases from Apache mirroring system
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/14137 ) Change subject: IMPALA-8886: Delete old releases from Apache mirroring system .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14137 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: asf-site Gerrit-MessageType: comment Gerrit-Change-Id: Ibc45c423379e4c5459604b9bdbbd1ea0b850d568 Gerrit-Change-Number: 14137 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 28 Aug 2019 17:01:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8901: Fix # links on /catalog page.
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/14153 ) Change subject: IMPALA-8901: Fix # links on /catalog page. .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14153 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I895742dcaeaf71ecf097b0fec69cc19a3b6f86f4 Gerrit-Change-Number: 14153 Gerrit-PatchSet: 1 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Wed, 28 Aug 2019 17:04:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8579: Ignore trivial alter table/partition events.
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14145 ) Change subject: IMPALA-8579: Ignore trivial alter table/partition events. .. Patch Set 4: Code-Review+2 Thanks for all the changes. Looks good to me. -- To view, visit http://gerrit.cloudera.org:8080/14145 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01a59d5170accc014f76f14eb526d96ddcf61f76 Gerrit-Change-Number: 14145 Gerrit-PatchSet: 4 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Aug 2019 17:20:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8579: Ignore trivial alter table/partition events.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14145 ) Change subject: IMPALA-8579: Ignore trivial alter table/partition events. .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14145 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01a59d5170accc014f76f14eb526d96ddcf61f76 Gerrit-Change-Number: 14145 Gerrit-PatchSet: 5 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Aug 2019 17:22:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8579: Ignore trivial alter table/partition events.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14145 ) Change subject: IMPALA-8579: Ignore trivial alter table/partition events. .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4865/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14145 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01a59d5170accc014f76f14eb526d96ddcf61f76 Gerrit-Change-Number: 14145 Gerrit-PatchSet: 5 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Aug 2019 17:22:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8803: Coordinator should release admitted memory per-backend
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14104 ) Change subject: IMPALA-8803: Coordinator should release admitted memory per-backend .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4417/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14104 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I88bb11e0ede7574568020e0277dd8ac8d2586dc9 Gerrit-Change-Number: 14104 Gerrit-PatchSet: 6 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 28 Aug 2019 17:22:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8902: Bump timeout in test result spooling:::test spilling
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14155 ) Change subject: IMPALA-8902: Bump timeout in test_result_spooling:::test_spilling .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4418/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14155 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iabac8d7273735079dca48a1e0ecd4f341ea690a0 Gerrit-Change-Number: 14155 Gerrit-PatchSet: 1 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 28 Aug 2019 17:23:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8901: Fix # links on /catalog page.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14153 ) Change subject: IMPALA-8901: Fix # links on /catalog page. .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4866/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14153 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I895742dcaeaf71ecf097b0fec69cc19a3b6f86f4 Gerrit-Change-Number: 14153 Gerrit-PatchSet: 1 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Wed, 28 Aug 2019 17:27:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8902: Bump timeout in test result spooling:::test spilling
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14155 ) Change subject: IMPALA-8902: Bump timeout in test_result_spooling:::test_spilling .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14155 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iabac8d7273735079dca48a1e0ecd4f341ea690a0 Gerrit-Change-Number: 14155 Gerrit-PatchSet: 1 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 28 Aug 2019 17:46:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8902: Bump timeout in test result spooling:::test spilling
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14155 ) Change subject: IMPALA-8902: Bump timeout in test_result_spooling:::test_spilling .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4867/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14155 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iabac8d7273735079dca48a1e0ecd4f341ea690a0 Gerrit-Change-Number: 14155 Gerrit-PatchSet: 1 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 28 Aug 2019 17:48:30 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] CONCAT function returns NULL if any argument is NULL
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14156 Change subject: [DOCS] CONCAT function returns NULL if any argument is NULL .. [DOCS] CONCAT function returns NULL if any argument is NULL Change-Id: Ieadd249881b45944a870a616abf75ff4386615ea --- M docs/topics/impala_string_functions.xml 1 file changed, 10 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/56/14156/1 -- To view, visit http://gerrit.cloudera.org:8080/14156 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ieadd249881b45944a870a616abf75ff4386615ea Gerrit-Change-Number: 14156 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] [DOCS] CONCAT function returns NULL if any argument is NULL
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14156 ) Change subject: [DOCS] CONCAT function returns NULL if any argument is NULL .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/471/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/14156 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieadd249881b45944a870a616abf75ff4386615ea Gerrit-Change-Number: 14156 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 28 Aug 2019 17:52:55 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] CONCAT function returns NULL if any argument is NULL
Greg Rahn has posted comments on this change. ( http://gerrit.cloudera.org:8080/14156 ) Change subject: [DOCS] CONCAT function returns NULL if any argument is NULL .. Patch Set 1: Code-Review+2 LGTM - Thanks! -- To view, visit http://gerrit.cloudera.org:8080/14156 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieadd249881b45944a870a616abf75ff4386615ea Gerrit-Change-Number: 14156 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 28 Aug 2019 18:13:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8851: Do not throw authorization exception in drop if exists queries
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14121 ) Change subject: IMPALA-8851: Do not throw authorization exception in drop if exists queries .. Patch Set 8: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/14121 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iba068935e5da92d71e16e2321afdb8e7b781086a Gerrit-Change-Number: 14121 Gerrit-PatchSet: 8 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Aug 2019 18:18:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8851: Do not throw authorization exception in drop if exists queries
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14121 ) Change subject: IMPALA-8851: Do not throw authorization exception in drop if exists queries .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/14121/6/fe/src/main/java/org/apache/impala/analysis/DropTableOrViewStmt.java File fe/src/main/java/org/apache/impala/analysis/DropTableOrViewStmt.java: http://gerrit.cloudera.org:8080/#/c/14121/6/fe/src/main/java/org/apache/impala/analysis/DropTableOrViewStmt.java@117 PS6, Line 117: tableName_ > Yeah maybe we should require that callers pass in dbName_ and tableName_, s I confirmed that the current solution works and the gets the correct database from the session. Thanks for checking that the case of the table name is not a problem. -- To view, visit http://gerrit.cloudera.org:8080/14121 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iba068935e5da92d71e16e2321afdb8e7b781086a Gerrit-Change-Number: 14121 Gerrit-PatchSet: 6 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Aug 2019 18:17:25 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8851: Do not throw authorization exception in drop if exists queries
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14121 ) Change subject: IMPALA-8851: Do not throw authorization exception in drop if exists queries .. Patch Set 8: > Patch Set 8: Code-Review+1 would be good if we have Tim's +1 as well since this relates to authorization of drop command. It looks to me from my side. -- To view, visit http://gerrit.cloudera.org:8080/14121 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iba068935e5da92d71e16e2321afdb8e7b781086a Gerrit-Change-Number: 14121 Gerrit-PatchSet: 8 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Aug 2019 18:18:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8851: Do not throw authorization exception in drop if exists queries
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14121 ) Change subject: IMPALA-8851: Do not throw authorization exception in drop if exists queries .. Patch Set 8: > > Patch Set 8: Code-Review+1 > > would be good if we have Tim's +1 as well since this relates to > authorization of drop command. It looks good to me from my side. -- To view, visit http://gerrit.cloudera.org:8080/14121 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iba068935e5da92d71e16e2321afdb8e7b781086a Gerrit-Change-Number: 14121 Gerrit-PatchSet: 8 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Aug 2019 18:19:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8795 : Enable event polling by default in tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13922 ) Change subject: IMPALA-8795 : Enable event polling by default in tests .. Patch Set 9: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4868/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13922 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7279349d4900e24fbcf558f290549496844ce138 Gerrit-Change-Number: 13922 Gerrit-PatchSet: 9 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Aug 2019 18:20:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7604: part 1: tests for agg cardinality
Tim Armstrong has removed a vote on this change. Change subject: IMPALA-7604: part 1: tests for agg cardinality .. Removed Verified-1 by Impala Public Jenkins -- To view, visit http://gerrit.cloudera.org:8080/14131 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: I59eaddbc5be253793293af064bb2d28a425564e1 Gerrit-Change-Number: 14131 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] [DOCS] CONCAT function returns NULL if any argument is NULL
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14156 ) Change subject: [DOCS] CONCAT function returns NULL if any argument is NULL .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/471/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/14156 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieadd249881b45944a870a616abf75ff4386615ea Gerrit-Change-Number: 14156 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 28 Aug 2019 19:22:10 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] CONCAT function returns NULL if any argument is NULL
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14156 ) Change subject: [DOCS] CONCAT function returns NULL if any argument is NULL .. [DOCS] CONCAT function returns NULL if any argument is NULL Change-Id: Ieadd249881b45944a870a616abf75ff4386615ea Reviewed-on: http://gerrit.cloudera.org:8080/14156 Reviewed-by: Greg Rahn Tested-by: Impala Public Jenkins --- M docs/topics/impala_string_functions.xml 1 file changed, 10 insertions(+), 1 deletion(-) Approvals: Greg Rahn: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/14156 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ieadd249881b45944a870a616abf75ff4386615ea Gerrit-Change-Number: 14156 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8851: Do not throw authorization exception in drop if exists queries
Hello Vihang Karajgaonkar, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14121 to look at the new patch set (#9). Change subject: IMPALA-8851: Do not throw authorization exception in drop if exists queries .. IMPALA-8851: Do not throw authorization exception in drop if exists queries Note that this is the continuation of work in https://github.com/vihangk1/impala/commits/IMPALA-8851 This patch's goal is to change Impala's behavior in the following case: - the query is a DROP TABLE/VIEW/DATABASE/FUNCTIONS IF EXISTS statement - the given object does not exist - the user has some kind of privilege on the object, which imples the privilege to know whether object exists, but does not have DROP privilege on the object Until now this lead to an authorization exception, while it will be allowed with this change. An example where this is useful is a user who has CREATE privilege on a database, and creates table t_owned, and gets ownership of the table. In this case DROP TABLE IF EXISTS was non idempotent: DROP TABLE IF EXISTS t_owned; -> success DROP TABLE IF EXISTS t_owned; -> authorization error, as the privileges for the table were deleted when the table was successfully dropped After this change the second statement will be also successful. The authorization logic has to avoid leaking information that the user has no right to know. For this reason DROP IF EXISTS has to return the same error message regardless whether the object exists or not if the user has no right to know it's existence. This is achieved with the following pattern: - in the IF EXISTS case first an ANY privilege is registered, then the existence of the object is checked and if it doesn't exist, the analysis returns successfully - if the object exists, the DROP privilege is registered (if there is no IF EXISTS in the query, this always happens) - as the authorization logic checks privileges in the order of registration, first the ANY will be checked, and DROP will be only checked if the user has ANY privileges Testing: - Added a new test case in the sentry tests which confirms that the authorization exception is not thrown when a drop if exists query is issued on a object which does not exist. - Changed several tests affected by the new behavior. - Ran core tests. Change-Id: Iba068935e5da92d71e16e2321afdb8e7b781086a --- M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/DropDbStmt.java M fe/src/main/java/org/apache/impala/analysis/DropFunctionStmt.java M fe/src/main/java/org/apache/impala/analysis/DropTableOrViewStmt.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java M fe/src/test/java/org/apache/impala/testutil/TestSentryGroupMapper.java M testdata/workloads/functional-query/queries/QueryTest/create-database.test M tests/authorization/test_owner_privileges.py M tests/common/sentry_cache_test_suite.py 11 files changed, 282 insertions(+), 38 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/14121/9 -- To view, visit http://gerrit.cloudera.org:8080/14121 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iba068935e5da92d71e16e2321afdb8e7b781086a Gerrit-Change-Number: 14121 Gerrit-PatchSet: 9 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8851: Do not throw authorization exception in drop if exists queries
Hello Vihang Karajgaonkar, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14121 to look at the new patch set (#10). Change subject: IMPALA-8851: Do not throw authorization exception in drop if exists queries .. IMPALA-8851: Do not throw authorization exception in drop if exists queries Note that this is the continuation of work in https://github.com/vihangk1/impala/commits/IMPALA-8851 This patch's goal is to change Impala's behavior in the following case: - the query is a DROP TABLE/VIEW/DATABASE/FUNCTIONS IF EXISTS statement - the given object does not exist - the user has some kind of privilege on the object, which imples the privilege to know whether object exists, but does not have DROP privilege on the object Until now this lead to an authorization exception, while it will be allowed with this change. An example where this is useful is a user who has CREATE privilege on a database, and creates table t_owned, and gets ownership of the table. In this case DROP TABLE IF EXISTS was non idempotent: DROP TABLE IF EXISTS t_owned; -> success DROP TABLE IF EXISTS t_owned; -> authorization error, as the privileges for the table were deleted when the table was successfully dropped After this change the second statement will be also successful. The authorization logic has to avoid leaking information that the user has no right to know. For this reason DROP IF EXISTS has to return the same error message regardless whether the object exists or not if the user has no right to know it's existence. This is achieved with the following pattern: - in the IF EXISTS case first an ANY privilege is registered, then the existence of the object is checked and if it doesn't exist, the analysis returns successfully - if the object exists, the DROP privilege is registered (if there is no IF EXISTS in the query, this always happens) - as the authorization logic checks privileges in the order of registration, first the ANY will be checked, and DROP will be only checked if the user has ANY privileges Testing: - Added a new test case in the sentry tests which confirms that the authorization exception is not thrown when a drop if exists query is issued on a object which does not exist. - Changed several tests affected by the new behavior. - Ran core tests. Change-Id: Iba068935e5da92d71e16e2321afdb8e7b781086a --- M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/DropDbStmt.java M fe/src/main/java/org/apache/impala/analysis/DropFunctionStmt.java M fe/src/main/java/org/apache/impala/analysis/DropTableOrViewStmt.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java M fe/src/test/java/org/apache/impala/testutil/TestSentryGroupMapper.java M testdata/workloads/functional-query/queries/QueryTest/create-database.test M tests/authorization/test_owner_privileges.py M tests/common/sentry_cache_test_suite.py 11 files changed, 280 insertions(+), 38 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/14121/10 -- To view, visit http://gerrit.cloudera.org:8080/14121 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iba068935e5da92d71e16e2321afdb8e7b781086a Gerrit-Change-Number: 14121 Gerrit-PatchSet: 10 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8851: Do not throw authorization exception in drop if exists queries
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/14121 ) Change subject: IMPALA-8851: Do not throw authorization exception in drop if exists queries .. Patch Set 10: (2 comments) PS 9 added new tests + some fixes for the issues caused by those tests. PS 10 is just removing some new lines. Carry +1 from Vihang http://gerrit.cloudera.org:8080/#/c/14121/10/fe/src/main/java/org/apache/impala/analysis/Analyzer.java File fe/src/main/java/org/apache/impala/analysis/Analyzer.java: http://gerrit.cloudera.org:8080/#/c/14121/10/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@2668 PS10, Line 2668: return getCatalog().getDb(dbName) != null; stmtTableCache.dbs turned out to be empty unless tables were loaded. http://gerrit.cloudera.org:8080/#/c/14121/10/fe/src/main/java/org/apache/impala/analysis/DropDbStmt.java File fe/src/main/java/org/apache/impala/analysis/DropDbStmt.java: http://gerrit.cloudera.org:8080/#/c/14121/10/fe/src/main/java/org/apache/impala/analysis/DropDbStmt.java@72 PS10, Line 72: // available in the toThrift() method. : serverName_ = analyzer.getServerName(); serverName_ has to be set before returning from the function. The new tests caught a null pointer exception in catalogd when this didn't happen. -- To view, visit http://gerrit.cloudera.org:8080/14121 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iba068935e5da92d71e16e2321afdb8e7b781086a Gerrit-Change-Number: 14121 Gerrit-PatchSet: 10 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Aug 2019 19:48:20 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6663 Expose current DDL metrics on WebUI
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/13806 ) Change subject: IMPALA-6663 Expose current DDL metrics on WebUI .. Patch Set 7: (1 comment) > (1 comment) > > I have started adding the table names to this change, will need > some additional time to finish. I will be on vacation for a month. I would not be able to take a look at this patch until I come back. May ask Bharath to take a look? http://gerrit.cloudera.org:8080/#/c/13806/7/fe/src/main/java/org/apache/impala/service/JniCatalog.java File fe/src/main/java/org/apache/impala/service/JniCatalog.java: http://gerrit.cloudera.org:8080/#/c/13806/7/fe/src/main/java/org/apache/impala/service/JniCatalog.java@190 PS7, Line 190: incrementResetMetadataCounter there are quite a few variations of reset metadata (global invalidate, refresh table, refresh functions, refresh authorization). I think it will be useful to have them track them individually -- To view, visit http://gerrit.cloudera.org:8080/13806 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0ed76f134bad6d3b3d4dce132365a53a01e9512a Gerrit-Change-Number: 13806 Gerrit-PatchSet: 7 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Aug 2019 20:08:12 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7312: Non-blocking mode for Fetch() RPC
Sahil Takiar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14157 Change subject: IMPALA-7312: Non-blocking mode for Fetch() RPC .. IMPALA-7312: Non-blocking mode for Fetch() RPC Adds the query option FETCH_ROWS_TIMEOUT_MS to control the client timeout when fetching rows. Set to 10 seconds by default to avoid unnecessary fetch requests. Timeout applies when result spooling is enabled or disabled. The timeout controls how long the client thread will wait for a single RowBatch to be produced by the coordinator fragment. Since result spooling allows clients to fetch multiple RowBatches at a time, the timeout is reset when each RowBatch is read by the client thread. Testing: * Added new tests to test_fetch.py * Running core tests Change-Id: I331acaba23a65dab43cca48e9dc0dc957b9c632d --- M be/src/exec/blocking-plan-root-sink.cc M be/src/exec/buffered-plan-root-sink.cc M be/src/exec/plan-root-sink.cc M be/src/exec/plan-root-sink.h M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M tests/hs2/test_fetch.py 9 files changed, 109 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/57/14157/1 -- To view, visit http://gerrit.cloudera.org:8080/14157 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I331acaba23a65dab43cca48e9dc0dc957b9c632d Gerrit-Change-Number: 14157 Gerrit-PatchSet: 1 Gerrit-Owner: Sahil Takiar
[Impala-ASF-CR] IMPALA-7312: Non-blocking mode for Fetch() RPC
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14157 ) Change subject: IMPALA-7312: Non-blocking mode for Fetch() RPC .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/14157/1/be/src/exec/blocking-plan-root-sink.cc File be/src/exec/blocking-plan-root-sink.cc: http://gerrit.cloudera.org:8080/#/c/14157/1/be/src/exec/blocking-plan-root-sink.cc@117 PS1, Line 117: // True if the consumer timed out waiting for the producer to send rows, false otherwise. line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/14157/1/be/src/exec/buffered-plan-root-sink.cc File be/src/exec/buffered-plan-root-sink.cc: http://gerrit.cloudera.org:8080/#/c/14157/1/be/src/exec/buffered-plan-root-sink.cc@155 PS1, Line 155: // True if the consumer timed out waiting for the producer to send rows, false otherwise. line too long (93 > 90) -- To view, visit http://gerrit.cloudera.org:8080/14157 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I331acaba23a65dab43cca48e9dc0dc957b9c632d Gerrit-Change-Number: 14157 Gerrit-PatchSet: 1 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 28 Aug 2019 20:15:57 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7312: Non-blocking mode for Fetch() RPC
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14157 to look at the new patch set (#2). Change subject: IMPALA-7312: Non-blocking mode for Fetch() RPC .. IMPALA-7312: Non-blocking mode for Fetch() RPC Adds the query option FETCH_ROWS_TIMEOUT_MS to control the client timeout when fetching rows. Set to 10 seconds by default to avoid unnecessary fetch requests. Timeout applies when result spooling is enabled or disabled. The timeout controls how long the client thread will wait for a single RowBatch to be produced by the coordinator fragment. Since result spooling allows clients to fetch multiple RowBatches at a time, the timeout is reset when each RowBatch is read by the client thread. Testing: * Added new tests to test_fetch.py * Running core tests Change-Id: I331acaba23a65dab43cca48e9dc0dc957b9c632d --- M be/src/exec/blocking-plan-root-sink.cc M be/src/exec/buffered-plan-root-sink.cc M be/src/exec/plan-root-sink.cc M be/src/exec/plan-root-sink.h M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M tests/hs2/test_fetch.py 9 files changed, 111 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/57/14157/2 -- To view, visit http://gerrit.cloudera.org:8080/14157 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I331acaba23a65dab43cca48e9dc0dc957b9c632d Gerrit-Change-Number: 14157 Gerrit-PatchSet: 2 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-7312: Non-blocking mode for Fetch() RPC
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14157 ) Change subject: IMPALA-7312: Non-blocking mode for Fetch() RPC .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/14157/1/be/src/exec/blocking-plan-root-sink.cc File be/src/exec/blocking-plan-root-sink.cc: http://gerrit.cloudera.org:8080/#/c/14157/1/be/src/exec/blocking-plan-root-sink.cc@117 PS1, Line 117: // True if the consumer timed out waiting for the producer to send rows, false otherwise. > line too long (91 > 90) Done http://gerrit.cloudera.org:8080/#/c/14157/1/be/src/exec/buffered-plan-root-sink.cc File be/src/exec/buffered-plan-root-sink.cc: http://gerrit.cloudera.org:8080/#/c/14157/1/be/src/exec/buffered-plan-root-sink.cc@155 PS1, Line 155: // True if the consumer timed out waiting for the producer to send rows, false otherwise. > line too long (93 > 90) Done -- To view, visit http://gerrit.cloudera.org:8080/14157 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I331acaba23a65dab43cca48e9dc0dc957b9c632d Gerrit-Change-Number: 14157 Gerrit-PatchSet: 1 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Comment-Date: Wed, 28 Aug 2019 20:17:33 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8851: Do not throw authorization exception in drop if exists queries
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14121 ) Change subject: IMPALA-8851: Do not throw authorization exception in drop if exists queries .. Patch Set 9: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4419/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14121 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iba068935e5da92d71e16e2321afdb8e7b781086a Gerrit-Change-Number: 14121 Gerrit-PatchSet: 9 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Aug 2019 20:18:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8851: Do not throw authorization exception in drop if exists queries
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14121 ) Change subject: IMPALA-8851: Do not throw authorization exception in drop if exists queries .. Patch Set 10: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4420/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14121 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iba068935e5da92d71e16e2321afdb8e7b781086a Gerrit-Change-Number: 14121 Gerrit-PatchSet: 10 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Aug 2019 20:24:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8897: Fully qualify all links on the webserver
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14151 to look at the new patch set (#2). Change subject: IMPALA-8897: Fully qualify all links on the webserver .. IMPALA-8897: Fully qualify all links on the webserver This patch adds the 'host:port' to all links on the webserver. This will facilitate proxying connections to the debug webui through Knox by allowing us to create rewrite rules that do the transform: ... => ... which allows us to have a single IMPALAUI Knox service that can proxy connections to any impalad/statestored/catalogd webui in a cluster. Note that this works because currently all of the links on Impala's webui are within the same webserver (it would also be possible to add links to other Impala daemon webuis within a cluster, eg. if we wanted to add webui links on the /backends page). If we ever need to add links to external pages, the Knox service definition will likely need to be modified. This patch also adds hidden fields to all forms for the scheme, host, and port value, so that GET requests from forms will result in the same form as the transformed url shown above. Testing: - Ran the webserver and manually clicked around on a bunch of links to ensure everything works as expected. - Ran in a cluster and verified the new Knox service defintion works as intended with this change. - Added a test that uses a regex to check for template files that don't conform to the requirements. Change-Id: If1195709a0f21f39d9a1e484880a0c46c9967ed2 --- M be/src/service/impala-beeswax-server.cc M be/src/service/impala-hs2-server.cc M be/src/util/webserver.cc M be/src/util/webserver.h M tests/common/file_utils.py M tests/webserver/test_web_pages.py M www/admission_controller.tmpl M www/catalog.tmpl M www/common-header.tmpl A www/form-hidden-inputs.tmpl M www/log_level.tmpl M www/queries.tmpl M www/query_detail_tabs.tmpl M www/query_profile.tmpl M www/sessions.tmpl M www/threadz.tmpl M www/threadz_tabs.tmpl 17 files changed, 116 insertions(+), 69 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/51/14151/2 -- To view, visit http://gerrit.cloudera.org:8080/14151 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If1195709a0f21f39d9a1e484880a0c46c9967ed2 Gerrit-Change-Number: 14151 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8897: Fully qualify all links on the webserver
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14151 ) Change subject: IMPALA-8897: Fully qualify all links on the webserver .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/14151/2/be/src/util/webserver.cc File be/src/util/webserver.cc: http://gerrit.cloudera.org:8080/#/c/14151/2/be/src/util/webserver.cc@430 PS2, Line 430: url_ = Substitute("$0://$1:$2", IsSecure() ? "https" : "http", hostname_, http_address_.port); line too long (96 > 90) -- To view, visit http://gerrit.cloudera.org:8080/14151 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If1195709a0f21f39d9a1e484880a0c46c9967ed2 Gerrit-Change-Number: 14151 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 28 Aug 2019 20:41:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14151 to look at the new patch set (#3). Change subject: IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules .. IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules This patch adds the 'host:port' to all links on the webserver. This will facilitate proxying connections to the debug webui through Knox by allowing us to create rewrite rules that do the transform: ... => ... which allows us to have a single IMPALAUI Knox service that can proxy connections to any impalad/statestored/catalogd webui in a cluster. Note that this works because currently all of the links on Impala's webui are within the same webserver (it would also be possible to add links to other Impala daemon webuis within a cluster, eg. if we wanted to add webui links on the /backends page). If we ever need to add links to external pages, the Knox service definition will likely need to be modified. This patch also adds hidden fields to all forms for the scheme, host, and port value, so that GET requests from forms will result in the same form as the transformed url shown above. Testing: - Ran the webserver and manually clicked around on a bunch of links to ensure everything works as expected. - Ran in a cluster and verified the new Knox service defintion works as intended with this change. - Added a test that uses a regex to check for template files that don't conform to the requirements. Change-Id: If1195709a0f21f39d9a1e484880a0c46c9967ed2 --- M be/src/service/impala-beeswax-server.cc M be/src/service/impala-hs2-server.cc M be/src/util/webserver.cc M be/src/util/webserver.h M tests/common/file_utils.py M tests/webserver/test_web_pages.py M www/admission_controller.tmpl M www/catalog.tmpl M www/common-header.tmpl A www/form-hidden-inputs.tmpl M www/log_level.tmpl M www/queries.tmpl M www/query_detail_tabs.tmpl M www/query_profile.tmpl M www/sessions.tmpl M www/threadz.tmpl M www/threadz_tabs.tmpl 17 files changed, 116 insertions(+), 69 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/51/14151/3 -- To view, visit http://gerrit.cloudera.org:8080/14151 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If1195709a0f21f39d9a1e484880a0c46c9967ed2 Gerrit-Change-Number: 14151 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8851: Do not throw authorization exception in drop if exists queries
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14121 ) Change subject: IMPALA-8851: Do not throw authorization exception in drop if exists queries .. Patch Set 10: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4869/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/14121 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iba068935e5da92d71e16e2321afdb8e7b781086a Gerrit-Change-Number: 14121 Gerrit-PatchSet: 10 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Aug 2019 20:54:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7312: Non-blocking mode for Fetch() RPC
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14157 ) Change subject: IMPALA-7312: Non-blocking mode for Fetch() RPC .. Patch Set 1: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/4421/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/14157 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I331acaba23a65dab43cca48e9dc0dc957b9c632d Gerrit-Change-Number: 14157 Gerrit-PatchSet: 1 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 28 Aug 2019 20:55:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7312: Non-blocking mode for Fetch() RPC
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14157 ) Change subject: IMPALA-7312: Non-blocking mode for Fetch() RPC .. Patch Set 2: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/4422/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/14157 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I331acaba23a65dab43cca48e9dc0dc957b9c632d Gerrit-Change-Number: 14157 Gerrit-PatchSet: 2 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 28 Aug 2019 20:58:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7975(continued): Improve supportability of the automatic invalidate feature.
Anurag Mantripragada has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14159 Change subject: IMPALA-7975(continued): Improve supportability of the automatic invalidate feature. .. IMPALA-7975(continued): Improve supportability of the automatic invalidate feature. This work was done by Sharanitha Harish. This commit adds support to display the metric last-synced-event-id as /metrics#events page whereas previously it was displayed only on the /events page. Testing: Added tests toMetaStoreEventsProcessorTest to verify: 1. If the metric updated as it should when new events were processed. 2. If the metric was not set when the event processor was not active. Change-Id: I6c468f2c3b456399ed67196ee27a9908f7376974 --- M be/src/util/event-metrics.cc M be/src/util/event-metrics.h M common/thrift/JniCatalog.thrift M common/thrift/metrics.json M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 6 files changed, 33 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/59/14159/1 -- To view, visit http://gerrit.cloudera.org:8080/14159 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I6c468f2c3b456399ed67196ee27a9908f7376974 Gerrit-Change-Number: 14159 Gerrit-PatchSet: 1 Gerrit-Owner: Anurag Mantripragada
[Impala-ASF-CR] IMPALA-7975(continued): Improve supportability of the automatic invalidate feature.
Anurag Mantripragada has abandoned this change. ( http://gerrit.cloudera.org:8080/14159 ) Change subject: IMPALA-7975(continued): Improve supportability of the automatic invalidate feature. .. Abandoned Duplicate -- To view, visit http://gerrit.cloudera.org:8080/14159 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I6c468f2c3b456399ed67196ee27a9908f7376974 Gerrit-Change-Number: 14159 Gerrit-PatchSet: 1 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14151 ) Change subject: IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules .. Patch Set 2: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/4423/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/14151 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If1195709a0f21f39d9a1e484880a0c46c9967ed2 Gerrit-Change-Number: 14151 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 28 Aug 2019 21:22:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8579: Ignore trivial alter table/partition events.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14145 ) Change subject: IMPALA-8579: Ignore trivial alter table/partition events. .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14145 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01a59d5170accc014f76f14eb526d96ddcf61f76 Gerrit-Change-Number: 14145 Gerrit-PatchSet: 5 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Aug 2019 21:23:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8579: Ignore trivial alter table/partition events.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14145 ) Change subject: IMPALA-8579: Ignore trivial alter table/partition events. .. IMPALA-8579: Ignore trivial alter table/partition events. Hive generates certain trivial alter events for eg: change only "transient_lastDdlTime". This is seen in the alter events accompanying an INSERT event. MetastoreEventProcessor should ignore such events as these trivial properties are not used by Impala and they cause unnecessary invalidates/refreshes. This change will also potentially reduce the likelihood of IMPALA-8877 as back to back invalidates after INSERT operations are avoided. Testing: Added tests respectively to testPartitionEvents() and testAlterTableEvent() in MetastoreEventProcessorTest class. Change-Id: I01a59d5170accc014f76f14eb526d96ddcf61f76 Reviewed-on: http://gerrit.cloudera.org:8080/14145 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 2 files changed, 150 insertions(+), 0 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/14145 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I01a59d5170accc014f76f14eb526d96ddcf61f76 Gerrit-Change-Number: 14145 Gerrit-PatchSet: 6 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14151 ) Change subject: IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules .. Patch Set 3: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/4424/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/14151 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If1195709a0f21f39d9a1e484880a0c46c9967ed2 Gerrit-Change-Number: 14151 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 28 Aug 2019 21:25:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8901: Fix # links on /catalog page.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14153 ) Change subject: IMPALA-8901: Fix # links on /catalog page. .. IMPALA-8901: Fix # links on /catalog page. After IMPALA-7935, the database links on the top of /catalog page that use # links to jump to the part of the page corresponding to the particular database is broken because the id is ommitted. This is fixed by moving id to a tag which doesn't change if local catalog is turned on. Testing: Manually tested the links are working in both local catalog mode and V1 mode. Change-Id: I895742dcaeaf71ecf097b0fec69cc19a3b6f86f4 Reviewed-on: http://gerrit.cloudera.org:8080/14153 Reviewed-by: Thomas Tauber-Marshall Tested-by: Impala Public Jenkins --- M www/catalog.tmpl 1 file changed, 2 insertions(+), 2 deletions(-) Approvals: Thomas Tauber-Marshall: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/14153 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I895742dcaeaf71ecf097b0fec69cc19a3b6f86f4 Gerrit-Change-Number: 14153 Gerrit-PatchSet: 2 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-8901: Fix # links on /catalog page.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14153 ) Change subject: IMPALA-8901: Fix # links on /catalog page. .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14153 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I895742dcaeaf71ecf097b0fec69cc19a3b6f86f4 Gerrit-Change-Number: 14153 Gerrit-PatchSet: 1 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Wed, 28 Aug 2019 21:34:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7975(continued): Improve supportability of the automatic invalidate feature.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14159 ) Change subject: IMPALA-7975(continued): Improve supportability of the automatic invalidate feature. .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4425/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14159 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6c468f2c3b456399ed67196ee27a9908f7376974 Gerrit-Change-Number: 14159 Gerrit-PatchSet: 1 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 28 Aug 2019 21:54:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8902: Bump timeout in test result spooling:::test spilling
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14155 ) Change subject: IMPALA-8902: Bump timeout in test_result_spooling:::test_spilling .. Patch Set 1: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4867/ -- To view, visit http://gerrit.cloudera.org:8080/14155 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iabac8d7273735079dca48a1e0ecd4f341ea690a0 Gerrit-Change-Number: 14155 Gerrit-PatchSet: 1 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 28 Aug 2019 22:00:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7604: part 1: tests for agg cardinality
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/14131 ) Change subject: IMPALA-7604: part 1: tests for agg cardinality .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14131 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I59eaddbc5be253793293af064bb2d28a425564e1 Gerrit-Change-Number: 14131 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 28 Aug 2019 22:51:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7604: part 1: tests for agg cardinality
Tim Armstrong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14131 ) Change subject: IMPALA-7604: part 1: tests for agg cardinality .. IMPALA-7604: part 1: tests for agg cardinality Add tests that demonstrate bugs with overflow and multiple aggregation classes and provide a baseline to demonstrate the effects of the bugfixes. Testing: Added planner tests: * Cardinality estimates with multiple groups (both grouping and non-grouping aggs). * Overflow case from multiplication of grouping expr cardinality. * Overflow handling of addition of multiple groups. Change-Id: I59eaddbc5be253793293af064bb2d28a425564e1 Reviewed-on: http://gerrit.cloudera.org:8080/14131 Reviewed-by: Bikramjeet Vig Tested-by: Tim Armstrong --- M fe/src/test/java/org/apache/impala/planner/PlannerTest.java A testdata/workloads/functional-planner/queries/PlannerTest/card-agg.test 2 files changed, 669 insertions(+), 0 deletions(-) Approvals: Bikramjeet Vig: Looks good to me, approved Tim Armstrong: Verified -- To view, visit http://gerrit.cloudera.org:8080/14131 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I59eaddbc5be253793293af064bb2d28a425564e1 Gerrit-Change-Number: 14131 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7604: part 2: fixes for AggregationNode cardinality
Tim Armstrong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14132 ) Change subject: IMPALA-7604: part 2: fixes for AggregationNode cardinality .. IMPALA-7604: part 2: fixes for AggregationNode cardinality * Use saturating arithmetic in Expr.getNumDistinctValues() to avoid overflows. * Avoid double-adding with checkedAdd() * Fix incorrect logic with multiple groups - each group cannot return more than the input rows, but with multiple groups it can add up to more than the input rows. Testing: Updated planner tests from part 1 to reflect bugfixes. Added targeted cardinality tests to verify behaviour with and without stats. Updated other planner tests that changed as a result of this fixed. Ran exhaustive tests. Change-Id: Ieed41d60c0e0dfeca64035e919cb8c28a054a9ab Reviewed-on: http://gerrit.cloudera.org:8080/14132 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/main/java/org/apache/impala/planner/AggregationNode.java M fe/src/test/java/org/apache/impala/planner/CardinalityTest.java M testdata/workloads/functional-planner/queries/PlannerTest/card-agg.test M testdata/workloads/functional-planner/queries/PlannerTest/empty.test M testdata/workloads/functional-planner/queries/PlannerTest/insert-sort-by.test M testdata/workloads/functional-planner/queries/PlannerTest/joins.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds-all.test 8 files changed, 964 insertions(+), 855 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/14132 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ieed41d60c0e0dfeca64035e919cb8c28a054a9ab Gerrit-Change-Number: 14132 Gerrit-PatchSet: 11 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7604: part 1: tests for agg cardinality
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14131 ) Change subject: IMPALA-7604: part 1: tests for agg cardinality .. Patch Set 6: Verified+1 Follow-on patch passed precommit, will apply verification to this too. -- To view, visit http://gerrit.cloudera.org:8080/14131 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I59eaddbc5be253793293af064bb2d28a425564e1 Gerrit-Change-Number: 14131 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 28 Aug 2019 22:52:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8851: Do not throw authorization exception in drop if exists queries
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14121 ) Change subject: IMPALA-8851: Do not throw authorization exception in drop if exists queries .. Patch Set 10: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14121 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iba068935e5da92d71e16e2321afdb8e7b781086a Gerrit-Change-Number: 14121 Gerrit-PatchSet: 10 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Aug 2019 22:59:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8795 : Enable event polling by default in tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13922 ) Change subject: IMPALA-8795 : Enable event polling by default in tests .. Patch Set 9: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4868/ -- To view, visit http://gerrit.cloudera.org:8080/13922 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7279349d4900e24fbcf558f290549496844ce138 Gerrit-Change-Number: 13922 Gerrit-PatchSet: 9 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 28 Aug 2019 23:15:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14151 ) Change subject: IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14151 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If1195709a0f21f39d9a1e484880a0c46c9967ed2 Gerrit-Change-Number: 14151 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 28 Aug 2019 23:35:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7312: Non-blocking mode for Fetch() RPC
Hello Michael Ho, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14157 to look at the new patch set (#3). Change subject: IMPALA-7312: Non-blocking mode for Fetch() RPC .. IMPALA-7312: Non-blocking mode for Fetch() RPC Adds the query option FETCH_ROWS_TIMEOUT_MS to control the client timeout when fetching rows. Set to 10 seconds by default to avoid unnecessary fetch requests. Timeout applies when result spooling is enabled or disabled. The timeout controls how long the client thread will wait for a single RowBatch to be produced by the coordinator fragment. Since result spooling allows clients to fetch multiple RowBatches at a time, the timeout is reset when each RowBatch is read by the client thread. Testing: * Added new tests to test_fetch.py * Running core tests Change-Id: I331acaba23a65dab43cca48e9dc0dc957b9c632d --- M be/src/exec/blocking-plan-root-sink.cc M be/src/exec/buffered-plan-root-sink.cc M be/src/exec/plan-root-sink.cc M be/src/exec/plan-root-sink.h M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M tests/hs2/test_fetch.py 9 files changed, 111 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/57/14157/3 -- To view, visit http://gerrit.cloudera.org:8080/14157 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I331acaba23a65dab43cca48e9dc0dc957b9c632d Gerrit-Change-Number: 14157 Gerrit-PatchSet: 3 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8902: Bump timeout in test result spooling:::test spilling
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14155 ) Change subject: IMPALA-8902: Bump timeout in test_result_spooling:::test_spilling .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4870/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14155 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iabac8d7273735079dca48a1e0ecd4f341ea690a0 Gerrit-Change-Number: 14155 Gerrit-PatchSet: 2 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 29 Aug 2019 00:20:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8902: Bump timeout in test result spooling:::test spilling
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14155 ) Change subject: IMPALA-8902: Bump timeout in test_result_spooling:::test_spilling .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14155 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iabac8d7273735079dca48a1e0ecd4f341ea690a0 Gerrit-Change-Number: 14155 Gerrit-PatchSet: 2 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 29 Aug 2019 00:20:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14129 ) Change subject: IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes .. Patch Set 5: Code-Review+2 (1 comment) LGTM, might be able to simplify further so just wanted to mention that. http://gerrit.cloudera.org:8080/#/c/14129/4/be/src/exec/buffered-plan-root-sink.h File be/src/exec/buffered-plan-root-sink.h: http://gerrit.cloudera.org:8080/#/c/14129/4/be/src/exec/buffered-plan-root-sink.h@145 PS4, Line 145: bool current_batch_empty_ = true; Is it sufficient to check that current_batch_row_ == current_batch_->num_rows()? I think this might be redundant with that state. -- To view, visit http://gerrit.cloudera.org:8080/14129 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8dd4b397ab6457a4f85e635f239b2c67130fcce4 Gerrit-Change-Number: 14129 Gerrit-PatchSet: 5 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 29 Aug 2019 00:32:32 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8572: Log query events before Unregister() call.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14143 ) Change subject: IMPALA-8572: Log query events before Unregister() call. .. Patch Set 4: (2 comments) http://gerrit.cloudera.org:8080/#/c/14143/4//COMMIT_MSG Commit Message: PS4: I had a couple of high level questions. * Do we know why the log-only-if-results-were-fetched behaviour exists? It seems simpler to me to log anything that gets into the FINISHED state, and I'm not seeing an obvious downside. * Could we do the logging inline in the Wait() thread? Would be nice to avoid creating another thread. http://gerrit.cloudera.org:8080/#/c/14143/4/be/src/service/client-request-state.cc File be/src/service/client-request-state.cc: http://gerrit.cloudera.org:8080/#/c/14143/4/be/src/service/client-request-state.cc@1412 PS4, Line 1412: statemens statements -- To view, visit http://gerrit.cloudera.org:8080/14143 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I639b9c1acb9806b29292cd85be2863688453ca2e Gerrit-Change-Number: 14143 Gerrit-PatchSet: 4 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Thu, 29 Aug 2019 00:43:56 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8851: Do not throw authorization exception in drop if exists queries
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14121 ) Change subject: IMPALA-8851: Do not throw authorization exception in drop if exists queries .. Patch Set 10: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4869/ -- To view, visit http://gerrit.cloudera.org:8080/14121 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iba068935e5da92d71e16e2321afdb8e7b781086a Gerrit-Change-Number: 14121 Gerrit-PatchSet: 10 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 29 Aug 2019 00:58:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7312: Non-blocking mode for Fetch() RPC
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14157 ) Change subject: IMPALA-7312: Non-blocking mode for Fetch() RPC .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4426/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14157 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I331acaba23a65dab43cca48e9dc0dc957b9c632d Gerrit-Change-Number: 14157 Gerrit-PatchSet: 3 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 29 Aug 2019 01:00:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8879: upgrade bootstrap for debug page to 4.3.1
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14119 ) Change subject: IMPALA-8879: upgrade bootstrap for debug page to 4.3.1 .. Patch Set 4: Do you have a chance to do another pass? -- To view, visit http://gerrit.cloudera.org:8080/14119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib58f407574f590825d208424a8c0fd101b0a19a7 Gerrit-Change-Number: 14119 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 29 Aug 2019 01:16:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8858: Add observability of the query load on executor groups
Hello Andrew Sherman, Lars Volker, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14103 to look at the new patch set (#6). Change subject: IMPALA-8858: Add observability of the query load on executor groups .. IMPALA-8858: Add observability of the query load on executor groups With this patch, all healthy executor groups will have a metric added that display the number of queries (admitted by the local coordinator) running on them. The metric for a group is added when a it is healthy and removed when its either not healthy or no longer exists. It gets updated when either the cluster membership changes or a query gets admitted or released by the admission controller. Also adds the ability to delete metrics from a metric group after registration. Testing: Added a custom cluster test and a BE metric test. Change-Id: I58cde8699c33af8b87273437e9d8bf6371a34539 --- M be/src/scheduling/admission-controller.cc M be/src/scheduling/admission-controller.h M be/src/scheduling/cluster-membership-mgr.cc M be/src/scheduling/cluster-membership-mgr.h M be/src/util/metrics-test.cc M be/src/util/metrics.cc M be/src/util/metrics.h M common/thrift/metrics.json M tests/custom_cluster/test_executor_groups.py 9 files changed, 210 insertions(+), 34 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/03/14103/6 -- To view, visit http://gerrit.cloudera.org:8080/14103 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I58cde8699c33af8b87273437e9d8bf6371a34539 Gerrit-Change-Number: 14103 Gerrit-PatchSet: 6 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker
[Impala-ASF-CR] IMPALA-8858: Add observability of the query load on executor groups
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/14103 ) Change subject: IMPALA-8858: Add observability of the query load on executor groups .. Patch Set 6: (4 comments) http://gerrit.cloudera.org:8080/#/c/14103/5/be/src/util/metrics.h File be/src/util/metrics.h: http://gerrit.cloudera.org:8080/#/c/14103/5/be/src/util/metrics.h@334 PS5, Line 334: invalidat > nit: reword, something like "users of this class", or just mention that del Done http://gerrit.cloudera.org:8080/#/c/14103/5/be/src/util/metrics.h@375 PS5, Line 375: > nit: It logs an error though, so a user shouldn't just call this without be As suggested, removed the concept of transient metrics http://gerrit.cloudera.org:8080/#/c/14103/5/be/src/util/metrics.h@456 PS5, Line 456: MetricMap metric_map_; > The separation between the pool and the map seems complicated. Should we co Good idea. I quickly scanned through the metrics in kudu/util and that seems to use something similar instead of an objectPool. http://gerrit.cloudera.org:8080/#/c/14103/5/be/src/util/metrics.h@462 PS5, Line 462: Produces a > why not use a unique_ptr? we need a type smart pointer that supports type erasure so we can store multiple subclass Metric types in the same map. This post explains it pretty well https://stackoverflow.com/a/39288979 -- To view, visit http://gerrit.cloudera.org:8080/14103 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I58cde8699c33af8b87273437e9d8bf6371a34539 Gerrit-Change-Number: 14103 Gerrit-PatchSet: 6 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Comment-Date: Thu, 29 Aug 2019 01:40:50 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8858: Add observability of the query load on executor groups
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14103 ) Change subject: IMPALA-8858: Add observability of the query load on executor groups .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4427/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14103 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I58cde8699c33af8b87273437e9d8bf6371a34539 Gerrit-Change-Number: 14103 Gerrit-PatchSet: 6 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Comment-Date: Thu, 29 Aug 2019 02:19:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7312: Non-blocking mode for Fetch() RPC
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14157 ) Change subject: IMPALA-7312: Non-blocking mode for Fetch() RPC .. Patch Set 3: (3 comments) Cool! I didn't expect to see this so soon. http://gerrit.cloudera.org:8080/#/c/14157/3/be/src/exec/blocking-plan-root-sink.cc File be/src/exec/blocking-plan-root-sink.cc: http://gerrit.cloudera.org:8080/#/c/14157/3/be/src/exec/blocking-plan-root-sink.cc@126 PS3, Line 126: timed_out = true; At this point is it possible that results had some rows appended to it? I think it's at very least subtle whether the timeout can expire and WaitFor() return false in the case when the producer appended and signalled. Might be good to comment on whether this code path needs to handle that case (it looks like it handles it correctly already). http://gerrit.cloudera.org:8080/#/c/14157/3/be/src/exec/buffered-plan-root-sink.cc File be/src/exec/buffered-plan-root-sink.cc: http://gerrit.cloudera.org:8080/#/c/14157/3/be/src/exec/buffered-plan-root-sink.cc@169 PS3, Line 169: if (!rows_available_.WaitFor(l, PlanRootSink::fetch_rows_timeout_us())) { Intuitively I would have expected the timeout to apply to the cumulative time spent blocked here - the RPC could still block for a long time if data was trickling in within the timeout period. http://gerrit.cloudera.org:8080/#/c/14157/3/be/src/service/query-options.h File be/src/service/query-options.h: http://gerrit.cloudera.org:8080/#/c/14157/3/be/src/service/query-options.h@189 PS3, Line 189: TQueryOptionLevel::DEVELOPMENT) I think this one should be ok to expose immediately, right, since it's being used in the non-spooling path - it could be advanced? -- To view, visit http://gerrit.cloudera.org:8080/14157 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I331acaba23a65dab43cca48e9dc0dc957b9c632d Gerrit-Change-Number: 14157 Gerrit-PatchSet: 3 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 29 Aug 2019 02:23:38 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes
Hello Michael Ho, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14129 to look at the new patch set (#6). Change subject: IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes .. IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes Adds support for non-default fetch sizes when result spooling is enabled (the default is to return BATCH_SIZE rows for each fetch request). When result spooling is disabled, Impala can only return up to BATCH_SIZE rows because it only buffers a single RowBatch at a time. When result spooling is enabled, each fetch request returns exactly the number of rows requested assuming there are that many rows left in the result set. There is also an upper limit on the fetch size to prevent the resulting QueryResultSet from getting too big. Unlike the behavior when result spooling is disabled, fetches do not break on RowBatch boundaries. For example, when result spooling is disabled, if the fetch size is 10 and the batch size is 15, the second fetch will return 5 rows. However, when result spooling is enabled the second fetch will return 10 rows (assuming there is another RowBatch to read). Testing: * Ran core tests * Added new tests to test_result_spooling.py * Added new tests to buffered-tuple-stream-test to validate writing to a BufferedTupleStream before releasing row batches with 'attach_on_read' set to true. Change-Id: I8dd4b397ab6457a4f85e635f239b2c67130fcce4 --- M be/src/exec/buffered-plan-root-sink.cc M be/src/exec/buffered-plan-root-sink.h M be/src/runtime/buffered-tuple-stream-test.cc M be/src/runtime/query-state.h M tests/query_test/test_result_spooling.py 5 files changed, 306 insertions(+), 56 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/14129/6 -- To view, visit http://gerrit.cloudera.org:8080/14129 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8dd4b397ab6457a4f85e635f239b2c67130fcce4 Gerrit-Change-Number: 14129 Gerrit-PatchSet: 6 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14129 ) Change subject: IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/14129/4/be/src/exec/buffered-plan-root-sink.h File be/src/exec/buffered-plan-root-sink.h: http://gerrit.cloudera.org:8080/#/c/14129/4/be/src/exec/buffered-plan-root-sink.h@145 PS4, Line 145: bool current_batch_empty_ = true; > Is it sufficient to check that current_batch_row_ == current_batch_->num_ro Yeah, good point. It is sufficient to check if 'current_batch_row_' equals / not equals 0. Updated the patch. -- To view, visit http://gerrit.cloudera.org:8080/14129 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8dd4b397ab6457a4f85e635f239b2c67130fcce4 Gerrit-Change-Number: 14129 Gerrit-PatchSet: 4 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 29 Aug 2019 03:41:13 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14129 ) Change subject: IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes .. Patch Set 6: Code-Review+2 Carrying +2. -- To view, visit http://gerrit.cloudera.org:8080/14129 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8dd4b397ab6457a4f85e635f239b2c67130fcce4 Gerrit-Change-Number: 14129 Gerrit-PatchSet: 6 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 29 Aug 2019 03:41:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14129 ) Change subject: IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4428/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14129 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8dd4b397ab6457a4f85e635f239b2c67130fcce4 Gerrit-Change-Number: 14129 Gerrit-PatchSet: 6 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 29 Aug 2019 04:18:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8902: Bump timeout in test result spooling:::test spilling
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14155 ) Change subject: IMPALA-8902: Bump timeout in test_result_spooling:::test_spilling .. IMPALA-8902: Bump timeout in test_result_spooling:::test_spilling De-flake test_result_spooling.py::TestResultSpooling::test_spilling. Bump up the timeout that controls how long the test waits for result spooling to start spilling to disk (detected by the presence of PeakUnpinnedBytes in the PLAN_ROOT_SINK section of the runtime profile). Testing: * Looped TestResultSpooling::test_spilling on an ASAN build for an hour without any failure. Change-Id: Iabac8d7273735079dca48a1e0ecd4f341ea690a0 Reviewed-on: http://gerrit.cloudera.org:8080/14155 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M tests/query_test/test_result_spooling.py 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/14155 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Iabac8d7273735079dca48a1e0ecd4f341ea690a0 Gerrit-Change-Number: 14155 Gerrit-PatchSet: 3 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8902: Bump timeout in test result spooling:::test spilling
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14155 ) Change subject: IMPALA-8902: Bump timeout in test_result_spooling:::test_spilling .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14155 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iabac8d7273735079dca48a1e0ecd4f341ea690a0 Gerrit-Change-Number: 14155 Gerrit-PatchSet: 2 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 29 Aug 2019 04:26:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules
Hello Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14151 to look at the new patch set (#4). Change subject: IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules .. IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules This patch adds the 'host:port' to all links on the webserver. This will facilitate proxying connections to the debug webui through Knox by allowing us to create rewrite rules that do the transform: ... => ... which allows us to have a single IMPALAUI Knox service that can proxy connections to any impalad/statestored/catalogd webui in a cluster. Note that this works because currently all of the links on Impala's webui are within the same webserver (it would also be possible to add links to other Impala daemon webuis within a cluster, eg. if we wanted to add webui links on the /backends page). If we ever need to add links to external pages, the Knox service definition will likely need to be modified. This patch also adds hidden fields to all forms for the scheme, host, and port value, so that GET requests from forms will result in the same form as the transformed url shown above. Testing: - Ran the webserver and manually clicked around on a bunch of links to ensure everything works as expected. - Ran in a cluster and verified the new Knox service defintion works as intended with this change. - Added a test that uses a regex to check for template files that don't conform to the requirements. Change-Id: If1195709a0f21f39d9a1e484880a0c46c9967ed2 --- M be/src/service/impala-beeswax-server.cc M be/src/service/impala-hs2-server.cc M be/src/util/webserver.cc M be/src/util/webserver.h M bin/rat_exclude_files.txt M tests/common/file_utils.py M tests/webserver/test_web_pages.py M www/admission_controller.tmpl M www/catalog.tmpl M www/common-header.tmpl A www/form-hidden-inputs.tmpl M www/log_level.tmpl M www/queries.tmpl M www/query_detail_tabs.tmpl M www/query_profile.tmpl M www/rpcz.tmpl M www/sessions.tmpl M www/threadz.tmpl M www/threadz_tabs.tmpl 19 files changed, 136 insertions(+), 74 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/51/14151/4 -- To view, visit http://gerrit.cloudera.org:8080/14151 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If1195709a0f21f39d9a1e484880a0c46c9967ed2 Gerrit-Change-Number: 14151 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules
Hello Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14151 to look at the new patch set (#5). Change subject: IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules .. IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules This patch adds the 'host:port' to all links on the webserver. This will facilitate proxying connections to the debug webui through Knox by allowing us to create rewrite rules that do the transform: ... => ... which allows us to have a single IMPALAUI Knox service that can proxy connections to any impalad/statestored/catalogd webui in a cluster. Note that this works because currently all of the links on Impala's webui are within the same webserver (it would also be possible to add links to other Impala daemon webuis within a cluster, eg. if we wanted to add webui links on the /backends page). If we ever need to add links to external pages, the Knox service definition will likely need to be modified. This patch also adds hidden fields to all forms for the scheme, host, and port value, so that GET requests from forms will result in the same form as the transformed url shown above. Testing: - Ran the webserver and manually clicked around on a bunch of links to ensure everything works as expected. - Ran in a cluster and verified the new Knox service defintion works as intended with this change. - Added a test that uses a regex to check for template files that don't conform to the requirements. Change-Id: If1195709a0f21f39d9a1e484880a0c46c9967ed2 --- M be/src/service/impala-beeswax-server.cc M be/src/service/impala-hs2-server.cc M be/src/util/webserver.cc M be/src/util/webserver.h M bin/rat_exclude_files.txt M tests/common/file_utils.py M tests/webserver/test_web_pages.py M www/admission_controller.tmpl M www/catalog.tmpl M www/common-header.tmpl A www/form-hidden-inputs.tmpl M www/log_level.tmpl M www/queries.tmpl M www/query_detail_tabs.tmpl M www/query_profile.tmpl M www/sessions.tmpl M www/threadz.tmpl M www/threadz_tabs.tmpl 18 files changed, 117 insertions(+), 69 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/51/14151/5 -- To view, visit http://gerrit.cloudera.org:8080/14151 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If1195709a0f21f39d9a1e484880a0c46c9967ed2 Gerrit-Change-Number: 14151 Gerrit-PatchSet: 5 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules
Hello Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14151 to look at the new patch set (#6). Change subject: IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules .. IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules This patch adds the 'host:port' to all links on the webserver. This will facilitate proxying connections to the debug webui through Knox by allowing us to create rewrite rules that do the transform: ... => ... which allows us to have a single IMPALAUI Knox service that can proxy connections to any impalad/statestored/catalogd webui in a cluster. Note that this works because currently all of the links on Impala's webui are within the same webserver (it would also be possible to add links to other Impala daemon webuis within a cluster, eg. if we wanted to add webui links on the /backends page). If we ever need to add links to external pages, the Knox service definition will likely need to be modified. This patch also adds hidden fields to all forms for the scheme, host, and port value, so that GET requests from forms will result in the same form as the transformed url shown above. Testing: - Ran the webserver and manually clicked around on a bunch of links to ensure everything works as expected. - Ran in a cluster and verified the new Knox service defintion works as intended with this change. - Added a test that uses a regex to check for template files that don't conform to the requirements. Change-Id: If1195709a0f21f39d9a1e484880a0c46c9967ed2 --- M be/src/service/impala-beeswax-server.cc M be/src/service/impala-hs2-server.cc M be/src/util/webserver.cc M be/src/util/webserver.h M bin/rat_exclude_files.txt M tests/common/file_utils.py M tests/webserver/test_web_pages.py M www/admission_controller.tmpl M www/catalog.tmpl M www/common-header.tmpl A www/form-hidden-inputs.tmpl M www/log_level.tmpl M www/queries.tmpl M www/query_detail_tabs.tmpl M www/query_profile.tmpl M www/sessions.tmpl M www/threadz.tmpl M www/threadz_tabs.tmpl 18 files changed, 117 insertions(+), 69 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/51/14151/6 -- To view, visit http://gerrit.cloudera.org:8080/14151 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If1195709a0f21f39d9a1e484880a0c46c9967ed2 Gerrit-Change-Number: 14151 Gerrit-PatchSet: 6 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/14151 ) Change subject: IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules .. Patch Set 6: Code-Review+2 carrying forward -- To view, visit http://gerrit.cloudera.org:8080/14151 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If1195709a0f21f39d9a1e484880a0c46c9967ed2 Gerrit-Change-Number: 14151 Gerrit-PatchSet: 6 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 29 Aug 2019 05:13:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14151 ) Change subject: IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4872/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14151 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If1195709a0f21f39d9a1e484880a0c46c9967ed2 Gerrit-Change-Number: 14151 Gerrit-PatchSet: 6 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 29 Aug 2019 05:13:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14151 ) Change subject: IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules .. Patch Set 5: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/4430/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/14151 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If1195709a0f21f39d9a1e484880a0c46c9967ed2 Gerrit-Change-Number: 14151 Gerrit-PatchSet: 5 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 29 Aug 2019 05:49:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14151 ) Change subject: IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules .. Patch Set 4: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/4429/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/14151 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If1195709a0f21f39d9a1e484880a0c46c9967ed2 Gerrit-Change-Number: 14151 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 29 Aug 2019 05:45:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14151 ) Change subject: IMPALA-8897: Update debug webui to help with Apache Knox rewrite rules .. Patch Set 6: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/4431/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/14151 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If1195709a0f21f39d9a1e484880a0c46c9967ed2 Gerrit-Change-Number: 14151 Gerrit-PatchSet: 6 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 29 Aug 2019 05:51:59 + Gerrit-HasComments: No
[native-toolchain-CR] Add Python 3.7.4 to toolchain
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14161 Change subject: Add Python 3.7.4 to toolchain .. Add Python 3.7.4 to toolchain This will be useful to experiment with Python 3 on all of the platforms we support. Testing: Did a docker toolchain build. Change-Id: I3883fe2803670d896cc6ade642f3d3c4e83c53b0 --- M buildall.sh M docker/debian7.df M docker/debian8.df M docker/ubuntu1204.df M docker/ubuntu1404.df M docker/ubuntu1604.df M docker/ubuntu1804.df 7 files changed, 8 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/61/14161/1 -- To view, visit http://gerrit.cloudera.org:8080/14161 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I3883fe2803670d896cc6ade642f3d3c4e83c53b0 Gerrit-Change-Number: 14161 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong