[Impala-ASF-CR] IMPALA-7363: Fix buffer usage in ScannerContext::Stream::ReadVLong()
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11828 ) Change subject: IMPALA-7363: Fix buffer usage in ScannerContext::Stream::ReadVLong() .. IMPALA-7363: Fix buffer usage in ScannerContext::Stream::ReadVLong() Previously, the ReadVLong() function would read a byte and use it after subsequent calls to ReadBytes() function. As a result, the value was invalidated depending on the contents of the boundary_buffer_. This change ensures that all the functions on the buffer are invoked before subsequent calls to ReadBytes(). Testing: Enabled test_tpch_scan_ranges() for sequence files and ran it multiple times to ensure it always produces the desired result. Manually verified the buffer usage of other calls to ReadBytes() and GetBytes(). Change-Id: Ic2b2ffe4b1d67c63cfcea2baedbeff48e65ca417 Reviewed-on: http://gerrit.cloudera.org:8080/11828 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/exec/scanner-context.h M be/src/exec/scanner-context.inline.h M tests/query_test/test_scanners.py 3 files changed, 11 insertions(+), 11 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/11828 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ic2b2ffe4b1d67c63cfcea2baedbeff48e65ca417 Gerrit-Change-Number: 11828 Gerrit-PatchSet: 7 Gerrit-Owner: Pooja Nilangekar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7363: Fix buffer usage in ScannerContext::Stream::ReadVLong()
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11828 ) Change subject: IMPALA-7363: Fix buffer usage in ScannerContext::Stream::ReadVLong() .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11828 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic2b2ffe4b1d67c63cfcea2baedbeff48e65ca417 Gerrit-Change-Number: 11828 Gerrit-PatchSet: 6 Gerrit-Owner: Pooja Nilangekar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 01 Nov 2018 04:41:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5950: fix TPC-DS Q35a and Q48 queries
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11833 ) Change subject: IMPALA-5950: fix TPC-DS Q35a and Q48 queries .. Patch Set 7: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11833 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic7c737f68adf616738d6eb6e5a02593af25bcbaf Gerrit-Change-Number: 11833 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 01 Nov 2018 03:54:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5950: fix TPC-DS Q35a and Q48 queries
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11833 ) Change subject: IMPALA-5950: fix TPC-DS Q35a and Q48 queries .. Patch Set 6: Code-Review+2 Greg set me straight - he is a committer (not sure how I didn't know that). -- To view, visit http://gerrit.cloudera.org:8080/11833 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic7c737f68adf616738d6eb6e5a02593af25bcbaf Gerrit-Change-Number: 11833 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 01 Nov 2018 03:54:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5950: fix TPC-DS Q35a and Q48 queries
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11833 ) Change subject: IMPALA-5950: fix TPC-DS Q35a and Q48 queries .. Patch Set 7: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3399/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11833 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic7c737f68adf616738d6eb6e5a02593af25bcbaf Gerrit-Change-Number: 11833 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 01 Nov 2018 03:54:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 ) Change subject: IMPALA-7792: fix disabling of ORC scans .. Patch Set 7: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609 Gerrit-Change-Number: 11835 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 01 Nov 2018 03:51:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 ) Change subject: IMPALA-7792: fix disabling of ORC scans .. Patch Set 7: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3398/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609 Gerrit-Change-Number: 11835 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 01 Nov 2018 03:51:50 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6323 Allow constant analytic window expressions.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11556 ) Change subject: IMPALA-6323 Allow constant analytic window expressions. .. Patch Set 7: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1244/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11556 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idf129026c45120e9470df601268863634037908c Gerrit-Change-Number: 11556 Gerrit-PatchSet: 7 Gerrit-Owner: Michal Ostrowski Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 01 Nov 2018 03:43:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 ) Change subject: IMPALA-7792: fix disabling of ORC scans .. Patch Set 6: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/3395/ -- To view, visit http://gerrit.cloudera.org:8080/11835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609 Gerrit-Change-Number: 11835 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 01 Nov 2018 03:14:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6323 Allow constant analytic window expressions.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11556 ) Change subject: IMPALA-6323 Allow constant analytic window expressions. .. Patch Set 7: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3397/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/11556 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idf129026c45120e9470df601268863634037908c Gerrit-Change-Number: 11556 Gerrit-PatchSet: 7 Gerrit-Owner: Michal Ostrowski Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 01 Nov 2018 03:11:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6323 Allow constant analytic window expressions.
Hello Michael Ho, Tim Armstrong, Impala Public Jenkins, Vuk Ercegovac, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11556 to look at the new patch set (#7). Change subject: IMPALA-6323 Allow constant analytic window expressions. .. IMPALA-6323 Allow constant analytic window expressions. The constraint imposed by IMPALA-1354 was artificial. If there are constant "partition by" expressions, simply drop them, they are no-ops. Constant "order by" expressions can be ignored as well, though in effect they should be accounted for as null expressions in the backend, with the effect that combine all rows in the same window (i.e. no window breaks). Change-Id: Idf129026c45120e9470df601268863634037908c --- M be/src/exec/analytic-eval-node.cc M fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java M fe/src/main/java/org/apache/impala/planner/AnalyticPlanner.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java M testdata/workloads/functional-planner/queries/PlannerTest/analytic-fns.test 5 files changed, 56 insertions(+), 26 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/56/11556/7 -- To view, visit http://gerrit.cloudera.org:8080/11556 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Idf129026c45120e9470df601268863634037908c Gerrit-Change-Number: 11556 Gerrit-PatchSet: 7 Gerrit-Owner: Michal Ostrowski Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-2566: Remove ability to access impala builtin cast functions directly
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11830 ) Change subject: IMPALA-2566: Remove ability to access impala builtin cast functions directly .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11830 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6f562607aaaf728fc417066565811bbd269baaa8 Gerrit-Change-Number: 11830 Gerrit-PatchSet: 4 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 01 Nov 2018 03:02:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-2566: Remove ability to access impala builtin cast functions directly
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11830 ) Change subject: IMPALA-2566: Remove ability to access impala builtin cast functions directly .. IMPALA-2566: Remove ability to access impala builtin cast functions directly Currently, all impala builtin cast functions are exposed to the user and can be used directly in sql statements. These methods are used internally for casting and sometimes require context not available when used directly. This patch removes direct access to those methods by throwing an analysis exception. Testing: Added relevant frontend tests. Change-Id: I6f562607aaaf728fc417066565811bbd269baaa8 Reviewed-on: http://gerrit.cloudera.org:8080/11830 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/analysis/CastExpr.java M fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java 3 files changed, 43 insertions(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/11830 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I6f562607aaaf728fc417066565811bbd269baaa8 Gerrit-Change-Number: 11830 Gerrit-PatchSet: 5 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7586: fix predicate pushdown of escaped strings
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11814 ) Change subject: IMPALA-7586: fix predicate pushdown of escaped strings .. Patch Set 7: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/3393/ -- To view, visit http://gerrit.cloudera.org:8080/11814 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I53d6e20dd48ab6837ddd325db8a9d49ee04fed28 Gerrit-Change-Number: 11814 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 01 Nov 2018 02:16:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7786: Start HMS and Sentry in debug mode in the minicluster
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11826 ) Change subject: IMPALA-7786: Start HMS and Sentry in debug mode in the minicluster .. IMPALA-7786: Start HMS and Sentry in debug mode in the minicluster This patch updates the HMS and Sentry run scripts to start HMS and Sentry in debug mode in the minicluster to make it easier to debug issues related to HMS and Sentry. HMS debug port: 30010 Sentry debug port: 30020 Testing: - Connected the debugger to both HMS and Sentry. Change-Id: I29b025cbde36ef398ea36fbe69eff26e27d93e48 Reviewed-on: http://gerrit.cloudera.org:8080/11826 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M testdata/bin/run-hive-server.sh M testdata/bin/run-sentry-service.sh 2 files changed, 4 insertions(+), 0 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/11826 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I29b025cbde36ef398ea36fbe69eff26e27d93e48 Gerrit-Change-Number: 11826 Gerrit-PatchSet: 7 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger
[Impala-ASF-CR] IMPALA-7786: Start HMS and Sentry in debug mode in the minicluster
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11826 ) Change subject: IMPALA-7786: Start HMS and Sentry in debug mode in the minicluster .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11826 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I29b025cbde36ef398ea36fbe69eff26e27d93e48 Gerrit-Change-Number: 11826 Gerrit-PatchSet: 6 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Thu, 01 Nov 2018 02:01:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7244: [DOCS] Remove unsupported format writer support
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11842 ) Change subject: IMPALA-7244: [DOCS] Remove unsupported format writer support .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/128/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/11842 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I363accf5f284d2a1535cea0652b2b579379b9588 Gerrit-Change-Number: 11842 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 01 Nov 2018 01:57:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7244: [DOCS] Remove unsupported format writer support
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11842 ) Change subject: IMPALA-7244: [DOCS] Remove unsupported format writer support .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/128/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/11842 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I363accf5f284d2a1535cea0652b2b579379b9588 Gerrit-Change-Number: 11842 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 01 Nov 2018 01:47:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7244: [DOCS] Remove unsupported format writer support
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11842 Change subject: IMPALA-7244: [DOCS] Remove unsupported format writer support .. IMPALA-7244: [DOCS] Remove unsupported format writer support - Added a "removed" note for the ALLOW_UNSUPPORTED_FORMATS and SEQ_COMPRESSION_MODE query options. - Will remove the above options from the docs at the next compatibility breaking release. Change-Id: I363accf5f284d2a1535cea0652b2b579379b9588 --- M docs/topics/impala_allow_unsupported_formats.xml M docs/topics/impala_avro.xml M docs/topics/impala_file_formats.xml M docs/topics/impala_seq_compression_mode.xml M docs/topics/impala_seqfile.xml M docs/topics/impala_txtfile.xml 6 files changed, 43 insertions(+), 148 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/11842/1 -- To view, visit http://gerrit.cloudera.org:8080/11842 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I363accf5f284d2a1535cea0652b2b579379b9588 Gerrit-Change-Number: 11842 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-5821: Add query with implicit casts to extended explain output.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11719 ) Change subject: IMPALA-5821: Add query with implicit casts to extended explain output. .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1243/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11719 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I55c3bdacc295137f66b2316a912fc347da30d6b0 Gerrit-Change-Number: 11719 Gerrit-PatchSet: 5 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 01 Nov 2018 01:18:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7363: Fix buffer usage in ScannerContext::Stream::ReadVLong()
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11828 ) Change subject: IMPALA-7363: Fix buffer usage in ScannerContext::Stream::ReadVLong() .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11828 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic2b2ffe4b1d67c63cfcea2baedbeff48e65ca417 Gerrit-Change-Number: 11828 Gerrit-PatchSet: 6 Gerrit-Owner: Pooja Nilangekar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 01 Nov 2018 00:45:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7363: Fix buffer usage in ScannerContext::Stream::ReadVLong()
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11828 ) Change subject: IMPALA-7363: Fix buffer usage in ScannerContext::Stream::ReadVLong() .. Patch Set 6: Hit the OpenJDK version issue -- To view, visit http://gerrit.cloudera.org:8080/11828 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic2b2ffe4b1d67c63cfcea2baedbeff48e65ca417 Gerrit-Change-Number: 11828 Gerrit-PatchSet: 6 Gerrit-Owner: Pooja Nilangekar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 01 Nov 2018 00:45:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7363: Fix buffer usage in ScannerContext::Stream::ReadVLong()
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11828 ) Change subject: IMPALA-7363: Fix buffer usage in ScannerContext::Stream::ReadVLong() .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3396/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11828 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic2b2ffe4b1d67c63cfcea2baedbeff48e65ca417 Gerrit-Change-Number: 11828 Gerrit-PatchSet: 6 Gerrit-Owner: Pooja Nilangekar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 01 Nov 2018 00:45:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7363: Fix buffer usage in ScannerContext::Stream::ReadVLong()
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11828 ) Change subject: IMPALA-7363: Fix buffer usage in ScannerContext::Stream::ReadVLong() .. Patch Set 5: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/3391/ -- To view, visit http://gerrit.cloudera.org:8080/11828 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic2b2ffe4b1d67c63cfcea2baedbeff48e65ca417 Gerrit-Change-Number: 11828 Gerrit-PatchSet: 5 Gerrit-Owner: Pooja Nilangekar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 01 Nov 2018 00:42:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5821: Add query with implicit casts to extended explain output.
Andrew Sherman has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/11719 ) Change subject: IMPALA-5821: Add query with implicit casts to extended explain output. .. IMPALA-5821: Add query with implicit casts to extended explain output. If explain_level is at 'extended' level or higher, then enhance the output from the explain command. (1) Show the analyzed sql in the explain header, this is the rewritten sql, which includes implicit casts, and literals are printed with a cast so that their type is visible. (2) When predicates are shown in the plan these are shown in the same format. The toSql() method can be called on a ParseNode tree to return the sql corresponding ot the tree. In the past toSQl() has been enhanced to print rewritten sql by partially overloading toSql() [with toSql(boolean)]. This current change requires changing toSQl() in many places as NumericLiteral can appear at different points in ia parse tree. To avoid many new fragile overloads of toSql() I added toSql(ToSqlOptions), where ToSqlOptions is an enum which controls the form of the Sql that is returned. This changes many files but is safer and means that any future options to toSql() can be added painlessly. If SHOW_IMPLICIT_CASTS is passed to toSql() then - in CastExpr print the implicit cast - in NumericLiteral print the literal with a cast to show the type Add a PlannerTestOption directive that will force the query text showing implicit casts to be included in the PLAN section of a .test file. The analyzed query text is wrapped at 80 characters. Note that the analyzed query cannot always be executed as queries rewritten to use LEFT SEMI JOIN are not legal sql. In addition some space characters may be removed from the query for prettier display. Documentation of this change will be done as IMPALA-7718 EXAMPLE OUTPUT: [localhost:21000] default> set explain_level=2; EXPLAIN_LEVEL set to 2 [localhost:21000] default> explain select * from functional_kudu.alltypestiny where bigint_col < 1000 / 100; Query: explain select * from functional_kudu.alltypestiny where bigint_col < 1000 / 100 Max Per-Host Resource Reservation: Memory=0B Threads=2 Per-Host Resource Estimates: Memory=10MB Codegen disabled by planner Analyzed query: SELECT * FROM functional_kudu.alltypestiny WHERE CAST(bigint_col AS DOUBLE) < CAST(10 AS DOUBLE) "" F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | Per-Host Resources: mem-estimate=4.88MB mem-reservation=0B thread-reservation=2 PLAN-ROOT SINK | mem-estimate=0B mem-reservation=0B thread-reservation=0 | 00:SCAN KUDU [functional_kudu.alltypestiny] predicates: CAST(bigint_col AS DOUBLE) < CAST(10 AS DOUBLE) mem-estimate=4.88MB mem-reservation=0B thread-reservation=1 tuple-ids=0 row-size=97B cardinality=1 in pipelines: 00(GETNEXT) Fetched 16 row(s) in 0.03s TESTING: All end-to-end tests pass. Added a new test in ExprRewriterTest which prints sql with implict casts for some interesting queries. Add a unit test for the code which wraps text at 60 characters. The output of some Planner Tests in .test files has been updated to include the Analyzed sql that is printed when explain_level is at at least 'extended' level. Change-Id: I55c3bdacc295137f66b2316a912fc347da30d6b0 --- M fe/src/main/java/org/apache/impala/analysis/AdminFnStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableAddDropRangePartitionStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableAddPartitionStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableDropPartitionStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterViewStmt.java M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java M fe/src/main/java/org/apache/impala/analysis/AnalyticWindow.java M fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java M fe/src/main/java/org/apache/impala/analysis/BaseTableRef.java M fe/src/main/java/org/apache/impala/analysis/BetweenPredicate.java M fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java M fe/src/main/java/org/apache/impala/analysis/BoolLiteral.java M fe/src/main/java/org/apache/impala/analysis/CaseExpr.java M fe/src/main/java/org/apache/impala/analysis/CastExpr.java M fe/src/main/java/org/apache/impala/analysis/CompoundPredicate.java M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateDataSrcStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateDbStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateDropRoleStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateOrAlterViewStmtBase.java M fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeFileStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java M fe/src
[Impala-ASF-CR] IMPALA-5821: Add query with implicit casts to extended explain output.
Andrew Sherman has posted comments on this change. (
http://gerrit.cloudera.org:8080/11719 )
Change subject: IMPALA-5821: Add query with implicit casts to extended explain
output.
..
Patch Set 4:
(21 comments)
Thanks for review comments
http://gerrit.cloudera.org:8080/#/c/11719/4//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/11719/4//COMMIT_MSG@51
PS4, Line 51: ""
> This quote seems strange here - is it included in the output?
This is what you get when you run 'impala-shell.sh -B'
http://gerrit.cloudera.org:8080/#/c/11719/4//COMMIT_MSG@69
PS4, Line 69: 80
> Isn't it 60?
yes
http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/analysis/ParseNode.java
File fe/src/main/java/org/apache/impala/analysis/ParseNode.java:
http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/analysis/ParseNode.java@32
PS4, Line 32: sql
> nit: pls use consistent case (SQL, sql, Sql)
Thanks
http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/analysis/ParseNode.java@32
PS4, Line 32: @param
> pls omit these javadoc annotations.
Done
http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/analysis/ParseNode.java@39
PS4, Line 39:* This should return the same result as calling
toSql(ToSqlOptions.DEFAULT).
> would be good to do this via a default interface method. pls add a todo for
Clever idea
http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/analysis/ToSqlOptions.java
File fe/src/main/java/org/apache/impala/analysis/ToSqlOptions.java:
http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/analysis/ToSqlOptions.java@25
PS4, Line 25: default, normal
> "normal" makes sense if you know what is currently done. perhaps "original
Done
http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/analysis/ToSqlOptions.java@38
PS4, Line 38: // This does have the consequence that the sql with implict
casts may not pssibly fail
> nit: spelling
Done
http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/analysis/ToSqlOptions.java@38
PS4, Line 38: may not pssibly fail
: // to parse if resubmitted as
> I found this difficult to understand. Did you mean that Sql produced in thi
Yes, rewritten SQL is already not always valid SQL.
EXISTS queries that are rewritten as semi-joins are not legal SQL.
I clarified the comment slightly
http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/analysis/TypeDef.java
File fe/src/main/java/org/apache/impala/analysis/TypeDef.java:
http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/analysis/TypeDef.java@165
PS4, Line 165: return parsedType_.toSql();
> pass down here?
It looks like it should, but the parsedType is not a ParseNode so I did not
propagate ToSqlOptions to there.
http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/common/PrintUtils.java
File fe/src/main/java/org/apache/impala/common/PrintUtils.java:
http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/common/PrintUtils.java@110
PS4, Line 110: 32
> what's this?
Just some padding for if lines expand. Maybe this is too ugly.
http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/common/PrintUtils.java@115
PS4, Line 115: exiting
> sp. existing?
Done
http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/test/java/org/apache/impala/analysis/ExprRewriterTest.java
File fe/src/test/java/org/apache/impala/analysis/ExprRewriterTest.java:
http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/test/java/org/apache/impala/analysis/ExprRewriterTest.java@424
PS4, Line 424: @pa
> remove
I don't really understand what is wrong with param?
But done.
http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/test/java/org/apache/impala/analysis/ExprRewriterTest.java@451
PS4, Line 451: // AnalyzesOk(stmt.toSql(true), ctx);
> rm?
Done
http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java
File fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java:
http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java@750
PS4, Line 750: @para
> remove the comment annotations.
Done
http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java@772
PS4, Line 772: if (line.contains("Analyzed query:")) {
: builder.append(line).append("\n");
: inImplictCasts = true;
: } else if (inImplictCasts) {
: // Keep copying the query with implicit casts.
: // This works because this is the last thing in the
header.
: builder.append(line).
[Impala-ASF-CR] IMPALA-5050: Add support to read TIMESTAMP MILLIS and TIMESTAMP MICROS from Parquet
Tim Armstrong has posted comments on this change. (
http://gerrit.cloudera.org:8080/11057 )
Change subject: IMPALA-5050: Add support to read TIMESTAMP_MILLIS and
TIMESTAMP_MICROS from Parquet
..
Patch Set 18:
(6 comments)
http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/exec/parquet-column-readers.cc
File be/src/exec/parquet-column-readers.cc:
http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/exec/parquet-column-readers.cc@768
PS18, Line 768: if (UNLIKELY(!TimestampValue::IsValidDate(val->date( {
A short comment hinting at why IsValidTime() is not needed might be useful for
future readers - otherwise it looks asymmetrical with above. E.g
// Time extracted from int64 unix timestamp is always in range therefore does
not need to be validated.
http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/exec/parquet-common.h
File be/src/exec/parquet-common.h:
http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/exec/parquet-common.h@433
PS18, Line 433: ParquetTimestampHelper
The name is ok but a little vague. I wonder if ParquetTimestampDecoder would be
clearer.
http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/exec/parquet-common.cc
File be/src/exec/parquet-common.cc:
http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/exec/parquet-common.cc@114
PS18, Line 114: if (needs_conversion) {
nit: can fit conditional on one line
http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/exec/parquet-common.cc@123
PS18, Line 123: v->UtcToLocal(*timezone_, &repeated_period_start);
Ah, this is clever.
http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/util/dict-encoding.h
File be/src/util/dict-encoding.h:
http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/util/dict-encoding.h@383
PS18, Line 383: inline
nit: inline is already implied for functions in the class body
http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/util/dict-encoding.h@383
PS18, Line 383: ResetInner
I feel like Decode() or DecodeHelper() would be clearer, since it describes
what the function does.
--
To view, visit http://gerrit.cloudera.org:8080/11057
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4c7c01fffa31b3d2ca3480adf6ff851137dadac3
Gerrit-Change-Number: 11057
Gerrit-PatchSet: 18
Gerrit-Owner: Csaba Ringhofer
Gerrit-Reviewer: Attila Jeges
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Thu, 01 Nov 2018 00:08:58 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6658: improve Parquet RLE for low bit widths
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11582 ) Change subject: IMPALA-6658: improve Parquet RLE for low bit widths .. Patch Set 8: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1242/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11582 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I191a581d3f699b6669e48ac9dc39c76ed77c4a76 Gerrit-Change-Number: 11582 Gerrit-PatchSet: 8 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 01 Nov 2018 00:07:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7655: Rewrite if, isnull to use CASE
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/11760 ) Change subject: IMPALA-7655: Rewrite if, isnull to use CASE .. Patch Set 11: (4 comments) http://gerrit.cloudera.org:8080/#/c/11760/11//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11760/11//COMMIT_MSG@42 PS11, Line 42: While the desire was to replace the BE implementation of if and isnull, This section summarizes limitations. They are: (1) no guarantee that the fns are removed (list dependent bugs, disabled rewrites), (2) missed simplifications (list dependent bugs). I think this can be made more concise. http://gerrit.cloudera.org:8080/#/c/11760/11//COMMIT_MSG@45 PS11, Line 45: ORDER BY clause, and 2) if the user disables rewrites. prior, you mentioned a potential performance regression-- I think I saw an example of this in the current tests, pls confirm. http://gerrit.cloudera.org:8080/#/c/11760/11//COMMIT_MSG@50 PS11, Line 50: Coalesce() is implemented, but disabled. The old rewrites are retained : until b as mentioned in the source files, I'd remove this. http://gerrit.cloudera.org:8080/#/c/11760/11/testdata/workloads/functional-planner/queries/PlannerTest/analytic-fns.test File testdata/workloads/functional-planner/queries/PlannerTest/analytic-fns.test: http://gerrit.cloudera.org:8080/#/c/11760/11/testdata/workloads/functional-planner/queries/PlannerTest/analytic-fns.test@2238 PS11, Line 2238: int_col did you want to mention the repeated sub-expression evaluation for examples like this (int_col could be an expression)? -- To view, visit http://gerrit.cloudera.org:8080/11760 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I526654d8546e17b2545c42cc59dab66d9fe1b163 Gerrit-Change-Number: 11760 Gerrit-PatchSet: 11 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 31 Oct 2018 23:53:24 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 ) Change subject: IMPALA-7792: fix disabling of ORC scans .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1241/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609 Gerrit-Change-Number: 11835 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 23:44:24 + Gerrit-HasComments: No
[Impala-ASF-CR] Detect bad OpenJDK version and install last known good
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/11838 ) Change subject: Detect bad OpenJDK version and install last known good .. Patch Set 7: > Are we sure that this particular version is good? I think we may > need "b11" The pre-commit job passed so I think whatever we were hitting is fixed. What makes you think this particular version is bad? -- To view, visit http://gerrit.cloudera.org:8080/11838 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I39b786c37ad836aae5a9fdfee3187303c9536956 Gerrit-Change-Number: 11838 Gerrit-PatchSet: 7 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Wed, 31 Oct 2018 23:44:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6658: improve Parquet RLE for low bit widths
Andrew Sherman has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/11582 ) Change subject: IMPALA-6658: improve Parquet RLE for low bit widths .. IMPALA-6658: improve Parquet RLE for low bit widths RleEncoder buffers values in its own cache to detect run lengths that can be efficiently encoded. When a run is detected it is written with an indicator byte which encodes the length of the run. So an encoded run always has an overhead of at least one byte. This means that for single bit values, encoding 8 values as a run is inefficient. Change RleEncoder to have the ability to use run lengths other than 8. A new parameter to the constructor (min_run_length) allows test callers (only) to set the minimum run length. By default RleEncoder will now use run length encoding for runs of length 16 for single bit values. All other bit widths will use the existing length 8 runs. Internally RleEncoder must buffer more values so that the longer runs can be detected. The internal buffer “buffered_values_” is larger and is now a circular buffer so that the first 8 bytes of the buffer can be separately flushed to BitWriter. Testing: All end-to-end and unit tests pass The unit test rle-test is enhanced to run all tests against RleEncoders using all possible values of min_run_length. In Addition, rle-test is refactored so that the Rle tests are in a class that inherits from ::testing::Test so that a SetUp() method can be used. The Overflow test is enhanced to be more exhaustive (while still completing in a second or two). Change-Id: I191a581d3f699b6669e48ac9dc39c76ed77c4a76 --- M be/src/util/rle-encoding.h M be/src/util/rle-test.cc 2 files changed, 582 insertions(+), 256 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/11582/8 -- To view, visit http://gerrit.cloudera.org:8080/11582 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I191a581d3f699b6669e48ac9dc39c76ed77c4a76 Gerrit-Change-Number: 11582 Gerrit-PatchSet: 8 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-6658: improve Parquet RLE for low bit widths
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/11582 ) Change subject: IMPALA-6658: improve Parquet RLE for low bit widths .. Patch Set 4: (2 comments) http://gerrit.cloudera.org:8080/#/c/11582/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11582/2//COMMIT_MSG@20 PS2, Line 20: length 16 for single bit values. All other bit widths will use the : existing length 8 runs. : > Yes, it looks good, but I think that the worst case (alternating L8 R16) wo OK http://gerrit.cloudera.org:8080/#/c/11582/4/be/src/util/rle-encoding.h File be/src/util/rle-encoding.h: http://gerrit.cloudera.org:8080/#/c/11582/4/be/src/util/rle-encoding.h@265 PS4, Line 265: return std::max(MinBufferSize(bit_width), literal_max_size); > I think that is still not always the worst case - the example you wrote for You are right, and I have updated MaxBufferSize (and added a test) in next patch -- To view, visit http://gerrit.cloudera.org:8080/11582 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I191a581d3f699b6669e48ac9dc39c76ed77c4a76 Gerrit-Change-Number: 11582 Gerrit-PatchSet: 4 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Wed, 31 Oct 2018 23:36:13 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 ) Change subject: IMPALA-7792: fix disabling of ORC scans .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3395/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609 Gerrit-Change-Number: 11835 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 23:31:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-2566: Remove ability to access impala builtin cast functions directly
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11830 ) Change subject: IMPALA-2566: Remove ability to access impala builtin cast functions directly .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1240/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11830 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6f562607aaaf728fc417066565811bbd269baaa8 Gerrit-Change-Number: 11830 Gerrit-PatchSet: 3 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 23:31:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 ) Change subject: IMPALA-7792: fix disabling of ORC scans .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609 Gerrit-Change-Number: 11835 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 23:31:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7655: Rewrite if, isnull to use CASE
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/11760 ) Change subject: IMPALA-7655: Rewrite if, isnull to use CASE .. Patch Set 11: (13 comments) http://gerrit.cloudera.org:8080/#/c/11760/11/fe/src/main/java/org/apache/impala/rewrite/RewriteConditionalFnsRule.java File fe/src/main/java/org/apache/impala/rewrite/RewriteConditionalFnsRule.java: http://gerrit.cloudera.org:8080/#/c/11760/11/fe/src/main/java/org/apache/impala/rewrite/RewriteConditionalFnsRule.java@39 PS11, Line 39: coalesce(v1, v2, ...) pls remove. here's a good place to put one todo for the coalesce work. http://gerrit.cloudera.org:8080/#/c/11760/11/fe/src/main/java/org/apache/impala/rewrite/RewriteConditionalFnsRule.java@43 PS11, Line 43: nullif where's this handled? http://gerrit.cloudera.org:8080/#/c/11760/11/fe/src/main/java/org/apache/impala/rewrite/RewriteConditionalFnsRule.java@65 PS11, Line 65: coalesce stale. http://gerrit.cloudera.org:8080/#/c/11760/11/fe/src/main/java/org/apache/impala/rewrite/RewriteConditionalFnsRule.java@99 PS11, Line 99: // IMPALA-7793: CASE statement does not handle NULL from UDF overflow : // Thus, CASE cannot be made to act as coalesce() when handling Decimal : // overflow, so can't do the rewrite. : //case "coalesce": : // return rewriteCoalesceFn(expr); lets get rid of this. example changes for coalesce can be revived from this review. http://gerrit.cloudera.org:8080/#/c/11760/11/fe/src/main/java/org/apache/impala/rewrite/RewriteConditionalFnsRule.java@164 PS11, Line 164: rewriteCoalesceFn pls remove. http://gerrit.cloudera.org:8080/#/c/11760/10/fe/src/main/java/org/apache/impala/rewrite/SimplifyConditionalsRule.java File fe/src/main/java/org/apache/impala/rewrite/SimplifyConditionalsRule.java: http://gerrit.cloudera.org:8080/#/c/11760/10/fe/src/main/java/org/apache/impala/rewrite/SimplifyConditionalsRule.java@217 PS10, Line 217: // Note: retaining the current simplification because of bug: : // IMPALA-7793: CASE statement does not handle NULL from UDF overflow : // Which makes it impossible to rewrite coalesce() to CASE in : // RewriteConditionalFnsRule without loss of functionality. : // pls remove and optionally put this as a todo with the new rewrites. when coalesce is properly removed, removing this will follow. http://gerrit.cloudera.org:8080/#/c/11760/11/fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java File fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java: http://gerrit.cloudera.org:8080/#/c/11760/11/fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java@a587 PS11, Line 587: why are these removed? http://gerrit.cloudera.org:8080/#/c/11760/11/fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java@296 PS11, Line 296: // Note: retaining the current simplification because of bug: : // IMPALA-7793: CASE statement does not handle NULL from UDF overflow : // Which makes it impossible to rewrite coalesce() to CASE in : // RewriteConditionalFnsRule without loss of functionality. : // This implementation is temporary until that bug is fixed. : // Once it is, remove these tests and enable those in : // RewriteConditionalFnsRuleTest and FullRewriteTest. pls remove. this is more of a note for the reviewer and not necessary to stick with the code as currently written. http://gerrit.cloudera.org:8080/#/c/11760/11/fe/src/test/java/org/apache/impala/analysis/FullRewriteTest.java File fe/src/test/java/org/apache/impala/analysis/FullRewriteTest.java: http://gerrit.cloudera.org:8080/#/c/11760/11/fe/src/test/java/org/apache/impala/analysis/FullRewriteTest.java@37 PS11, Line 37: r coalesce since its rewrite handles all the : * required simplification. stale http://gerrit.cloudera.org:8080/#/c/11760/11/fe/src/test/java/org/apache/impala/analysis/FullRewriteTest.java@119 PS11, Line 119: //v thank you for including tests that should work for bugs that will be fixed later. nit: pls turn these into block comments so that they are not at the start of the line. http://gerrit.cloudera.org:8080/#/c/11760/11/fe/src/test/java/org/apache/impala/analysis/FullRewriteTest.java@163 PS11, Line 163: i nit: capitalize http://gerrit.cloudera.org:8080/#/c/11760/11/fe/src/test/java/org/apache/impala/analysis/FullRewriteTest.java@223 PS11, Line 223: TestNullif() what's going on with the duplication with the test on L240? http://gerrit.cloudera.org:8080/#/c/11760/11/fe/src/test/java/org/apache/impala/analysis/FullRewriteTest.java@257 PS11, Line 257: Where were you planning to have a standard battery of expressions that you then apply to different parts of the statement? it does not look
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Bharath Vissapragada has posted comments on this change. (
http://gerrit.cloudera.org:8080/11835 )
Change subject: IMPALA-7792: fix disabling of ORC scans
..
Patch Set 5: Code-Review+2
(2 comments)
http://gerrit.cloudera.org:8080/#/c/11835/4/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
File fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java:
http://gerrit.cloudera.org:8080/#/c/11835/4/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@414
PS4, Line 414: if (!BackendConfig.INSTANCE.isOrcScannerEnabled()) {
: for (FeFsPartition part: partitions_) {
: if (part.getInputFormatDescriptor().getFileFormat() ==
HdfsFileFormat.ORC) {
: throw new NotImplementedException(
: "ORC scans are disabled by --enable_orc_scanner
flag");
: }
: }
: }
> I should have removed the block below. The below check doesn't work because
ah, ya!
http://gerrit.cloudera.org:8080/#/c/11835/4/tests/custom_cluster/test_disable_features.py
File tests/custom_cluster/test_disable_features.py:
http://gerrit.cloudera.org:8080/#/c/11835/4/tests/custom_cluster/test_disable_features.py@32
PS4, Line 32: vector
> I copied this from a different custom cluster test. Are you saying that it'
wfm, I think even with vector it only runs a specific dimension and not all
combinations like in a normal e-e test.
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Philip Zeyliger
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Wed, 31 Oct 2018 23:28:07 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-2566: Remove ability to access impala builtin cast functions directly
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11830 ) Change subject: IMPALA-2566: Remove ability to access impala builtin cast functions directly .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11830 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6f562607aaaf728fc417066565811bbd269baaa8 Gerrit-Change-Number: 11830 Gerrit-PatchSet: 4 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 23:11:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-2566: Remove ability to access impala builtin cast functions directly
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11830 ) Change subject: IMPALA-2566: Remove ability to access impala builtin cast functions directly .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1239/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11830 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6f562607aaaf728fc417066565811bbd269baaa8 Gerrit-Change-Number: 11830 Gerrit-PatchSet: 2 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 23:22:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5950: fix TPC-DS Q35a and Q48 queries
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11833 ) Change subject: IMPALA-5950: fix TPC-DS Q35a and Q48 queries .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1238/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11833 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic7c737f68adf616738d6eb6e5a02593af25bcbaf Gerrit-Change-Number: 11833 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 23:17:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Tim Armstrong has posted comments on this change. (
http://gerrit.cloudera.org:8080/11835 )
Change subject: IMPALA-7792: fix disabling of ORC scans
..
Patch Set 4:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/11835/4/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
File fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java:
http://gerrit.cloudera.org:8080/#/c/11835/4/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@414
PS4, Line 414: if (!BackendConfig.INSTANCE.isOrcScannerEnabled()) {
: for (FeFsPartition part: partitions_) {
: if (part.getInputFormatDescriptor().getFileFormat() ==
HdfsFileFormat.ORC) {
: throw new NotImplementedException(
: "ORC scans are disabled by --enable_orc_scanner
flag");
: }
: }
: }
> Do we need this block? Shouldn't the check below inside the loop work?
I should have removed the block below. The below check doesn't work because we
bail out early if there's no complex typed column.
http://gerrit.cloudera.org:8080/#/c/11835/4/tests/custom_cluster/test_disable_features.py
File tests/custom_cluster/test_disable_features.py:
http://gerrit.cloudera.org:8080/#/c/11835/4/tests/custom_cluster/test_disable_features.py@32
PS4, Line 32: vector
> nit: Not sure if the intention here was to run with multiple dimensions but
I copied this from a different custom cluster test. Are you saying that it's
unnecessary? I'd totally believe we just copied this blindly. It seems to work
fine though :)
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Philip Zeyliger
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Wed, 31 Oct 2018 23:16:05 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Hello Bharath Vissapragada, Philip Zeyliger, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11835 to look at the new patch set (#5). Change subject: IMPALA-7792: fix disabling of ORC scans .. IMPALA-7792: fix disabling of ORC scans The previous approach could lead to hangs or cryptic error messages because it removed the ORC data type from a lookup table. Instead check explicitly in the planner for ORC scans and throw a more helpful error message. Testing: Added custom cluster test to exercise code and check error message. Change-Id: I209e79b18745c48d0182800a916d6566083f4609 --- M fe/src/main/java/org/apache/impala/catalog/HdfsFileFormat.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java A testdata/workloads/functional-query/queries/QueryTest/disable-orc-scanner.test A tests/custom_cluster/test_disable_features.py 5 files changed, 71 insertions(+), 20 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/11835/5 -- To view, visit http://gerrit.cloudera.org:8080/11835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609 Gerrit-Change-Number: 11835 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-2566: Remove ability to access impala builtin cast functions directly
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11830 ) Change subject: IMPALA-2566: Remove ability to access impala builtin cast functions directly .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3394/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11830 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6f562607aaaf728fc417066565811bbd269baaa8 Gerrit-Change-Number: 11830 Gerrit-PatchSet: 4 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 23:11:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Bharath Vissapragada has posted comments on this change. (
http://gerrit.cloudera.org:8080/11835 )
Change subject: IMPALA-7792: fix disabling of ORC scans
..
Patch Set 4:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/11835/4/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
File fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java:
http://gerrit.cloudera.org:8080/#/c/11835/4/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@414
PS4, Line 414: if (!BackendConfig.INSTANCE.isOrcScannerEnabled()) {
: for (FeFsPartition part: partitions_) {
: if (part.getInputFormatDescriptor().getFileFormat() ==
HdfsFileFormat.ORC) {
: throw new NotImplementedException(
: "ORC scans are disabled by --enable_orc_scanner
flag");
: }
: }
: }
Do we need this block? Shouldn't the check below inside the loop work?
http://gerrit.cloudera.org:8080/#/c/11835/4/tests/custom_cluster/test_disable_features.py
File tests/custom_cluster/test_disable_features.py:
http://gerrit.cloudera.org:8080/#/c/11835/4/tests/custom_cluster/test_disable_features.py@32
PS4, Line 32: vector
nit: Not sure if the intention here was to run with multiple dimensions but I
think custom cluster by default limits to only specific set of dimensions (I
could be wrong on this).
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Philip Zeyliger
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Wed, 31 Oct 2018 23:11:04 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-2566: Remove ability to access impala builtin cast functions directly
Hello Tianyi Wang, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11830 to look at the new patch set (#3). Change subject: IMPALA-2566: Remove ability to access impala builtin cast functions directly .. IMPALA-2566: Remove ability to access impala builtin cast functions directly Currently, all impala builtin cast functions are exposed to the user and can be used directly in sql statements. These methods are used internally for casting and sometimes require context not available when used directly. This patch removes direct access to those methods by throwing an analysis exception. Testing: Added relevant frontend tests. Change-Id: I6f562607aaaf728fc417066565811bbd269baaa8 --- M fe/src/main/java/org/apache/impala/analysis/CastExpr.java M fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java 3 files changed, 43 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/30/11830/3 -- To view, visit http://gerrit.cloudera.org:8080/11830 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6f562607aaaf728fc417066565811bbd269baaa8 Gerrit-Change-Number: 11830 Gerrit-PatchSet: 3 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-2566: Remove ability to access impala builtin cast functions directly
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/11830 ) Change subject: IMPALA-2566: Remove ability to access impala builtin cast functions directly .. Patch Set 3: Code-Review+2 carrying over +2 -- To view, visit http://gerrit.cloudera.org:8080/11830 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6f562607aaaf728fc417066565811bbd269baaa8 Gerrit-Change-Number: 11830 Gerrit-PatchSet: 3 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 23:10:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-2566: Remove ability to access impala builtin cast functions directly
Bikramjeet Vig has posted comments on this change. (
http://gerrit.cloudera.org:8080/11830 )
Change subject: IMPALA-2566: Remove ability to access impala builtin cast
functions directly
..
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/11830/2/fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java
File fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java:
http://gerrit.cloudera.org:8080/#/c/11830/2/fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java@184
PS2, Line 184: private boolean isBuiltinCastFunction(){
> space before {
Done
--
To view, visit http://gerrit.cloudera.org:8080/11830
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6f562607aaaf728fc417066565811bbd269baaa8
Gerrit-Change-Number: 11830
Gerrit-PatchSet: 2
Gerrit-Owner: Bikramjeet Vig
Gerrit-Reviewer: Bikramjeet Vig
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tianyi Wang
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Wed, 31 Oct 2018 23:09:20 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 ) Change subject: IMPALA-7792: fix disabling of ORC scans .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1237/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609 Gerrit-Change-Number: 11835 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 23:09:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-2566: Remove ability to access impala builtin cast functions directly
Tim Armstrong has posted comments on this change. (
http://gerrit.cloudera.org:8080/11830 )
Change subject: IMPALA-2566: Remove ability to access impala builtin cast
functions directly
..
Patch Set 2: Code-Review+2
(1 comment)
http://gerrit.cloudera.org:8080/#/c/11830/2/fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java
File fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java:
http://gerrit.cloudera.org:8080/#/c/11830/2/fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java@184
PS2, Line 184: private boolean isBuiltinCastFunction(){
space before {
--
To view, visit http://gerrit.cloudera.org:8080/11830
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6f562607aaaf728fc417066565811bbd269baaa8
Gerrit-Change-Number: 11830
Gerrit-PatchSet: 2
Gerrit-Owner: Bikramjeet Vig
Gerrit-Reviewer: Bikramjeet Vig
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tianyi Wang
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Wed, 31 Oct 2018 22:43:33 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6249: Expose several build flags via web UI
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11410 ) Change subject: IMPALA-6249: Expose several build flags via web UI .. Patch Set 12: According to JIRA IMPALA-7691 is a blocker for 3.1. Do we plan to move this review along or should we change the target for IMPALA-7691? -- To view, visit http://gerrit.cloudera.org:8080/11410 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I47e3ad4cbf844909bdaf22a6f9d7bd915dce3f19 Gerrit-Change-Number: 11410 Gerrit-PatchSet: 12 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 22:36:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-2566: Remove ability to access impala builtin cast functions directly
Hello Tianyi Wang, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11830 to look at the new patch set (#2). Change subject: IMPALA-2566: Remove ability to access impala builtin cast functions directly .. IMPALA-2566: Remove ability to access impala builtin cast functions directly Currently, all impala builtin cast functions are exposed to the user and can be used directly in sql statements. These methods are used internally for casting and sometimes require context not available when used directly. This patch removes direct access to those methods by throwing an analysis exception. Testing: Added relevant frontend tests. Change-Id: I6f562607aaaf728fc417066565811bbd269baaa8 --- M fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java 2 files changed, 38 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/30/11830/2 -- To view, visit http://gerrit.cloudera.org:8080/11830 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6f562607aaaf728fc417066565811bbd269baaa8 Gerrit-Change-Number: 11830 Gerrit-PatchSet: 2 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 ) Change subject: IMPALA-7792: fix disabling of ORC scans .. Patch Set 4: I can review this one. Lemme dig into the previous approach. -- To view, visit http://gerrit.cloudera.org:8080/11835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609 Gerrit-Change-Number: 11835 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 22:32:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-2566: Remove ability to access impala builtin cast functions directly
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/11830 ) Change subject: IMPALA-2566: Remove ability to access impala builtin cast functions directly .. Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/11830/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11830/1//COMMIT_MSG@15 PS1, Line 15: > It's weird that we're still showing all these functions in _impala_builtins As discussed offline, we are going ahead with this solution due the extent of invasive changes required for the alternatives. http://gerrit.cloudera.org:8080/#/c/11830/1/fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java File fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java: http://gerrit.cloudera.org:8080/#/c/11830/1/fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java@500 PS1, Line 500: Impala builtin cast functions as reserved for internal use only > Maybe something like: Done http://gerrit.cloudera.org:8080/#/c/11830/1/fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java File fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java: http://gerrit.cloudera.org:8080/#/c/11830/1/fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java@3962 PS1, Line 3962: String expectedError = "Impala builtin cast functions as reserved for internal use only"; > line too long (93 > 90) Done -- To view, visit http://gerrit.cloudera.org:8080/11830 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6f562607aaaf728fc417066565811bbd269baaa8 Gerrit-Change-Number: 11830 Gerrit-PatchSet: 1 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 22:32:32 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5950: fix TPC-DS Q35a and Q48 queries
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11833 ) Change subject: IMPALA-5950: fix TPC-DS Q35a and Q48 queries .. Patch Set 6: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/11833 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic7c737f68adf616738d6eb6e5a02593af25bcbaf Gerrit-Change-Number: 11833 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 22:22:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5950: fix TPC-DS Q35a and Q48 queries
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11833 ) Change subject: IMPALA-5950: fix TPC-DS Q35a and Q48 queries .. Patch Set 6: Carry Greg's +2 as a +1 since he's not a committer (sorry Greg!) -- To view, visit http://gerrit.cloudera.org:8080/11833 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic7c737f68adf616738d6eb6e5a02593af25bcbaf Gerrit-Change-Number: 11833 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 22:22:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5950: fix TPC-DS Q35a and Q48 queries
Hello Greg Rahn, David Knupp, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11833 to look at the new patch set (#6). Change subject: IMPALA-5950: fix TPC-DS Q35a and Q48 queries .. IMPALA-5950: fix TPC-DS Q35a and Q48 queries The query text (for Q48) and substitution parameters didn't match the TPC-DS standard for qualification queries. After fixing that, the queries return the results expected by the TPC-DS standards. Note that this may affect the performance of perf workloads running tpcds-unmodified. Change-Id: Ic7c737f68adf616738d6eb6e5a02593af25bcbaf --- A testdata/workloads/tpcds-unmodified/queries/tpcds-q35a.test M testdata/workloads/tpcds-unmodified/queries/tpcds-q48.test M tests/query_test/test_tpcds_queries.py 3 files changed, 268 insertions(+), 32 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/11833/6 -- To view, visit http://gerrit.cloudera.org:8080/11833 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic7c737f68adf616738d6eb6e5a02593af25bcbaf Gerrit-Change-Number: 11833 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 ) Change subject: IMPALA-7792: fix disabling of ORC scans .. Patch Set 4: Any chance you could review this Bharath? I can provide any context if needed. -- To view, visit http://gerrit.cloudera.org:8080/11835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609 Gerrit-Change-Number: 11835 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 22:19:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 ) Change subject: IMPALA-7792: fix disabling of ORC scans .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/11835/3/tests/custom_cluster/test_disable_features.py File tests/custom_cluster/test_disable_features.py: http://gerrit.cloudera.org:8080/#/c/11835/3/tests/custom_cluster/test_disable_features.py@22 PS3, Line 22: > flake8: E302 expected 2 blank lines, found 1 Done -- To view, visit http://gerrit.cloudera.org:8080/11835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609 Gerrit-Change-Number: 11835 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 22:18:14 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Hello Philip Zeyliger, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11835 to look at the new patch set (#4). Change subject: IMPALA-7792: fix disabling of ORC scans .. IMPALA-7792: fix disabling of ORC scans The previous approach could lead to hangs or cryptic error messages because it removed the ORC data type from a lookup table. Instead check explicitly in the planner for ORC scans and throw a more helpful error message. Testing: Added custom cluster test to exercise code and check error message. Change-Id: I209e79b18745c48d0182800a916d6566083f4609 --- M fe/src/main/java/org/apache/impala/catalog/HdfsFileFormat.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java A testdata/workloads/functional-query/queries/QueryTest/disable-orc-scanner.test A tests/custom_cluster/test_disable_features.py 5 files changed, 78 insertions(+), 20 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/11835/4 -- To view, visit http://gerrit.cloudera.org:8080/11835 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609 Gerrit-Change-Number: 11835 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger
[Impala-ASF-CR] Detect bad OpenJDK version and install last known good
Michal Ostrowski has posted comments on this change. ( http://gerrit.cloudera.org:8080/11838 ) Change subject: Detect bad OpenJDK version and install last known good .. Patch Set 6: Are we sure that this particular version is good? I think we may need "b11" -- To view, visit http://gerrit.cloudera.org:8080/11838 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I39b786c37ad836aae5a9fdfee3187303c9536956 Gerrit-Change-Number: 11838 Gerrit-PatchSet: 6 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Wed, 31 Oct 2018 22:14:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7586: fix predicate pushdown of escaped strings
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11814 ) Change subject: IMPALA-7586: fix predicate pushdown of escaped strings .. Patch Set 7: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3393/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11814 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I53d6e20dd48ab6837ddd325db8a9d49ee04fed28 Gerrit-Change-Number: 11814 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 31 Oct 2018 22:14:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7586: fix predicate pushdown of escaped strings
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11814 ) Change subject: IMPALA-7586: fix predicate pushdown of escaped strings .. Patch Set 7: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11814 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I53d6e20dd48ab6837ddd325db8a9d49ee04fed28 Gerrit-Change-Number: 11814 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 31 Oct 2018 22:14:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7655: Rewrite if, isnull to use CASE
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11760 ) Change subject: IMPALA-7655: Rewrite if, isnull to use CASE .. Patch Set 11: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1236/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11760 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I526654d8546e17b2545c42cc59dab66d9fe1b163 Gerrit-Change-Number: 11760 Gerrit-PatchSet: 11 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 31 Oct 2018 22:12:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7786: Start HMS and Sentry in debug mode in the minicluster
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11826 ) Change subject: IMPALA-7786: Start HMS and Sentry in debug mode in the minicluster .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11826 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I29b025cbde36ef398ea36fbe69eff26e27d93e48 Gerrit-Change-Number: 11826 Gerrit-PatchSet: 6 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Wed, 31 Oct 2018 22:12:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7786: Start HMS and Sentry in debug mode in the minicluster
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11826 ) Change subject: IMPALA-7786: Start HMS and Sentry in debug mode in the minicluster .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3392/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11826 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I29b025cbde36ef398ea36fbe69eff26e27d93e48 Gerrit-Change-Number: 11826 Gerrit-PatchSet: 6 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Wed, 31 Oct 2018 22:12:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7586: fix predicate pushdown of escaped strings
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/11814 ) Change subject: IMPALA-7586: fix predicate pushdown of escaped strings .. Patch Set 6: Code-Review+2 Fix looks simple. +2ing myself to unblock it. -- To view, visit http://gerrit.cloudera.org:8080/11814 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I53d6e20dd48ab6837ddd325db8a9d49ee04fed28 Gerrit-Change-Number: 11814 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 31 Oct 2018 22:01:27 + Gerrit-HasComments: No
[Impala-ASF-CR] Detect bad OpenJDK version and install last known good
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11838 ) Change subject: Detect bad OpenJDK version and install last known good .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11838 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I39b786c37ad836aae5a9fdfee3187303c9536956 Gerrit-Change-Number: 11838 Gerrit-PatchSet: 6 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Wed, 31 Oct 2018 21:52:12 + Gerrit-HasComments: No
[Impala-ASF-CR] Detect bad OpenJDK version and install last known good
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11838 ) Change subject: Detect bad OpenJDK version and install last known good .. Detect bad OpenJDK version and install last known good A recent bug in OpenJDK caused surefire to fail. As a workaround for the affected version (8u181-b13-1ubuntu0.16.04.1) we download the last known good version and downgrade it. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=911925 https://bugs.launchpad.net/ubuntu/+source/openjdk-8/+bug/1800792 Change-Id: I39b786c37ad836aae5a9fdfee3187303c9536956 Reviewed-on: http://gerrit.cloudera.org:8080/11838 Reviewed-by: Philip Zeyliger Reviewed-by: Michal Ostrowski Tested-by: Impala Public Jenkins --- M bin/bootstrap_system.sh 1 file changed, 23 insertions(+), 0 deletions(-) Approvals: Philip Zeyliger: Looks good to me, approved Michal Ostrowski: Looks good to me, but someone else must approve Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/11838 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I39b786c37ad836aae5a9fdfee3187303c9536956 Gerrit-Change-Number: 11838 Gerrit-PatchSet: 7 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Philip Zeyliger
[Impala-ASF-CR] IMPALA-7655: Rewrite if, isnull, coalesce to use CASE
Paul Rogers has posted comments on this change. ( http://gerrit.cloudera.org:8080/11760 ) Change subject: IMPALA-7655: Rewrite if, isnull, coalesce to use CASE .. Patch Set 10: (10 comments) http://gerrit.cloudera.org:8080/#/c/11760/8//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11760/8//COMMIT_MSG@43 PS8, Line 43: (IMPALA-7737) > are there examples of these fns in our benchmarks to quantify the regressio Done http://gerrit.cloudera.org:8080/#/c/11760/8//COMMIT_MSG@45 PS8, Line 45: Still, the fix provides most of what the JIRA ticket requested > I'd skip these next three paragraphs. Done http://gerrit.cloudera.org:8080/#/c/11760/8//COMMIT_MSG@57 PS8, Line 57: > pls make a section for this called "Testing" so its easier to jump to. also Done http://gerrit.cloudera.org:8080/#/c/11760/8/be/src/exprs/conditional-functions.h File be/src/exprs/conditional-functions.h: http://gerrit.cloudera.org:8080/#/c/11760/8/be/src/exprs/conditional-functions.h@76 PS8, Line 76: since : /// various bugs mean that this implementation is still sometimes used. But : /// the goal is to remove these classes at some point. > simpler: "until their use is eliminated by the frontend". Done http://gerrit.cloudera.org:8080/#/c/11760/8/fe/src/main/java/org/apache/impala/rewrite/RewriteConditionalFnsRule.java File fe/src/main/java/org/apache/impala/rewrite/RewriteConditionalFnsRule.java: http://gerrit.cloudera.org:8080/#/c/11760/8/fe/src/main/java/org/apache/impala/rewrite/RewriteConditionalFnsRule.java@37 PS8, Line 37: vanish > is this accurate given the comments in the commit message about order by? Yes, they vanish after the rewrite. If the rewrite does not occur (due to bugs or options), then the functions do not vanish. http://gerrit.cloudera.org:8080/#/c/11760/8/fe/src/main/java/org/apache/impala/rewrite/RewriteConditionalFnsRule.java@48 PS8, Line 48: planner runs the rule to simplify CASE : * after this rule. Where that other rule can perform simplifications, : * those simplifications are omitted here > simplify and use the specific rule name for concreteness. Rewrote. Since many rules apply, did not seem to add value to try to enumerate them. http://gerrit.cloudera.org:8080/#/c/11760/8/fe/src/main/java/org/apache/impala/rewrite/RewriteConditionalFnsRule.java@106 PS8, Line 106:return rewriteCoalesceFn(e > clarify whether you think this happens after the rewrite or before. If its Done http://gerrit.cloudera.org:8080/#/c/11760/8/fe/src/main/java/org/apache/impala/rewrite/RewriteConditionalFnsRule.java@130 PS8, Line 130:Lists.ne > isn't this all that's done here (the most general case) and we'll depend on Rewrote comment http://gerrit.cloudera.org:8080/#/c/11760/8/fe/src/main/java/org/apache/impala/rewrite/RewriteConditionalFnsRule.java@172 PS8, Line 172: ESCE(a, literal) when literal > The simplest rewrite here would be to not look at the child exprs for the v I found that I could strip this function down as you suggest, but the results were sub-optimal due to weaknesses in other parts of the code. coalesce(id) --> CASE ELSE id END (expected: id) coalesce(id, null) --> CASE WHEN id IS NOT NULL THEN ID ELSE NULL END (expected: id) coalesce(null, id) --> same as above coalesce(null is null, id) --> 1 (expected: TRUE) coalesce(10 + null, id) --> CASE WHEN NULL IS NOT NULL THEN NULL ELSE id END (expected: id) coalesce(42, min(distinct tinyint_col)) --> CASE WHEN FALSE THEN NULL WHEN TRUE THEN 42 ELSE min(distinct tinyint_col, 42)) END (Expected: CASE WHEN min(distinct tinyint_col) IS NOT NULL THEN min(distinct tinyint_col) ELSE 42 END) And so on. One solution is to restore the original coalesce() function rewrite which produces somewhat better results. Even that function has holes which this version fixed. So, we'd lose those optimizations. As it turns out, all this is moot because the BE CASE implementation has a bug so we can't even use CASE to replace coalesce()... For now, I did rework the function to the simple form requested, then disabled it. We can tackle any other concerns if/when it is a priority. http://gerrit.cloudera.org:8080/#/c/11760/8/testdata/workloads/functional-planner/queries/PlannerTest/predicate-propagation.test File testdata/workloads/functional-planner/queries/PlannerTest/predicate-propagation.test: http://gerrit.cloudera.org:8080/#/c/11760/8/testdata/workloads/functional-planner/queries/PlannerTest/predicate-propagation.test@1137 PS8, Line 1137: | other predicates: functional.alltypestiny.tinyint_col + functional.alltypestiny.smallint_col + functional.alltypestiny.int_col > 10, CASE WHEN functional.alltypestiny.tinyint_col + functional.alltypestiny.bigint_col IS NULL THEN 1 ELSE functional.alltypestiny.tinyint_col + functional.alltypestiny.bigint_col END = 1 > so this is the example of the performance regression (same w
[Impala-ASF-CR] IMPALA-7655: Rewrite if, isnull to use CASE
Hello Bharath Vissapragada, Philip Zeyliger, Tim Armstrong, Csaba Ringhofer, Impala Public Jenkins, Vuk Ercegovac, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11760 to look at the new patch set (#11). Change subject: IMPALA-7655: Rewrite if, isnull to use CASE .. IMPALA-7655: Rewrite if, isnull to use CASE See IMPALA-7655 for backgound. Tim found that the current interpreted forms of if, isnull and coalesce are slow compared to the code-generated CASE statement. This patch rewrites the if and isnull functions into the equivalent CASE structure. Coalesce is omitted due to limitations within the code. Caveats: * IMPALA-7753: Conditionals in the top-level ORDER BY clause are not rewritten, but those one or more levels down are. The result is that PlannerTest files still show uses of if and isnull despite this fix. * IMPALA-7785: Similar, but says that the GROUP BY clause is not analyzed prior to rewrites, so the code again forces analysis. There are some odd "analyze()" calls in the code to work around this limitation snd the next two. * IMPALA-7769: Some expressions involving NULL are not simplified. As a result, some of the rewrites don't result in the most optimal result. * IMPALA-7754: The rewrite engine ignores unanalyzed expressions, yet the rewrite engine does not, in general, re-analyze the expressions it produces, causing simplifications to be skipped. * IMPALA-4356: Even after this change, code gen won't occur for SELECT expressions executed in teh root fragment (the most common case in simple tests.) Because of the above, some if and isnull calls are not rewritten, and others are not rewritten to the simplest forms. The tests contain commented-out tests, and comments with bug numbers, to record these known issues. Still, the current fix is useful enough by itself to proceed despite the above. While the desire was to replace the BE implementation of if and isnull, the caveats prevent doing so at present, so the BE retains the original interpreted forms. They are used when: 1) top-level conditions in the ORDER BY clause, and 2) if the user disables rewrites. Conditional function rewrites are moved into a new class, RewriteConditionalsRule in order to keep things simple. Coalesce() is implemented, but disabled. The old rewrites are retained until blocking issues are fixed. Testing: * Tests for conditional functions are pulled out into a new test class. * A new base class holds code common to the new class and the existing ExprRewriterTest. * A new FullRewriteTest class tests rewrites that require multiple passes thorugh teh rewrite engine. * Planner Test expected results were updated based on the rewritten expressions. Change-Id: I526654d8546e17b2545c42cc59dab66d9fe1b163 --- M be/src/exprs/conditional-functions.h M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java A fe/src/main/java/org/apache/impala/rewrite/RewriteConditionalFnsRule.java M fe/src/main/java/org/apache/impala/rewrite/SimplifyConditionalsRule.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java A fe/src/test/java/org/apache/impala/analysis/BaseRewriteRulesTest.java M fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java A fe/src/test/java/org/apache/impala/analysis/FullRewriteTest.java A fe/src/test/java/org/apache/impala/analysis/RewriteConditionalFnsRuleTest.java M testdata/workloads/functional-planner/queries/PlannerTest/analytic-fns.test M testdata/workloads/functional-planner/queries/PlannerTest/predicate-propagation.test M testdata/workloads/functional-planner/queries/PlannerTest/runtime-filter-propagation.test 13 files changed, 939 insertions(+), 256 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/60/11760/11 -- To view, visit http://gerrit.cloudera.org:8080/11760 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I526654d8546e17b2545c42cc59dab66d9fe1b163 Gerrit-Change-Number: 11760 Gerrit-PatchSet: 11 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-5821: Add query with implicit casts to extended explain output.
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/11719 ) Change subject: IMPALA-5821: Add query with implicit casts to extended explain output. .. Patch Set 4: (15 comments) overall good cleanup. mostly minor comments from my end. http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/analysis/ParseNode.java File fe/src/main/java/org/apache/impala/analysis/ParseNode.java: http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/analysis/ParseNode.java@32 PS4, Line 32: sql nit: pls use consistent case (SQL, sql, Sql) http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/analysis/ParseNode.java@32 PS4, Line 32: @param pls omit these javadoc annotations. http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/analysis/ParseNode.java@39 PS4, Line 39:* This should return the same result as calling toSql(ToSqlOptions.DEFAULT). would be good to do this via a default interface method. pls add a todo for when we fully move to Java 8. http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/analysis/ToSqlOptions.java File fe/src/main/java/org/apache/impala/analysis/ToSqlOptions.java: http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/analysis/ToSqlOptions.java@25 PS4, Line 25: default, normal "normal" makes sense if you know what is currently done. perhaps "original query without rewrites"? http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/analysis/ToSqlOptions.java@38 PS4, Line 38: // This does have the consequence that the sql with implict casts may not pssibly fail nit: spelling http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/analysis/ToSqlOptions.java@38 PS4, Line 38: may not pssibly fail : // to parse if resubmitted as I found this difficult to understand. Did you mean that Sql produced in this mode may not be valid Sql? I'd prefer the comment be merged with the comment block on L35. http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/analysis/TypeDef.java File fe/src/main/java/org/apache/impala/analysis/TypeDef.java: http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/analysis/TypeDef.java@165 PS4, Line 165: return parsedType_.toSql(); pass down here? http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/common/PrintUtils.java File fe/src/main/java/org/apache/impala/common/PrintUtils.java: http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/common/PrintUtils.java@110 PS4, Line 110: 32 what's this? http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/main/java/org/apache/impala/common/PrintUtils.java@115 PS4, Line 115: exiting sp. existing? http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/test/java/org/apache/impala/analysis/ExprRewriterTest.java File fe/src/test/java/org/apache/impala/analysis/ExprRewriterTest.java: http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/test/java/org/apache/impala/analysis/ExprRewriterTest.java@424 PS4, Line 424: @pa remove http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/test/java/org/apache/impala/analysis/ExprRewriterTest.java@451 PS4, Line 451: // AnalyzesOk(stmt.toSql(true), ctx); rm? http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java File fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java: http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java@750 PS4, Line 750: @para remove the comment annotations. http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java@801 PS4, Line 801: This would also be included if both more direct: Equivalent to enabling INCLUDE_EXPLAIN_HEADER and EXTENDED_EXPLAIN. http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/test/java/org/apache/impala/util/PrintUtilsTest.java File fe/src/test/java/org/apache/impala/util/PrintUtilsTest.java: http://gerrit.cloudera.org:8080/#/c/11719/4/fe/src/test/java/org/apache/impala/util/PrintUtilsTest.java@128 PS4, Line 128: @pa rm http://gerrit.cloudera.org:8080/#/c/11719/4/testdata/workloads/functional-planner/queries/PlannerTest/spillable-buffer-sizing.test File testdata/workloads/functional-planner/queries/PlannerTest/spillable-buffer-sizing.test: http://gerrit.cloudera.org:8080/#/c/11719/4/testdata/workloads/functional-planner/queries/PlannerTest/spillable-buffer-sizing.test@9 PS4, Line 9: -- +straight_join : * were you expecting the hint to print this way? -- To view, visit http://gerrit.cloudera.org:8080/11719 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment
[Impala-ASF-CR] IMPALA-7363: Fix buffer usage in ScannerContext::Stream::ReadVLong()
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11828 ) Change subject: IMPALA-7363: Fix buffer usage in ScannerContext::Stream::ReadVLong() .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11828 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic2b2ffe4b1d67c63cfcea2baedbeff48e65ca417 Gerrit-Change-Number: 11828 Gerrit-PatchSet: 5 Gerrit-Owner: Pooja Nilangekar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 20:48:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7363: Fix buffer usage in ScannerContext::Stream::ReadVLong()
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11828 ) Change subject: IMPALA-7363: Fix buffer usage in ScannerContext::Stream::ReadVLong() .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3391/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11828 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic2b2ffe4b1d67c63cfcea2baedbeff48e65ca417 Gerrit-Change-Number: 11828 Gerrit-PatchSet: 5 Gerrit-Owner: Pooja Nilangekar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 20:48:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5950: fix TPC-DS Q35a and Q48 queries
Greg Rahn has posted comments on this change. ( http://gerrit.cloudera.org:8080/11833 ) Change subject: IMPALA-5950: fix TPC-DS Q35a and Q48 queries .. Patch Set 5: Code-Review+2 Both queries and results LGTM -- To view, visit http://gerrit.cloudera.org:8080/11833 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic7c737f68adf616738d6eb6e5a02593af25bcbaf Gerrit-Change-Number: 11833 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 20:13:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5950: fix TPC-DS Q35a and Q48 queries
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11833 ) Change subject: IMPALA-5950: fix TPC-DS Q35a and Q48 queries .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1235/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11833 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic7c737f68adf616738d6eb6e5a02593af25bcbaf Gerrit-Change-Number: 11833 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 19:25:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5950: fix TPC-DS Q35a and Q48 queries
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11833 ) Change subject: IMPALA-5950: fix TPC-DS Q35a and Q48 queries .. Patch Set 4: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/1234/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/11833 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic7c737f68adf616738d6eb6e5a02593af25bcbaf Gerrit-Change-Number: 11833 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 19:18:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7614: [DOCS] Part 2: Document the New Invalidate option
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11839 ) Change subject: IMPALA-7614: [DOCS] Part 2: Document the New Invalidate option .. IMPALA-7614: [DOCS] Part 2: Document the New Invalidate option Change-Id: I1a825bff4a09a66b28138a8ece4ee6a60868b189 Reviewed-on: http://gerrit.cloudera.org:8080/11839 Tested-by: Impala Public Jenkins Reviewed-by: Vuk Ercegovac --- M docs/topics/impala_config_options.xml 1 file changed, 9 insertions(+), 8 deletions(-) Approvals: Impala Public Jenkins: Verified Vuk Ercegovac: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/11839 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I1a825bff4a09a66b28138a8ece4ee6a60868b189 Gerrit-Change-Number: 11839 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Adrian Ng (389) Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-7614: [DOCS] Part 2: Document the New Invalidate option
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/11839 ) Change subject: IMPALA-7614: [DOCS] Part 2: Document the New Invalidate option .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11839 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1a825bff4a09a66b28138a8ece4ee6a60868b189 Gerrit-Change-Number: 11839 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Adrian Ng (389) Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 31 Oct 2018 19:05:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7614: [DOCS] Part 2: Document the New Invalidate option
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11839 ) Change subject: IMPALA-7614: [DOCS] Part 2: Document the New Invalidate option .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/127/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/11839 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1a825bff4a09a66b28138a8ece4ee6a60868b189 Gerrit-Change-Number: 11839 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Adrian Ng (389) Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 31 Oct 2018 19:02:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6661 Make NaN values equal for grouping purposes.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11535 ) Change subject: IMPALA-6661 Make NaN values equal for grouping purposes. .. Patch Set 20: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/3387/ -- To view, visit http://gerrit.cloudera.org:8080/11535 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I996c4a2e1934fd887046ed0c55457b7285375086 Gerrit-Change-Number: 11535 Gerrit-PatchSet: 20 Gerrit-Owner: Michal Ostrowski Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 18:59:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7614: [DOCS] Part 2: Document the New Invalidate option
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/11839 ) Change subject: IMPALA-7614: [DOCS] Part 2: Document the New Invalidate option .. Patch Set 1: This patch has the updates that Vuk and Adrian suggested. -- To view, visit http://gerrit.cloudera.org:8080/11839 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1a825bff4a09a66b28138a8ece4ee6a60868b189 Gerrit-Change-Number: 11839 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Adrian Ng (389) Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 31 Oct 2018 18:50:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7614: [DOCS] Part 2: Document the New Invalidate option
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11839 ) Change subject: IMPALA-7614: [DOCS] Part 2: Document the New Invalidate option .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/127/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/11839 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1a825bff4a09a66b28138a8ece4ee6a60868b189 Gerrit-Change-Number: 11839 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 31 Oct 2018 18:49:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7614: [DOCS] Part 2: Document the New Invalidate option
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11839 Change subject: IMPALA-7614: [DOCS] Part 2: Document the New Invalidate option .. IMPALA-7614: [DOCS] Part 2: Document the New Invalidate option Change-Id: I1a825bff4a09a66b28138a8ece4ee6a60868b189 --- M docs/topics/impala_config_options.xml 1 file changed, 9 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/39/11839/1 -- To view, visit http://gerrit.cloudera.org:8080/11839 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I1a825bff4a09a66b28138a8ece4ee6a60868b189 Gerrit-Change-Number: 11839 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-5950: fix TPC-DS Q35a and Q48 queries
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11833 ) Change subject: IMPALA-5950: fix TPC-DS Q35a and Q48 queries .. Patch Set 5: (5 comments) http://gerrit.cloudera.org:8080/#/c/11833/5/tests/query_test/test_tpcds_queries.py File tests/query_test/test_tpcds_queries.py: http://gerrit.cloudera.org:8080/#/c/11833/5/tests/query_test/test_tpcds_queries.py@537 PS5, Line 537: \ flake8: E502 the backslash is redundant between brackets http://gerrit.cloudera.org:8080/#/c/11833/5/tests/query_test/test_tpcds_queries.py@538 PS5, Line 538: \ flake8: E502 the backslash is redundant between brackets http://gerrit.cloudera.org:8080/#/c/11833/5/tests/query_test/test_tpcds_queries.py@539 PS5, Line 539: \ flake8: E502 the backslash is redundant between brackets http://gerrit.cloudera.org:8080/#/c/11833/5/tests/query_test/test_tpcds_queries.py@545 PS5, Line 545: \ flake8: E502 the backslash is redundant between brackets http://gerrit.cloudera.org:8080/#/c/11833/5/tests/query_test/test_tpcds_queries.py@548 PS5, Line 548: \ flake8: E502 the backslash is redundant between brackets -- To view, visit http://gerrit.cloudera.org:8080/11833 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic7c737f68adf616738d6eb6e5a02593af25bcbaf Gerrit-Change-Number: 11833 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 18:47:43 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5950: fix TPC-DS Q35a and Q48 queries
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11833 ) Change subject: IMPALA-5950: fix TPC-DS Q35a and Q48 queries .. Patch Set 4: (5 comments) http://gerrit.cloudera.org:8080/#/c/11833/4/tests/query_test/test_tpcds_queries.py File tests/query_test/test_tpcds_queries.py: http://gerrit.cloudera.org:8080/#/c/11833/4/tests/query_test/test_tpcds_queries.py@537 PS4, Line 537: \ flake8: E502 the backslash is redundant between brackets http://gerrit.cloudera.org:8080/#/c/11833/4/tests/query_test/test_tpcds_queries.py@538 PS4, Line 538: \ flake8: E502 the backslash is redundant between brackets http://gerrit.cloudera.org:8080/#/c/11833/4/tests/query_test/test_tpcds_queries.py@539 PS4, Line 539: \ flake8: E502 the backslash is redundant between brackets http://gerrit.cloudera.org:8080/#/c/11833/4/tests/query_test/test_tpcds_queries.py@545 PS4, Line 545: \ flake8: E502 the backslash is redundant between brackets http://gerrit.cloudera.org:8080/#/c/11833/4/tests/query_test/test_tpcds_queries.py@548 PS4, Line 548: \ flake8: E502 the backslash is redundant between brackets -- To view, visit http://gerrit.cloudera.org:8080/11833 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic7c737f68adf616738d6eb6e5a02593af25bcbaf Gerrit-Change-Number: 11833 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 18:46:51 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7614: [DOCS] Document the New Invalidate Options
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/11809 ) Change subject: IMPALA-7614: [DOCS] Document the New Invalidate Options .. Patch Set 2: (6 comments) This patch was merged with Tiany's +2, so I am uploading a new patch with the updates requested by Adrian and Vuk. http://gerrit.cloudera.org:8080/#/c/11809/2/docs/topics/impala_config_options.xml File docs/topics/impala_config_options.xml: http://gerrit.cloudera.org:8080/#/c/11809/2/docs/topics/impala_config_options.xml@367 PS2, Line 367: is high > We can say "reaches 60% of JVM heap size" to be specific. Done http://gerrit.cloudera.org:8080/#/c/11809/2/docs/topics/impala_config_options.xml@368 PS2, Line 368: certain > We can say invalidates 10% of the least recently used tables. Done http://gerrit.cloudera.org:8080/#/c/11809/3/docs/topics/impala_config_options.xml File docs/topics/impala_config_options.xml: http://gerrit.cloudera.org:8080/#/c/11809/3/docs/topics/impala_config_options.xml@355 PS3, Line 355: small > bounded Done http://gerrit.cloudera.org:8080/#/c/11809/3/docs/topics/impala_config_options.xml@363 PS3, Line 363: specified time period. T > in seconds. Done http://gerrit.cloudera.org:8080/#/c/11809/3/docs/topics/impala_config_options.xml@365 PS3, Line 365: Java garbage collection-based > simplify to: Memory-based Done http://gerrit.cloudera.org:8080/#/c/11809/3/docs/topics/impala_config_options.xml@373 PS3, Line 373: but the feature could potentially : cause performance risks > do we have standardized phrasing around memory knobs? something like, "may Changed to: the feature could potentially cause performance issues and may require tuning. -- To view, visit http://gerrit.cloudera.org:8080/11809 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I40c552eeaee81ee6528d9f725bd416b51d8ab837 Gerrit-Change-Number: 11809 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Adrian Ng (389) Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 31 Oct 2018 18:46:37 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5950: fix TPC-DS Q35a and Q48 queries
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11833 ) Change subject: IMPALA-5950: fix TPC-DS Q35a and Q48 queries .. Patch Set 4: This seems like the right thing to do for functional testing but may invalidate some perf baselines. I'd appreciate any input on how to handle that. -- To view, visit http://gerrit.cloudera.org:8080/11833 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic7c737f68adf616738d6eb6e5a02593af25bcbaf Gerrit-Change-Number: 11833 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 18:46:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5950: fix TPC-DS Q35a and Q48 queries
Hello Greg Rahn, David Knupp, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11833 to look at the new patch set (#5). Change subject: IMPALA-5950: fix TPC-DS Q35a and Q48 queries .. IMPALA-5950: fix TPC-DS Q35a and Q48 queries The query text (for Q48) and substitution parameters didn't match the TPC-DS standard for qualification queries. After fixing that, the queries return the results expected by the TPC-DS standards. Note that this may affect the performance of perf workloads running tpcds-unmodified. Change-Id: Ic7c737f68adf616738d6eb6e5a02593af25bcbaf --- A testdata/workloads/tpcds-unmodified/queries/tpcds-q35a.test M testdata/workloads/tpcds-unmodified/queries/tpcds-q48.test M tests/query_test/test_tpcds_queries.py 3 files changed, 257 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/11833/5 -- To view, visit http://gerrit.cloudera.org:8080/11833 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic7c737f68adf616738d6eb6e5a02593af25bcbaf Gerrit-Change-Number: 11833 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-5950: fix TPC-DS Q35a and Q48 queries
Hello Greg Rahn, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11833 to look at the new patch set (#4). Change subject: IMPALA-5950: fix TPC-DS Q35a and Q48 queries .. IMPALA-5950: fix TPC-DS Q35a and Q48 queries The query text (for Q48) and substitution parameters didn't match the TPC-DS standard for qualification queries. After fixing that, the queries return the results expected by the TPC-DS standards. Note that this may affect the performance of perf workloads running tpcds-unmodified. Change-Id: Ic7c737f68adf616738d6eb6e5a02593af25bcbaf --- A testdata/workloads/tpcds-unmodified/queries/tpcds-q35a.test M testdata/workloads/tpcds-unmodified/queries/tpcds-q48.test M tests/query_test/test_tpcds_queries.py 3 files changed, 259 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/11833/4 -- To view, visit http://gerrit.cloudera.org:8080/11833 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic7c737f68adf616738d6eb6e5a02593af25bcbaf Gerrit-Change-Number: 11833 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] Detect bad OpenJDK version and install last known good
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11838 ) Change subject: Detect bad OpenJDK version and install last known good .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1233/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11838 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I39b786c37ad836aae5a9fdfee3187303c9536956 Gerrit-Change-Number: 11838 Gerrit-PatchSet: 6 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Wed, 31 Oct 2018 18:18:10 + Gerrit-HasComments: No
[Impala-ASF-CR] Detect bad OpenJDK version and install last known good
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11838 ) Change subject: Detect bad OpenJDK version and install last known good .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1232/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11838 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I39b786c37ad836aae5a9fdfee3187303c9536956 Gerrit-Change-Number: 11838 Gerrit-PatchSet: 5 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Wed, 31 Oct 2018 18:14:17 + Gerrit-HasComments: No
[Impala-ASF-CR] Detect bad OpenJDK version and install last known good
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11838 ) Change subject: Detect bad OpenJDK version and install last known good .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1231/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11838 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I39b786c37ad836aae5a9fdfee3187303c9536956 Gerrit-Change-Number: 11838 Gerrit-PatchSet: 4 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Wed, 31 Oct 2018 18:03:04 + Gerrit-HasComments: No
[Impala-ASF-CR] Detect bad OpenJDK version and install last known good
Michal Ostrowski has posted comments on this change. ( http://gerrit.cloudera.org:8080/11838 ) Change subject: Detect bad OpenJDK version and install last known good .. Patch Set 6: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/11838 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I39b786c37ad836aae5a9fdfee3187303c9536956 Gerrit-Change-Number: 11838 Gerrit-PatchSet: 6 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Wed, 31 Oct 2018 17:59:52 + Gerrit-HasComments: No
[Impala-ASF-CR] Detect bad OpenJDK version and install last known good
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11838 ) Change subject: Detect bad OpenJDK version and install last known good .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3390/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11838 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I39b786c37ad836aae5a9fdfee3187303c9536956 Gerrit-Change-Number: 11838 Gerrit-PatchSet: 6 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Wed, 31 Oct 2018 17:55:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7761: Add multiple DISTINCT to targeted stress and perf
Michael Brown has posted comments on this change. ( http://gerrit.cloudera.org:8080/11805 ) Change subject: IMPALA-7761: Add multiple DISTINCT to targeted stress and perf .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/11805/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11805/1//COMMIT_MSG@16 PS1, Line 16: Testing: : - Ran the test file locally. > By "run locally" I mean that I put together a python test that run the file You could: 1. Put the queries into the tpch workload directory 2. Get them running regularly in query_test/test_tpch.py so that your queries run in regular CI immediately 3. Alter the stress test regex and unit test that makes sure all expected queries are there Commentary: The easiest way to get them into the stress test is to add queries, 1 query per .test file, to the tpch workload, and extend the regex to match the files you added. To prevent the regex/matching from regressing, there are CI tests to make sure we match an expected number of files (at least two bugs in the past have busted the matching when people altered the countents of the tpc workload directories). That test is here: https://github.com/apache/impala/blob/master/tests/infra/test_stress_infra.py#L53 . You could also run the queries in query_test/test_tpch.py, at which point you'll have regular CI on these queries as opposed to them being put into odd workloads. -- To view, visit http://gerrit.cloudera.org:8080/11805 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I400aaf6b6620b4001895eafff785956bffb312c9 Gerrit-Change-Number: 11805 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 31 Oct 2018 17:54:08 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Detect bad OpenJDK version and install last known good
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11838 ) Change subject: Detect bad OpenJDK version and install last known good .. Patch Set 3: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/1230/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/11838 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I39b786c37ad836aae5a9fdfee3187303c9536956 Gerrit-Change-Number: 11838 Gerrit-PatchSet: 3 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Wed, 31 Oct 2018 17:54:22 + Gerrit-HasComments: No
[Impala-ASF-CR] Detect bad OpenJDK version and install last known good
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11838 ) Change subject: Detect bad OpenJDK version and install last known good .. Patch Set 1: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/3388/ -- To view, visit http://gerrit.cloudera.org:8080/11838 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I39b786c37ad836aae5a9fdfee3187303c9536956 Gerrit-Change-Number: 11838 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Wed, 31 Oct 2018 17:47:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7794: Rewrite flaky ownership authorization tests
Adam Holley has posted comments on this change. ( http://gerrit.cloudera.org:8080/11837 ) Change subject: IMPALA-7794: Rewrite flaky ownership authorization tests .. Patch Set 4: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/11837 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic98f8dbec41360261fd0339d835f3ce6b504ee29 Gerrit-Change-Number: 11837 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 31 Oct 2018 17:45:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7794: Rewrite flaky ownership authorization tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11837 ) Change subject: IMPALA-7794: Rewrite flaky ownership authorization tests .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1229/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11837 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic98f8dbec41360261fd0339d835f3ce6b504ee29 Gerrit-Change-Number: 11837 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 31 Oct 2018 17:45:53 + Gerrit-HasComments: No
