[Impala-ASF-CR] IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Adam Tamas has uploaded this change for review. (
http://gerrit.cloudera.org:8080/16418
Change subject: IMPALA-10012: ds_hll_sketch() results ascii codec decoding
error fix
..
IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
While the ds_hll_sketch() generates a string value as output the data
is not an ascii encoded text but a bitsketch, because of this, when
the shell get this data it disconnect while it tries to decode it.
The issue can be reproduced with a simple method like using unhex
with a wrong input.
Example: SELECT unhex("aa");
This patch contains a solution, where we replace any not UTF-8
decodable characters if we run into an UnicodeDecodeError after
fetching it.
This solution is working with the Thrift 0.9.3 autogenerated gen-py
but still fails with Thrift 0.11.0.
For Thrift 0.11.0 the error is catched and an error message is sent
(not working with beeswax protocol, because it generates a different
error (TypeError) which can come for other reasons too).
Testing:
-manual testing with these protocols: 'hs2-http', 'hs2', 'beeswax'
Change-Id: I0c5f1290356e21aed8ca7f896f953541942aed05
---
M shell/impala_client.py
M shell/impala_shell.py
M shell/shell_output.py
3 files changed, 37 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/18/16418/1
--
To view, visit http://gerrit.cloudera.org:8080/16418
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I0c5f1290356e21aed8ca7f896f953541942aed05
Gerrit-Change-Number: 16418
Gerrit-PatchSet: 1
Gerrit-Owner: Adam Tamas
[Impala-ASF-CR] IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Adam Tamas has uploaded a new patch set (#10). (
http://gerrit.cloudera.org:8080/16305 )
Change subject: IMPALA-10012: ds_hll_sketch() results ascii codec decoding
error fix
..
IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix
While the ds_hll_sketch() generates a string value as output the data
is not an ascii encoded text but a bitsketch, because of this, when
the shell get this data it disconnect while it tries to decode it.
The issue can be reproduced with a simple method like using unhex
with a wrong input.
Example: SELECT unhex("aa");
This patch contains a solution, where we replace any not UTF-8
decodable characters if we run into an UnicodeDecodeError after
fetching it.
This solution is working with the Thrift 0.9.3 autogenerated gen-py
but still fails with Thrift 0.11.0.
For Thrift 0.11.0 the error is catched and an error message is sent
(not working with beeswax protocol, because it generates a different
error (TypeError) which can come for other reasons too).
Testing:
-manual testing with these protocols: 'hs2-http', 'hs2', 'beeswax'
Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
---
M shell/impala_client.py
M shell/impala_shell.py
M shell/shell_output.py
3 files changed, 37 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/05/16305/10
--
To view, visit http://gerrit.cloudera.org:8080/16305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8
Gerrit-Change-Number: 16305
Gerrit-PatchSet: 10
Gerrit-Owner: Adam Tamas
Gerrit-Reviewer: Adam Tamas
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/16305 ) Change subject: IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix .. Patch Set 10: (6 comments) http://gerrit.cloudera.org:8080/#/c/16305/9//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16305/9//COMMIT_MSG@20 PS9, Line 20: > typo Done http://gerrit.cloudera.org:8080/#/c/16305/9//COMMIT_MSG@21 PS9, Line 21: This solution is working with the Th > Can you be more specific about the error thrown? Done http://gerrit.cloudera.org:8080/#/c/16305/9//COMMIT_MSG@24 PS9, Line 24: For Thrift 0.11.0 the error is catched and an error message is sen > can you add a bit more info, e.g. the issue can be reproduced with SELECT u Done http://gerrit.cloudera.org:8080/#/c/16305/9/shell/impala_shell.py File shell/impala_shell.py: http://gerrit.cloudera.org:8080/#/c/16305/9/shell/impala_shell.py@1232 PS9, Line 1232: 'to find the possible source of the error.' % (e,), file=sys.stderr) > nit: +4 indent Done http://gerrit.cloudera.org:8080/#/c/16305/9/shell/shell_output.py File shell/shell_output.py: http://gerrit.cloudera.org:8080/#/c/16305/9/shell/shell_output.py@40 PS9, Line 40: decodable. > nit: decodable Done http://gerrit.cloudera.org:8080/#/c/16305/9/shell/shell_output.py@41 PS9, Line 41: row an > typo Done -- To view, visit http://gerrit.cloudera.org:8080/16305 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8 Gerrit-Change-Number: 16305 Gerrit-PatchSet: 10 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 04 Sep 2020 11:41:26 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/16305 ) Change subject: IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix .. Patch Set 9: (5 comments) http://gerrit.cloudera.org:8080/#/c/16305/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16305/5//COMMIT_MSG@14 PS5, Line 14: > nit: please wrap at 72 Done http://gerrit.cloudera.org:8080/#/c/16305/5//COMMIT_MSG@14 PS5, Line 14: decodable > typo: I think it is decodable Done http://gerrit.cloudera.org:8080/#/c/16305/5//COMMIT_MSG@17 PS5, Line 17: autogenerated > typo Done http://gerrit.cloudera.org:8080/#/c/16305/5//COMMIT_MSG@18 PS5, Line 18: but still fails with Thrift 0.11.0 > Can you create a ticket for this? As we have discussed, fixing this would n Done (IMPALA-10145) http://gerrit.cloudera.org:8080/#/c/16305/5/shell/impala_shell.py File shell/impala_shell.py: http://gerrit.cloudera.org:8080/#/c/16305/5/shell/impala_shell.py@1187 PS5, Line 1187: self.output_stream.write(rows) > nit: extra line Done -- To view, visit http://gerrit.cloudera.org:8080/16305 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8 Gerrit-Change-Number: 16305 Gerrit-PatchSet: 9 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 04 Sep 2020 10:53:57 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10066: Fix test cancellation mid command fails
Adam Tamas has abandoned this change. ( http://gerrit.cloudera.org:8080/16322 ) Change subject: IMPALA-10066: Fix test_cancellation_mid_command fails .. Abandoned -- To view, visit http://gerrit.cloudera.org:8080/16322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: Ib80706d52a85d2c19b13fbbe5695934658c0bf7e Gerrit-Change-Number: 16322 Gerrit-PatchSet: 6 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Adam Tamas has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/16305 ) Change subject: IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix .. IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix While the ds_hll_sketch() generates a string value as output the data is not an ascii encoded text but a bitsketch, because of this, when the shell get this data it disconnect while it tries to decode it. This patch contains a solution, where we replace any not UTF-8 decodable characters if we run into an UnicodeDecodeError after fetching it. This solution is working with the Thrift 0.9.3 autogenerated gen-py but still fails with Thrift 0.11.0. For Thrift 0.11.0 the error is catched and an error messenge is sent (not working with beeswax protocol). Testing: -manual testing with these protocols: 'hs2-http', 'hs2', 'beeswax' Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8 --- M shell/impala_client.py M shell/impala_shell.py M shell/shell_output.py 3 files changed, 37 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/05/16305/9 -- To view, visit http://gerrit.cloudera.org:8080/16305 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8 Gerrit-Change-Number: 16305 Gerrit-PatchSet: 9 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Adam Tamas has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/16305 ) Change subject: IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix .. IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix While the ds_hll_sketch() generates a string value as output the data is not an ascii encoded text but a bitsketch, because of this, when the shell get this data it disconnect while it tries to decode it. This patch contains a solution, where we replace any not UTF-8 decodable characters if we run into an UnicodeDecodeError after fetching it. This solution is working with the Thrift 0.9.3 autogenerated gen-py but still fails with Thrift 0.11.0. For Thrift 0.11.0 the error is catched and an error messenge is sent (not working with beeswax protocol). Testing: -manual testing with these protocols: 'hs2-http', 'hs2', 'beeswax' Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8 --- M shell/impala_client.py M shell/impala_shell.py M shell/shell_output.py 3 files changed, 37 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/05/16305/8 -- To view, visit http://gerrit.cloudera.org:8080/16305 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8 Gerrit-Change-Number: 16305 Gerrit-PatchSet: 8 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10133:Implement ds hll stringify function.
Adam Tamas has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/16382 ) Change subject: IMPALA-10133:Implement ds_hll_stringify function. .. IMPALA-10133:Implement ds_hll_stringify function. This function receives a string that is a serialized Apache DataSketches HLL sketch and returns its stringified format. A stringified format should look like and contains the following data: select ds_hll_stringify(ds_hll_sketch(float_col)) from functional_parquet.alltypestiny; ++ | ds_hll_stringify(ds_hll_sketch(float_col)) | ++ | ### HLL sketch summary:| | Log Config K : 12 | | Hll Target : HLL_4 | | Current Mode : LIST| | LB : 2 | | Estimate : 2 | | UB : 2.0001 | | OutOfOrder flag: false | | Coupon count : 2 | | ### End HLL sketch summary | || ++ Change-Id: I85dbf20b5114dd75c300eef0accabe90eac240a0 --- M be/src/exprs/datasketches-functions-ir.cc M be/src/exprs/datasketches-functions.h M common/function-registry/impala_functions.py M testdata/workloads/functional-query/queries/QueryTest/datasketches-hll.test 4 files changed, 59 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/16382/5 -- To view, visit http://gerrit.cloudera.org:8080/16382 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I85dbf20b5114dd75c300eef0accabe90eac240a0 Gerrit-Change-Number: 16382 Gerrit-PatchSet: 5 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10107:(1/3)Implement ds hll stringify function.
Adam Tamas has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/16382 ) Change subject: IMPALA-10107:(1/3)Implement ds_hll_stringify function. .. IMPALA-10107:(1/3)Implement ds_hll_stringify function. This function receives a string that is a serialized Apache DataSketches HLL sketch and returns its stringified format. A stringified format should look like and contains the following data: select ds_hll_stringify(ds_hll_sketch(float_col)) from functional_parquet.alltypestiny; ++ | ds_hll_stringify(ds_hll_sketch(float_col)) | ++ | ### HLL sketch summary:| | Log Config K : 12 | | Hll Target : HLL_4 | | Current Mode : LIST| | LB : 2 | | Estimate : 2 | | UB : 2.0001 | | OutOfOrder flag: false | | Coupon count : 2 | | ### End HLL sketch summary | || ++ Change-Id: I85dbf20b5114dd75c300eef0accabe90eac240a0 --- M be/src/exprs/datasketches-functions-ir.cc M be/src/exprs/datasketches-functions.h M common/function-registry/impala_functions.py M testdata/workloads/functional-query/queries/QueryTest/datasketches-hll.test 4 files changed, 59 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/16382/2 -- To view, visit http://gerrit.cloudera.org:8080/16382 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I85dbf20b5114dd75c300eef0accabe90eac240a0 Gerrit-Change-Number: 16382 Gerrit-PatchSet: 2 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10107:(1/3)Implement ds hll stringify function.
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/16382 ) Change subject: IMPALA-10107:(1/3)Implement ds_hll_stringify function. .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/16382/1/be/src/exprs/datasketches-functions.h File be/src/exprs/datasketches-functions.h: http://gerrit.cloudera.org:8080/#/c/16382/1/be/src/exprs/datasketches-functions.h@37 PS1, Line 37: > nit: leave an empty line before starting the comment. Done http://gerrit.cloudera.org:8080/#/c/16382/1/be/src/exprs/datasketches-functions.h@39 PS1, Line 39: then the query fai > DataSketches HLL sketch Done -- To view, visit http://gerrit.cloudera.org:8080/16382 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I85dbf20b5114dd75c300eef0accabe90eac240a0 Gerrit-Change-Number: 16382 Gerrit-PatchSet: 2 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 02 Sep 2020 08:56:24 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10012: Test for undecodable binary display with impala-shell
Adam Tamas has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16393 Change subject: IMPALA-10012: Test for undecodable binary display with impala-shell .. IMPALA-10012: Test for undecodable binary display with impala-shell A test for making sure the impala-shell is working perfectly after trying to display an undecodable binary code. Only working if both the original IMPALA-10012 and the binary type is already implemeted (IMPALA-9482)! Should work for HS2 and HS2-http (undecodable binary display is NOT working with beeswax protocol) Change-Id: Ic17bcc30a9d948fb1adc8ba5ea5bdeaded32d7ff --- M tests/shell/test_shell_interactive.py 1 file changed, 27 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/93/16393/1 -- To view, visit http://gerrit.cloudera.org:8080/16393 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ic17bcc30a9d948fb1adc8ba5ea5bdeaded32d7ff Gerrit-Change-Number: 16393 Gerrit-PatchSet: 1 Gerrit-Owner: Adam Tamas
[Impala-ASF-CR] IMPALA-10107:(1/3)Implement ds hll stringify function.
Adam Tamas has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16382 Change subject: IMPALA-10107:(1/3)Implement ds_hll_stringify function. .. IMPALA-10107:(1/3)Implement ds_hll_stringify function. This function receives a string that is a serialized Apache DataSketches HLL sketch and returns its stringified format. A stringified format should look like and contains the following data: select ds_hll_stringify(ds_hll_sketch(float_col)) from functional_parquet.alltypestiny; ++ | ds_hll_stringify(ds_hll_sketch(float_col)) | ++ | ### HLL sketch summary:| | Log Config K : 12 | | Hll Target : HLL_4 | | Current Mode : LIST| | LB : 2 | | Estimate : 2 | | UB : 2.0001 | | OutOfOrder flag: false | | Coupon count : 2 | | ### End HLL sketch summary | || ++ Change-Id: I85dbf20b5114dd75c300eef0accabe90eac240a0 --- M be/src/exprs/datasketches-functions-ir.cc M be/src/exprs/datasketches-functions.h M common/function-registry/impala_functions.py M testdata/workloads/functional-query/queries/QueryTest/datasketches-hll.test 4 files changed, 58 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/16382/1 -- To view, visit http://gerrit.cloudera.org:8080/16382 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I85dbf20b5114dd75c300eef0accabe90eac240a0 Gerrit-Change-Number: 16382 Gerrit-PatchSet: 1 Gerrit-Owner: Adam Tamas
[Impala-ASF-CR] IMPALA-10108: Implement ds kll stringify function
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/16370 ) Change subject: IMPALA-10108: Implement ds_kll_stringify function .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/16370/4/be/src/exprs/datasketches-functions.h File be/src/exprs/datasketches-functions.h: http://gerrit.cloudera.org:8080/#/c/16370/4/be/src/exprs/datasketches-functions.h@98 PS4, Line 98: const StringVal& ser > nit: this fits in the previous line. Done -- To view, visit http://gerrit.cloudera.org:8080/16370 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I97f654a4838bf91e3e0bed6a00d78b2c7aa96f75 Gerrit-Change-Number: 16370 Gerrit-PatchSet: 5 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 28 Aug 2020 07:03:58 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10108: Implement ds kll stringify function
Adam Tamas has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/16370 ) Change subject: IMPALA-10108: Implement ds_kll_stringify function .. IMPALA-10108: Implement ds_kll_stringify function This function receives a string that is a serialized Apache DataSketches KLL sketch and returns its stringified format. A stringified format should look like and contains the following data: select ds_kll_stringify(ds_kll_sketch(float_col)) from functional_parquet.alltypestiny; ++ | ds_kll_stringify(ds_kll_sketch(float_col)) | ++ | ### KLL sketch summary:| |K : 200| |min K : 200| |M : 8 | |N : 8 | |Epsilon: 1.33% | |Epsilon PMF: 1.65% | |Empty : false | |Estimation mode: false | |Levels : 1 | |Sorted : false | |Capacity items : 200| |Retained items : 8 | |Storage bytes : 64 | |Min value : 0 | |Max value : 1.1| | ### End sketch summary | || ++ Change-Id: I97f654a4838bf91e3e0bed6a00d78b2c7aa96f75 --- M be/src/exprs/datasketches-functions-ir.cc M be/src/exprs/datasketches-functions.h M common/function-registry/impala_functions.py M testdata/workloads/functional-query/queries/QueryTest/datasketches-kll.test 4 files changed, 59 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/70/16370/5 -- To view, visit http://gerrit.cloudera.org:8080/16370 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I97f654a4838bf91e3e0bed6a00d78b2c7aa96f75 Gerrit-Change-Number: 16370 Gerrit-PatchSet: 5 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10108: Implement ds kll stringify function
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/16370 ) Change subject: IMPALA-10108: Implement ds_kll_stringify function .. Patch Set 4: (5 comments) http://gerrit.cloudera.org:8080/#/c/16370/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16370/3//COMMIT_MSG@10 PS3, Line 10: stringified format > Could you mention what does this stringified format look like? What informa Done http://gerrit.cloudera.org:8080/#/c/16370/3/be/src/exprs/datasketches-functions-ir.cc File be/src/exprs/datasketches-functions-ir.cc: http://gerrit.cloudera.org:8080/#/c/16370/3/be/src/exprs/datasketches-functions-ir.cc@144 PS3, Line 144: const StringVal& ser > nit: this param would fit in the previous line, right? Right, done. http://gerrit.cloudera.org:8080/#/c/16370/3/be/src/exprs/datasketches-functions.h File be/src/exprs/datasketches-functions.h: http://gerrit.cloudera.org:8080/#/c/16370/3/be/src/exprs/datasketches-functions.h@109 PS3, Line 109: c > nit: an Done http://gerrit.cloudera.org:8080/#/c/16370/3/be/src/exprs/datasketches-functions.h@110 PS3, Line 110: PMFCDF mode); > Is this meant to be in the private section? I missed that it went here after the merge, fixed. http://gerrit.cloudera.org:8080/#/c/16370/3/testdata/workloads/functional-query/queries/QueryTest/datasketches-kll.test File testdata/workloads/functional-query/queries/QueryTest/datasketches-kll.test: http://gerrit.cloudera.org:8080/#/c/16370/3/testdata/workloads/functional-query/queries/QueryTest/datasketches-kll.test@604 PS3, Line 604: row_regex: .*### KLL sketch summary:.*Epsilon.*Epsilon PMF.*Em > What is the content of the sketch summary? Can't you assert on anything tha An alternative would look like for the whole summary: '### KLL sketch summary:\n K : 200\n min K : 200\n M : 8\n N : 8\n Epsilon: 1.33%%\n Epsilon PMF: 1.65%%\n Empty : false\n Estimation mode: false\n Levels : 1\n Sorted : false\n Capacity items : 200\n Retained items : 8\n Storage bytes : 64\n Min value : 0\n Max value : 1.1\n### End sketch summary\n' Modified it so it look for more parts from the sketch summary. -- To view, visit http://gerrit.cloudera.org:8080/16370 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I97f654a4838bf91e3e0bed6a00d78b2c7aa96f75 Gerrit-Change-Number: 16370 Gerrit-PatchSet: 4 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 26 Aug 2020 16:20:10 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10108: Implement ds kll stringify function
Adam Tamas has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/16370 ) Change subject: IMPALA-10108: Implement ds_kll_stringify function .. IMPALA-10108: Implement ds_kll_stringify function This function receives a string that is a serialized Apache DataSketches KLL sketch and returns its stringified format. A stringified format should look like and contains the following data: select ds_kll_stringify(ds_kll_sketch(float_col)) from functional_parquet.alltypestiny; ++ | ds_kll_stringify(ds_kll_sketch(float_col)) | ++ | ### KLL sketch summary:| |K : 200| |min K : 200| |M : 8 | |N : 8 | |Epsilon: 1.33% | |Epsilon PMF: 1.65% | |Empty : false | |Estimation mode: false | |Levels : 1 | |Sorted : false | |Capacity items : 200| |Retained items : 8 | |Storage bytes : 64 | |Min value : 0 | |Max value : 1.1| | ### End sketch summary | || ++ Change-Id: I97f654a4838bf91e3e0bed6a00d78b2c7aa96f75 --- M be/src/exprs/datasketches-functions-ir.cc M be/src/exprs/datasketches-functions.h M common/function-registry/impala_functions.py M testdata/workloads/functional-query/queries/QueryTest/datasketches-kll.test 4 files changed, 59 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/70/16370/4 -- To view, visit http://gerrit.cloudera.org:8080/16370 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I97f654a4838bf91e3e0bed6a00d78b2c7aa96f75 Gerrit-Change-Number: 16370 Gerrit-PatchSet: 4 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10106: Upgrade DataSketches to version 2.1.0
Adam Tamas has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/16360 ) Change subject: IMPALA-10106: Upgrade DataSketches to version 2.1.0 .. IMPALA-10106: Upgrade DataSketches to version 2.1.0 Upgrade the external DataSketches files for HLL/KLL to version 2.1.0 tests: -Ran the tests from tests/query_test/test_datasketches.py Change-Id: I4faa31c0b628a62c7e56a6c4b9549d0aaa8a02ff --- M be/src/thirdparty/datasketches/README.md M be/src/thirdparty/datasketches/kll_quantile_calculator.hpp M be/src/thirdparty/datasketches/kll_quantile_calculator_impl.hpp M be/src/thirdparty/datasketches/kll_sketch_impl.hpp 4 files changed, 106 insertions(+), 120 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/60/16360/4 -- To view, visit http://gerrit.cloudera.org:8080/16360 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4faa31c0b628a62c7e56a6c4b9549d0aaa8a02ff Gerrit-Change-Number: 16360 Gerrit-PatchSet: 4 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10106: Upgrade DataSketches to version 2.1.0
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/16360 ) Change subject: IMPALA-10106: Upgrade DataSketches to version 2.1.0 .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/16360/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16360/3//COMMIT_MSG@7 PS3, Line 7: to version 2.1.0 > This is not what I meant. Please add the DataSketches version here that we Ah, sorry, I misunderstood, done -- To view, visit http://gerrit.cloudera.org:8080/16360 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4faa31c0b628a62c7e56a6c4b9549d0aaa8a02ff Gerrit-Change-Number: 16360 Gerrit-PatchSet: 4 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 26 Aug 2020 13:45:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10108: Implement ds kll stringify function
Adam Tamas has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/16370 ) Change subject: IMPALA-10108: Implement ds_kll_stringify function .. IMPALA-10108: Implement ds_kll_stringify function This function receives a string that is a serialized Apache DataSketches KLL sketch and returns its stringified format. Change-Id: I97f654a4838bf91e3e0bed6a00d78b2c7aa96f75 --- M be/src/exprs/datasketches-functions-ir.cc M be/src/exprs/datasketches-functions.h M common/function-registry/impala_functions.py M testdata/workloads/functional-query/queries/QueryTest/datasketches-kll.test 4 files changed, 59 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/70/16370/3 -- To view, visit http://gerrit.cloudera.org:8080/16370 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I97f654a4838bf91e3e0bed6a00d78b2c7aa96f75 Gerrit-Change-Number: 16370 Gerrit-PatchSet: 3 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10108: Implement ds kll stringify function
Adam Tamas has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/16370 ) Change subject: IMPALA-10108: Implement ds_kll_stringify function .. IMPALA-10108: Implement ds_kll_stringify function This function receives a string that is a serialized Apache DataSketches KLL sketch and returns its stringified format. Change-Id: I97f654a4838bf91e3e0bed6a00d78b2c7aa96f75 --- M be/src/exprs/datasketches-functions-ir.cc M be/src/exprs/datasketches-functions.h M common/function-registry/impala_functions.py M testdata/workloads/functional-query/queries/QueryTest/datasketches-kll.test 4 files changed, 59 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/70/16370/2 -- To view, visit http://gerrit.cloudera.org:8080/16370 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I97f654a4838bf91e3e0bed6a00d78b2c7aa96f75 Gerrit-Change-Number: 16370 Gerrit-PatchSet: 2 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10108: Implement ds kll stringify function
Adam Tamas has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16370 Change subject: IMPALA-10108: Implement ds_kll_stringify function .. IMPALA-10108: Implement ds_kll_stringify function This function receives a string that is a serialized Apache DataSketches KLL sketch and returns its stringified format. Change-Id: I97f654a4838bf91e3e0bed6a00d78b2c7aa96f75 --- M be/src/exprs/datasketches-functions-ir.cc M be/src/exprs/datasketches-functions.h M common/function-registry/impala_functions.py M testdata/workloads/functional-query/queries/QueryTest/datasketches-kll.test 4 files changed, 59 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/70/16370/1 -- To view, visit http://gerrit.cloudera.org:8080/16370 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I97f654a4838bf91e3e0bed6a00d78b2c7aa96f75 Gerrit-Change-Number: 16370 Gerrit-PatchSet: 1 Gerrit-Owner: Adam Tamas
[Impala-ASF-CR] IMPALA-10106: Upgrade DataSketches in be/src/thirdparty
Adam Tamas has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/16360 ) Change subject: IMPALA-10106: Upgrade DataSketches in be/src/thirdparty .. IMPALA-10106: Upgrade DataSketches in be/src/thirdparty Upgrade the external DataSketches files for HLL/KLL to version 2.1.0 tests: -Ran the tests from tests/query_test/test_datasketches.py Change-Id: I4faa31c0b628a62c7e56a6c4b9549d0aaa8a02ff --- M be/src/thirdparty/datasketches/README.md M be/src/thirdparty/datasketches/kll_quantile_calculator.hpp M be/src/thirdparty/datasketches/kll_quantile_calculator_impl.hpp M be/src/thirdparty/datasketches/kll_sketch_impl.hpp 4 files changed, 106 insertions(+), 120 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/60/16360/3 -- To view, visit http://gerrit.cloudera.org:8080/16360 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4faa31c0b628a62c7e56a6c4b9549d0aaa8a02ff Gerrit-Change-Number: 16360 Gerrit-PatchSet: 3 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10106: Upgrade DataSketches in be/src/thirdparty
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/16360 ) Change subject: IMPALA-10106: Upgrade DataSketches in be/src/thirdparty .. Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/16360/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16360/2//COMMIT_MSG@7 PS2, Line 7: Upgrade DataSketche > could you be more specific in the title of the commit message? You can add Done and done. http://gerrit.cloudera.org:8080/#/c/16360/2/be/src/thirdparty/datasketches/README.md File be/src/thirdparty/datasketches/README.md: http://gerrit.cloudera.org:8080/#/c/16360/2/be/src/thirdparty/datasketches/README.md@11 PS2, Line 11: 2.1.0-incubating > As far as I know branches ending with ".x" are not final and they can still Ah, yes, sorry, corrected. (2.1.x is the same as the 2.1.0-rc1 tagged one at the moment, so I will only change the commit message and readme for now and we can wait until it became the official 2.1.0) -- To view, visit http://gerrit.cloudera.org:8080/16360 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4faa31c0b628a62c7e56a6c4b9549d0aaa8a02ff Gerrit-Change-Number: 16360 Gerrit-PatchSet: 3 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 25 Aug 2020 14:03:26 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10106: Update DataSketches
Adam Tamas has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16360 Change subject: IMPALA-10106: Update DataSketches .. IMPALA-10106: Update DataSketches Update the external DataSketches files for HLL/KLL to version 2.1.x tests: -Ran the tests from tests/query_test/test_datasketches.py Change-Id: I4faa31c0b628a62c7e56a6c4b9549d0aaa8a02ff --- M be/src/thirdparty/datasketches/README.md M be/src/thirdparty/datasketches/kll_quantile_calculator.hpp M be/src/thirdparty/datasketches/kll_quantile_calculator_impl.hpp M be/src/thirdparty/datasketches/kll_sketch_impl.hpp 4 files changed, 106 insertions(+), 120 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/60/16360/2 -- To view, visit http://gerrit.cloudera.org:8080/16360 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I4faa31c0b628a62c7e56a6c4b9549d0aaa8a02ff Gerrit-Change-Number: 16360 Gerrit-PatchSet: 2 Gerrit-Owner: Adam Tamas
[Impala-ASF-CR] IMPALA-9982: Fix flakyness in test dateless timestamp text
Adam Tamas has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16334 Change subject: IMPALA-9982: Fix flakyness in test_dateless_timestamp_text .. IMPALA-9982: Fix flakyness in test_dateless_timestamp_text In this test there is no need to check for "Error parsing row" since the "Error converting column" is enought to be sure we are no longer able to read dateless timestamps. Change-Id: Ia97490288dae81561969d260739a07ec42571f48 --- M testdata/workloads/functional-query/queries/QueryTest/dateless_timestamp_text.test 1 file changed, 0 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/34/16334/1 -- To view, visit http://gerrit.cloudera.org:8080/16334 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ia97490288dae81561969d260739a07ec42571f48 Gerrit-Change-Number: 16334 Gerrit-PatchSet: 1 Gerrit-Owner: Adam Tamas
[Impala-ASF-CR] WIP IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/16305 ) Change subject: WIP IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix .. Patch Set 4: (5 comments) http://gerrit.cloudera.org:8080/#/c/16305/3/shell/impala_client.py File shell/impala_client.py: http://gerrit.cloudera.org:8080/#/c/16305/3/shell/impala_client.py@1100 PS3, Line 1100: try: > It could be potentially faster if the exception was caught per row instead Done http://gerrit.cloudera.org:8080/#/c/16305/3/shell/impala_shell.py File shell/impala_shell.py: http://gerrit.cloudera.org:8080/#/c/16305/3/shell/impala_shell.py@1191 PS3, Line 1191: # retrieve the error log > Where does the exception come from? Here, it can come from 2 different sources depending on which formatter we are using (DelimitedOutputFormatter or PrettyOutputFormatter from the shell_output.py) Moved the check there. This is only true, if we are using the gen-py generated by thrift 0.9.3, if it is 0.11.0 the error is coming from the thrift autogenerated files. (Here is no solution for thrift 0.11.0 yet) http://gerrit.cloudera.org:8080/#/c/16305/3/shell/impala_shell.py@1197 PS3, Line 1197: if > What is "i", a character? I would prefer a name like "ch" in that case, as THANKS! This was a problem I was looking a solution for. (The Dwarves delved too greedily and too deep) http://gerrit.cloudera.org:8080/#/c/16305/3/tests/shell/test_shell_interactive.py File tests/shell/test_shell_interactive.py: http://gerrit.cloudera.org:8080/#/c/16305/3/tests/shell/test_shell_interactive.py@289 PS3, Line 289: > I would prefer to test against a predefined string, e.g. one that contains While it is theoretically done, since the tests use thrift 0.11.0 this test will fail for now. http://gerrit.cloudera.org:8080/#/c/16305/3/tests/shell/test_shell_interactive.py@292 PS3, Line 292: child_proc = spawn_shell(shell_cmd) > This path tests the shell when it uses the pretty printer - can you also te Done -- To view, visit http://gerrit.cloudera.org:8080/16305 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8 Gerrit-Change-Number: 16305 Gerrit-PatchSet: 4 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 13 Aug 2020 10:05:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] WIP IMPALA-10012: ds hll sketch() results ascii codec decoding error fix
Adam Tamas has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16305 Change subject: WIP IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix .. WIP IMPALA-10012: ds_hll_sketch() results ascii codec decoding error fix While the ds_hll_sketch() generates a string value as output the date is not an ascii encoded text but a bitsketch, because of this, when the shell get this data it disconnect while it tries to decode it. This patch contains a solution, where we replace any not UTF-8 decodeable characters if we run into an UnicodeDecodeError. This solution is working with the Thrift 0.9.3 autogegnerated gen-py but still fails with Thrift 0.11.0. Testing: -manual testing with these protocols: 'hs2-http', 'hs2', 'beeswax' (-test added to tests/shell/test_shell_interactive.py) Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8 --- M shell/impala_client.py M shell/impala_shell.py M shell/shell_output.py M tests/shell/test_shell_interactive.py 4 files changed, 44 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/05/16305/4 -- To view, visit http://gerrit.cloudera.org:8080/16305 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ic5cfb907871ca83e5f04a39ca9d7a8e138d711a8 Gerrit-Change-Number: 16305 Gerrit-PatchSet: 4 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab
[Impala-ASF-CR] IMPALA-9962: Implement ds kll quantiles() function
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/16324 ) Change subject: IMPALA-9962: Implement ds_kll_quantiles() function .. Patch Set 3: Code-Review+1 LGTM! -- To view, visit http://gerrit.cloudera.org:8080/16324 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76f6039977f4e14ded89a3ee4bc4e6ff855f5e7f Gerrit-Change-Number: 16324 Gerrit-PatchSet: 3 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 12 Aug 2020 10:06:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9962: Implement ds kll quantiles() function
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/16324 ) Change subject: IMPALA-9962: Implement ds_kll_quantiles() function .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/16324/2/testdata/workloads/functional-query/queries/QueryTest/datasketches-kll.test File testdata/workloads/functional-query/queries/QueryTest/datasketches-kll.test: http://gerrit.cloudera.org:8080/#/c/16324/2/testdata/workloads/functional-query/queries/QueryTest/datasketches-kll.test@357 PS2, Line 357: : QUERY : select : ds_kll_quantiles(ds_kll_sketch(float_col), 0, 0.2, NULL, 0.8, 1) : from functional_parquet.alltypessmall; : RESULTS : '0,1.1,0,7.7,9.9' : TYPES : STRING : : QUERY : select : ds_kll_quantiles(ds_kll_sketch(float_col), NULL) : from functional_parquet.alltypessmall; : RESULTS : 'NULL' : TYPES : STRING : Is this alright that if there is a NULL argument given, then the result is 0/NULL depending if there are other arguments? Shouldn't this be generalized? -- To view, visit http://gerrit.cloudera.org:8080/16324 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76f6039977f4e14ded89a3ee4bc4e6ff855f5e7f Gerrit-Change-Number: 16324 Gerrit-PatchSet: 2 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 11 Aug 2020 13:33:12 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10066: Fix test cancellation mid command fails
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/16322 ) Change subject: IMPALA-10066: Fix test_cancellation_mid_command fails .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/16322/3/tests/shell/test_shell_interactive.py File tests/shell/test_shell_interactive.py: http://gerrit.cloudera.org:8080/#/c/16322/3/tests/shell/test_shell_interactive.py@a265 PS3, Line 265: > What's the reason this can now run in parallel with other tests? Shouldn't While it is a cancellation test, we are cancelling a query in its writing phase and not while it's already in the flight, so it should only affect the shell. Since we are using pexpect spawn, this test has its own impala shell to work with. -- To view, visit http://gerrit.cloudera.org:8080/16322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib80706d52a85d2c19b13fbbe5695934658c0bf7e Gerrit-Change-Number: 16322 Gerrit-PatchSet: 3 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 11 Aug 2020 12:23:21 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10066: Fix test cancellation mid command fails
Adam Tamas has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16322 Change subject: IMPALA-10066: Fix test_cancellation_mid_command fails .. IMPALA-10066: Fix test_cancellation_mid_command fails If there is a delay between the shell "quit;" command and the Pexpect's wait() then the child process will be already dead and we end up with a Pexpect error. Since this error has nothing to do with the essence of the test we can ignore it. Testing: -ran the test_cancellation_mid_command() in a 1..500 loop Change-Id: Ib80706d52a85d2c19b13fbbe5695934658c0bf7e --- M tests/shell/test_shell_interactive.py 1 file changed, 10 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/16322/3 -- To view, visit http://gerrit.cloudera.org:8080/16322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ib80706d52a85d2c19b13fbbe5695934658c0bf7e Gerrit-Change-Number: 16322 Gerrit-PatchSet: 3 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Gabor Kaszab
[Impala-ASF-CR] IMPALA-9963: Implement ds kll n() function
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/16259 ) Change subject: IMPALA-9963: Implement ds_kll_n() function .. Patch Set 4: Code-Review+1 LGTM! -- To view, visit http://gerrit.cloudera.org:8080/16259 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I166e87a468e68e888ac15fca7429ac2552dbb781 Gerrit-Change-Number: 16259 Gerrit-PatchSet: 4 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 05 Aug 2020 12:28:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10018: Implement ds kll rank() function
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/16283 ) Change subject: IMPALA-10018: Implement ds_kll_rank() function .. Patch Set 2: Code-Review+1 LGTM! -- To view, visit http://gerrit.cloudera.org:8080/16283 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I95857886dfbb8c84aeeaf718c0e610012fda4be0 Gerrit-Change-Number: 16283 Gerrit-PatchSet: 2 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 04 Aug 2020 12:12:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10018: Implement ds kll rank() function
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/16283 ) Change subject: IMPALA-10018: Implement ds_kll_rank() function .. Patch Set 1: (1 comment) Hi Gabor, Thank you for the good work with the KLL functions. Apart from a general nit, it looks good to me. http://gerrit.cloudera.org:8080/#/c/16283/1/be/src/exprs/datasketches-functions.h File be/src/exprs/datasketches-functions.h: http://gerrit.cloudera.org:8080/#/c/16283/1/be/src/exprs/datasketches-functions.h@47 PS1, Line 47: not then nit: missing comma As far as I see, it is missing in every comment where this sentence is used. -- To view, visit http://gerrit.cloudera.org:8080/16283 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I95857886dfbb8c84aeeaf718c0e610012fda4be0 Gerrit-Change-Number: 16283 Gerrit-PatchSet: 1 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 04 Aug 2020 08:39:11 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9963: Implement ds kll n() function
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/16259 ) Change subject: IMPALA-9963: Implement ds_kll_n() function .. Patch Set 1: Code-Review+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/16259/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16259/1//COMMIT_MSG@9 PS1, Line 9: s I think this should be in singular. -- To view, visit http://gerrit.cloudera.org:8080/16259 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I166e87a468e68e888ac15fca7429ac2552dbb781 Gerrit-Change-Number: 16259 Gerrit-PatchSet: 1 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 30 Jul 2020 15:13:25 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9942: DataSketches HLL shouldn't take empty strings as distinct values
Adam Tamas has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/16226 ) Change subject: IMPALA-9942: DataSketches HLL shouldn't take empty strings as distinct values .. IMPALA-9942: DataSketches HLL shouldn't take empty strings as distinct values In Hive empty strings doesn't count as separate values when querying count(distinct) estimates using Apache DataSketches HLL algorithm on strings and varchars. For compatibility's sake Impala should not take it either. Tests: -added extra tests for hll with empty strings Change-Id: Ie7648217bbe2f66b817788f131c062f349b1e9ad --- M be/src/exprs/aggregate-functions-ir.cc M testdata/workloads/functional-query/queries/QueryTest/datasketches-hll.test 2 files changed, 28 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/16226/5 -- To view, visit http://gerrit.cloudera.org:8080/16226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie7648217bbe2f66b817788f131c062f349b1e9ad Gerrit-Change-Number: 16226 Gerrit-PatchSet: 5 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9942: DataSketches HLL shouldn't take empty strings as distinct values
Adam Tamas has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/16226 ) Change subject: IMPALA-9942: DataSketches HLL shouldn't take empty strings as distinct values .. IMPALA-9942: DataSketches HLL shouldn't take empty strings as distinct values In Hive empty strings doesn't count as separate values when querying count(distinct) estimates using Apache DataSketches HLL algorithm on strings and varchars. For compatibility's sake Impala should not take it either. Tests: -added extra tests for hll with empty strings Change-Id: Ie7648217bbe2f66b817788f131c062f349b1e9ad --- M be/src/exprs/aggregate-functions-ir.cc M testdata/workloads/functional-query/queries/QueryTest/datasketches-hll.test 2 files changed, 26 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/16226/3 -- To view, visit http://gerrit.cloudera.org:8080/16226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie7648217bbe2f66b817788f131c062f349b1e9ad Gerrit-Change-Number: 16226 Gerrit-PatchSet: 3 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9942: DataSketches HLL shouldn't take empty strings as distinct values
Adam Tamas has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/16226 ) Change subject: IMPALA-9942: DataSketches HLL shouldn't take empty strings as distinct values .. IMPALA-9942: DataSketches HLL shouldn't take empty strings as distinct values In Hive empty strings doesn't count as separate values when querying count(distinct) estimates using Apache DataSketches HLL algorithm on strings and varchars. For compatibility's sake Impala should not take it either. Tests: -added extra tests for hll with empty strings Change-Id: Ie7648217bbe2f66b817788f131c062f349b1e9ad --- M be/src/exprs/aggregate-functions-ir.cc M testdata/workloads/functional-query/queries/QueryTest/datasketches-hll.test 2 files changed, 26 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/16226/4 -- To view, visit http://gerrit.cloudera.org:8080/16226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie7648217bbe2f66b817788f131c062f349b1e9ad Gerrit-Change-Number: 16226 Gerrit-PatchSet: 4 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9942: DataSketches HLL shouldn't take empty strings as distinct values
Adam Tamas has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16226 Change subject: IMPALA-9942: DataSketches HLL shouldn't take empty strings as distinct values .. IMPALA-9942: DataSketches HLL shouldn't take empty strings as distinct values On Hive DataSketches HLL empty string did not count to the end result on strings and varchars and for compatibility's shake Impala should not take it either. Tests: -added extra tests for hll with empty strings Change-Id: Ie7648217bbe2f66b817788f131c062f349b1e9ad --- M be/src/exprs/aggregate-functions-ir.cc M testdata/workloads/functional-query/queries/QueryTest/datasketches-hll.test 2 files changed, 24 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/16226/2 -- To view, visit http://gerrit.cloudera.org:8080/16226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ie7648217bbe2f66b817788f131c062f349b1e9ad Gerrit-Change-Number: 16226 Gerrit-PatchSet: 2 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Gabor Kaszab
[Impala-ASF-CR] IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS
Adam Tamas has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/16199 ) Change subject: IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS .. IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS In "show tables" ANY privilege was used, whereas in "show functions" the required privilege was VIEW_METADATA. To solve the inconsistency "show functions" will use ANY instead of VIEW_METADATA similar to "show tables". After this, an user granted only the privilege of CREATE is now able to execute "show functions" after this patch, making it easier for the user to manage the functions it creates. Testing: -Ran CORE tests. -Added new tests to check the privilege. Change-Id: I9ae7546c206daaf98ecc3de449069027c43c6e1a --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/ShowFunctionsStmt.java M fe/src/test/java/org/apache/impala/analysis/AuditingTest.java M tests/authorization/test_ranger.py 4 files changed, 58 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/16199/8 -- To view, visit http://gerrit.cloudera.org:8080/16199 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9ae7546c206daaf98ecc3de449069027c43c6e1a Gerrit-Change-Number: 16199 Gerrit-PatchSet: 8 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS
Adam Tamas has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/16199 ) Change subject: IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS .. IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS In "show tables" ANY privilege was used, whereas in "show functions" the required privilege was VIEW_METADATA. To solve the inconsistency "show functions" will use ANY instead of VIEW_METADATA similar to "show tables". After this, an user granted only the privilege of CREATE is now able to execute "show functions" after this patch, making it easier for the user to manage the functions it creates. Testing: -Ran CORE tests. -Added new tests to check the privilege. Change-Id: I9ae7546c206daaf98ecc3de449069027c43c6e1a --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/ShowFunctionsStmt.java M fe/src/test/java/org/apache/impala/analysis/AuditingTest.java M tests/authorization/test_ranger.py 4 files changed, 61 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/16199/7 -- To view, visit http://gerrit.cloudera.org:8080/16199 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9ae7546c206daaf98ecc3de449069027c43c6e1a Gerrit-Change-Number: 16199 Gerrit-PatchSet: 7 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS
Adam Tamas has posted comments on this change. (
http://gerrit.cloudera.org:8080/16199 )
Change subject: IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES
and SHOW FUNCTIONS
..
Patch Set 7:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/16199/6/tests/authorization/test_ranger.py
File tests/authorization/test_ranger.py:
http://gerrit.cloudera.org:8080/#/c/16199/6/tests/authorization/test_ranger.py@830
PS6, Line 830: admin_client.execute("create database
{0}".format(unique_database), user=ADMIN)
> Could you create a function in unique_database and verify it in the show fu
Thanks for the suggestion. Done.
http://gerrit.cloudera.org:8080/#/c/16199/6/tests/authorization/test_ranger.py@841
PS6, Line 841:
: try:
> Why should we need "with grant option" here?
We don't.
--
To view, visit http://gerrit.cloudera.org:8080/16199
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9ae7546c206daaf98ecc3de449069027c43c6e1a
Gerrit-Change-Number: 16199
Gerrit-PatchSet: 7
Gerrit-Owner: Adam Tamas
Gerrit-Reviewer: Adam Tamas
Gerrit-Reviewer: Fang-Yu Rao
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Quanlong Huang
Gerrit-Comment-Date: Fri, 17 Jul 2020 13:56:13 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS
Adam Tamas has posted comments on this change. (
http://gerrit.cloudera.org:8080/16199 )
Change subject: IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES
and SHOW FUNCTIONS
..
Patch Set 6:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/16199/3/tests/authorization/test_ranger.py
File tests/authorization/test_ranger.py:
http://gerrit.cloudera.org:8080/#/c/16199/3/tests/authorization/test_ranger.py@841
PS3, Line 841: "grant {0} on
database {1} to user {2} with "
> The variable of 'result' is not used afterwards. Maybe we could remove it.
Ah, I missed it.
Thank you for bringing this to my attention.
Done.
--
To view, visit http://gerrit.cloudera.org:8080/16199
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9ae7546c206daaf98ecc3de449069027c43c6e1a
Gerrit-Change-Number: 16199
Gerrit-PatchSet: 6
Gerrit-Owner: Adam Tamas
Gerrit-Reviewer: Adam Tamas
Gerrit-Reviewer: Fang-Yu Rao
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Thu, 16 Jul 2020 19:19:54 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS
Adam Tamas has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/16199 ) Change subject: IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS .. IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS In "show tables" ANY privilege was used, whereas in "show functions" the required privilege was VIEW_METADATA. To solve the inconsistency "show functions" will use ANY instead of VIEW_METADATA similar to "show tables". After this, an user granted only the privilege of CREATE is now able to execute "show functions" after this patch, making it easier for the user to manage the functions it creates. Testing: -Ran CORE tests. -Added new tests to check the privilege. Change-Id: I9ae7546c206daaf98ecc3de449069027c43c6e1a --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/ShowFunctionsStmt.java M fe/src/test/java/org/apache/impala/analysis/AuditingTest.java M tests/authorization/test_ranger.py 4 files changed, 45 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/16199/3 -- To view, visit http://gerrit.cloudera.org:8080/16199 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9ae7546c206daaf98ecc3de449069027c43c6e1a Gerrit-Change-Number: 16199 Gerrit-PatchSet: 3 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS
Adam Tamas has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/16199 ) Change subject: IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS .. IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS In "show tables" ANY privilege was used, whereas in "show functions" the required privilege was VIEW_METADATA. To solve the inconsistency "show functions" will use ANY instead of VIEW_METADATA similar to "show tables". After this, an user granted only the privilege of CREATE is now able to execute "show functions" after this patch, making it easier for the user to manage the functions it creates. Testing: -Ran CORE tests. -Added new tests to check the privilege. Change-Id: I9ae7546c206daaf98ecc3de449069027c43c6e1a --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/ShowFunctionsStmt.java M fe/src/test/java/org/apache/impala/analysis/AuditingTest.java M tests/authorization/test_ranger.py 4 files changed, 53 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/16199/6 -- To view, visit http://gerrit.cloudera.org:8080/16199 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9ae7546c206daaf98ecc3de449069027c43c6e1a Gerrit-Change-Number: 16199 Gerrit-PatchSet: 6 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS
Adam Tamas has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/16199 ) Change subject: IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS .. IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS In "show tables" ANY privilege was used, whereas in "show functions" the required privilege was VIEW_METADATA. To solve the inconsistency "show functions" will use ANY instead of VIEW_METADATA similar to "show tables". After this, an user granted only the privilege of CREATE is now able to execute "show functions" after this patch, making it easier for the user to manage the functions it creates. Testing: -Ran CORE tests. -Added new tests to check the privilege. Change-Id: I9ae7546c206daaf98ecc3de449069027c43c6e1a --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/ShowFunctionsStmt.java M fe/src/test/java/org/apache/impala/analysis/AuditingTest.java M tests/authorization/test_ranger.py 4 files changed, 53 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/16199/5 -- To view, visit http://gerrit.cloudera.org:8080/16199 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9ae7546c206daaf98ecc3de449069027c43c6e1a Gerrit-Change-Number: 16199 Gerrit-PatchSet: 5 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS
Adam Tamas has posted comments on this change. (
http://gerrit.cloudera.org:8080/16199 )
Change subject: IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES
and SHOW FUNCTIONS
..
Patch Set 4:
> Hi Adam, thanks for working on this patch!
>
> The patch looks good to me since you have implemented what Fredy
> had suggested at https://issues.apache.org/jira/browse/IMPALA-7001.
> I only have two minor comments regarding the test and the commit
> message.
>
> Specifically, after your patch, a user granted only the privilege
> of CREATE on a specified database, e.g., functional, would be able
> to execute a statement like "SHOW FUNCTIONS IN functional", since
> according to
> https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/Privilege.java
> and
> https://github.com/apache/impala/blob/3a6022ce80ca1cedb629400b18caaf0d1f54137c/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java#L431-L453,
> such a statement would succeed as long as the user is granted any
> privilege in the set {ALL, OWNER, ALTER, DROP, CREATE, INSERT,
> SELECT, REFRESH}.
>
> Before your patch, in order for the statement above to succeed, a
> user has to be granted any privilege in the set {INSERT, SELECT,
> REFRESH}. Thus I think it would be good to add one more test case
> in
> https://github.com/apache/impala/blob/master/tests/authorization/test_ranger.py,
> where we 1) grant the privilege of CREATE to a user (as
> admin_client), and 2) execute a statement like "SHOW FUNCTIONS IN
> unique_database" to verify there is no exception thrown.
>
> On the other hand, I think it may also be good to provide more
> detail of the difference before and after the patch. For instance,
> we could mention that a user granted only the privilege of CREATE
> is now able to execute that SQL statement above after this patch,
> making it easier for the user to manage the functions it creates.
Hi and thank you for the review, I updated it based on the suggestions.
--
To view, visit http://gerrit.cloudera.org:8080/16199
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9ae7546c206daaf98ecc3de449069027c43c6e1a
Gerrit-Change-Number: 16199
Gerrit-PatchSet: 4
Gerrit-Owner: Adam Tamas
Gerrit-Reviewer: Adam Tamas
Gerrit-Reviewer: Fang-Yu Rao
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Thu, 16 Jul 2020 16:29:55 +
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS
Adam Tamas has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/16199 ) Change subject: IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS .. IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS In "show tables" ANY privilege was used, whereas in "show functions" the required privilege was VIEW_METADATA. To solve the inconsistency "show functions" will use ANY instead of VIEW_METADATA similar to "show tables". After this, an user granted only the privilege of CREATE is now able to execute "show functions" after this patch, making it easier for the user to manage the functions it creates. Testing: -Ran CORE tests. -Added new tests to check the privilege. Change-Id: I9ae7546c206daaf98ecc3de449069027c43c6e1a --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/ShowFunctionsStmt.java M fe/src/test/java/org/apache/impala/analysis/AuditingTest.java M tests/authorization/test_ranger.py 4 files changed, 50 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/16199/4 -- To view, visit http://gerrit.cloudera.org:8080/16199 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9ae7546c206daaf98ecc3de449069027c43c6e1a Gerrit-Change-Number: 16199 Gerrit-PatchSet: 4 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS
Adam Tamas has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/16199 ) Change subject: IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS .. IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS In "show tables" ANY privilege was used, whereas in "show functions" the required privilege was VIEW_METADATA. To solve the inconsistency "show functions" will use ANY instead of VIEW_METADATA similar to "show tables". Testing: -Ran CORE tests. -Added new test to check the privilege. Change-Id: I9ae7546c206daaf98ecc3de449069027c43c6e1a --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/ShowFunctionsStmt.java M fe/src/test/java/org/apache/impala/analysis/AuditingTest.java 3 files changed, 15 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/16199/2 -- To view, visit http://gerrit.cloudera.org:8080/16199 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9ae7546c206daaf98ecc3de449069027c43c6e1a Gerrit-Change-Number: 16199 Gerrit-PatchSet: 2 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS
Adam Tamas has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16199 Change subject: IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS .. IMPALA-7001: Fix Privilege inconsistency between SHOW TABLES and SHOW FUNCTIONS In "show tables" ANY privilege was used, whereas in "show functions" the required privilege was VIEW_METADATA. To solve the inconsistency "show functions" will use ANY instead of VIEW_METADATA similar to "show tables". Testing: -Ran CORE tests. -Added new test to check the privilege. Change-Id: I9ae7546c206daaf98ecc3de449069027c43c6e1a --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/ShowFunctionsStmt.java M fe/src/test/java/org/apache/impala/analysis/AuditingTest.java 3 files changed, 13 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/16199/1 -- To view, visit http://gerrit.cloudera.org:8080/16199 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I9ae7546c206daaf98ecc3de449069027c43c6e1a Gerrit-Change-Number: 16199 Gerrit-PatchSet: 1 Gerrit-Owner: Adam Tamas
[Impala-ASF-CR] IMPALA-9941: ExprTest.CastExprs fails when running with ASAN fix
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/16179 ) Change subject: IMPALA-9941: ExprTest.CastExprs fails when running with ASAN fix .. Patch Set 6: > (2 comments) > > The patch seems fine. Could you comment on the Jira for IMPALA-9531 > that this (IMPALA-9941) is also required? Done and thank you for reviewing! -- To view, visit http://gerrit.cloudera.org:8080/16179 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I75bd47b6627a2e338c46dc354f7e67d34c1abbcf Gerrit-Change-Number: 16179 Gerrit-PatchSet: 6 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 14 Jul 2020 09:10:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9941: ExprTest.CastExprs fails when running with ASAN fix
Adam Tamas has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/16179 ) Change subject: IMPALA-9941: ExprTest.CastExprs fails when running with ASAN fix .. IMPALA-9941: ExprTest.CastExprs fails when running with ASAN fix Fixed the issue with the ASAN build where too short tokens in timestamps fails because of indexing out of bounds because of a missing check. Updated some missed files connected to the drop of dateless timestamps (IMPALA-9531) regarding tests, comments. Testing: - Ran CORE tests - Ran ASAN with EE_TEST_SHARDS=6 (with the help of IMPALA-9887) Change-Id: I75bd47b6627a2e338c46dc354f7e67d34c1abbcf --- M be/src/exprs/expr-test.cc M be/src/runtime/date-parse-util.cc M be/src/runtime/datetime-simple-date-format-parser.cc M be/src/runtime/datetime-simple-date-format-parser.h M be/src/runtime/timestamp-parse-util.cc M be/src/runtime/timestamp-parse-util.h 6 files changed, 67 insertions(+), 72 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/79/16179/5 -- To view, visit http://gerrit.cloudera.org:8080/16179 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I75bd47b6627a2e338c46dc354f7e67d34c1abbcf Gerrit-Change-Number: 16179 Gerrit-PatchSet: 5 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9941: ExprTest.CastExprs fails when running with ASAN fix
Adam Tamas has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/16179 ) Change subject: IMPALA-9941: ExprTest.CastExprs fails when running with ASAN fix .. IMPALA-9941: ExprTest.CastExprs fails when running with ASAN fix Fixed the issue with the ASAN build where too short tokens in a timestamps fails becouse of indexing out of bounds becouse of a missing check. Updated files connected to the drop of dateless timestamps. Testing: - Ran CORE tests - Ran ASAN with EE_TEST_SHARDS=6 (with the help of IMPALA-9887) Change-Id: I75bd47b6627a2e338c46dc354f7e67d34c1abbcf --- M be/src/exprs/expr-test.cc M be/src/runtime/date-parse-util.cc M be/src/runtime/datetime-simple-date-format-parser.cc M be/src/runtime/datetime-simple-date-format-parser.h M be/src/runtime/timestamp-parse-util.cc M be/src/runtime/timestamp-parse-util.h 6 files changed, 67 insertions(+), 72 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/79/16179/4 -- To view, visit http://gerrit.cloudera.org:8080/16179 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I75bd47b6627a2e338c46dc354f7e67d34c1abbcf Gerrit-Change-Number: 16179 Gerrit-PatchSet: 4 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9941: ExprTest.CastExprs fails when running with ASAN fix
Adam Tamas has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/16179 ) Change subject: IMPALA-9941: ExprTest.CastExprs fails when running with ASAN fix .. IMPALA-9941: ExprTest.CastExprs fails when running with ASAN fix Fixed the issue with the ASAN build where only date tokens with wrong format in a timestamps fails. Updated files connected to the drop of dateless timestamps. Testxing: - Ran CORE tests - Ran ASAN with EE_TEST_SHARDS=6 (with the help of IMPALA-9887) Change-Id: I75bd47b6627a2e338c46dc354f7e67d34c1abbcf --- M be/src/exprs/expr-test.cc M be/src/runtime/date-parse-util.cc M be/src/runtime/datetime-simple-date-format-parser.cc M be/src/runtime/datetime-simple-date-format-parser.h M be/src/runtime/timestamp-parse-util.cc M be/src/runtime/timestamp-parse-util.h 6 files changed, 64 insertions(+), 70 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/79/16179/3 -- To view, visit http://gerrit.cloudera.org:8080/16179 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I75bd47b6627a2e338c46dc354f7e67d34c1abbcf Gerrit-Change-Number: 16179 Gerrit-PatchSet: 3 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] WIP IMPALA-9941: ExprTest.CastExprs fails when running with ASAN fix
Adam Tamas has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16179 Change subject: WIP IMPALA-9941: ExprTest.CastExprs fails when running with ASAN fix .. WIP IMPALA-9941: ExprTest.CastExprs fails when running with ASAN fix Testxing: - Ran ASAN with EE_TEST_SHARDS=6 Change-Id: I75bd47b6627a2e338c46dc354f7e67d34c1abbcf --- M be/src/exprs/expr-test.cc M be/src/runtime/date-parse-util.cc M be/src/runtime/datetime-simple-date-format-parser.cc M be/src/runtime/datetime-simple-date-format-parser.h M be/src/runtime/timestamp-parse-util.cc M be/src/runtime/timestamp-parse-util.h 6 files changed, 64 insertions(+), 70 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/79/16179/2 -- To view, visit http://gerrit.cloudera.org:8080/16179 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I75bd47b6627a2e338c46dc354f7e67d34c1abbcf Gerrit-Change-Number: 16179 Gerrit-PatchSet: 2 Gerrit-Owner: Adam Tamas
[Impala-ASF-CR] IMPALA-9531: Dropped support for dateless timestamps
Adam Tamas has uploaded a new patch set (#12). (
http://gerrit.cloudera.org:8080/15866 )
Change subject: IMPALA-9531: Dropped support for dateless timestamps
..
IMPALA-9531: Dropped support for dateless timestamps
Removed the support for dateless timestamps.
During dateless timestamp casts if the format doesn't contain
date part we get an error during tokenization of the format.
If the input str doesn't contain a date part then we get null result.
Examples:
select cast('01:02:59' as timestamp);
This will come back as NULL value.
select to_timestamp('01:01:01', 'HH:mm:ss');
select cast('01:02:59' as timestamp format 'HH12:MI:SS');
select cast('12 AM' as timestamp FORMAT 'AM.HH12');
These will come back with a parsing errors.
Casting from a table will generate similar results.
Testing:
Modified the previous tests related to dateless timestamps.
Added test to read fromtables which are still containing dateless
timestamps and covered timestamp to string path when no date tokens
are requested in the output string.
Change-Id: I48c49bf027cc4b917849b3d58518facba372b322
---
M be/src/benchmarks/convert-timestamp-benchmark.cc
M be/src/benchmarks/parse-timestamp-benchmark.cc
M be/src/exec/text-converter.inline.h
M be/src/exprs/cast-functions-ir.cc
M be/src/exprs/expr-test.cc
M be/src/exprs/scalar-expr-evaluator.cc
M be/src/exprs/timestamp-functions-ir.cc
M be/src/exprs/timestamp-functions.cc
M be/src/exprs/timestamp-functions.h
M be/src/runtime/date-parse-util.cc
M be/src/runtime/date-test.cc
M be/src/runtime/datetime-iso-sql-format-tokenizer.cc
M be/src/runtime/datetime-parser-common.cc
M be/src/runtime/datetime-parser-common.h
M be/src/runtime/datetime-simple-date-format-parser.cc
M be/src/runtime/datetime-simple-date-format-parser.h
M be/src/runtime/timestamp-parse-util.cc
M be/src/runtime/timestamp-test.cc
M be/src/runtime/timestamp-value.h
M bin/rat_exclude_files.txt
M common/function-registry/impala_functions.py
M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
M testdata/data/README
A testdata/data/dateless_timestamps.parq
A testdata/data/dateless_timestamps.txt
M testdata/data/lazy_timestamp.csv
M testdata/workloads/functional-query/queries/QueryTest/date.test
A
testdata/workloads/functional-query/queries/QueryTest/dateless_timestamp_parquet.test
A
testdata/workloads/functional-query/queries/QueryTest/dateless_timestamp_text.test
M testdata/workloads/functional-query/queries/QueryTest/exprs.test
M
testdata/workloads/functional-query/queries/QueryTest/select-lazy-timestamp.test
M tests/data_errors/test_data_errors.py
M tests/query_test/test_cast_with_format.py
M tests/query_test/test_scanners.py
34 files changed, 278 insertions(+), 231 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/66/15866/12
--
To view, visit http://gerrit.cloudera.org:8080/15866
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I48c49bf027cc4b917849b3d58518facba372b322
Gerrit-Change-Number: 15866
Gerrit-PatchSet: 12
Gerrit-Owner: Adam Tamas
Gerrit-Reviewer: Adam Tamas
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9531: Dropped support for dateless timestamps
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/15866 ) Change subject: IMPALA-9531: Dropped support for dateless timestamps .. Patch Set 10: (2 comments) http://gerrit.cloudera.org:8080/#/c/15866/7/be/src/exprs/timestamp-functions.cc File be/src/exprs/timestamp-functions.cc: http://gerrit.cloudera.org:8080/#/c/15866/7/be/src/exprs/timestamp-functions.cc@154 PS7, Line 154: // The purpose of making this .cc only is to avoid moving the code of > I'd rather say something to indicate that the purpose of making this .cc on Done http://gerrit.cloudera.org:8080/#/c/15866/7/be/src/runtime/datetime-simple-date-format-parser.cc File be/src/runtime/datetime-simple-date-format-parser.cc: http://gerrit.cloudera.org:8080/#/c/15866/7/be/src/runtime/datetime-simple-date-format-parser.cc@56 PS7, Line 56: Tokenize(&DEFAULT_DATE_TIME_CTX[i], PARSE); > It's no longer needed to give the 3rd parameter in case it's true, right? L Done -- To view, visit http://gerrit.cloudera.org:8080/15866 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I48c49bf027cc4b917849b3d58518facba372b322 Gerrit-Change-Number: 15866 Gerrit-PatchSet: 10 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 07 Jul 2020 11:27:41 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9531: Dropped support for dateless timestamps
Adam Tamas has uploaded a new patch set (#10). (
http://gerrit.cloudera.org:8080/15866 )
Change subject: IMPALA-9531: Dropped support for dateless timestamps
..
IMPALA-9531: Dropped support for dateless timestamps
Removed the support for dateless timestamps.
During dateless timestamp casts if the format doesn't contain
date part we get an error during tokenization of the format.
If the input str doesn't contain a date part then we get null result.
Examples:
select cast('01:02:59' as timestamp);
This will come back as NULL value.
select to_timestamp('01:01:01', 'HH:mm:ss');
select cast('01:02:59' as timestamp format 'HH12:MI:SS');
select cast('12 AM' as timestamp FORMAT 'AM.HH12');
These will come back with a parsing errors.
Casting from a table will generate similar results.
Testing:
Modified the previous tests related to dateless timestamps.
Added test to read fromtables which are still containing dateless
timestamps and covered timestamp to string path when no date tokens
are requested in the output string.
Change-Id: I48c49bf027cc4b917849b3d58518facba372b322
---
M be/src/benchmarks/convert-timestamp-benchmark.cc
M be/src/benchmarks/parse-timestamp-benchmark.cc
M be/src/exec/text-converter.inline.h
M be/src/exprs/cast-functions-ir.cc
M be/src/exprs/expr-test.cc
M be/src/exprs/scalar-expr-evaluator.cc
M be/src/exprs/timestamp-functions-ir.cc
M be/src/exprs/timestamp-functions.cc
M be/src/exprs/timestamp-functions.h
M be/src/runtime/date-parse-util.cc
M be/src/runtime/date-test.cc
M be/src/runtime/datetime-iso-sql-format-tokenizer.cc
M be/src/runtime/datetime-parser-common.cc
M be/src/runtime/datetime-parser-common.h
M be/src/runtime/datetime-simple-date-format-parser.cc
M be/src/runtime/datetime-simple-date-format-parser.h
M be/src/runtime/timestamp-parse-util.cc
M be/src/runtime/timestamp-test.cc
M be/src/runtime/timestamp-value.h
M bin/rat_exclude_files.txt
M common/function-registry/impala_functions.py
M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
M testdata/data/README
A testdata/data/dateless_timestamps.parq
A testdata/data/dateless_timestamps.txt
M testdata/data/lazy_timestamp.csv
M testdata/workloads/functional-query/queries/QueryTest/date.test
A
testdata/workloads/functional-query/queries/QueryTest/dateless_timestamp_parquet.test
A
testdata/workloads/functional-query/queries/QueryTest/dateless_timestamp_text.test
M testdata/workloads/functional-query/queries/QueryTest/exprs.test
M
testdata/workloads/functional-query/queries/QueryTest/select-lazy-timestamp.test
M tests/data_errors/test_data_errors.py
M tests/query_test/test_cast_with_format.py
M tests/query_test/test_scanners.py
34 files changed, 277 insertions(+), 231 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/66/15866/10
--
To view, visit http://gerrit.cloudera.org:8080/15866
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I48c49bf027cc4b917849b3d58518facba372b322
Gerrit-Change-Number: 15866
Gerrit-PatchSet: 10
Gerrit-Owner: Adam Tamas
Gerrit-Reviewer: Adam Tamas
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9531: Dropped support for dateless timestamps
Adam Tamas has uploaded a new patch set (#8). (
http://gerrit.cloudera.org:8080/15866 )
Change subject: IMPALA-9531: Dropped support for dateless timestamps
..
IMPALA-9531: Dropped support for dateless timestamps
Removed the support for dateless timestamps.
During dateless timestamp casts if the format doesn't contain
date part we get an error during tokenization of the format.
If the input str doesn't contain a date part then we get null result.
Examples:
select cast('01:02:59' as timestamp);
This will come back as NULL value.
select to_timestamp('01:01:01', 'HH:mm:ss');
select cast('01:02:59' as timestamp format 'HH12:MI:SS');
select cast('12 AM' as timestamp FORMAT 'AM.HH12');
These will come back with a parsing errors.
Casting from a table will generate similar results.
Testing:
Modified the previous tests related to dateless timestamps.
Added test to read fromtables which are still containing dateless
timestamps and covered timestamp to string path when no date tokens
are requested in the output string.
Change-Id: I48c49bf027cc4b917849b3d58518facba372b322
---
M be/src/benchmarks/convert-timestamp-benchmark.cc
M be/src/benchmarks/parse-timestamp-benchmark.cc
M be/src/exec/text-converter.inline.h
M be/src/exprs/cast-functions-ir.cc
M be/src/exprs/expr-test.cc
M be/src/exprs/scalar-expr-evaluator.cc
M be/src/exprs/timestamp-functions-ir.cc
M be/src/exprs/timestamp-functions.cc
M be/src/exprs/timestamp-functions.h
M be/src/runtime/date-parse-util.cc
M be/src/runtime/date-test.cc
M be/src/runtime/datetime-iso-sql-format-tokenizer.cc
M be/src/runtime/datetime-parser-common.cc
M be/src/runtime/datetime-parser-common.h
M be/src/runtime/datetime-simple-date-format-parser.cc
M be/src/runtime/datetime-simple-date-format-parser.h
M be/src/runtime/timestamp-parse-util.cc
M be/src/runtime/timestamp-test.cc
M be/src/runtime/timestamp-value.h
M bin/rat_exclude_files.txt
M common/function-registry/impala_functions.py
M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
M testdata/data/README
A testdata/data/dateless_timestamps.parq
A testdata/data/dateless_timestamps.txt
M testdata/data/lazy_timestamp.csv
M testdata/workloads/functional-query/queries/QueryTest/date.test
A
testdata/workloads/functional-query/queries/QueryTest/dateless_timestamp_parquet.test
A
testdata/workloads/functional-query/queries/QueryTest/dateless_timestamp_text.test
M testdata/workloads/functional-query/queries/QueryTest/exprs.test
M
testdata/workloads/functional-query/queries/QueryTest/select-lazy-timestamp.test
M tests/data_errors/test_data_errors.py
M tests/query_test/test_cast_with_format.py
M tests/query_test/test_scanners.py
34 files changed, 278 insertions(+), 231 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/66/15866/8
--
To view, visit http://gerrit.cloudera.org:8080/15866
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I48c49bf027cc4b917849b3d58518facba372b322
Gerrit-Change-Number: 15866
Gerrit-PatchSet: 8
Gerrit-Owner: Adam Tamas
Gerrit-Reviewer: Adam Tamas
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9531: Dropped support for dateless timestamps
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/15866 ) Change subject: IMPALA-9531: Dropped support for dateless timestamps .. Patch Set 6: There was a rebase between patch set 5 and 6. The only thing that is changed is bin/rat_exclude_files.txt. -- To view, visit http://gerrit.cloudera.org:8080/15866 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I48c49bf027cc4b917849b3d58518facba372b322 Gerrit-Change-Number: 15866 Gerrit-PatchSet: 6 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 22 Jun 2020 08:35:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9531: Dropped support for dateless timestamps
Adam Tamas has uploaded a new patch set (#6). (
http://gerrit.cloudera.org:8080/15866 )
Change subject: IMPALA-9531: Dropped support for dateless timestamps
..
IMPALA-9531: Dropped support for dateless timestamps
Removed the support for dateless timestamps.
During dateless timestamp casts if the format doesn't contain
date part we get an error during tokenization of the format.
If the input str doesn't contain a date part then we get null result.
Examples:
select cast('01:02:59' as timestamp);
This will come back as NULL value.
select to_timestamp('01:01:01', 'HH:mm:ss');
select cast('01:02:59' as timestamp format 'HH12:MI:SS');
select cast('12 AM' as timestamp FORMAT 'AM.HH12');
These will come back with a parsing errors.
Casting from a table will generate similar results.
Testing:
Modified the previous tests related to dateless timestamps.
Added test to read fromtables which are still containing dateless
timestamps and covered timestamp to string path when no date tokens
are requested in the output string.
Change-Id: I48c49bf027cc4b917849b3d58518facba372b322
---
M be/src/benchmarks/convert-timestamp-benchmark.cc
M be/src/benchmarks/parse-timestamp-benchmark.cc
M be/src/exec/text-converter.inline.h
M be/src/exprs/cast-functions-ir.cc
M be/src/exprs/expr-test.cc
M be/src/exprs/scalar-expr-evaluator.cc
M be/src/exprs/timestamp-functions-ir.cc
M be/src/exprs/timestamp-functions.cc
M be/src/exprs/timestamp-functions.h
M be/src/runtime/date-parse-util.cc
M be/src/runtime/date-test.cc
M be/src/runtime/datetime-iso-sql-format-tokenizer.cc
M be/src/runtime/datetime-parser-common.cc
M be/src/runtime/datetime-parser-common.h
M be/src/runtime/datetime-simple-date-format-parser.cc
M be/src/runtime/datetime-simple-date-format-parser.h
M be/src/runtime/timestamp-parse-util.cc
M be/src/runtime/timestamp-test.cc
M be/src/runtime/timestamp-value.h
M bin/rat_exclude_files.txt
M common/function-registry/impala_functions.py
M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
M testdata/data/README
A testdata/data/dateless_timestamps.parq
A testdata/data/dateless_timestamps.txt
M testdata/data/lazy_timestamp.csv
M testdata/workloads/functional-query/queries/QueryTest/date.test
A
testdata/workloads/functional-query/queries/QueryTest/dateless_timestamp_parquet.test
A
testdata/workloads/functional-query/queries/QueryTest/dateless_timestamp_text.test
M testdata/workloads/functional-query/queries/QueryTest/exprs.test
M
testdata/workloads/functional-query/queries/QueryTest/select-lazy-timestamp.test
M tests/data_errors/test_data_errors.py
M tests/query_test/test_cast_with_format.py
M tests/query_test/test_scanners.py
34 files changed, 275 insertions(+), 230 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/66/15866/6
--
To view, visit http://gerrit.cloudera.org:8080/15866
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I48c49bf027cc4b917849b3d58518facba372b322
Gerrit-Change-Number: 15866
Gerrit-PatchSet: 6
Gerrit-Owner: Adam Tamas
Gerrit-Reviewer: Adam Tamas
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9531: Dropped support for dateless timestamps
Adam Tamas has posted comments on this change. (
http://gerrit.cloudera.org:8080/15866 )
Change subject: IMPALA-9531: Dropped support for dateless timestamps
..
Patch Set 5:
(10 comments)
http://gerrit.cloudera.org:8080/#/c/15866/4/be/src/exprs/scalar-expr-evaluator.cc
File be/src/exprs/scalar-expr-evaluator.cc:
http://gerrit.cloudera.org:8080/#/c/15866/4/be/src/exprs/scalar-expr-evaluator.cc@444
PS4, Line 444: TimestampFunctions::FromUnixPrepare(nullptr,
FunctionContext::FRAGMENT_LOCAL);
> line too long (93 > 90)
Done
http://gerrit.cloudera.org:8080/#/c/15866/3/be/src/exprs/timestamp-functions-ir.cc
File be/src/exprs/timestamp-functions-ir.cc:
http://gerrit.cloudera.org:8080/#/c/15866/3/be/src/exprs/timestamp-functions-ir.cc@167
PS3, Line 167: if (!context->IsArgConstant(1)) {
> Did you find out what this check is for? Don't you break anything with simp
Done
http://gerrit.cloudera.org:8080/#/c/15866/4/be/src/exprs/timestamp-functions-ir.cc
File be/src/exprs/timestamp-functions-ir.cc:
http://gerrit.cloudera.org:8080/#/c/15866/4/be/src/exprs/timestamp-functions-ir.cc@168
PS4, Line 168: dt_ctx->Reset(reinterpret_cast(fmt.ptr), fmt.len)
> Anyway, if you tokenize the format here then you do the tokenization for ea
Done
http://gerrit.cloudera.org:8080/#/c/15866/4/be/src/exprs/timestamp-functions.h
File be/src/exprs/timestamp-functions.h:
http://gerrit.cloudera.org:8080/#/c/15866/4/be/src/exprs/timestamp-functions.h@104
PS4, Line 104:
> Please use datetime_parse_util::CastDirection
We can't reach datetime_parse_util::CastDirection from here that is why I used
bool.
I spoke with Csaba Ringhofer about it and in the end we come to the conclusion
that it is not necessary to be declared here, it is enough in the .cc because
it is just there to avoid code duplication.
http://gerrit.cloudera.org:8080/#/c/15866/4/be/src/exprs/timestamp-functions.cc
File be/src/exprs/timestamp-functions.cc:
http://gerrit.cloudera.org:8080/#/c/15866/4/be/src/exprs/timestamp-functions.cc@155
PS4, Line 155: CastDirection cast
> Why don't you use the parameter type that could be directly passed to Token
Done
http://gerrit.cloudera.org:8080/#/c/15866/4/be/src/exprs/timestamp-functions.cc@167
PS4, Line 167: if (!parse_result) {
: delete dt_ctx;
: ReportBadFormat(context,
datetime_parse_util::GENERAL_ERROR, fmt_val, true);
: return;
> Please check how to format if-else in Impala
Done
http://gerrit.cloudera.org:8080/#/c/15866/4/be/src/runtime/datetime-simple-date-format-parser.h
File be/src/runtime/datetime-simple-date-format-parser.h:
http://gerrit.cloudera.org:8080/#/c/15866/4/be/src/runtime/datetime-simple-date-format-parser.h@88
PS4, Line 88:
> I wouldn't give this a default value unless you have any specific reason ma
Done
http://gerrit.cloudera.org:8080/#/c/15866/4/testdata/data/README
File testdata/data/README:
http://gerrit.cloudera.org:8080/#/c/15866/4/testdata/data/README@503
PS4, Line 503: dateless_timestamps.pa
> The file name is not that verbose. After reading it it doesn't give any hin
Done
http://gerrit.cloudera.org:8080/#/c/15866/4/testdata/data/timestamp_text_test.txt
File testdata/data/timestamp_text_test.txt:
http://gerrit.cloudera.org:8080/#/c/15866/4/testdata/data/timestamp_text_test.txt@1
PS4, Line 1:
> Same comment for the file name as for the parquet file: "dateless_timestamp
Done
http://gerrit.cloudera.org:8080/#/c/15866/4/tests/query_test/test_scanners.py
File tests/query_test/test_scanners.py:
http://gerrit.cloudera.org:8080/#/c/15866/4/tests/query_test/test_scanners.py@391
PS4, Line 391:
> flake8: E501 line too long (98 > 90 characters)
Done
--
To view, visit http://gerrit.cloudera.org:8080/15866
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I48c49bf027cc4b917849b3d58518facba372b322
Gerrit-Change-Number: 15866
Gerrit-PatchSet: 5
Gerrit-Owner: Adam Tamas
Gerrit-Reviewer: Adam Tamas
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Sun, 21 Jun 2020 10:58:49 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9531: Dropped support for dateless timestamps
Adam Tamas has uploaded a new patch set (#5). (
http://gerrit.cloudera.org:8080/15866 )
Change subject: IMPALA-9531: Dropped support for dateless timestamps
..
IMPALA-9531: Dropped support for dateless timestamps
Removed the support for dateless timestamps.
During dateless timestamp casts if the format doesn't contain
date part we get an error during tokenization of the format.
If the input str doesn't contain a date part then we get null result.
Examples:
select cast('01:02:59' as timestamp);
This will come back as NULL value.
select to_timestamp('01:01:01', 'HH:mm:ss');
select cast('01:02:59' as timestamp format 'HH12:MI:SS');
select cast('12 AM' as timestamp FORMAT 'AM.HH12');
These will come back with a parsing errors.
Casting from a table will generate similar results.
Testing:
Modified the previous tests related to dateless timestamps.
Added test to read fromtables which are still containing dateless
timestamps and covered timestamp to string path when no date tokens
are requested in the output string.
Change-Id: I48c49bf027cc4b917849b3d58518facba372b322
---
M be/src/benchmarks/convert-timestamp-benchmark.cc
M be/src/benchmarks/parse-timestamp-benchmark.cc
M be/src/exec/text-converter.inline.h
M be/src/exprs/cast-functions-ir.cc
M be/src/exprs/expr-test.cc
M be/src/exprs/scalar-expr-evaluator.cc
M be/src/exprs/timestamp-functions-ir.cc
M be/src/exprs/timestamp-functions.cc
M be/src/exprs/timestamp-functions.h
M be/src/runtime/date-parse-util.cc
M be/src/runtime/date-test.cc
M be/src/runtime/datetime-iso-sql-format-tokenizer.cc
M be/src/runtime/datetime-parser-common.cc
M be/src/runtime/datetime-parser-common.h
M be/src/runtime/datetime-simple-date-format-parser.cc
M be/src/runtime/datetime-simple-date-format-parser.h
M be/src/runtime/timestamp-parse-util.cc
M be/src/runtime/timestamp-test.cc
M be/src/runtime/timestamp-value.h
M common/function-registry/impala_functions.py
M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
M testdata/data/README
A testdata/data/dateless_timestamps.parq
A testdata/data/dateless_timestamps.txt
M testdata/data/lazy_timestamp.csv
M testdata/workloads/functional-query/queries/QueryTest/date.test
A
testdata/workloads/functional-query/queries/QueryTest/dateless_timestamp_parquet.test
A
testdata/workloads/functional-query/queries/QueryTest/dateless_timestamp_text.test
M testdata/workloads/functional-query/queries/QueryTest/exprs.test
M
testdata/workloads/functional-query/queries/QueryTest/select-lazy-timestamp.test
M tests/data_errors/test_data_errors.py
M tests/query_test/test_cast_with_format.py
M tests/query_test/test_scanners.py
33 files changed, 274 insertions(+), 230 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/66/15866/5
--
To view, visit http://gerrit.cloudera.org:8080/15866
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I48c49bf027cc4b917849b3d58518facba372b322
Gerrit-Change-Number: 15866
Gerrit-PatchSet: 5
Gerrit-Owner: Adam Tamas
Gerrit-Reviewer: Adam Tamas
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9531: Dropped support for dateless timestamps
Adam Tamas has posted comments on this change. (
http://gerrit.cloudera.org:8080/15866 )
Change subject: IMPALA-9531: Dropped support for dateless timestamps
..
Patch Set 4:
(18 comments)
http://gerrit.cloudera.org:8080/#/c/15866/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/15866/3//COMMIT_MSG@15
PS3, Line 15: select cast('01:02:59' as timestamp);
> This should return parse error because the format itself doesn't contain da
Done
http://gerrit.cloudera.org:8080/#/c/15866/3//COMMIT_MSG@17
PS3, Line 17:
> nit: dot at the end of a sentence.
Done
http://gerrit.cloudera.org:8080/#/c/15866/3//COMMIT_MSG@17
PS3, Line 17:
> nit: This should be plural in my opinion as it stands for 2 separate exampl
Done
http://gerrit.cloudera.org:8080/#/c/15866/3//COMMIT_MSG@28
PS3, Line 28: timesta
> I still don't see tests where you use a file populated by dateless timestam
Done
http://gerrit.cloudera.org:8080/#/c/15866/3//COMMIT_MSG@29
PS3, Line 29: are requ
> nit: start a sentence with capital letter.
Done
http://gerrit.cloudera.org:8080/#/c/15866/3//COMMIT_MSG@29
PS3, Line 29: utpu
> nit: tests
Done
http://gerrit.cloudera.org:8080/#/c/15866/3//COMMIT_MSG@30
PS3, Line 30:
> It's just a matter of your point of view which direction you call backwards
Done
http://gerrit.cloudera.org:8080/#/c/15866/3/be/src/exprs/expr-test.cc
File be/src/exprs/expr-test.cc:
http://gerrit.cloudera.org:8080/#/c/15866/3/be/src/exprs/expr-test.cc@4507
PS3, Line 4507: 1.00
> Is this related to your change or does it come with a rebase you've made in
Sorry for this one, I really did a rebase between the commits.
http://gerrit.cloudera.org:8080/#/c/15866/3/be/src/exprs/expr-test.cc@7536
PS3, Line 7536: H:mm:ss'
> Actually, we should get an error when parsing the format and see that there
Done
http://gerrit.cloudera.org:8080/#/c/15866/3/be/src/runtime/datetime-simple-date-format-parser.h
File be/src/runtime/datetime-simple-date-format-parser.h:
http://gerrit.cloudera.org:8080/#/c/15866/3/be/src/runtime/datetime-simple-date-format-parser.h@88
PS3, Line 88:
> Can you re-use the CastDirection type similarly as it is used in e.g. ISO S
Done
http://gerrit.cloudera.org:8080/#/c/15866/3/be/src/runtime/datetime-simple-date-format-parser.cc
File be/src/runtime/datetime-simple-date-format-parser.cc:
http://gerrit.cloudera.org:8080/#/c/15866/3/be/src/runtime/datetime-simple-date-format-parser.cc@179
PS3, Line 179: if (cast_mode == PARSE) return (dt_ctx->has_date_toks);
> Please don't leave commented code
Done
http://gerrit.cloudera.org:8080/#/c/15866/3/be/src/runtime/datetime-simple-date-format-parser.cc@180
PS3, Line 180: return (dt_ctx->has_date_toks || dt_ctx->has_time_toks);
> This seems logically correct but not that readable. Additionally, this is n
Done
http://gerrit.cloudera.org:8080/#/c/15866/3/be/src/runtime/timestamp-test.cc
File be/src/runtime/timestamp-test.cc:
http://gerrit.cloudera.org:8080/#/c/15866/3/be/src/runtime/timestamp-test.cc@610
PS3, Line 610: (TimestampTC("-MM-dd HH:m:ss", "2020-05-11 1:24:34",
false, true))
> Here the intention was to also cover the case when the minute part of the i
Here we are checking the format tokens not the given string themselves which
has a short minute part("-MM-dd HH:m:ss")
We use the short form here because we expect it to fail.
http://gerrit.cloudera.org:8080/#/c/15866/3/be/src/runtime/timestamp-test.cc@612
PS3, Line 612: (TimestampTC("-MM-dd HH:mm:s", "2020-05-11 14:24:34", false,
true, true, 2020,
: 05, 11, 14, 24, 34))
> Similarly to the other tests, please add another one where the second part
Done
http://gerrit.cloudera.org:8080/#/c/15866/3/be/src/runtime/timestamp-test.cc@679
PS3, Line 679: mestampFormatTC(1382337792, "yyy
> nit: This sentence sounds a bit weird for me.
Done
http://gerrit.cloudera.org:8080/#/c/15866/3/be/src/runtime/timestamp-test.cc@695
PS3, Line 695: Test padding on double di
> nit: same as above
Done
http://gerrit.cloudera.org:8080/#/c/15866/3/testdata/workloads/functional-query/queries/QueryTest/exprs.test
File testdata/workloads/functional-query/queries/QueryTest/exprs.test:
http://gerrit.cloudera.org:8080/#/c/15866/3/testdata/workloads/functional-query/queries/QueryTest/exprs.test@2976
PS3, Line 2976: select to_timestamp('01:01:01', 'HH:mm:ss');
> As mentioned elsewhere, this should result a parse error as the format does
Done
http://gerrit.cloudera.org:8080/#/c/15866/3/testdata/workloads/functional-query/queries/QueryTest/exprs.test@2981
PS3, Line 2981: select to_timestamp('01:01:01.123', 'HH:mm:ss.SSS');
> same as above
Done
--
To view, visit http://gerrit.cloudera.org:8080/15866
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I48c49bf027cc4b917849b3d58518facba372b322
Gerrit-Change-Number: 158
[Impala-ASF-CR] IMPALA-9531: Dropped support for dateless timestamps
Adam Tamas has uploaded a new patch set (#4). (
http://gerrit.cloudera.org:8080/15866 )
Change subject: IMPALA-9531: Dropped support for dateless timestamps
..
IMPALA-9531: Dropped support for dateless timestamps
Removed the support for dateless timestamps.
During dateless timestamp casts if the format doesn't contain
date part we get an error during tokenization of the format.
If the input str doesn't contain a date part then we get null result.
Examples:
select cast('01:02:59' as timestamp);
This will come back as NULL value.
select to_timestamp('01:01:01', 'HH:mm:ss');
select cast('01:02:59' as timestamp format 'HH12:MI:SS');
select cast('12 AM' as timestamp FORMAT 'AM.HH12');
These will come back with a parsing errors.
Casting from a table will generate similar results.
Testing:
Modified the previous tests related to dateless timestamps.
Added test to read fromtables which are still containing dateless
timestamps and covered timestamp to string path when no date tokens
are requested in the output string.
Change-Id: I48c49bf027cc4b917849b3d58518facba372b322
---
M be/src/exec/text-converter.inline.h
M be/src/exprs/cast-functions-ir.cc
M be/src/exprs/expr-test.cc
M be/src/exprs/scalar-expr-evaluator.cc
M be/src/exprs/timestamp-functions-ir.cc
M be/src/exprs/timestamp-functions.cc
M be/src/exprs/timestamp-functions.h
M be/src/runtime/date-parse-util.cc
M be/src/runtime/datetime-iso-sql-format-tokenizer.cc
M be/src/runtime/datetime-parser-common.cc
M be/src/runtime/datetime-parser-common.h
M be/src/runtime/datetime-simple-date-format-parser.cc
M be/src/runtime/datetime-simple-date-format-parser.h
M be/src/runtime/timestamp-parse-util.cc
M be/src/runtime/timestamp-test.cc
M be/src/runtime/timestamp-value.h
M common/function-registry/impala_functions.py
M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
M testdata/data/README
M testdata/data/lazy_timestamp.csv
A testdata/data/timestamp_parquet_test.parq
A testdata/data/timestamp_text_test.txt
M testdata/workloads/functional-query/queries/QueryTest/date.test
A
testdata/workloads/functional-query/queries/QueryTest/dateless_timestamp_parquet.test
A
testdata/workloads/functional-query/queries/QueryTest/dateless_timestamp_text.test
M testdata/workloads/functional-query/queries/QueryTest/exprs.test
M
testdata/workloads/functional-query/queries/QueryTest/select-lazy-timestamp.test
M tests/data_errors/test_data_errors.py
M tests/query_test/test_cast_with_format.py
M tests/query_test/test_scanners.py
30 files changed, 263 insertions(+), 217 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/66/15866/4
--
To view, visit http://gerrit.cloudera.org:8080/15866
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I48c49bf027cc4b917849b3d58518facba372b322
Gerrit-Change-Number: 15866
Gerrit-PatchSet: 4
Gerrit-Owner: Adam Tamas
Gerrit-Reviewer: Adam Tamas
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9531: Dropped support for dateless timestamps
Adam Tamas has uploaded a new patch set (#3). (
http://gerrit.cloudera.org:8080/15866 )
Change subject: IMPALA-9531: Dropped support for dateless timestamps
..
IMPALA-9531: Dropped support for dateless timestamps
Removed the support for dateless timestamps.
During dateless timestamp casts if the format doesn't contain
date part we get an error during tokenization of the format.
If the input str doesn't contain a date part then we get null result.
Examples:
select to_timestamp('01:01:01', 'HH:mm:ss');
select cast('01:02:59' as timestamp);
This will come back as NULL values
select cast(string_col as timestamp) from table_name;
Casting from a table with a similar method like this
will also give back NULL values similar to the above example.
select cast('01:02:59' as timestamp format 'HH12:MI:SS');
select cast('12 AM' as timestamp FORMAT 'AM.HH12');
These will come back with a parsing error which is:
ERROR: PARSE ERROR: No date tokens provided.
Testing:
modified the previous test related to dateless timestamps and
added new tests to check if the conversions backwards(timestamp to
string) is still working
Change-Id: I48c49bf027cc4b917849b3d58518facba372b322
---
M be/src/exec/text-converter.inline.h
M be/src/exprs/cast-functions-ir.cc
M be/src/exprs/expr-test.cc
M be/src/exprs/timestamp-functions-ir.cc
M be/src/runtime/date-parse-util.cc
M be/src/runtime/datetime-iso-sql-format-tokenizer.cc
M be/src/runtime/datetime-parser-common.cc
M be/src/runtime/datetime-parser-common.h
M be/src/runtime/datetime-simple-date-format-parser.cc
M be/src/runtime/datetime-simple-date-format-parser.h
M be/src/runtime/timestamp-parse-util.cc
M be/src/runtime/timestamp-test.cc
M be/src/runtime/timestamp-value.h
M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
M testdata/data/lazy_timestamp.csv
M testdata/workloads/functional-query/queries/QueryTest/date.test
M testdata/workloads/functional-query/queries/QueryTest/exprs.test
M
testdata/workloads/functional-query/queries/QueryTest/select-lazy-timestamp.test
M tests/data_errors/test_data_errors.py
M tests/query_test/test_cast_with_format.py
20 files changed, 131 insertions(+), 198 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/66/15866/3
--
To view, visit http://gerrit.cloudera.org:8080/15866
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I48c49bf027cc4b917849b3d58518facba372b322
Gerrit-Change-Number: 15866
Gerrit-PatchSet: 3
Gerrit-Owner: Adam Tamas
Gerrit-Reviewer: Adam Tamas
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9531: Dropped support for dateless timestamps
Adam Tamas has uploaded this change for review. (
http://gerrit.cloudera.org:8080/15866
Change subject: IMPALA-9531: Dropped support for dateless timestamps
..
IMPALA-9531: Dropped support for dateless timestamps
Removed the support for dateless timestamps.
During dateless timestamp casts if the format doesn't contain
date part we get an error during tokenization of the format.
If the input str doesn't contain a date part then we get null result.
Examples:
select to_timestamp('01:01:01', 'HH:mm:ss');
select cast('01:02:59' as timestamp);
This will come back as NULL values
select cast(string_col as timestamp) from table_name;
Casting from a table with a similar method like this
will also give back NULL values similar to the above example.
select cast('01:02:59' as timestamp format 'HH12:MI:SS');
select cast('12 AM' as timestamp FORMAT 'AM.HH12');
These will come back with a parsing error which is:
ERROR: PARSE ERROR: No date tokens provided.
Testing:
modified the previous test related to dateless timestamps and
added new tests to check if the conversions backwards(timestamp to
string) is still working
Change-Id: I48c49bf027cc4b917849b3d58518facba372b322
---
M be/src/exec/text-converter.inline.h
M be/src/exprs/cast-functions-ir.cc
M be/src/exprs/expr-test.cc
M be/src/exprs/timestamp-functions-ir.cc
M be/src/runtime/date-parse-util.cc
M be/src/runtime/datetime-iso-sql-format-tokenizer.cc
M be/src/runtime/datetime-parser-common.cc
M be/src/runtime/datetime-parser-common.h
M be/src/runtime/datetime-simple-date-format-parser.cc
M be/src/runtime/datetime-simple-date-format-parser.h
M be/src/runtime/timestamp-parse-util.cc
M be/src/runtime/timestamp-test.cc
M be/src/runtime/timestamp-value.h
M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
M testdata/data/lazy_timestamp.csv
M testdata/workloads/functional-query/queries/QueryTest/date.test
M testdata/workloads/functional-query/queries/QueryTest/exprs.test
M
testdata/workloads/functional-query/queries/QueryTest/select-lazy-timestamp.test
M tests/data_errors/test_data_errors.py
M tests/query_test/test_cast_with_format.py
20 files changed, 131 insertions(+), 198 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/66/15866/2
--
To view, visit http://gerrit.cloudera.org:8080/15866
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I48c49bf027cc4b917849b3d58518facba372b322
Gerrit-Change-Number: 15866
Gerrit-PatchSet: 2
Gerrit-Owner: Adam Tamas
Gerrit-Reviewer: Gabor Kaszab
[Impala-ASF-CR] IMPALA-9531: Dropped support for dateless timestamps
Adam Tamas has posted comments on this change. (
http://gerrit.cloudera.org:8080/15866 )
Change subject: IMPALA-9531: Dropped support for dateless timestamps
..
Patch Set 2:
(19 comments)
http://gerrit.cloudera.org:8080/#/c/15866/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/15866/1//COMMIT_MSG@7
PS1, Line 7: d
> no need for quotation marks
Done
http://gerrit.cloudera.org:8080/#/c/15866/1//COMMIT_MSG@10
PS1, Line 10: During dateless timestamp casts if the format doesn't contain
: date part we get an error during tokenization of the format
> I'd rather mention that when a format is provided for a cast then if the fo
Done
http://gerrit.cloudera.org:8080/#/c/15866/1//COMMIT_MSG@14
PS1, Line 14: s:
> I'd use this example:
Done
http://gerrit.cloudera.org:8080/#/c/15866/1//COMMIT_MSG@15
PS1, Line 15: estam
> I'd use an example as '01:02:59'
Done
http://gerrit.cloudera.org:8080/#/c/15866/1//COMMIT_MSG@16
PS1, Line 16: select cast('01:02:59' as timestamp);
> I'd also add an example for to_timestamp(input_str, format_str)
Done
http://gerrit.cloudera.org:8080/#/c/15866/1//COMMIT_MSG@17
PS1, Line 17: This will come back as NULL values
> Another example:
Done
http://gerrit.cloudera.org:8080/#/c/15866/1//COMMIT_MSG@23
PS1, Line 23: select cast('01:02:59' as timestamp format 'HH12:MI:SS');
: select cast('12 AM' as timestamp FORMAT 'AM.HH12');
: These will come back with a parsing error which is:
: ERROR: PARSE ERROR: No date tokens provid
> No need to list the modified test files here. If there is something in part
Done
http://gerrit.cloudera.org:8080/#/c/15866/1/be/src/exprs/expr-test.cc
File be/src/exprs/expr-test.cc:
http://gerrit.cloudera.org:8080/#/c/15866/1/be/src/exprs/expr-test.cc@6777
PS1, Line 6777: TestValue("dayofweek(cast('2011-12-22' as timestamp))",
TYPE_INT, 5);
: TestValue("dayofweek(cast('2011-12-24' as timestamp))",
TYPE_INT, 7);
: TestStringValue(
: "to_date(cast('2011-12-22 09:10:11.12345678' as
timestamp))", "2011-12-22");
:
> these are the same tests as in L6759-6762. No need to duplicate them.
Done
http://gerrit.cloudera.org:8080/#/c/15866/1/be/src/exprs/expr-test.cc@7155
PS1, Line 7155:
: TestStringValue(
> These 2 tests lost their purpose with your change, A comment in L7151 refer
If we call trunc() on timestamp is gives back everything that stands before
that value and the value that stands in the given place, as I see we can no
longer create a valid test which will give back NULL value since we dropped the
timeless timestamps.
I will remove these tests since they are indeed misleading and there are other
test case where we trunc() a NULL value.
http://gerrit.cloudera.org:8080/#/c/15866/1/be/src/runtime/datetime-iso-sql-format-tokenizer.cc
File be/src/runtime/datetime-iso-sql-format-tokenizer.cc:
http://gerrit.cloudera.org:8080/#/c/15866/1/be/src/runtime/datetime-iso-sql-format-tokenizer.cc@195
PS1, Line 195:
> I'd move this check after L-197-202. We exit there if cast_mode_ == FORMAT
Done
http://gerrit.cloudera.org:8080/#/c/15866/1/be/src/runtime/datetime-simple-date-format-parser.h
File be/src/runtime/datetime-simple-date-format-parser.h:
http://gerrit.cloudera.org:8080/#/c/15866/1/be/src/runtime/datetime-simple-date-format-parser.h@77
PS1, Line 77: DEFAULT_SHORT_DATE_T
> Is this needed anymore?
Done
http://gerrit.cloudera.org:8080/#/c/15866/1/be/src/runtime/datetime-simple-date-format-parser.cc
File be/src/runtime/datetime-simple-date-format-parser.cc:
http://gerrit.cloudera.org:8080/#/c/15866/1/be/src/runtime/datetime-simple-date-format-parser.cc@38
PS1, Line 38: DEFAULT_SHORT_DATE_T
> Is this still needed?
Done
http://gerrit.cloudera.org:8080/#/c/15866/1/be/src/runtime/datetime-simple-date-format-parser.cc@101
PS1, Line 101: bool accept_time_toks, bool parse) {
> It would be nice to see tests for to_timestamp as well. I haven't seem any
I looked around a bit and as I seen there was no test for to_timestamp() with
dateless conversions, but I tried it out with manual tests and it is no longer
accepting dateless formats.
I will look around and add a failing test with dateless timestamp to the
to_timestamp() tests.
http://gerrit.cloudera.org:8080/#/c/15866/1/be/src/runtime/datetime-simple-date-format-parser.cc@344
PS1, Line 344: // Check if this string starts with a date
> I don't think that this check is needed.
Done
http://gerrit.cloudera.org:8080/#/c/15866/1/be/src/runtime/timestamp-test.cc
File be/src/runtime/timestamp-test.cc:
http://gerrit.cloudera.org:8080/#/c/15866/1/be/src/runtime/timestamp-test.cc@a617
PS1, Line 617:
:
:
:
:
:
:
:
:
:
:
:
:
:
> Thes
[Impala-ASF-CR] IMPALA-9680: Fixed compressed inserts failing
Adam Tamas has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/15816 ) Change subject: IMPALA-9680: Fixed compressed inserts failing .. IMPALA-9680: Fixed compressed inserts failing Modified the insert testfiles to get which database they need to use for 'CREATE TABLE LIKE' dynamically. Tests: Did targeted exhaustive testruns in test_insert.py and test_mt_dop.py and did a full exhaustive testrun. Change-Id: Ib3c7ba02190f57a7ed40311c95a3dd9eca9b474d --- M testdata/workloads/functional-query/queries/QueryTest/insert-mem-limit.test M testdata/workloads/functional-query/queries/QueryTest/insert.test M testdata/workloads/functional-query/queries/QueryTest/insert_bad_expr.test M testdata/workloads/functional-query/queries/QueryTest/insert_null.test M testdata/workloads/functional-query/queries/QueryTest/insert_overwrite.test M tests/common/impala_test_suite.py M tests/custom_cluster/test_mt_dop.py M tests/query_test/test_insert.py M tests/query_test/test_mt_dop.py 9 files changed, 49 insertions(+), 28 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/16/15816/8 -- To view, visit http://gerrit.cloudera.org:8080/15816 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib3c7ba02190f57a7ed40311c95a3dd9eca9b474d Gerrit-Change-Number: 15816 Gerrit-PatchSet: 8 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9680: Fixed compressed inserts failing
Adam Tamas has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/15816 ) Change subject: IMPALA-9680: Fixed compressed inserts failing .. IMPALA-9680: Fixed compressed inserts failing Modified the insert testfiles to get which database they need to use for 'CREATE TABLE LIKE' dynamically. Tests: Did targeted exhaustive testruns in test_insert.py and test_mt_dop.py and did a full exhaustive testrun. Change-Id: Ib3c7ba02190f57a7ed40311c95a3dd9eca9b474d --- M testdata/workloads/functional-query/queries/QueryTest/insert-mem-limit.test M testdata/workloads/functional-query/queries/QueryTest/insert.test M testdata/workloads/functional-query/queries/QueryTest/insert_bad_expr.test M testdata/workloads/functional-query/queries/QueryTest/insert_null.test M testdata/workloads/functional-query/queries/QueryTest/insert_overwrite.test M tests/common/impala_test_suite.py M tests/custom_cluster/test_mt_dop.py M tests/query_test/test_insert.py M tests/query_test/test_mt_dop.py 9 files changed, 49 insertions(+), 28 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/16/15816/5 -- To view, visit http://gerrit.cloudera.org:8080/15816 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib3c7ba02190f57a7ed40311c95a3dd9eca9b474d Gerrit-Change-Number: 15816 Gerrit-PatchSet: 5 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9680 Fixed compressed inserts failing
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/15816 ) Change subject: IMPALA-9680 Fixed compressed inserts failing .. Patch Set 4: (3 comments) http://gerrit.cloudera.org:8080/#/c/15816/3/tests/common/custom_cluster_test_suite.py File tests/common/custom_cluster_test_suite.py: http://gerrit.cloudera.org:8080/#/c/15816/3/tests/common/custom_cluster_test_suite.py@69 PS3, Line 69: @classmethod : def add_test_dimensions(cls): : super(CustomClusterTestSuite, cls).add_test_dimensions() : cls.add_custom_cl > Why do we need this here if is already in ImpalaTestSuite? Done http://gerrit.cloudera.org:8080/#/c/15816/3/tests/common/impala_test_suite.py File tests/common/impala_test_suite.py: http://gerrit.cloudera.org:8080/#/c/15816/3/tests/common/impala_test_suite.py@769 PS3, Line 769: return QueryTestSectionReader.get_db_name(table_format, scale_factor) : > nit: this fits to one line Done http://gerrit.cloudera.org:8080/#/c/15816/2/tests/custom_cluster/test_mt_dop.py File tests/custom_cluster/test_mt_dop.py: http://gerrit.cloudera.org:8080/#/c/15816/2/tests/custom_cluster/test_mt_dop.py@38 PS2, Line 38: @CustomClusterTestSuite.with_args(impalad_args="--mt_dop_auto_fallback=true") > Ah, I see - but this is a CustomClusterTest, so it makes sense not to run i These tests are flaky if not run serially. for example, test_insert_large_string sometimes come back with an error that it can't allocate enough memory. -- To view, visit http://gerrit.cloudera.org:8080/15816 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib3c7ba02190f57a7ed40311c95a3dd9eca9b474d Gerrit-Change-Number: 15816 Gerrit-PatchSet: 4 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 04 May 2020 08:49:58 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9680 Fixed compressed inserts failing
Adam Tamas has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/15816 ) Change subject: IMPALA-9680 Fixed compressed inserts failing .. IMPALA-9680 Fixed compressed inserts failing Modified the insert testfiles to get which database they need to use for 'CREATE TABLE LIKE' dynamically. Tests: Did targeted exhaustive testruns in test_insert.py and test_mt_dop.py and did a full exhaustive testrun. Change-Id: Ib3c7ba02190f57a7ed40311c95a3dd9eca9b474d --- M testdata/workloads/functional-query/queries/QueryTest/insert-mem-limit.test M testdata/workloads/functional-query/queries/QueryTest/insert.test M testdata/workloads/functional-query/queries/QueryTest/insert_bad_expr.test M testdata/workloads/functional-query/queries/QueryTest/insert_null.test M testdata/workloads/functional-query/queries/QueryTest/insert_overwrite.test M tests/common/impala_test_suite.py M tests/custom_cluster/test_mt_dop.py M tests/query_test/test_insert.py M tests/query_test/test_mt_dop.py 9 files changed, 49 insertions(+), 28 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/16/15816/4 -- To view, visit http://gerrit.cloudera.org:8080/15816 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib3c7ba02190f57a7ed40311c95a3dd9eca9b474d Gerrit-Change-Number: 15816 Gerrit-PatchSet: 4 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9680 Fixed compressed inserts failing
Adam Tamas has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/15816 ) Change subject: IMPALA-9680 Fixed compressed inserts failing .. IMPALA-9680 Fixed compressed inserts failing Modified the insert testfiles to get which database they need to use for 'CREATE TABLE LIKE' dynamically. Tests: Did targeted exhaustive testruns in test_insert.py and test_mt_dop.py and did a full exhaustive testrun. Change-Id: Ib3c7ba02190f57a7ed40311c95a3dd9eca9b474d --- M testdata/workloads/functional-query/queries/QueryTest/insert-mem-limit.test M testdata/workloads/functional-query/queries/QueryTest/insert.test M testdata/workloads/functional-query/queries/QueryTest/insert_bad_expr.test M testdata/workloads/functional-query/queries/QueryTest/insert_null.test M testdata/workloads/functional-query/queries/QueryTest/insert_overwrite.test M tests/common/custom_cluster_test_suite.py M tests/common/impala_test_suite.py M tests/custom_cluster/test_mt_dop.py M tests/query_test/test_insert.py M tests/query_test/test_mt_dop.py 10 files changed, 55 insertions(+), 28 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/16/15816/3 -- To view, visit http://gerrit.cloudera.org:8080/15816 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib3c7ba02190f57a7ed40311c95a3dd9eca9b474d Gerrit-Change-Number: 15816 Gerrit-PatchSet: 3 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9680 Fixed compressed inserts failing
Adam Tamas has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15816 Change subject: IMPALA-9680 Fixed compressed inserts failing .. IMPALA-9680 Fixed compressed inserts failing Modified the insert testfiles to get which database they need to use for 'CREATE TABLE LIKE' dynamically. Tests: Did targeted exhaustive testruns in test_insert.py and test_mt_dop.py and did a full exhaustive testrun. Change-Id: Ib3c7ba02190f57a7ed40311c95a3dd9eca9b474d --- M testdata/workloads/functional-query/queries/QueryTest/insert-mem-limit.test M testdata/workloads/functional-query/queries/QueryTest/insert.test M testdata/workloads/functional-query/queries/QueryTest/insert_bad_expr.test M testdata/workloads/functional-query/queries/QueryTest/insert_null.test M testdata/workloads/functional-query/queries/QueryTest/insert_overwrite.test M tests/custom_cluster/test_mt_dop.py M tests/query_test/test_insert.py M tests/query_test/test_mt_dop.py 8 files changed, 48 insertions(+), 28 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/16/15816/2 -- To view, visit http://gerrit.cloudera.org:8080/15816 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ib3c7ba02190f57a7ed40311c95a3dd9eca9b474d Gerrit-Change-Number: 15816 Gerrit-PatchSet: 2 Gerrit-Owner: Adam Tamas
[Impala-ASF-CR] IMPALA-9665: Fixed database not found errors in query test.test insert
Adam Tamas has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15766 Change subject: IMPALA-9665: Fixed database not found errors in query_test.test_insert .. IMPALA-9665: Fixed database not found errors in query_test.test_insert Fixed the usage of the unique_database in the test_insert.py to wait with the tests until the database is synced. Testing: -tests/run-tests.py query_test/test_insert.py --exploration_strategy=exhaustive Change-Id: I9b7aa3775dd4375f536d76f2e236ce126f8c78cd --- M tests/query_test/test_insert.py 1 file changed, 4 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/66/15766/1 -- To view, visit http://gerrit.cloudera.org:8080/15766 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I9b7aa3775dd4375f536d76f2e236ce126f8c78cd Gerrit-Change-Number: 15766 Gerrit-PatchSet: 1 Gerrit-Owner: Adam Tamas
[Impala-ASF-CR] IMPALA-8980: Remove functional*.alltypesinsert from EE tests
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/15529 ) Change subject: IMPALA-8980: Remove functional*.alltypesinsert from EE tests .. Patch Set 5: The job failed because of https://issues.apache.org/jira/browse/IMPALA-9596 , it seems the test is flaky. -- To view, visit http://gerrit.cloudera.org:8080/15529 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I257e936868917a2fcc6c030f6c855b247e8a0eea Gerrit-Change-Number: 15529 Gerrit-PatchSet: 5 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 14 Apr 2020 08:07:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8980: Remove functional*.alltypesinsert from EE tests
Adam Tamas has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/15529 ) Change subject: IMPALA-8980: Remove functional*.alltypesinsert from EE tests .. IMPALA-8980: Remove functional*.alltypesinsert from EE tests -Modified the ‘test_insert.py’ so the tests can run parallel. -Every test will create its own temporary tables for insert testing. -Swapped out the SETUP tags to Truncate table QUERY statement. -Becouse the SETUP tag is not used anymore, the correspondig code was removed. -A test query in ‘insert.test’. The test was incorrect so modified to test for the right behavior. Testing: -tests/run-tests.py query_test/test_insert.py -impala-py.test tests/query_test/test_insert.py -the same for test_insert_permutation.py and test_load.py Change-Id: I257e936868917a2fcc6c030f6c855b247e8a0eea --- M testdata/datasets/functional/functional_schema_template.sql M testdata/workloads/functional-query/queries/QueryTest/insert-mem-limit.test M testdata/workloads/functional-query/queries/QueryTest/insert.test M testdata/workloads/functional-query/queries/QueryTest/insert_bad_expr.test M testdata/workloads/functional-query/queries/QueryTest/insert_null.test M testdata/workloads/functional-query/queries/QueryTest/insert_overwrite.test M testdata/workloads/functional-query/queries/QueryTest/insert_permutation.test M testdata/workloads/functional-query/queries/QueryTest/load.test M testdata/workloads/functional-query/queries/QueryTest/show.test M tests/common/impala_test_suite.py M tests/custom_cluster/test_mt_dop.py M tests/query_test/test_insert.py M tests/query_test/test_mt_dop.py 13 files changed, 147 insertions(+), 266 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/15529/4 -- To view, visit http://gerrit.cloudera.org:8080/15529 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I257e936868917a2fcc6c030f6c855b247e8a0eea Gerrit-Change-Number: 15529 Gerrit-PatchSet: 4 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8980: Remove functional*.alltypesinsert from EE tests
Adam Tamas has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/15529 ) Change subject: IMPALA-8980: Remove functional*.alltypesinsert from EE tests .. IMPALA-8980: Remove functional*.alltypesinsert from EE tests -Modified the ‘test_insert.py’ so the tests can run parallel. -Every test will create its own temporary tables for insert testing. -Swapped out the SETUP tags to Truncate table QUERY statement. -Becouse the SETUP tag is not used anymore, the correspondig code was removed. -A test query in ‘insert.test’. The test was incorrect so modified to test for the right behavior. Testing: -tests/run-tests.py query_test/test_insert.py -impala-py.test tests/query_test/test_insert.py -the same for test_insert_permutation.py and test_load.py Change-Id: I257e936868917a2fcc6c030f6c855b247e8a0eea --- M testdata/datasets/functional/functional_schema_template.sql M testdata/workloads/functional-query/queries/QueryTest/insert-mem-limit.test M testdata/workloads/functional-query/queries/QueryTest/insert.test M testdata/workloads/functional-query/queries/QueryTest/insert_bad_expr.test M testdata/workloads/functional-query/queries/QueryTest/insert_null.test M testdata/workloads/functional-query/queries/QueryTest/insert_overwrite.test M testdata/workloads/functional-query/queries/QueryTest/insert_permutation.test M testdata/workloads/functional-query/queries/QueryTest/load.test M testdata/workloads/functional-query/queries/QueryTest/show.test M tests/common/impala_test_suite.py M tests/custom_cluster/test_mt_dop.py M tests/query_test/test_insert.py M tests/query_test/test_mt_dop.py 13 files changed, 142 insertions(+), 251 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/15529/3 -- To view, visit http://gerrit.cloudera.org:8080/15529 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I257e936868917a2fcc6c030f6c855b247e8a0eea Gerrit-Change-Number: 15529 Gerrit-PatchSet: 3 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8980: Remove functional*.alltypesinsert from EE tests
Adam Tamas has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/15529 ) Change subject: IMPALA-8980: Remove functional*.alltypesinsert from EE tests .. IMPALA-8980: Remove functional*.alltypesinsert from EE tests -Modified the ‘test_insert.py’ so the tests can run parallel. -Every test will create its own temporary tables for insert testing. -Swapped out the SETUP tags to Truncate table QUERY statement. -Becouse the SETUP tag is not used anymore, the correspondig code was removed. -A test query in ‘insert.test’. The test was incorrect so modified to test for the right behavior. Testing: -tests/run-tests.py query_test/test_insert.py -impala-py.test tests/query_test/test_insert.py -the same for test_insert_permutation.py and test_load.py Change-Id: I257e936868917a2fcc6c030f6c855b247e8a0eea --- M testdata/datasets/functional/functional_schema_template.sql M testdata/workloads/functional-query/queries/QueryTest/insert-mem-limit.test M testdata/workloads/functional-query/queries/QueryTest/insert.test M testdata/workloads/functional-query/queries/QueryTest/insert_bad_expr.test M testdata/workloads/functional-query/queries/QueryTest/insert_null.test M testdata/workloads/functional-query/queries/QueryTest/insert_overwrite.test M testdata/workloads/functional-query/queries/QueryTest/insert_permutation.test M testdata/workloads/functional-query/queries/QueryTest/load.test M testdata/workloads/functional-query/queries/QueryTest/show.test M tests/common/impala_test_suite.py M tests/custom_cluster/test_mt_dop.py M tests/query_test/test_insert.py M tests/query_test/test_mt_dop.py 13 files changed, 142 insertions(+), 251 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/15529/2 -- To view, visit http://gerrit.cloudera.org:8080/15529 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I257e936868917a2fcc6c030f6c855b247e8a0eea Gerrit-Change-Number: 15529 Gerrit-PatchSet: 2 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8980: Remove functional*.alltypesinsert from EE tests
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/15529 ) Change subject: IMPALA-8980: Remove functional*.alltypesinsert from EE tests .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/15529/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/15529/1//COMMIT_MSG@11 PS1, Line 11: -Swapped out the Reset table and Drop partition SETUP tags to Truncate table QUERY statement. > Do we still use SETUP anywhere in the tests? If not, then it would be great There are 2 more tests where we use SETUP. Should I swap them out too and remove this part? http://gerrit.cloudera.org:8080/#/c/15529/1/testdata/workloads/functional-query/queries/QueryTest/insert.test File testdata/workloads/functional-query/queries/QueryTest/insert.test: http://gerrit.cloudera.org:8080/#/c/15529/1/testdata/workloads/functional-query/queries/QueryTest/insert.test@a675 PS1, Line 675: : : : : : : : : : : : : : : : > Instead of deleting we could also for test the current behavior. This test case was added when IMPALA-89 got resolved which implies that it should work as hive, which would not clear the table in this case either. This is what I based on that it is not a bug, but a wrongly added test. -- To view, visit http://gerrit.cloudera.org:8080/15529 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I257e936868917a2fcc6c030f6c855b247e8a0eea Gerrit-Change-Number: 15529 Gerrit-PatchSet: 1 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 24 Mar 2020 09:23:39 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8980: Remove functional*.alltypesinsert from EE tests
Adam Tamas has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15529 Change subject: IMPALA-8980: Remove functional*.alltypesinsert from EE tests .. IMPALA-8980: Remove functional*.alltypesinsert from EE tests -Modified the ‘test_insert.py’ so the tests can run parallel. -Every test will create its own temporary tables for insert testing. -Swapped out the Reset table and Drop partition SETUP tags to Truncate table QUERY statement. -Deleted a test query in ‘insert.test’. The test incorrectly tried to clear out a table which was not working, and should not work either. Testing: -tests/run-tests.py query_test/test_insert.py -impala-py.test tests/query_test/test_insert.py Change-Id: I257e936868917a2fcc6c030f6c855b247e8a0eea --- M testdata/workloads/functional-query/queries/QueryTest/insert-mem-limit.test M testdata/workloads/functional-query/queries/QueryTest/insert.test M testdata/workloads/functional-query/queries/QueryTest/insert_bad_expr.test M testdata/workloads/functional-query/queries/QueryTest/insert_null.test M testdata/workloads/functional-query/queries/QueryTest/insert_overwrite.test M tests/query_test/test_insert.py 6 files changed, 106 insertions(+), 162 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/15529/1 -- To view, visit http://gerrit.cloudera.org:8080/15529 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I257e936868917a2fcc6c030f6c855b247e8a0eea Gerrit-Change-Number: 15529 Gerrit-PatchSet: 1 Gerrit-Owner: Adam Tamas
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala WebUI by default
Adam Tamas has uploaded a new patch set (#16). ( http://gerrit.cloudera.org:8080/15288 ) Change subject: IMPALA-6360: Don't show full query statement on Impala WebUI by default .. IMPALA-6360: Don't show full query statement on Impala WebUI by default In the WebUI's query list the query statements are trimmed, but the full query statement can be seen in the details page. The default statement length is 250 chars and it can be adjusted by the query_stmt_size flag that can be set when the cluster starts. Example: bin/start-impala-cluster.py -s1 --impalad_args --query_stmt_size=10 Testing: -manual testing in the WebUI. -added 'test_query_stmt()' to test_web_pages.py -added test to the custom cluster webserver tests to check without truncate and with custom length truncate Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 --- M be/src/service/impala-http-handler.cc M be/src/service/impala-server.cc M tests/custom_cluster/test_web_pages.py M tests/webserver/test_web_pages.py M www/queries.tmpl 5 files changed, 62 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/15288/16 -- To view, visit http://gerrit.cloudera.org:8080/15288 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 Gerrit-Change-Number: 15288 Gerrit-PatchSet: 16 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala WebUI by default
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/15288 ) Change subject: IMPALA-6360: Don't show full query statement on Impala WebUI by default .. Patch Set 15: (3 comments) http://gerrit.cloudera.org:8080/#/c/15288/15/tests/custom_cluster/test_web_pages.py File tests/custom_cluster/test_web_pages.py: http://gerrit.cloudera.org:8080/#/c/15288/15/tests/custom_cluster/test_web_pages.py@114 PS15, Line 114: imput > nit: typo Done http://gerrit.cloudera.org:8080/#/c/15288/15/tests/custom_cluster/test_web_pages.py@130 PS15, Line 130: part > nit: partial Done http://gerrit.cloudera.org:8080/#/c/15288/15/tests/webserver/test_web_pages.py File tests/webserver/test_web_pages.py: http://gerrit.cloudera.org:8080/#/c/15288/15/tests/webserver/test_web_pages.py@422 PS15, Line 422: x > nit: I would explicitly write it as "x " (notice the space after the x), li Done -- To view, visit http://gerrit.cloudera.org:8080/15288 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 Gerrit-Change-Number: 15288 Gerrit-PatchSet: 15 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Mon, 23 Mar 2020 09:59:37 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala WebUI by default
Adam Tamas has uploaded a new patch set (#15). ( http://gerrit.cloudera.org:8080/15288 ) Change subject: IMPALA-6360: Don't show full query statement on Impala WebUI by default .. IMPALA-6360: Don't show full query statement on Impala WebUI by default In the WebUI's query list the query statements are trimmed, but the full query statement can be seen in the details page. The default statement length is 250 chars and it can be adjusted by the query_stmt_size flag that can be set when the cluster starts. Example: bin/start-impala-cluster.py -s1 --impalad_args --query_stmt_size=10 Testing: -manual testing in the WebUI. -added 'test_query_stmt()' to test_web_pages.py -added test to the custom cluster webserver tests to check without truncate and with custom length truncate Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 --- M be/src/service/impala-http-handler.cc M be/src/service/impala-server.cc M tests/custom_cluster/test_web_pages.py M tests/webserver/test_web_pages.py M www/queries.tmpl 5 files changed, 62 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/15288/15 -- To view, visit http://gerrit.cloudera.org:8080/15288 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 Gerrit-Change-Number: 15288 Gerrit-PatchSet: 15 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala WebUI by default
Adam Tamas has uploaded a new patch set (#14). ( http://gerrit.cloudera.org:8080/15288 ) Change subject: IMPALA-6360: Don't show full query statement on Impala WebUI by default .. IMPALA-6360: Don't show full query statement on Impala WebUI by default In the WebUI's query list the query statements are trimmed, but the full query statement can be seen in the details page. The default statement length is 250 chars and it can be adjusted by the query_stmt_size flag that can be set when the cluster starts. Example: bin/start-impala-cluster.py -s1 --impalad_args --query_stmt_size=10 Testing: -manual testing in the WebUI. -added 'test_query_stmt()' to test_web_pages.py -added test to the custom cluster webserver tests to check without truncate and with custom length truncate Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 --- M be/src/service/impala-http-handler.cc M be/src/service/impala-server.cc M tests/custom_cluster/test_web_pages.py M tests/webserver/test_web_pages.py M www/queries.tmpl 5 files changed, 63 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/15288/14 -- To view, visit http://gerrit.cloudera.org:8080/15288 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 Gerrit-Change-Number: 15288 Gerrit-PatchSet: 14 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala WebUI by default
Adam Tamas has uploaded a new patch set (#13). ( http://gerrit.cloudera.org:8080/15288 ) Change subject: IMPALA-6360: Don't show full query statement on Impala WebUI by default .. IMPALA-6360: Don't show full query statement on Impala WebUI by default In the WebUI's query list the query statements are trimmed, but the full query statement can be seen in the details page. The default statement length is 250 chars and it can be adjusted by the query_stmt_size flag that can be set when the cluster starts. Example: bin/start-impala-cluster.py -s1 --impalad_args --query_stmt_size=10 Testing: -manual testing in the WebUI. -added 'test_query_stmt()' to test_web_pages.py -added test to the custom cluster webserver tests to check without truncate and with custom length truncate Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 --- M be/src/service/impala-http-handler.cc M be/src/service/impala-server.cc M tests/custom_cluster/test_web_pages.py M tests/webserver/test_web_pages.py M www/queries.tmpl 5 files changed, 63 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/15288/13 -- To view, visit http://gerrit.cloudera.org:8080/15288 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 Gerrit-Change-Number: 15288 Gerrit-PatchSet: 13 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala WebUI by default
Adam Tamas has uploaded a new patch set (#12). ( http://gerrit.cloudera.org:8080/15288 ) Change subject: IMPALA-6360: Don't show full query statement on Impala WebUI by default .. IMPALA-6360: Don't show full query statement on Impala WebUI by default In the WebUI's query list the query statements are trimmed, but the full query statement can be seen in the details page. The default statement length is 250 chars and it can be adjusted by the query_stmt_size flag that can be set when the cluster starts. Example: bin/start-impala-cluster.py -s1 --impalad_args --query_stmt_size=10 Testing: -manual testing in the WebUI. -added 'test_query_stmt()' to test_web_pages.py Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 --- M be/src/service/impala-http-handler.cc M be/src/service/impala-server.cc M tests/webserver/test_web_pages.py 3 files changed, 34 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/15288/12 -- To view, visit http://gerrit.cloudera.org:8080/15288 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 Gerrit-Change-Number: 15288 Gerrit-PatchSet: 12 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-7686: Allow RANGE() clause before HASH() clause for PARTITION BY
Adam Tamas has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/15332 ) Change subject: IMPALA-7686: Allow RANGE() clause before HASH() clause for PARTITION BY .. IMPALA-7686: Allow RANGE() clause before HASH() clause for PARTITION BY Modified the sql_parser.cup to accept the reversed syntax. Testing: -Added extra analyzer tests to cover the case when RANGE() is before HASH() for Kudu tables. Change-Id: I914e340e9acfb0f49c7d4f78705dbd9bde0aec8c --- M fe/src/main/cup/sql-parser.cup M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java 3 files changed, 19 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/32/15332/7 -- To view, visit http://gerrit.cloudera.org:8080/15332 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I914e340e9acfb0f49c7d4f78705dbd9bde0aec8c Gerrit-Change-Number: 15332 Gerrit-PatchSet: 7 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala WebUI by default
Adam Tamas has uploaded a new patch set (#11). ( http://gerrit.cloudera.org:8080/15288 ) Change subject: IMPALA-6360: Don't show full query statement on Impala WebUI by default .. IMPALA-6360: Don't show full query statement on Impala WebUI by default In the WebUI's query list the query statements are trimmed, but the full query statement can be seen in the details page. The default statement length is 250 chars and it can be adjusted by the query_stmt_size flag that can be set when the cluster starts. Example: bin/start-impala-cluster.py -s1 --impalad_args --query_stmt_size=10 Testing: -manual testing in the WebUI. -added 'test_query_stmt()' to test_web_pages.py Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 --- M be/src/service/impala-http-handler.cc M be/src/service/impala-server.cc M tests/webserver/test_web_pages.py 3 files changed, 33 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/15288/11 -- To view, visit http://gerrit.cloudera.org:8080/15288 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 Gerrit-Change-Number: 15288 Gerrit-PatchSet: 11 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-7686: Allow RANGE() clause before HASH() clause for PARTITION BY
Adam Tamas has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/15332 ) Change subject: IMPALA-7686: Allow RANGE() clause before HASH() clause for PARTITION BY .. IMPALA-7686: Allow RANGE() clause before HASH() clause for PARTITION BY Modified the sql_parser.cup to accept the reversed syntax. Testing: -Added extra analyzer tests to cover the case when RANGE() is before HASH() for Kudu tables. Change-Id: I914e340e9acfb0f49c7d4f78705dbd9bde0aec8c --- M fe/src/main/cup/sql-parser.cup M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java 3 files changed, 19 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/32/15332/5 -- To view, visit http://gerrit.cloudera.org:8080/15332 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I914e340e9acfb0f49c7d4f78705dbd9bde0aec8c Gerrit-Change-Number: 15332 Gerrit-PatchSet: 5 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala WebUI by default
Adam Tamas has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/15288 ) Change subject: IMPALA-6360: Don't show full query statement on Impala WebUI by default .. IMPALA-6360: Don't show full query statement on Impala WebUI by default In the WebUI's query list the query statements are trimmed, but the full query statement can be seen in the details page. The default statement length is 250 chars and it can be adjusted by the query_stmt_size flag that can be set when the cluster starts. Example: bin/start-impala-cluster.py -s1 --impalad_args --query_stmt_size=10 Testing: -manual testing in the WebUI. -added 'test_query_stmt()' to test_web_pages.py Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 --- M be/src/service/impala-http-handler.cc M be/src/service/impala-server.cc M tests/webserver/test_web_pages.py 3 files changed, 33 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/15288/10 -- To view, visit http://gerrit.cloudera.org:8080/15288 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 Gerrit-Change-Number: 15288 Gerrit-PatchSet: 10 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala WebUI by default
Adam Tamas has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/15288 ) Change subject: IMPALA-6360: Don't show full query statement on Impala WebUI by default .. IMPALA-6360: Don't show full query statement on Impala WebUI by default In the WebUI's query list the query statements are trimmed, but the full query statement can be seen in the details page. The default statement length is 250 chars and it can be adjusted by the query_stmt_size flag that can be set when the cluster starts. Example: bin/start-impala-cluster.py -s1 --impalad_args --query_stmt_size=10 Testing: -manual testing in the WebUI. -added 'test_query_stmt()' to test_web_pages.py Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 --- M be/src/service/impala-http-handler.cc M be/src/service/impala-server.cc M tests/webserver/test_web_pages.py 3 files changed, 33 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/15288/9 -- To view, visit http://gerrit.cloudera.org:8080/15288 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 Gerrit-Change-Number: 15288 Gerrit-PatchSet: 9 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-7686: Allow RANGE() clause before HASH() clause for PARTITION BY
Adam Tamas has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/15332 ) Change subject: IMPALA-7686: Allow RANGE() clause before HASH() clause for PARTITION BY .. IMPALA-7686: Allow RANGE() clause before HASH() clause for PARTITION BY Modified the sql_parser.cup to accept the reversed syntax. Testing: -Added extra analyzer tests to cover the case when RANGE() is before HASH() for Kudu tables. Change-Id: I914e340e9acfb0f49c7d4f78705dbd9bde0aec8c --- M fe/src/main/cup/sql-parser.cup M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java 2 files changed, 17 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/32/15332/4 -- To view, visit http://gerrit.cloudera.org:8080/15332 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I914e340e9acfb0f49c7d4f78705dbd9bde0aec8c Gerrit-Change-Number: 15332 Gerrit-PatchSet: 4 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-7686: Allow RANGE() clause before HASH() clause for PARTITION BY
Adam Tamas has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/15332 ) Change subject: IMPALA-7686: Allow RANGE() clause before HASH() clause for PARTITION BY .. IMPALA-7686: Allow RANGE() clause before HASH() clause for PARTITION BY Modified the sql_parser.cup to accept the reversed syntax. Testing: -Added an extra AnalyzesOk into ‘AnalyzeKuduDDLTest()’ -> ‘testDDlsOnKuduTable()’ in AnalyzeKuduDDLTest.java where the RANGE() and HASH() pair is swapped. Change-Id: I914e340e9acfb0f49c7d4f78705dbd9bde0aec8c --- M fe/src/main/cup/sql-parser.cup M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java 2 files changed, 10 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/32/15332/3 -- To view, visit http://gerrit.cloudera.org:8080/15332 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I914e340e9acfb0f49c7d4f78705dbd9bde0aec8c Gerrit-Change-Number: 15332 Gerrit-PatchSet: 3 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala webUI by default
Adam Tamas has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/15288 ) Change subject: IMPALA-6360: Don't show full query statement on Impala webUI by default .. IMPALA-6360: Don't show full query statement on Impala webUI by default Added the ‘query_stmt_size’ flag to impala-server.cc with default value of 250 and modified the ‘ImpalaHttpHandler::QueryStateToJson()’ to truncate the end of the statements if they are too long. Testing: -manual testing in the WebUI. -added 'test_query_stmt()' to test_web_pages.py Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 --- M be/src/service/impala-http-handler.cc M be/src/service/impala-server.cc M tests/webserver/test_web_pages.py 3 files changed, 33 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/15288/8 -- To view, visit http://gerrit.cloudera.org:8080/15288 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 Gerrit-Change-Number: 15288 Gerrit-PatchSet: 8 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-7686: Allow RANGE() clause before HASH() clause for PARTITION BY
Adam Tamas has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/15332 ) Change subject: IMPALA-7686: Allow RANGE() clause before HASH() clause for PARTITION BY .. IMPALA-7686: Allow RANGE() clause before HASH() clause for PARTITION BY Modified the sql_parser.cup to accept the reversed syntax. Testing: -Added an extra AnalyzesOk into ‘AnalyzeKuduDDLTest()’ -> ‘testDDlsOnKuduTable()’ in AnalyzeKuduDDLTest.java where the RANGE() and HASH() pair is swapped. Change-Id: I914e340e9acfb0f49c7d4f78705dbd9bde0aec8c --- M fe/src/main/cup/sql-parser.cup M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java 2 files changed, 10 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/32/15332/2 -- To view, visit http://gerrit.cloudera.org:8080/15332 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I914e340e9acfb0f49c7d4f78705dbd9bde0aec8c Gerrit-Change-Number: 15332 Gerrit-PatchSet: 2 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-7686: Allow RANGE() clause before HASH() clause for PARTITION BY
Adam Tamas has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15332 Change subject: IMPALA-7686: Allow RANGE() clause before HASH() clause for PARTITION BY .. IMPALA-7686: Allow RANGE() clause before HASH() clause for PARTITION BY Modified the sql_parser.cup to accept the reversed syntax. Testing: -swapped an instance of RANGE() + HASH() pair in test_kudu.py -> ‘test_primary_key_and_distribution()’ and left the rest to check if it is still create the right table with ‘assert_show_create_equals()’. Change-Id: I914e340e9acfb0f49c7d4f78705dbd9bde0aec8c --- M fe/src/main/cup/sql-parser.cup M tests/query_test/test_kudu.py 2 files changed, 10 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/32/15332/1 -- To view, visit http://gerrit.cloudera.org:8080/15332 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I914e340e9acfb0f49c7d4f78705dbd9bde0aec8c Gerrit-Change-Number: 15332 Gerrit-PatchSet: 1 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala webUI by default
Adam Tamas has posted comments on this change. ( http://gerrit.cloudera.org:8080/15288 ) Change subject: IMPALA-6360: Don't show full query statement on Impala webUI by default .. Patch Set 7: (1 comment) http://gerrit.cloudera.org:8080/#/c/15288/6/tests/webserver/test_web_pages.py File tests/webserver/test_web_pages.py: http://gerrit.cloudera.org:8080/#/c/15288/6/tests/webserver/test_web_pages.py@423 PS6, Line 423: > nit: This should be in the same line as the string, also the second line sh Done -- To view, visit http://gerrit.cloudera.org:8080/15288 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 Gerrit-Change-Number: 15288 Gerrit-PatchSet: 7 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Mon, 02 Mar 2020 13:48:28 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala webUI by default
Adam Tamas has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/15288 ) Change subject: IMPALA-6360: Don't show full query statement on Impala webUI by default .. IMPALA-6360: Don't show full query statement on Impala webUI by default Added the ‘query_stmt_size’ flag to impala-server.cc with default value of 250 and modified the ‘ImpalaHttpHandler::QueryStateToJson()’ to truncate the end of the statements if they are too long. Testing: -manual testing in the WebUI. -added 'test_query_stmt()' to test_web_pages.py Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 --- M be/src/service/impala-http-handler.cc M be/src/service/impala-server.cc M tests/webserver/test_web_pages.py 3 files changed, 32 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/15288/7 -- To view, visit http://gerrit.cloudera.org:8080/15288 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 Gerrit-Change-Number: 15288 Gerrit-PatchSet: 7 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala webUI by default
Adam Tamas has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/15288 ) Change subject: IMPALA-6360: Don't show full query statement on Impala webUI by default .. IMPALA-6360: Don't show full query statement on Impala webUI by default Added the ‘query_stmt_size’ flag to impala-server.cc with default value of 250 and modified the ‘ImpalaHttpHandler::QueryStateToJson()’ to truncate the end of the statements if they are too long. Testing: -manual testing in the WebUI. -added 'test_query_stmt()' to test_web_pages.py Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 --- M be/src/service/impala-http-handler.cc M be/src/service/impala-server.cc M tests/webserver/test_web_pages.py 3 files changed, 34 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/15288/6 -- To view, visit http://gerrit.cloudera.org:8080/15288 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 Gerrit-Change-Number: 15288 Gerrit-PatchSet: 6 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate
