[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. IMPALA-10051: impala-shell exits with ValueError with WITH clauses When a query contains WITH clause impala-shell tries to identify whether it is a DML query or not, so that later it can provide appropriate result messages. Earlier shlex was used to create tokens and assess the query type based on that. However shlex can misinterpret some query strings where whitespace charachters are mixed with quotes, because it splits the string based on whitespace charachters. In some scenarios 'ValueError: No closing quotation' error can occur. This change moves the tokenization from shlex to sqlparse. Testing: - Added unit test to cover queries that contain mixed whitespaces and strings Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Reviewed-on: http://gerrit.cloudera.org:8080/16389 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M shell/impala_shell.py M tests/shell/test_shell_interactive.py 2 files changed, 19 insertions(+), 9 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 7 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 6 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Fri, 18 Sep 2020 04:06:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 6 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Thu, 17 Sep 2020 22:35:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6442/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 6 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Thu, 17 Sep 2020 22:35:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 5: Thank you for the review Fredy. Could you start a dry-run=true or merge the change as well? I do not have the privileges to do it myself. -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 5 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Thu, 17 Sep 2020 06:55:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 5: > Patch Set 5: > > Hi Fredy, > Just checking in if you had some time to review the current version. LGTM. Thanks for fixing this! -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 5 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Tue, 15 Sep 2020 16:30:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 5 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Tue, 15 Sep 2020 16:30:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 5: Hi Fredy, Just checking in if you had some time to review the current version. -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 5 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Tue, 15 Sep 2020 14:53:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 5: Code-Review+1 Looks good! -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 5 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Fri, 04 Sep 2020 14:48:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 5 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Fri, 04 Sep 2020 13:23:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6401/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 5 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Fri, 04 Sep 2020 07:43:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 5: (2 comments) Thank you for the review Qifan. Updated the change. The previous build failure seem to be due to a flaky test, I have opened a Jira for that IMPALA-10143. http://gerrit.cloudera.org:8080/#/c/16389/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16389/4//COMMIT_MSG@9 PS4, Line 9: co > nit: removal? Done http://gerrit.cloudera.org:8080/#/c/16389/4/tests/shell/test_shell_interactive.py File tests/shell/test_shell_interactive.py: http://gerrit.cloudera.org:8080/#/c/16389/4/tests/shell/test_shell_interactive.py@1059 PS4, Line 1059: fix prevents Va > nit: reworded as 'prevents' Done -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 5 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Thu, 03 Sep 2020 19:14:33 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7088/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 5 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Thu, 03 Sep 2020 19:07:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Hello Qifan Chen, Fredy Wijaya, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16389 to look at the new patch set (#5). Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. IMPALA-10051: impala-shell exits with ValueError with WITH clauses When a query contains WITH clause impala-shell tries to identify whether it is a DML query or not, so that later it can provide appropriate result messages. Earlier shlex was used to create tokens and assess the query type based on that. However shlex can misinterpret some query strings where whitespace charachters are mixed with quotes, because it splits the string based on whitespace charachters. In some scenarios 'ValueError: No closing quotation' error can occur. This change moves the tokenization from shlex to sqlparse. Testing: - Added unit test to cover queries that contain mixed whitespaces and strings Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad --- M shell/impala_shell.py M tests/shell/test_shell_interactive.py 2 files changed, 19 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/16389/5 -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 5 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 4: (2 comments) Looks good to me. Use of sql parser to classify a query is the right way to go. http://gerrit.cloudera.org:8080/#/c/16389/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16389/4//COMMIT_MSG@9 PS4, Line 9: a nit: removal? http://gerrit.cloudera.org:8080/#/c/16389/4/tests/shell/test_shell_interactive.py File tests/shell/test_shell_interactive.py: http://gerrit.cloudera.org:8080/#/c/16389/4/tests/shell/test_shell_interactive.py@1059 PS4, Line 1059: applied to avoid nit: reworded as 'prevents' -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 4 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Thu, 03 Sep 2020 14:35:39 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 4 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Thu, 03 Sep 2020 14:12:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6395/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 4 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Thu, 03 Sep 2020 08:48:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 3: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/6375/ -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 3 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Tue, 01 Sep 2020 18:46:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 3 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Tue, 01 Sep 2020 14:28:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6375/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 3 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Tue, 01 Sep 2020 13:25:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7061/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 3 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Tue, 01 Sep 2020 12:57:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Tamas Mate has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. IMPALA-10051: impala-shell exits with ValueError with WITH clauses When a query a contains WITH clause impala-shell tries to identify whether it is a DML query or not, so that later it can provide appropriate result messages. Earlier shlex was used to create tokens and assess the query type based on that. However shlex can misinterpret some query strings where whitespace charachters are mixed with quotes, because it splits the string based on whitespace charachters. In some scenarios 'ValueError: No closing quotation' error can occur. This change moves the tokenization from shlex to sqlparse. Testing: - Added unit test to cover queries that contain mixed whitespaces and strings Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad --- M shell/impala_shell.py M tests/shell/test_shell_interactive.py 2 files changed, 19 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/16389/3 -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 3 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 2: (1 comment) Thanks for the review Fredy. Fixed the dry-run test, the assert was dev env specific, my test table was empty. http://gerrit.cloudera.org:8080/#/c/16389/2/shell/impala_shell.py File shell/impala_shell.py: http://gerrit.cloudera.org:8080/#/c/16389/2/shell/impala_shell.py@1280 PS2, Line 1280: if self.DML_REGEX.match(query_type.lower()): > looks like there were failed tests in the dry-run Right, it is cleaner this way. Done. -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 2 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Tue, 01 Sep 2020 11:54:45 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/16389/2/shell/impala_shell.py File shell/impala_shell.py: http://gerrit.cloudera.org:8080/#/c/16389/2/shell/impala_shell.py@1280 PS2, Line 1280: if self.DML_REGEX.match(query_type.lower()): looks like there were failed tests in the dry-run nit: this code can be simplified like below. is_dml = self.DML_REGEX.match(query_type.lower()) return self._execute_stmt(query, is_dml=is_dml, print_web_link=True) -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 2 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 31 Aug 2020 22:06:18 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 2: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/6365/ -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 2 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 31 Aug 2020 15:29:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6365/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 2 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 31 Aug 2020 10:17:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16389 ) Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7051/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 2 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 31 Aug 2020 10:13:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10051: impala-shell exits with ValueError with WITH clauses
Tamas Mate has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16389 Change subject: IMPALA-10051: impala-shell exits with ValueError with WITH clauses .. IMPALA-10051: impala-shell exits with ValueError with WITH clauses When a query a contains WITH clause impala-shell tries to identify whether it is a DML query or not, so that later it can provide appropriate result messages. Earlier shlex was used to create tokens and assess the query type based on that. However shlex can misinterpret some query strings where whitespace charachters are mixed with quotes, because it splits the string based on whitespace charachters. In some scenarios 'ValueError: No closing quotation' error can occur. This change moves the tokenization from shlex to sqlparse. Testing: - Added unit test to cover queries that contain mixed whitespaces and strings Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad --- M shell/impala_shell.py M tests/shell/test_shell_interactive.py 2 files changed, 21 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/16389/2 -- To view, visit http://gerrit.cloudera.org:8080/16389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I442d3bc65b90a55c73c847948d5179a8586d71ad Gerrit-Change-Number: 16389 Gerrit-PatchSet: 2 Gerrit-Owner: Tamas Mate
