[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Michael Smith has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. IMPALA-10086: Implicit cast comparing char and varchar Until IMPALA-7368, Impala allowed comparing char and varchar slots as in select * from functional.chars_tiny where cs = vc; IMPALA-7368 added DATE type support, and as part of that changed function call resolution to use a fit function based on the number of arguments that match the call types. Previously the comparison above would take the first matching function, which happened to be equality between STRING and STRING; CHAR and VARCHAR can both be implicitly cast to STRING, so this function worked. With the new function resolution, the best fit is equality between VARCHAR and VARCHAR, however implicit casting to VARCHAR(*) from CHAR wasn't allowed. The behavior before IMPALA-7368 was somewhat accidental; it depended on the order that builtin EQ functions are added via BinaryPredicate.initBuiltins -> Type.getSupportedTypes. Supported types happened to be ordered with STRING preceding VARCHAR and CHAR. The fit function makes sense and changing its behavior may have other consequences; it also makes sense that CHAR should be castable to VARCHAR. This change allows implicit cast between matching types. Functionally it only changes how we handle char/varchar comparison with wildcard char/varchar, because decimals are handled before checking for matching types and other type matching is the same as equals. It now allows casting to a compatible type when it is a char or varchar and the target type is a wildcard version of the same. Does not attempt to address differences from CHAR padding (IMPALA-1652). Testing: - Adds tests covering cast comparison and other implicit conversions. - Passed exhaustive test run. Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Reviewed-on: http://gerrit.cloudera.org:8080/20425 Reviewed-by: Peter Rozsa Reviewed-by: Daniel Becker Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java M tests/query_test/test_cast_with_format.py 3 files changed, 71 insertions(+), 9 deletions(-) Approvals: Peter Rozsa: Looks good to me, but someone else must approve Daniel Becker: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 7 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 6 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 12 Sep 2023 12:46:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9705/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 6 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 12 Sep 2023 08:25:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 6 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 12 Sep 2023 08:24:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Peter Rozsa has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 6: Code-Review+1 LGTM -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 6 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 12 Sep 2023 06:17:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 12 Sep 2023 01:04:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/20425/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/20425/5//COMMIT_MSG@29 PS5, Line 29: : This change allows implicit cast between matching types > Can you highlight that this change only affects char/varchar, as for other Done -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 6 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Mon, 11 Sep 2023 21:26:00 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Hello Quanlong Huang, Daniel Becker, Peter Rozsa, Joe McDonnell, Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20425 to look at the new patch set (#6). Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. IMPALA-10086: Implicit cast comparing char and varchar Until IMPALA-7368, Impala allowed comparing char and varchar slots as in select * from functional.chars_tiny where cs = vc; IMPALA-7368 added DATE type support, and as part of that changed function call resolution to use a fit function based on the number of arguments that match the call types. Previously the comparison above would take the first matching function, which happened to be equality between STRING and STRING; CHAR and VARCHAR can both be implicitly cast to STRING, so this function worked. With the new function resolution, the best fit is equality between VARCHAR and VARCHAR, however implicit casting to VARCHAR(*) from CHAR wasn't allowed. The behavior before IMPALA-7368 was somewhat accidental; it depended on the order that builtin EQ functions are added via BinaryPredicate.initBuiltins -> Type.getSupportedTypes. Supported types happened to be ordered with STRING preceding VARCHAR and CHAR. The fit function makes sense and changing its behavior may have other consequences; it also makes sense that CHAR should be castable to VARCHAR. This change allows implicit cast between matching types. Functionally it only changes how we handle char/varchar comparison with wildcard char/varchar, because decimals are handled before checking for matching types and other type matching is the same as equals. It now allows casting to a compatible type when it is a char or varchar and the target type is a wildcard version of the same. Does not attempt to address differences from CHAR padding (IMPALA-1652). Testing: - Adds tests covering cast comparison and other implicit conversions. - Passed exhaustive test run. Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 --- M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java M tests/query_test/test_cast_with_format.py 3 files changed, 71 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/25/20425/6 -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 6 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 5: Code-Review+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/20425/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/20425/5//COMMIT_MSG@29 PS5, Line 29: : This change allows implicit cast between matching types Can you highlight that this change only affects char/varchar, as for other types matching is the same as equals, and decimals are handled separately? -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Mon, 11 Sep 2023 21:19:32 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9701/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Mon, 11 Sep 2023 20:42:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/13970/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Mon, 11 Sep 2023 18:03:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/20425/4/fe/src/main/java/org/apache/impala/analysis/Expr.java File fe/src/main/java/org/apache/impala/analysis/Expr.java: http://gerrit.cloudera.org:8080/#/c/20425/4/fe/src/main/java/org/apache/impala/analysis/Expr.java@1576 PS4, Line 1576: ways returns a "real" (not wildcard) type. > This part is essentially the same as 'cast to the type both values can be a Done -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Mon, 11 Sep 2023 17:37:27 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Hello Quanlong Huang, Daniel Becker, Joe McDonnell, Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20425 to look at the new patch set (#5). Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. IMPALA-10086: Implicit cast comparing char and varchar Until IMPALA-7368, Impala allowed comparing char and varchar slots as in select * from functional.chars_tiny where cs = vc; IMPALA-7368 added DATE type support, and as part of that changed function call resolution to use a fit function based on the number of arguments that match the call types. Previously the comparison above would take the first matching function, which happened to be equality between STRING and STRING; CHAR and VARCHAR can both be implicitly cast to STRING, so this function worked. With the new function resolution, the best fit is equality between VARCHAR and VARCHAR, however implicit casting to VARCHAR(*) from CHAR wasn't allowed. The behavior before IMPALA-7368 was somewhat accidental; it depended on the order that builtin EQ functions are added via BinaryPredicate.initBuiltins -> Type.getSupportedTypes. Supported types happened to be ordered with STRING preceding VARCHAR and CHAR. The fit function makes sense and changing its behavior may have other consequences; it also makes sense that CHAR should be castable to VARCHAR. This change allows implicit cast between matching types. That includes equals, and adds casting to a compatible type when it is a char or varchar and the target type is a wildcard version of the same. Does not attempt to address differences from CHAR padding (IMPALA-1652). Testing: - Adds tests covering cast comparison and other implicit conversions. - Passed exhaustive test run. Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 --- M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java M tests/query_test/test_cast_with_format.py 3 files changed, 71 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/25/20425/5 -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Daniel Becker has posted comments on this change. (
http://gerrit.cloudera.org:8080/20425 )
Change subject: IMPALA-10086: Implicit cast comparing char and varchar
..
Patch Set 4:
(1 comment)
Thanks Michael.
http://gerrit.cloudera.org:8080/#/c/20425/4/fe/src/main/java/org/apache/impala/analysis/Expr.java
File fe/src/main/java/org/apache/impala/analysis/Expr.java:
http://gerrit.cloudera.org:8080/#/c/20425/4/fe/src/main/java/org/apache/impala/analysis/Expr.java@1576
PS4, Line 1576: Requested cast must be to assignment-compatible type
This part is essentially the same as 'cast to the type both values can be
assigned to', so we could delete it and move the parenthesised part ('which
implies no loss of precision') to the sentence above.
--
To view, visit http://gerrit.cloudera.org:8080/20425
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436
Gerrit-Change-Number: 20425
Gerrit-PatchSet: 4
Gerrit-Owner: Michael Smith
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Daniel Becker
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Michael Smith
Gerrit-Reviewer: Quanlong Huang
Gerrit-Comment-Date: Mon, 11 Sep 2023 11:15:46 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/13959/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 4 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 08 Sep 2023 22:14:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Michael Smith has posted comments on this change. (
http://gerrit.cloudera.org:8080/20425 )
Change subject: IMPALA-10086: Implicit cast comparing char and varchar
..
Patch Set 3:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/20425/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/20425/3//COMMIT_MSG@14
PS3, Line 14: number of
: arguments match
> Nit: 'number of arguments _that_ match'?
Done
http://gerrit.cloudera.org:8080/#/c/20425/3/fe/src/main/java/org/apache/impala/analysis/Expr.java
File fe/src/main/java/org/apache/impala/analysis/Expr.java:
http://gerrit.cloudera.org:8080/#/c/20425/3/fe/src/main/java/org/apache/impala/analysis/Expr.java@1574
PS3, Line 1574: more specific compatible type
> Do we know that 'type' is always the more specific type and not 'targetType
Done. Best I can reason is that Impala doesn't use wildcard types for storage,
so we shouldn't cast to them. And getAssignmentCompatibleType will always
return a char/varchar version that has real (positive) storage rather than the
wildcard version.
http://gerrit.cloudera.org:8080/#/c/20425/3/tests/query_test/test_cast_with_format.py
File tests/query_test/test_cast_with_format.py:
http://gerrit.cloudera.org:8080/#/c/20425/3/tests/query_test/test_cast_with_format.py@2191
PS3, Line 2191: self.execute_query("drop table if exists " + table)
> I don't think we need this, 'unique_database' will always start out as empt
Done
--
To view, visit http://gerrit.cloudera.org:8080/20425
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436
Gerrit-Change-Number: 20425
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Smith
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Daniel Becker
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Michael Smith
Gerrit-Reviewer: Quanlong Huang
Gerrit-Comment-Date: Fri, 08 Sep 2023 21:51:11 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Hello Quanlong Huang, Daniel Becker, Joe McDonnell, Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20425 to look at the new patch set (#4). Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. IMPALA-10086: Implicit cast comparing char and varchar Until IMPALA-7368, Impala allowed comparing char and varchar slots as in select * from functional.chars_tiny where cs = vc; IMPALA-7368 added DATE type support, and as part of that changed function call resolution to use a fit function based on the number of arguments that match the call types. Previously the comparison above would take the first matching function, which happened to be equality between STRING and STRING; CHAR and VARCHAR can both be implicitly cast to STRING, so this function worked. With the new function resolution, the best fit is equality between VARCHAR and VARCHAR, however implicit casting to VARCHAR(*) from CHAR wasn't allowed. The behavior before IMPALA-7368 was somewhat accidental; it depended on the order that builtin EQ functions are added via BinaryPredicate.initBuiltins -> Type.getSupportedTypes. Supported types happened to be ordered with STRING preceding VARCHAR and CHAR. The fit function makes sense and changing its behavior may have other consequences; it also makes sense that CHAR should be castable to VARCHAR. This change allows implicit cast between matching types. That includes equals, and adds casting to a compatible type when it is a char or varchar and the target type is a wildcard version of the same. Adds tests covering cast comparison and other implicit conversions. Does not attempt to address differences from CHAR padding (IMPALA-1652). Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 --- M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java M tests/query_test/test_cast_with_format.py 3 files changed, 72 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/25/20425/4 -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 4 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Daniel Becker has posted comments on this change. (
http://gerrit.cloudera.org:8080/20425 )
Change subject: IMPALA-10086: Implicit cast comparing char and varchar
..
Patch Set 3:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/20425/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/20425/3//COMMIT_MSG@14
PS3, Line 14: number of
: arguments match
Nit: 'number of arguments _that_ match'?
http://gerrit.cloudera.org:8080/#/c/20425/3/fe/src/main/java/org/apache/impala/analysis/Expr.java
File fe/src/main/java/org/apache/impala/analysis/Expr.java:
http://gerrit.cloudera.org:8080/#/c/20425/3/fe/src/main/java/org/apache/impala/analysis/Expr.java@1574
PS3, Line 1574: more specific compatible type
Do we know that 'type' is always the more specific type and not 'targetType'?
Is it because Type.getAssignmentCompatibleType() doesn't return wildcard types?
If so, it could be included in the comment.
http://gerrit.cloudera.org:8080/#/c/20425/3/tests/query_test/test_cast_with_format.py
File tests/query_test/test_cast_with_format.py:
http://gerrit.cloudera.org:8080/#/c/20425/3/tests/query_test/test_cast_with_format.py@2191
PS3, Line 2191: self.execute_query("drop table if exists " + table)
I don't think we need this, 'unique_database' will always start out as empty.
--
To view, visit http://gerrit.cloudera.org:8080/20425
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436
Gerrit-Change-Number: 20425
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Smith
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Daniel Becker
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Michael Smith
Gerrit-Reviewer: Quanlong Huang
Gerrit-Comment-Date: Fri, 08 Sep 2023 12:54:24 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 3 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 07 Sep 2023 23:10:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/20425/1/tests/query_test/test_cast_with_format.py File tests/query_test/test_cast_with_format.py: http://gerrit.cloudera.org:8080/#/c/20425/1/tests/query_test/test_cast_with_format.py@2197 PS1, Line 2197: # Compare char to varchar > I tracked this down to http://gerrit.cloudera.org:8080/12481, between Impal Updated the commit message with my findings. -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 3 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 07 Sep 2023 18:54:12 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9683/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 3 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 07 Sep 2023 18:52:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Hello Quanlong Huang, Daniel Becker, Joe McDonnell, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20425 to look at the new patch set (#3). Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. IMPALA-10086: Implicit cast comparing char and varchar Until IMPALA-7368, Impala allowed comparing char and varchar slots as in select * from functional.chars_tiny where cs = vc; IMPALA-7368 added DATE type support, and as part of that changed function call resolution to use a fit function based on the number of arguments match the call types. Previously the comparison above would take the first matching function, which happened to be equality between STRING and STRING; CHAR and VARCHAR can both be implicitly cast to STRING, so this function worked. With the new function resolution, the best fit is equality between VARCHAR and VARCHAR, however implicit casting to VARCHAR(*) from CHAR wasn't allowed. The behavior before IMPALA-7368 was somewhat accidental; it depended on the order that builtin EQ functions are added via BinaryPredicate.initBuiltins -> Type.getSupportedTypes. Supported types happened to be ordered with STRING preceding VARCHAR and CHAR. The fit function makes sense and changing its behavior may have other consequences; it also makes sense that CHAR should be castable to VARCHAR. This change allows implicit cast between matching types. That includes equals, and adds casting to a compatible type when it is a char or varchar and the target type is a wildcard version of the same. Adds tests covering cast comparison and other implicit conversions. Does not attempt to address differences from CHAR padding (IMPALA-1652). Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 --- M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java M tests/query_test/test_cast_with_format.py 3 files changed, 71 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/25/20425/3 -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 3 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/20425/1/tests/query_test/test_cast_with_format.py File tests/query_test/test_cast_with_format.py: http://gerrit.cloudera.org:8080/#/c/20425/1/tests/query_test/test_cast_with_format.py@2197 PS1, Line 2197: # Compare char to varchar > I didn't identify the specific JIRA or commit. Given that this change is in I tracked this down to http://gerrit.cloudera.org:8080/12481, between Impala 3.2 and 3.3. On first glance, I'd guess this is due to the change in best-fit logic for resolving function calls. I'll have to spend some time with that change to understand why it caused this. -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 07 Sep 2023 02:21:58 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/13921/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 06 Sep 2023 01:01:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 2: (3 comments) http://gerrit.cloudera.org:8080/#/c/20425/2/fe/src/main/java/org/apache/impala/analysis/Expr.java File fe/src/main/java/org/apache/impala/analysis/Expr.java: http://gerrit.cloudera.org:8080/#/c/20425/2/fe/src/main/java/org/apache/impala/analysis/Expr.java@1576 PS2, Line 1576: if (type.matchesType(targetType)) return uncheckedCastTo(type, compatibility); I've updated this to a version that I think makes better semantic sense (matching the comment) and passes the new and existing cast tests (still need to run the full test suite). This preserves the prior behavior around targetType.equals(type) (as long as equals is symmetric). matchesType adds the additional semantic that if type is char/varchar/decimal and targetType is a wildcard version of same, then they match and we cast to the compatible (non-wildcard) version. Casting to targetType tends to not work or crash in the backend, so I suspect representation of wildcard varchar is incomplete (if you create a table with a 'varchar' value, it represents it as STRING). Casting to the compatible type (which must have a length) makes sense to me. I still plan to track down what change broke this behavior. http://gerrit.cloudera.org:8080/#/c/20425/1/fe/src/main/java/org/apache/impala/catalog/DefaultCompatibility.java File fe/src/main/java/org/apache/impala/catalog/DefaultCompatibility.java: http://gerrit.cloudera.org:8080/#/c/20425/1/fe/src/main/java/org/apache/impala/catalog/DefaultCompatibility.java@151 PS1, Line 151: > Looks like I need to add a test case covering that too. Done http://gerrit.cloudera.org:8080/#/c/20425/1/tests/query_test/test_cast_with_format.py File tests/query_test/test_cast_with_format.py: http://gerrit.cloudera.org:8080/#/c/20425/1/tests/query_test/test_cast_with_format.py@2189 PS1, Line 2189: self > Ack Done -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 06 Sep 2023 00:37:55 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Hello Quanlong Huang, Daniel Becker, Joe McDonnell, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20425 to look at the new patch set (#2). Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. IMPALA-10086: Implicit cast comparing char and varchar Allow implicit cast between matching types. That includes equals, and adds casting to a compatible type when it is a char or varchar and the target type is a wildcard version of the same. Adds tests covering cast comparison and other implicit conversions. Does not attempt to address differences from CHAR padding (IMPALA-1652). Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 --- M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java M tests/query_test/test_cast_with_format.py 3 files changed, 71 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/25/20425/2 -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/20425/1/fe/src/main/java/org/apache/impala/catalog/DefaultCompatibility.java File fe/src/main/java/org/apache/impala/catalog/DefaultCompatibility.java: http://gerrit.cloudera.org:8080/#/c/20425/1/fe/src/main/java/org/apache/impala/catalog/DefaultCompatibility.java@151 PS1, Line 151: > Looks like we'll need to figure out why string and varchar backend implemen Looks like I need to add a test case covering that too. -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 05 Sep 2023 22:32:28 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/20425/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/20425/1//COMMIT_MSG@7 PS1, Line 7: IMPALA-10086 > What's the relationship of this change to https://gerrit.cloudera.org/#/c/1 It was attempting to provide a more complete fix. The SqlCastException was filed as a bug because in prior Impala (circa 3.2) it used to cast correctly. The fix in 18001 removed the exception, but didn't actually work. http://gerrit.cloudera.org:8080/#/c/20425/1/tests/query_test/test_cast_with_format.py File tests/query_test/test_cast_with_format.py: http://gerrit.cloudera.org:8080/#/c/20425/1/tests/query_test/test_cast_with_format.py@2189 PS1, Line 2189: self > nit: add 'unique_database' and create the table under it. Ack http://gerrit.cloudera.org:8080/#/c/20425/1/tests/query_test/test_cast_with_format.py@2197 PS1, Line 2197: assert ['test \ttest', 'tester\ttester'] == self.execute_query( > Do you know which JIRA introduces the regression? I didn't identify the specific JIRA or commit. Given that this change is insufficient, that may be my next step. -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 31 Aug 2023 01:48:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 1: Code-Review-1 (1 comment) http://gerrit.cloudera.org:8080/#/c/20425/1/fe/src/main/java/org/apache/impala/catalog/DefaultCompatibility.java File fe/src/main/java/org/apache/impala/catalog/DefaultCompatibility.java: http://gerrit.cloudera.org:8080/#/c/20425/1/fe/src/main/java/org/apache/impala/catalog/DefaultCompatibility.java@151 PS1, Line 151: > Choosing STRING as the common type of CHAR() and VARCHAR() is problematic i Looks like we'll need to figure out why string and varchar backend implementation seem to differ then. -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 31 Aug 2023 01:46:05 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Daniel Becker has posted comments on this change. (
http://gerrit.cloudera.org:8080/20425 )
Change subject: IMPALA-10086: Implicit cast comparing char and varchar
..
Patch Set 1:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/20425/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/20425/1//COMMIT_MSG@7
PS1, Line 7: IMPALA-10086
What's the relationship of this change to
https://gerrit.cloudera.org/#/c/18001/?
http://gerrit.cloudera.org:8080/#/c/20425/1/fe/src/main/java/org/apache/impala/catalog/DefaultCompatibility.java
File fe/src/main/java/org/apache/impala/catalog/DefaultCompatibility.java:
http://gerrit.cloudera.org:8080/#/c/20425/1/fe/src/main/java/org/apache/impala/catalog/DefaultCompatibility.java@151
PS1, Line 151:
Choosing STRING as the common type of CHAR() and VARCHAR() is problematic if
you want to insert the result into a VARCHAR(x) column. I ran into the problem
in this change: https://gerrit.cloudera.org/#/c/18999/
Quoting from the commit message:
"""We choose VARCHAR instead of STRING as the common type because VARCHAR
can be converted to any VARCHAR type shorter or the same length and also
to STRING, while STRING cannot safely be converted to VARCHAR because
its length is not bounded - we would therefore run into problems if the
common type were STRING and the destination column were VARCHAR."""
The following works on master but fails if I run it on this patch:
create table tmp (v varchar(6));
insert into tmp values (cast("aaabbb" as varchar(6))), (cast("cccddd" as
char(6)));
ERROR: AnalysisException: Possible loss of precision for target table
'default.tmp'.
Expression 'CAST('cccddd' AS CHAR(6))' (type: STRING) would need to be cast to
VARCHAR(6) for column 'v'
--
To view, visit http://gerrit.cloudera.org:8080/20425
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436
Gerrit-Change-Number: 20425
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith
Gerrit-Reviewer: Daniel Becker
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Michael Smith
Gerrit-Reviewer: Quanlong Huang
Gerrit-Comment-Date: Tue, 29 Aug 2023 13:12:50 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 1: (3 comments) > Patch Set 1: > > > Patch Set 1: > > > > I'll run a perf-AB-test, as this may have performance implications. > > https://jenkins.impala.io/job/perf-AB-test/473/ Unfortunately, we don't use char/varchar in our tpch/tpcds data sets: https://github.com/apache/impala/blob/master/testdata/datasets/tpch/tpch_schema_template.sql https://github.com/apache/impala/blob/master/testdata/datasets/tpcds/tpcds_schema_template.sql So the current perf-AB-test can't reveal any difference. Optional: we can add a targeted-perf test in testdata/workloads/targeted-perf/queries. http://gerrit.cloudera.org:8080/#/c/20425/1/fe/src/test/java/org/apache/impala/analysis/TypesUtilTest.java File fe/src/test/java/org/apache/impala/analysis/TypesUtilTest.java: http://gerrit.cloudera.org:8080/#/c/20425/1/fe/src/test/java/org/apache/impala/analysis/TypesUtilTest.java@255 PS1, Line 255: } What about adding some unit tests in TypesUtilTest.java? http://gerrit.cloudera.org:8080/#/c/20425/1/tests/query_test/test_cast_with_format.py File tests/query_test/test_cast_with_format.py: http://gerrit.cloudera.org:8080/#/c/20425/1/tests/query_test/test_cast_with_format.py@2189 PS1, Line 2189: self nit: add 'unique_database' and create the table under it. The 'unique_database' fixture will handle the cleanup so we don't need to drop the table at the beginning. https://github.com/apache/impala/blob/0c8fc997ef7df09b675180a7baa1482852d60b11/tests/conftest.py#L376 http://gerrit.cloudera.org:8080/#/c/20425/1/tests/query_test/test_cast_with_format.py@2197 PS1, Line 2197: assert ['test \ttest', 'tester\ttester'] == self.execute_query( > This test suite matches the behavior we used to have with Impala 3.2 (as se Do you know which JIRA introduces the regression? -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 29 Aug 2023 07:03:51 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/13850/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 25 Aug 2023 23:18:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 1: > Patch Set 1: > > I'll run a perf-AB-test, as this may have performance implications. https://jenkins.impala.io/job/perf-AB-test/473/ -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 25 Aug 2023 23:10:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/20425/1/tests/query_test/test_cast_with_format.py File tests/query_test/test_cast_with_format.py: http://gerrit.cloudera.org:8080/#/c/20425/1/tests/query_test/test_cast_with_format.py@2197 PS1, Line 2197: assert ['test \ttest', 'tester\ttester'] == self.execute_query( This test suite matches the behavior we used to have with Impala 3.2 (as seen is CDH 6.3). -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 25 Aug 2023 23:00:13 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10086: Implicit cast comparing char and varchar
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20425 ) Change subject: IMPALA-10086: Implicit cast comparing char and varchar .. Patch Set 1: I'll run a perf-AB-test, as this may have performance implications. -- To view, visit http://gerrit.cloudera.org:8080/20425 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89d0a391bc8f2152ecd9151c8872a01ba19c436 Gerrit-Change-Number: 20425 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 25 Aug 2023 22:55:53 + Gerrit-HasComments: No
