[GitHub] spark pull request #21871: [SPARK-24916][SQL] Fix type coercion for IN expre...

2018-07-25 Thread wangyum 
Github user wangyum closed the pull request at:

https://github.com/apache/spark/pull/21871


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21871: [SPARK-24916][SQL] Fix type coercion for IN expre...

2018-07-25 Thread HyukjinKwon 
Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/21871#discussion_r205068186
  
--- Diff: 
sql/core/src/test/resources/sql-tests/inputs/typeCoercion/native/inConversion.sql
 ---
@@ -328,3 +328,159 @@ SELECT cast('2017-12-12 09:30:00' as date) in 
(cast('2017-12-12 09:30:00' as dat
 SELECT cast('2017-12-12 09:30:00' as date) in (cast('2017-12-12 09:30:00' 
as date), cast(1 as boolean)) FROM t;
 SELECT cast('2017-12-12 09:30:00' as date) in (cast('2017-12-12 09:30:00' 
as date), cast('2017-12-11 09:30:00.0' as timestamp)) FROM t;
 SELECT cast('2017-12-12 09:30:00' as date) in (cast('2017-12-12 09:30:00' 
as date), cast('2017-12-11 09:30:00' as date)) FROM t;
+
+SELECT * FROM t WHERE (cast(1 as tinyint)) IN (SELECT cast(1 as tinyint) 
FROM t);
--- End diff --

Do we really need to test all the combinations? We need most of such logics 
should be tested in `findWiderTypeWithoutStringPromotionForTwo` and we could 
have just few end to end tests.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21871: [SPARK-24916][SQL] Fix type coercion for IN expre...

2018-07-25 Thread wangyum 
GitHub user wangyum opened a pull request:

https://github.com/apache/spark/pull/21871

[SPARK-24916][SQL] Fix type coercion for IN expression with subquery

## What changes were proposed in this pull request?

The below SQL will throw `AnalysisException`. but it can success on Spark 
2.1.x. This pr fix this issue.
```sql
CREATE TEMPORARY VIEW t4 AS SELECT * FROM VALUES
  (CAST(1 AS DOUBLE), CAST(2 AS STRING), CAST(3 AS STRING))
AS t1(t4a, t4b, t4c);

CREATE TEMPORARY VIEW t5 AS SELECT * FROM VALUES
  (CAST(1 AS DECIMAL(18, 0)), CAST(2 AS STRING), CAST(3 AS BIGINT))
AS t1(t5a, t5b, t5c);

SELECT * FROM t4
WHERE
(t4a, t4b, t4c) IN (SELECT t5a, t5b, t5c FROM t5);
```

## How was this patch tested?

unit tests


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/wangyum/spark SPARK-24916

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/21871.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #21871


commit d60185b159ffc1a0d74a8c5bfba0c11ceac4241b
Author: Yuming Wang 
Date:   2018-07-25T07:33:45Z

default: findCommonTypeForBinaryComparison(l.dataType, r.dataType, 
conf).orElse(findTightestCommonType(l.dataType, r.dataType))

commit 8a118b5bdf63a7f4b0f0033c0783aa220c9c1eb1
Author: Yuming Wang 
Date:   2018-07-25T07:37:03Z

findCommonTypeForBinaryComparison(l.dataType, r.dataType, 
conf).orElse(findWiderTypeForTwo(l.dataType, r.dataType))

commit c306810f0a0e701e6b46434db75bbd9813c7337c
Author: Yuming Wang 
Date:   2018-07-25T07:39:40Z

findWiderTypeForTwo(l.dataType, r.dataType)

commit daa120e15153c77c17a7966df7b727fcea4bb02b
Author: Yuming Wang 
Date:   2018-07-25T07:42:46Z

findCommonTypeForBinaryComparison(l.dataType, r.dataType, 
conf).orElse(findWiderTypeWithoutStringPromotionForTwo(l.dataType, r.dataType))

commit c84ba4d9823a50953f560e110638a9d4e094b17a
Author: Yuming Wang 
Date:   2018-07-25T07:45:43Z

findWiderTypeWithoutStringPromotionForTwo(l.dataType, r.dataType)

commit bc41b99b7548a22db1ed278fda1c741fd08b78ef
Author: Yuming Wang 
Date:   2018-07-25T08:03:26Z

findCommonTypeForBinaryComparison(l.dataType, r.dataType, 
conf).orElse(findTightestCommonType(l.dataType, 
r.dataType)).orElse(findWiderTypeForDecimal(l.dataType, r.dataType))

commit 8ef142f78c22b980fe60d836c56d7d18d221a958
Author: Yuming Wang 
Date:   2018-07-25T09:27:51Z

Fix type coercion for IN expression with subquery




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org