[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
Thanks @cloud-fan @gatorsmile @dilipbiswal 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[cloud-fan]

Github user cloud-fan commented on the issue:

https://github.com/apache/spark/pull/18968
  
LGTM, merging to master!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81071/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #81071 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81071/testReport)**
 for PR 18968 at commit 
[`9364d6e`](https://github.com/apache/spark/commit/9364d6e04b1f99d6b70e851b4405d790a26f58be).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[dilipbiswal]

Github user dilipbiswal commented on the issue:

https://github.com/apache/spark/pull/18968
  
Thanks Simon. Changes look good to me. cc @cloud-fan @gatorsmile for any 
additional comments. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #81071 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81071/testReport)**
 for PR 18968 at commit 
[`9364d6e`](https://github.com/apache/spark/commit/9364d6e04b1f99d6b70e851b4405d790a26f58be).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
We could. resolveSubquery will do at least one time analysis to resolve the
subquery. It might be a waste for already resolved subquery.

Let me try if I can remove the case with little changes.


On Aug 24, 2017 2:10 PM, "Dilip Biswal"  wrote:

*@dilipbiswal* commented on this pull request.
--

In sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/an
alysis/Analyzer.scala
:

> @@ -1286,8 +1286,16 @@ class Analyzer(
   resolveSubQuery(s, plans)(ScalarSubquery(_, _, exprId))
 case e @ Exists(sub, _, exprId) if !sub.resolved =>
   resolveSubQuery(e, plans)(Exists(_, _, exprId))
-case In(value, Seq(l @ ListQuery(sub, _, exprId))) if
value.resolved && !sub.resolved =>
-  val expr = resolveSubQuery(l, plans)(ListQuery(_, _, exprId))
+case In(value, Seq(l @ ListQuery(sub, _, exprId, _))) if
value.resolved && !sub.resolved =>

@viirya  If we modified to

case In(value, Seq(l @ ListQuery(sub, _, exprId, _))) if
value.resolved && !l.resolved

would we still require the following case statement ? The following case
looks a little
strange as we are in the resolveSubqueries routine and check for
sub.resolved == true.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
,
or mute
the thread


.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81028/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #81028 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81028/testReport)**
 for PR 18968 at commit 
[`dae01f1`](https://github.com/apache/spark/commit/dae01f1aff2603a7e8d8bbefdd4db4db470de6f3).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81027/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #81027 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81027/testReport)**
 for PR 18968 at commit 
[`66a193d`](https://github.com/apache/spark/commit/66a193dd9ef939c39279554c0ed45748fc72962b).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #81028 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81028/testReport)**
 for PR 18968 at commit 
[`dae01f1`](https://github.com/apache/spark/commit/dae01f1aff2603a7e8d8bbefdd4db4db470de6f3).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #81027 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81027/testReport)**
 for PR 18968 at commit 
[`66a193d`](https://github.com/apache/spark/commit/66a193dd9ef939c39279554c0ed45748fc72962b).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[cloud-fan]

Github user cloud-fan commented on the issue:

https://github.com/apache/spark/pull/18968
  
SGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
Maybe we can still have a case for ListQuery, but it is simpler and mainly 
for better message?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
@dilipbiswal @gatorsmile Regarding with the error message, I do think so.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80989/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80989 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80989/testReport)**
 for PR 18968 at commit 
[`498bd3b`](https://github.com/apache/spark/commit/498bd3b5429d22867a57c0d7b34fc0a20214aa87).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80988/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80988 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80988/testReport)**
 for PR 18968 at commit 
[`a7f6816`](https://github.com/apache/spark/commit/a7f681698ce6cbea6e0289747b893d2b3cf60415).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[dilipbiswal]

Github user dilipbiswal commented on the issue:

https://github.com/apache/spark/pull/18968
  
Thanks @viirya @cloud-fan. This looks much better. Can we not preserve the 
user facing error we raise today? I think the error we raise today is better 
for the user ? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80989 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80989/testReport)**
 for PR 18968 at commit 
[`498bd3b`](https://github.com/apache/spark/commit/498bd3b5429d22867a57c0d7b34fc0a20214aa87).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80988 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80988/testReport)**
 for PR 18968 at commit 
[`a7f6816`](https://github.com/apache/spark/commit/a7f681698ce6cbea6e0289747b893d2b3cf60415).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
@cloud-fan Based on my understanding, I revise this change. May you look if 
it is what you think? Thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
@cloud-fan OK. I think I got your point. Your proposal is to have 
`ListQuery.childOutputs` which keeps the original output of subquery. Even we 
extend the project list of subquery, `childOutputs` is unchanged.

If that is correct, my only concern is if we extend the project list by 
adding attributes which are not in condition, we aren't able to find it out.

This is basically a safe guard for the possibility we mess the query plan 
in optimization, if you think it is not necessary, I think your proposal is a 
good choice.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
@cloud-fan Are they the same issue? I think you propose to simplify the 
data type checking? This PR goes to avoid reporting unresolved plans.

In this query:

SELECT t1.a FROM t1
WHERE
t1.a IN (SELECT t2.c
FROM t2
WHERE t1.b < t2.d);

The left side of `In` is `t1.a`. After pulling correlated predicates, the 
right side is `(t2.c, t2.d)`. They don't match but we shouldn't fail the check 
and report it unresolved plan.

Am I thinking it incorrectly? Or I misunderstand you propose?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[cloud-fan]

Github user cloud-fan commented on the issue:

https://github.com/apache/spark/pull/18968
  
I think the refactor I proposed is a smaller patch than this PR...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
Oh, yes, I thought you said we don't need to check it. Right we can do it. 
I think we better have another PR to do it. What do you think?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[cloud-fan]

Github user cloud-fan commented on the issue:

https://github.com/apache/spark/pull/18968
  
doesn't the default type check logic work?
```
 val mismatchOpt = list.find(l => l.dataType != value.dataType)
 if (mismatchOpt.isDefined) {
   TypeCheckResult.TypeCheckFailure(s"Arguments must be same type 
but were: " +
 s"${value.dataType} != ${mismatchOpt.get.dataType}")
 } else {
   TypeUtils.checkForOrderingExpr(value.dataType, s"function 
$prettyName")
 }
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
Even we have correct data type, don't we still need to verify it matches 
the data type of `value` expressions in `In.checkInputDataTypes`? There is no 
guarantee that they will match, right?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[cloud-fan]

Github user cloud-fan commented on the issue:

https://github.com/apache/spark/pull/18968
  
After we correctly define the data type of `ListQuery`, can we remove the 
special handling of `ListQuery` in `In.checkInputDataTypes`?

we can add `ListQuery.childOutputs: Seq[Attribute]`, so that even we extend 
the project list of `ListQuery.plan`, we still keep the corrected data type:
```
def dataType = if (childOutputs.length > 1) childOutputs.toStructType else 
childOutputs.head.dataType
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
@gatorsmile @cloud-fan Any more comments on this change? Thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[dilipbiswal]

Github user dilipbiswal commented on the issue:

https://github.com/apache/spark/pull/18968
  
@viirya Thank you !!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
@dilipbiswal That's right. We don't really use the data type of 
`ListQuery`, because it is not an real expression actually. But maybe have an 
accurate one is good too.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[dilipbiswal]

Github user dilipbiswal commented on the issue:

https://github.com/apache/spark/pull/18968
  
@gatorsmile @cloud-fan @viirya I was thinking about cloud-fan's question. 
Actually we may not be representing the data type of the listquery expression 
correctly.  Would this represent this more accurately ? 
```sql
override def dataType: DataType =
if (plan.output.length > 1) plan.schema else 
plan.schema.fields.head.dataType
```
Actually we don't use the data type to know the return type for listquery 
instead we use the plan's output.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[gatorsmile]

Github user gatorsmile commented on the issue:

https://github.com/apache/spark/pull/18968
  
We do allow multiple columns in `IN subquery`. For example, 

```SQL
SELECT t1a,
   t1b,
   t1d
FROM   t1
WHERE  ( t1b, t1d ) IN (SELECT t2b,
   t2d
FROM   t2
WHERE  t2i IN (SELECT t3i
   FROM   t3
   WHERE  t2b > t3b))
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80930/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80930 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80930/testReport)**
 for PR 18968 at commit 
[`a96fb8d`](https://github.com/apache/spark/commit/a96fb8d056bde4ed8a5f02a2767d09a3c661051d).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
I'm not very sure how we use `ListQuery.dataType`, or do we actually use it 
since it might not be very meaningful to have a datatype for a IN subquery.

In order to make sure type matching, we do type checking in `In` between 
the output of subquery wrapped in `ListQuery` and the `value` expressions. We 
won't use `ListQuery` with other expression, I think.

Maybe it has `dataType` just because it needs to have one, as it inherits 
`Expression`?



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[cloud-fan]

Github user cloud-fan commented on the issue:

https://github.com/apache/spark/pull/18968
  
ah, if that query is valid, seems `ListQuery.dataType` is wrong?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80930 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80930/testReport)**
 for PR 18968 at commit 
[`a96fb8d`](https://github.com/apache/spark/commit/a96fb8d056bde4ed8a5f02a2767d09a3c661051d).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
Btw, what the purpose to have `ListQuery.childOuput: Attribute`?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
We can write IN subquery like:

SELECT * FROM t1
WHERE struct(t1.a, t1.b) IN (SELECT t2.d, t2.e
 FROM t2
 WHERE t1.c < t2.f)")

It is valid because the length of value expressions (t1.a, t1.b) matches 
the length of subquery output (t2.d, t2.e).

So seems `ListQuery.plan` can produce more than one column?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[cloud-fan]

Github user cloud-fan commented on the issue:

https://github.com/apache/spark/pull/18968
  
One thing I wanna discuss, is it better to have a `ListQuery.childOuput: 
Attribute`? Looking at `ListQuery.dataType`, seems we assume `ListQuery.plan` 
only produce one column, but we didn't enforce it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80925/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80925 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80925/testReport)**
 for PR 18968 at commit 
[`0e15cde`](https://github.com/apache/spark/commit/0e15cdeb5897b767de91bcfb38e7d15034bbd994).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80925 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80925/testReport)**
 for PR 18968 at commit 
[`0e15cde`](https://github.com/apache/spark/commit/0e15cdeb5897b767de91bcfb38e7d15034bbd994).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
retest this please.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80920/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80920 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80920/testReport)**
 for PR 18968 at commit 
[`0e15cde`](https://github.com/apache/spark/commit/0e15cdeb5897b767de91bcfb38e7d15034bbd994).
 * This patch **fails due to an unknown error code, -9**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80915/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80915 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80915/testReport)**
 for PR 18968 at commit 
[`dd48a9d`](https://github.com/apache/spark/commit/dd48a9d9476c7ba775df2a1764b6311d778891b9).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80920 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80920/testReport)**
 for PR 18968 at commit 
[`0e15cde`](https://github.com/apache/spark/commit/0e15cdeb5897b767de91bcfb38e7d15034bbd994).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80915 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80915/testReport)**
 for PR 18968 at commit 
[`dd48a9d`](https://github.com/apache/spark/commit/dd48a9d9476c7ba775df2a1764b6311d778891b9).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80895/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80895 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80895/testReport)**
 for PR 18968 at commit 
[`52f52bb`](https://github.com/apache/spark/commit/52f52bb2609bc019d3aa4d609118810eccb9caa2).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80895 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80895/testReport)**
 for PR 18968 at commit 
[`52f52bb`](https://github.com/apache/spark/commit/52f52bb2609bc019d3aa4d609118810eccb9caa2).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80893/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80893 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80893/testReport)**
 for PR 18968 at commit 
[`4e9cea8`](https://github.com/apache/spark/commit/4e9cea8e170ec6eadfa43ffacc08f58e0a506798).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80892/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80892 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80892/testReport)**
 for PR 18968 at commit 
[`0903838`](https://github.com/apache/spark/commit/0903838b738f5575826b16be039c73b47d3c84ff).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80891/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80891 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80891/testReport)**
 for PR 18968 at commit 
[`99d9570`](https://github.com/apache/spark/commit/99d9570a04b583cd1dc5180d3760c6ffcb1e8513).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80889/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80889 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80889/testReport)**
 for PR 18968 at commit 
[`998c70b`](https://github.com/apache/spark/commit/998c70bddd605fbd44a89db5f089a03014b37066).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80893 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80893/testReport)**
 for PR 18968 at commit 
[`4e9cea8`](https://github.com/apache/spark/commit/4e9cea8e170ec6eadfa43ffacc08f58e0a506798).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80892 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80892/testReport)**
 for PR 18968 at commit 
[`0903838`](https://github.com/apache/spark/commit/0903838b738f5575826b16be039c73b47d3c84ff).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
I'd moved the original check to `CheckAnalysis`. The valid IN subquery 
should have equal size value expressions and subquery output. If not, we should 
throw user-facing error.

To prevent possible issue brought by optimization rules, the additional 
check is put in `resolved`. This is not user-facing error. I think we can rely 
the newly proposed structural integrity check to detect this.






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80891 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80891/testReport)**
 for PR 18968 at commit 
[`99d9570`](https://github.com/apache/spark/commit/99d9570a04b583cd1dc5180d3760c6ffcb1e8513).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80889 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80889/testReport)**
 for PR 18968 at commit 
[`998c70b`](https://github.com/apache/spark/commit/998c70bddd605fbd44a89db5f089a03014b37066).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[dilipbiswal]

Github user dilipbiswal commented on the issue:

https://github.com/apache/spark/pull/18968
  
@viirya ok.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
That's right, if we combine the two rules, we won't produce the unresolved 
`In` predicate. I just don't know if we have the plan in near term to combine 
them together. Sounds like a non-trivial change.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[dilipbiswal]

Github user dilipbiswal commented on the issue:

https://github.com/apache/spark/pull/18968
  
@gatorsmile @viirya There was pr from Natt 
[pr](https://github.com/apache/spark/pull/17520). Is it possible to get some 
feedback on the idea ? If we do this, the next step was to combine the pullup 
and rewrite to one single rule. Actually i had this change made in my local 
branch a while back. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[dilipbiswal]

Github user dilipbiswal commented on the issue:

https://github.com/apache/spark/pull/18968
  
@viirya I agree that we should report violations. The only question i had 
is whether we should tie this particular check to the expression being resolved 
or not. In the old version of the code, we used to do this particular check in 
the analyzer. However after doing the check, we used to change the expression 
to PredicateSubquery after pulling up the correlated predicates.  I am fine 
with whatever you decide. If we keep the check, we should make it such that it 
reads a little better  than how it reads now :-)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
@dilipbiswal But we still need to detect such violation of check and report 
the error if optimization rules cause the side effect that makes the plan 
unresolved.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[dilipbiswal]

Github user dilipbiswal commented on the issue:

https://github.com/apache/spark/pull/18968
  
@viirya Isn't checkAnalysis supposed to catch such semantic errors ? In my 
thinking, this particular error to make sure the left hand side number of args 
matching the right hand side is to catch **user errors in the input query**. 
After that is done either during analyzer or post analysis phase such as 
checkAnalysis , we shouldn't be doing this particular check. My reason is that 
, if optimizer causes a side effect such that it makes the original check 
invalid, we shouldn't be returning the particular error that we return today as 
that wouldn't mean much to the user as thats not the query he typed in , 
correct ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
Re-thinking it, I agree that this kind of check should put in `resolved`. 
However, I doubt whether we should put in `checkAnalysis`. By doing this, we 
split the resolving check of `In` to two places, one in `checkAnalysis`, one in 
`In.resolved`.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
I agree that the original check should be in `checkAnalysis` instead of 
`checkInputDataTypes`.

The additional check added by this change can be put in `resolved`. Sounds 
good to me.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[gatorsmile]

Github user gatorsmile commented on the issue:

https://github.com/apache/spark/pull/18968
  
[Moving such checks from `In.checkInputDataTypes ` to 
`checkAnalysis`](https://github.com/dilipbiswal/spark/commit/185159e82572fd8c41f482bae54f791d2bf1b56a)
 looks cleaner to me. What we are doing in `In.checkInputDataTypes` does not 
belong to `checkInputDataTypes`.

If we want to capture the potential analysis errors introduced by the 
optimizer rule `PullupCorrelatedPredicates `, we should override `resolved` 
instead of doing all these checks in `checkInputDataTypes`. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
@dilipbiswal Thanks for comment.

This issue is happened at optimization phase, so I think it is supposed 
that we already pass `checkAnalysis`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[dilipbiswal]

Github user dilipbiswal commented on the issue:

https://github.com/apache/spark/pull/18968
  
@viirya Hi Simon, many thanks for finding this. Instead of adding the 
compensation code in the resolve logic for in-subquery expression can we 
consider to move the semantic checking of comparing the count of arguments in 
either side of in-subquery expression to checkAnalysis instead ? We do several 
other checks in checkAnalysis for subquery expression. I just feel this may be 
a little cleaner ?

For your reference, i quickly tried it 
[here](https://github.com/dilipbiswal/spark/tree/pr-18968-viirya)



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80812/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18968
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18968
  
**[Test build #80812 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80812/testReport)**
 for PR 18968 at commit 
[`476b4ab`](https://github.com/apache/spark/commit/476b4abe43495d626da2d23bdbeb6a5e37f8e981).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18968: [SPARK-21759][SQL] In.checkInputDataTypes should not wro...

[viirya]

Github user viirya commented on the issue:

https://github.com/apache/spark/pull/18968
  
@cloud-fan Thanks for the comments. I've updated the PR description and 
added a SQL statement.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org