[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755221#comment-15755221
]
Ryan Blue commented on SPARK-16032:
---
+1
> Audit semantics of various insertion operations related to
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755113#comment-15755113
]
Dongjoon Hyun commented on SPARK-16032:
---
Hi, All.
This issue is targeted on 2.1.0 and all subtasks
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353790#comment-15353790
]
Yin Huai commented on SPARK-16032:
--
Thank you [~rdblue] for the detailed reply. Those are super helpful.
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347172#comment-15347172
]
Ryan Blue commented on SPARK-16032:
---
bq. I am not sure apply by-name resolution just to partition
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345443#comment-15345443
]
Yin Huai commented on SPARK-16032:
--
Let's look at two examples (tests were done using the 2.0 branch
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15343243#comment-15343243
]
Ryan Blue commented on SPARK-16032:
---
[~cloud_fan], while I think by-name insertion is important in the
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15343036#comment-15343036
]
Wenchen Fan commented on SPARK-16032:
-
[~rdblue] I think the biggest problem is we don't have by-name
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15343029#comment-15343029
]
Wenchen Fan commented on SPARK-16032:
-
I think it's nonsense to use `partitionBy` with `insertInto`,
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15342963#comment-15342963
]
Ryan Blue commented on SPARK-16032:
---
I'm referring to disabling the use of {{partitionBy}} with
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15342856#comment-15342856
]
Yin Huai commented on SPARK-16032:
--
Regarding {{disabling Hive features}}, can you be more specific?
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15342224#comment-15342224
]
Ryan Blue commented on SPARK-16032:
---
bq. the most important issue we would like to address here is
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15342216#comment-15342216
]
Ryan Blue commented on SPARK-16032:
---
bq. I don't think the package matters, the pre-insert is still an
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15342135#comment-15342135
]
Ryan Blue commented on SPARK-16032:
---
I agree with the push to unify the Hive and DataSource
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341754#comment-15341754
]
Cheng Lian commented on SPARK-16032:
[~rdblue], I also migrated some test cases from your PR so that
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341658#comment-15341658
]
Cheng Lian commented on SPARK-16032:
Hey [~rdblue], [~yhuai] and [~cloud_fan] had already covered a
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341021#comment-15341021
]
Yin Huai commented on SPARK-16032:
--
[~rdblue] Thank you for your feedback. Those are great input for the
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340966#comment-15340966
]
Wenchen Fan commented on SPARK-16032:
-
> The pre-insert checks currently live in the
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340929#comment-15340929
]
Ryan Blue commented on SPARK-16032:
---
Overall, I'm *-1* on these changes going into Spark 2.0.
Looking
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340797#comment-15340797
]
Ryan Blue commented on SPARK-16032:
---
I'm going to put review comments here because the PRs are already
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340409#comment-15340409
]
Ryan Blue commented on SPARK-16032:
---
The changes here don't look like the rule that was added in
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340294#comment-15340294
]
Ryan Blue commented on SPARK-16032:
---
Sounds good, I'm glad to see that at least the cast changes were
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340290#comment-15340290
]
Yin Huai commented on SPARK-16032:
--
Wenchen's commit (https://github.com/apache/spark/pull/13754)
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340272#comment-15340272
]
Yin Huai commented on SPARK-16032:
--
Yea. Unfortunately, they are different right now. For 2.1, we need
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340185#comment-15340185
]
Ryan Blue commented on SPARK-16032:
---
Why does the DDL for the data source table differ from the Hive
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340138#comment-15340138
]
Ryan Blue commented on SPARK-16032:
---
[~yhuai], thanks for pinging me. I'll take a look at this today.
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340087#comment-15340087
]
Yin Huai commented on SPARK-16032:
--
[~rdblue] Since you have worked on this area, we'd like to let you
[
https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338066#comment-15338066
]
Yin Huai commented on SPARK-16032:
--
We will attach the report to here.
> Audit semantics of various
27 matches
Mail list logo
