[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325686#comment-15325686
]
Apache Spark commented on SPARK-15585:
--
User 'maropu' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323947#comment-15323947
]
Takeshi Yamamuro commented on SPARK-15585:
--
okay, I'll push later.
> Don't use null in data
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323946#comment-15323946
]
Reynold Xin commented on SPARK-15585:
-
Great let's update the documentation that way.
> Don't use
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323944#comment-15323944
]
Takeshi Yamamuro commented on SPARK-15585:
--
yea, I manually checked that it works well.
If we
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323940#comment-15323940
]
Reynold Xin commented on SPARK-15585:
-
Looks good. Does empty string actually work?
> Don't use
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323936#comment-15323936
]
Takeshi Yamamuro commented on SPARK-15585:
--
Understood.
Anyway, I think it's okay to update docs
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322992#comment-15322992
]
Reynold Xin commented on SPARK-15585:
-
They suffer from the same problem.
Before your patch,
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322670#comment-15322670
]
Takeshi Yamamuro commented on SPARK-15585:
--
I'm afraid the `sep` option for `csv` overrides the
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320133#comment-15320133
]
Reynold Xin commented on SPARK-15585:
-
It would woudln't it? Because the sep argument for the "csv"
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318132#comment-15318132
]
Takeshi Yamamuro commented on SPARK-15585:
--
btw, the behavior of `df.option("sep",
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318113#comment-15318113
]
Takeshi Yamamuro commented on SPARK-15585:
--
yea, it's okay to me just to add docs about
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316288#comment-15316288
]
Reynold Xin commented on SPARK-15585:
-
[~maropu] I think the best way is to advise users to pass
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305185#comment-15305185
]
Apache Spark commented on SPARK-15585:
--
User 'maropu' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305178#comment-15305178
]
Takeshi Yamamuro commented on SPARK-15585:
--
okay, I'll push soon.
> Don't use null in data
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305166#comment-15305166
]
Reynold Xin commented on SPARK-15585:
-
Feel free to create a pr with python changes and then we can
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305152#comment-15305152
]
Takeshi Yamamuro commented on SPARK-15585:
--
okay
> Don't use null in data source options to
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15304267#comment-15304267
]
Shivaram Venkataraman commented on SPARK-15585:
---
[~maropu] Can you also add test cases in
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303787#comment-15303787
]
Takeshi Yamamuro commented on SPARK-15585:
--
okay, I got your point.
I'll make a pr based on
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303722#comment-15303722
]
Reynold Xin commented on SPARK-15585:
-
I was suggesting setting the value to None directly rather
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303721#comment-15303721
]
Takeshi Yamamuro commented on SPARK-15585:
--
If quote is "NONE" in readwriter.py, I think no
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303692#comment-15303692
]
Reynold Xin commented on SPARK-15585:
-
"None" becomes null, doesn't it?
> Don't use null in data
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303690#comment-15303690
]
Takeshi Yamamuro commented on SPARK-15585:
--
We cannot pass `null` at `quote` for univocity
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303523#comment-15303523
]
Shivaram Venkataraman commented on SPARK-15585:
---
I am not sure i completely understand the
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303160#comment-15303160
]
Takeshi Yamamuro commented on SPARK-15585:
--
yea, If no problem, I'll take this.
> Don't use
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303142#comment-15303142
]
Reynold Xin commented on SPARK-15585:
-
cc [~maropu] interested in doing this?
> Don't use null in
[
https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303144#comment-15303144
]
Reynold Xin commented on SPARK-15585:
-
cc [~shivaram] / [~sunrui] / [~felixcheung] would this impact
26 matches
Mail list logo
