Github user HyukjinKwon commented on the pull request:
https://github.com/apache/spark/pull/12904#issuecomment-218976260
@sureshthalamati oh, the comments are not related with this PR but moving
the discussion to here was suggested. So, i did. Sorry for that if it was
confusing.
--
Github user sureshthalamati commented on the pull request:
https://github.com/apache/spark/pull/12904#issuecomment-218975151
@HyukjinKwon does your previous comment for meant for some other PR ?
This PR does not have any change you mentioned above. Am I missing some thing ?
Github user HyukjinKwon commented on the pull request:
https://github.com/apache/spark/pull/12904#issuecomment-218680978
After the discussion https://github.com/apache/spark/pull/13041, I think
the field below:
```scala
StructField("", StringType, nullable =true)
```
Github user HyukjinKwon commented on the pull request:
https://github.com/apache/spark/pull/12904#issuecomment-218635820
After the discussion here, https://github.com/apache/spark/pull/12904, then
I think
```scala
StructField("", StringType, nullable = true)
```
Github user HyukjinKwon commented on the pull request:
https://github.com/apache/spark/pull/12904#issuecomment-218633000
@andrewor14 Here it seems it is concluded that `""` is a string and a empty
string is `null`. Because `""` is a legitimate string, this can be a field name
whereas
Github user HyukjinKwon commented on the pull request:
https://github.com/apache/spark/pull/12904#issuecomment-217765823
+1 for @sureshthalamati
#12921 handles the inconsistent behaviour and this is why I think we should
hold off this until that PR is merged.
---
If your projec
Github user sureshthalamati commented on the pull request:
https://github.com/apache/spark/pull/12904#issuecomment-217765505
I am not sure what was the history behind returning empty String for null
value. In my opinion it should be null be default. current behavior is also
inconsis
Github user sureshthalamati commented on a diff in the pull request:
https://github.com/apache/spark/pull/12904#discussion_r62444274
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
---
@@ -555,4 +558,37 @@ class CSVSuite extends Quer
Github user HyukjinKwon commented on the pull request:
https://github.com/apache/spark/pull/12904#issuecomment-217605838
In case of writing, I think
```
Row("", "null", null)
```
should produce the CSV as below:
1. With the option, `nullValue` set to `"null
Github user HyukjinKwon commented on the pull request:
https://github.com/apache/spark/pull/12904#issuecomment-217605160
Here is what I think CSV datasource should handle `""`, empty string and
`nullValue`.
With the option, `nullValue` set to `"null"`, I think
```csv
Github user HyukjinKwon commented on the pull request:
https://github.com/apache/spark/pull/12904#issuecomment-217604986
@rxin @sureshthalamati Do you mind holding off this change until #12921 is
merged? That PR also handles `nullValue`. Apparently, I guess `nullValue` could
affect th
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/12904#discussion_r62411095
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
---
@@ -555,4 +558,37 @@ class CSVSuite extends QueryTes
Github user sureshthalamati commented on the pull request:
https://github.com/apache/spark/pull/12904#issuecomment-217555333
Thank you for the feedback , Reynold , HyukjinKwon. Update the PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear
13 matches
Mail list logo
