Github user nchammas commented on the pull request:
https://github.com/apache/spark/pull/10474#issuecomment-218892078
Is this PR really meant to modify 1,320 files?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user yhuai commented on the pull request:
https://github.com/apache/spark/pull/10474#issuecomment-170268671
Can you add tests and also make changes to the write path (parquet/orc)?
Otherwise, when you write a json dataset with null type to parquet, it will
fail.
---
If your p
Github user naveenminchu commented on a diff in the pull request:
https://github.com/apache/spark/pull/10474#discussion_r49264010
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/InferSchema.scala
---
@@ -169,7 +169,7 @@ private[json] object InferS
Github user yhuai commented on the pull request:
https://github.com/apache/spark/pull/10474#issuecomment-169396179
@naveenminchu I think the next step is to remove those null type fields
before we write data to parquet/orc. @liancheng Parquet does not have a null
type, right?
---
If
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/10474#discussion_r48982980
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/InferSchema.scala
---
@@ -169,7 +169,7 @@ private[json] object InferSchema {
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/10474#issuecomment-169252314
cc @yhuai
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enable
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10474#issuecomment-167198456
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your p
GitHub user naveenminchu opened a pull request:
https://github.com/apache/spark/pull/10474
[SPARK-12436][SQL] Now InferSchema should return NullType and
ArrayType(NullType)
a.) Currently I have fix for InferSchema to return NullType and
ArrayType(NullType)
b.) @rxin Can you co
