Github user fangshil commented on the issue:
https://github.com/apache/spark/pull/21310
To summarize our discussion in this pr:
Spark-avro is now merged into Spark as a built-in data source. Upstream
community is not merging the AvroEncoder to support Avro types in Dataset,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21310
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21310
I think this PR is blocked by adding UDT officially(it's currently
internal). Maybe we can start a thread about UDT in the dev list.
---
Github user fangshil commented on the issue:
https://github.com/apache/spark/pull/21310
@HyukjinKwon thanks for the update. What do you mean by "Avro was now in
Spark"? The PR I mentioned is
https://github.com/databricks/spark-avro/pull/215/files. I have been
maintaining this PR
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21310
@fangshil also can you point me out the PR not merged into spark-avro
please so that I can check when I have some time.
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21310
@fangshil, Avro was now in Spark. How does it relate to this PR? Should we
go forward?
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21310
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user fangshil commented on the issue:
https://github.com/apache/spark/pull/21310
@viirya thanks for the feedback. We internally customized the AvroEncoder
based on the open source PR, since it never gets merged into spark-avro. we
propose this feature since it should apply to
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21310
I'm not sure if I look into correct project. But seems spark-avro project
doesn't have `AvroEncoder` yet. The PR going to add `AvroEncoder` looks like
this one
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21310
I will take a look later today.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user fangshil commented on the issue:
https://github.com/apache/spark/pull/21310
@viirya @cloud-fan before I add test, could you guys take a look and
advise if the approach taken in this patch is acceptable?
---
Github user fangshil commented on the issue:
https://github.com/apache/spark/pull/21310
I will investigate how can we add test for this. thoughts are welcomed
---
-
To unsubscribe, e-mail:
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21310
You need to add tests first. Could you?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21310
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21310
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
15 matches
Mail list logo
