[
https://issues.apache.org/jira/browse/SPARK-21280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076466#comment-16076466
]
Eran Moscovici commented on SPARK-21280:
----------------------------------------
Thanks for your responses.
Unfortunately for our use-case a structured serialization which maintains the
inner structure of the object on disk, is needed (instead of melting the object
into one single byte array).
Encoders.bean does maintain the inner structure but BloomFilter cannot be used
with Encoders.bean since it is not bean compliant.
Encoders.javaSerialization works with BloomFilter but does not maintain the
inner structure on disk.
> org.apache.spark.util.sketch.BloomFilter not bean compliant
> -----------------------------------------------------------
>
> Key: SPARK-21280
> URL: https://issues.apache.org/jira/browse/SPARK-21280
> Project: Spark
> Issue Type: Improvement
> Components: Java API
> Affects Versions: 2.1.1
> Reporter: Eran Moscovici
> Priority: Minor
>
> Trying to work with Dataset<BloomFilter> fails in runtime with the 'not bean
> compliant' exception.
> This means that BloomFilter objects cannot be used as values to be handled
> within a Spark Dataset or saved (for example as a parquet file).
> One would expect an object within the Spark ecosystem
> ('org.apache.spark.util.sketch.BloomFilter') to be able to do that.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
