I'd like to create a Dataset using some classes from Geotools to do some geospatial analysis. In particular, I'm trying to use Spark to distribute the work based on ID and label fields that I extract from the polygon data.
My simplified case class looks like this: implicit val geometryEncoder: Encoder[Geometry] = Encoders.kryo[Geometry] case class IndexedGeometry(label: String, tract: Geometry) When I try to create a dataset using this case class, it give me this error message: Exception in thread "main" java.lang.UnsupportedOperationException: No Encoder found for com.vividsolutions.jts.geom.Geometry - field (class: "com.vividsolutions.jts.geom.Geometry", name: "tract") - root class: "org.me.HelloWorld.IndexedGeometry" If I add another encoder for my case class...: implicit val indexedGeometryEncoder: Encoder[IndexedGeometry] = Encoders.kryo[IndexedGeometry] ...it works, but now the entire dataset has a single field, "value", and it's a binary blob. Is there a way to do what I'm trying to do? I believe this PR is related, but it's been idle since December: https://github.com/apache/spark/pull/15918 -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Case-class-with-POJO-encoder-issues-tp28381.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
