I have a relatively complex Java object that I would like to use in a dataset
if I say Encoder<MyType> evidence = Encoders.kryo(MyType.class); JavaRDD<MyType> rddMyType= generateRDD(); // some code Dataset<MyType> datasetMyType= sqlCtx.createDataset( rddMyType.rdd(), evidence); I get one column - the whole object The object is a bean with all fields having getters and setters but some of the fields are other complex java objects - It would be fine to serielize the objects in these fields with Kryo or Java serialization but the Bean serializer treats all referenced objects as beans and some lack the required getter and setter fields How can I get my columns with bean serializer even if some of the values in the columns are not bean types
