Hello Apache Spark Support Team,
I am writing Spark on Java now. I use Dataset API and I face with an issue,
that I am doing something like that
public <K, T> Dataset<Tuple2<K, List>> groupByKey(Dataset<Tuple2<K,
T>> consumers, Class<K> kClass) {
consumers.groupBy("_1").agg(collect_list(col("_2"))).printSchema();
return
consumers.groupBy("_1").agg(collect_list(col("_2"))).as(Encoders.tuple(Encoders.bean(kClass),
Encoders.bean(List.class)));
}
And I faced the issue that I can not deserialize collect_list part.
https://spark.apache.org/docs/latest/sql-reference.html#data-types -
mapping ArrayType to java.util.List
Could you please give me any suggestions, wasted too much time trying to
fix it?
Best Regards,
Dmytro
