I did a try. the schema after flatMap is the same, which is expected.
What’s your Row encoder?
> On Jul 23, 2016, at 20:36, Julien Nauroy <julien.nau...@u-psud.fr> wrote:
>
> Hi,
>
> I'm trying to call flatMap on a Dataframe with Spark 2.0 (rc5).
> The code is the following:
> var data = spark.read.parquet(fileName).flatMap(x => List(x))
>
> Of course it's an overly simplified example, but the result is the same.
> The dataframe schema goes from this:
> root
> |-- field1: double (nullable = true)
> |-- field2: integer (nullable = true)
> (etc)
>
> to this:
> root
> |-- value: binary (nullable = true)
>
> Plus I have to provide an encoder for Row.
> I expect to get the same schema after calling flatMap.
> Any idea what I could be doing wrong?
>
>
> Best regards,
> Julien
>
>
