If you want to keep the dataset, maybe you can try to add a constructor to the
case class (through the companion objcet) that receives only the age.Bentzi
Sent from Yahoo Mail on Android
On Sat, May 9, 2020 at 17:50, Jorge Machado wrote:
Ok, I found a way to solve it.
Just pass the
Ok, I found a way to solve it.
Just pass the schema like this:
val schema = Encoders.product[Person].schema
spark.read.schema(schema).parquet(“input”)….
> On 9. May 2020, at 13:28, Jorge Machado wrote:
>
> Hello everyone,
>
> One question to the community.
>
> Imagine I have this
>
>
Hello everyone,
One question to the community.
Imagine I have this
Case class Person(age: int)
spark.read.parquet(“inputPath”).as[Person]
After a few weeks of coding I change the class to:
Case class Person(age: int, name: Option[String] = None)
Then when I run
