gt; to delete this message and destroy any printed copies.
>
>
> -Original Message-
> From: Nicolas Paris
> Sent: Tuesday, July 23, 2019 6:56 PM
> To: user@spark.apache.org
> Subject: Avro large binary read memory problem
>
> Hi
>
> I have those avro file wit
@spark.apache.org
Subject: Avro large binary read memory problem
Hi
I have those avro file with the schema id:Long, content:Binary
the binary are large image with a maximum of 2GB of size.
I d like to get a subset of row "where id in (...)"
Sadly I get memory errors even if the subset is 0 of size
Hi
I have those avro file with the schema id:Long, content:Binary
the binary are large image with a maximum of 2GB of size.
I d like to get a subset of row "where id in (...)"
Sadly I get memory errors even if the subset is 0 of size. It looks like
the reader stores the binary information
