Finally I was able to solve this issue by setting this conf.
"spark.driver.extraJavaOptions=-Dorg.xerial.snappy.tempdir=/my_user/temp_
folder"
Thanks all!
On Sat, 8 Jul 2023 at 3:45 AM, Brian Huynh wrote:
> Hi Khalid,
>
> Elango mentioned the file is working fine in our another environment
Hi Khalid,Elango mentioned the file is working fine in our another environment with the same driver and executor memoryBrianOn Jul 7, 2023, at 10:18 AM, Khalid Mammadov wrote:Perhaps that parquet file is corrupted or got that is in that folder?To check, try to read that file with pandas or other
Perhaps that parquet file is corrupted or got that is in that folder?
To check, try to read that file with pandas or other tools to see if you
can read without Spark.
On Wed, 5 Jul 2023, 07:25 elango vaidyanathan, wrote:
>
> Hi team,
>
> Any updates on this below issue
>
> On Mon, 3 Jul 2023 at
Hi team,
Any updates on this below issue
On Mon, 3 Jul 2023 at 6:18 PM, elango vaidyanathan
wrote:
>
>
> Hi all,
>
> I am reading a parquet file like this and it gives
> java.lang.IllegalArgumentException.
> However i can work with other parquet files (such as nyc taxi parquet
> files)
Hi all,
I am reading a parquet file like this and it gives
java.lang.IllegalArgumentException.
However i can work with other parquet files (such as nyc taxi parquet
files) without any issue. I have copied the full error log as well. Can you
please check once and let me know how to fix this?
