laid=6087618
From: Gera Shegalov
Sent: Wednesday, May 29, 2024 7:57:56 am
To: Prem Sahoo
Cc: eab...@163.com ; Vibhor Gupta ;
user @spark
Subject: Re: Re: EXT: Dual Write to HDFS and MinIO in faster way
I agree with the previous answers that (if requirements allow it
*Date:* 2024-05-22 00:38
>> *To:* Vibhor Gupta ; user
>>
>> *Subject:* Re: EXT: Dual Write to HDFS and MinIO in faster way
>>
>>
>> On Tue, May 21, 2024 at 6:58 AM Prem Sahoo wrote:
>>
>>> Hello Vibhor,
>>> Thanks for the suggestion .
>
eabour
>
>
> *From:* Prem Sahoo
> *Date:* 2024-05-22 00:38
> *To:* Vibhor Gupta ; user
>
> *Subject:* Re: EXT: Dual Write to HDFS and MinIO in faster way
>
>
> On Tue, May 21, 2024 at 6:58 AM Prem Sahoo wrote:
>
>> Hello Vibhor,
>> Thanks for the sugge
Hi，
I think you should write to HDFS then copy file (parquet or orc) from HDFS
to MinIO.
eabour
From: Prem Sahoo
Date: 2024-05-22 00:38
To: Vibhor Gupta; user
Subject: Re: EXT: Dual Write to HDFS and MinIO in faster way
On Tue, May 21, 2024 at 6:58 AM Prem Sahoo wrote:
Hello Vibhor
gt; This will prevent duplicate transformation.
>
>
>
> You can also try persisting the dataframe using the DISK_ONLY level.
>
>
>
> Regards,
>
> Vibhor
>
> *From: *Prem Sahoo
> *Date: *Tuesday, 21 May 2024 at 8:16 AM
> *To: *Spark dev list
> *Subject:
