Re: Spark :- Update record in partition.

2020-06-07 Thread ayan guha 
Hi

Please look at delta.io which is a companion open source project. It
addresses the exact use case you are after.

On Mon, Jun 8, 2020 at 2:35 AM Sunil Kalra  wrote:

> Hi All,
>
> If i have to update a record in partition using spark, do i have to read
> the whole partition and update the row and overwrite the partition?
>
> Is there a way to only update 1 row like DBMS. Otherwise 1 row update
> takes a long time to rewrite the whole partition ?
>
> Thanks
> Sunil
>
>
>
>
>

-- 
Best Regards,
Ayan Guha

Spark :- Update record in partition.

2020-06-07 Thread Sunil Kalra 
Hi All,

If i have to update a record in partition using spark, do i have to read
the whole partition and update the row and overwrite the partition?

Is there a way to only update 1 row like DBMS. Otherwise 1 row update takes
a long time to rewrite the whole partition ?

Thanks
Sunil