Hi All,
If i have to update a record in partition using spark, do i have to read
the whole partition and update the row and overwrite the partition?
Is there a way to only update 1 row like DBMS. Otherwise 1 row update takes
a long time to rewrite the whole partition ?
Thanks
Sunil
Ankit
Can you try reducing number of cores or increasing memory. Because with
below configuration your each core is getting ~3.5 GB. Otherwise your data
is skewed, that one of cores is getting too much data based key.
spark.executor.cores 6 spark.executor.memory 36g
On Sat, Sep 7, 2019 at 6:35
