Looks like the write to Aerospike is taking too long.
Could you try writing the rdd directly to filesystem, skipping the
Aerospike write.
foreachPartition at WriteToAerospike.java:47, took 338.345827 s
- Thanks, via mobile, excuse brevity.
On Jul 12, 2016 8:08 PM, "Saurav Sinha" wrote:
> Hi,
Hi,
I am getting into an issue where job is running in multiple partition
around 21000 parts.
Setting
Driver = 5G
Executor memory = 10G
Total executor core =32
It us falling when I am trying to write to aerospace earlier it is working
fine. I am suspecting number of partition as reason.
Kindly
