Hi Hyma!
Looks like you encountered a classic deadlock. It happens because you put
values into cache in arbitrary order.
This line causes this problem:
*companyDao.nameCache.putAll(kvs)*
So, when multiple threads try to acquire the same locks in different order,
then these operations will be wait
Below is the corresponding code where ignite step was in hung state.
logInfo("Populating the canonical name Cache on Ignite Nodes")
val time = System.currentTimeMillis()
companyVORDD.mapPartitions(x => {
val kvs = x.map(comp =>
(comp.wcaId,comp)).toMap[String,CompanyVO].asJava
Hi Hyma,
Could you please show a code snippet where it is hanged?
Thanks,
Mike.
2017-11-22 12:48 GMT+03:00 Hyma :
> Thanks Mikhail.
>
> I suspected to increase the spark heartbeat/network timeout. But my
> question
> here is if an executor is lost, corresponding ignite node also gets out of
> c
Thanks Mikhail.
I suspected to increase the spark heartbeat/network timeout. But my question
here is if an executor is lost, corresponding ignite node also gets out of
cluster. In that case, ignite takes care of re balancing between the other
active nodes right. My spark job was not killed and it
Hi Hyma,
looks like your job takes too much time, you hit some timeout and spark
killed your jobs.
I don't see any other errors or warnings from your logs, it's very likely
that you need to increase some time out in spark.
thanks,
Mike.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.
Hi,
When loading ignite cache, we saw the spark job went into hung state at this
step.
We see one of the executor task has been running for hours and below are the
logs from this executor that had the failure.
Stdout log
Launch class org.apache.spark.executor.CoarseGrainedExecutorBackend by
calli
