Re: Spark Issue with Istio in Distributed Mode

2022-09-02 Thread Ilan Filonenko
This might be a result of the idle_timeout that is configured in envoy. The default is an hour. On Sat, Sep 3, 2022 at 12:17 AM Deepak Sharma wrote: > Hi All, > In 1 of our cluster , we enabled Istio where spark is running in > distributed mode. > Spark works fine when we run it with Istio in

Spark Issue with Istio in Distributed Mode

2022-09-02 Thread Deepak Sharma
Hi All, In 1 of our cluster , we enabled Istio where spark is running in distributed mode. Spark works fine when we run it with Istio in standalone mode. In spark distributed mode , we are seeing that every 1 hour or so the workers are getting disassociated from master and then master is not able

Unsafe loop in CatalystTypeConverters

2022-09-02 Thread Edmondo Porcu
Hello Spark, as I tried to implement an improvement on Delta Lake by changing a signature of a command by making it return some data (see https://github.com/delta-io/delta/pull/1240/files#diff-f993636fe0ceb5f15de09ed4a0a71ea405a5e89ad66d36047099fcc1387a375aR123) the code produced a runtime