Re: Setting spark.kubernetes.driver.connectionTimeout, spark.kubernetes.submission.connectionTimeout to default spark.network.timeout

2022-08-02 Thread Pralabh Kumar
Hi Dongjoon Thx for replying and clarifying. Below are the errors in Spark32 pn K8s which occurred because of time out . io.fabric8.kubernetes.client.KubernetesClientException: Operation: [create] for kind: [ConfigMap] with name: [null] in namespace: [xyz] failed. at

Re: Setting spark.kubernetes.driver.connectionTimeout, spark.kubernetes.submission.connectionTimeout to default spark.network.timeout

2022-08-01 Thread Dongjoon Hyun
Hi, Pralabh. Could you elaborate on your situation more? I'm interested in your needs. Currently, the default value of spark.network.timeout, 120s, is quite bigger than the default value of spark.kubernetes.driver.connectionTimeout, 10s. It would be a breaking change if we increase

Setting spark.kubernetes.driver.connectionTimeout, spark.kubernetes.submission.connectionTimeout to default spark.network.timeout

2022-08-01 Thread Pralabh Kumar
Hi Dev team Since* spark.network.timeout* is default for all the network transactions . Shouldn’t *spark.kubernetes.driver.connectionTimeout*, *spark.kubernetes.submission.connectionTimeout* by default to be set spark.network.timeout . Users migrating from Yarn to K8s are familiar with