skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-533082486
thanks @holdenk @dongjoon-hyun I will target Spark 3 and try to implement
this for all managers.
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-520250390
@dongjoon-hyun sorry for the name spelling thing :) I am on vacations but
will slowly have a look to what @squito suggests ;)
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518871183
@dongjoon-hyun that was one of my options. So this PR was intended to
protect the user from the spark issues when things dont go well and the
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518871183
@@dongjoon-hyun that was one of my options. So this PR was intended to
protect the user from the spark issues when things dont go well and the
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518871183
@Dooyoung-Hwang that was one of my options. So this PR was intended to
protect the user from the spark issues when things dont go well and the
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517
@dongjoon-hyun a clarification here pls:
> By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p"
and it will be
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517
@dongjoon-hyun a clarification here pls:
> By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p"
and it will be
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517
@dongjoon-hyun a clarification here pls:
> By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p"
and it will be
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517
@dongjoon-hyun a clarification here pls:
> By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p"
and it will be
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517
@dongjoon-hyun a clarification here pls:
> By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p"
and it will be
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517
@dongjoon-hyun a clarification here pls:
> By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p"
and it will be
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517
@dongjoon-hyun a clarification here pls:
> By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p"
and it will be
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517
@dongjoon-hyun a clarification here pls:
> By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p"
and it will be
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517
@dongjoon-hyun a clarification here pls:
> By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p"
and it will be
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517
@dongjoon-hyun a clarification here pls:
> By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p"
and it will be
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518759232
@dongjoon-hyun
By default the user will not be able to pass any
spark.driver.extraJavaOptions settings since his properties are put in a
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518759232
@dongjoon-hyun
By default the user will not be able to pass any
spark.driver.extraJavaOptions settings since his properties are put in a
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518759232
@dongjoon-hyun
By default the user will not be able to pass any
spark.driver.extraJavaOptions settings since his properties are put in a
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518759232
@dongjoon-hyun
By default the user will not be able to pass any
spark.driver.extraJavaOptions settings since his properties are put in a
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518748224
@dongjoon-hyun one thing I noticed is that if I set:
```
case "$1" in
driver)
shift 1
VERBOSE_FLAG=$(get_verbose_flag)
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-518748224
@dongjoon-hyun one thing I noticed is that if I set:
```
case "$1" in
driver)
shift 1
VERBOSE_FLAG=$(get_verbose_flag)
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-516594015
@dongjoon-hyun I updated the PR.
Using the following submission cmd:
```
./bin/spark-submit --master k8s://https://192.168.2.5:8443\
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-516594015
@dongjoon-hyun I updated the PR.
Using the following submission cmd:
```
./bin/spark-submit --master k8s://https://192.168.2.5:8443\
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-515909297
@dongjoon-hyun Ok I will improve the PR, I had in mind to test
`spark.driver.extraJavaOptions` with this anyway,but missed it. I
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-515909297
@dongjoon-hyun Ok I will improve the PR, I had in mind to test
`spark.driver.extraJavaOptions` with this anyway,but missed it. I
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-515909297
@dongjoon-hyun Ok I will improve the PR, I had in mind to test
`spark.driver.extraJavaOptions` with this anyway,but missed it. I
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-515909297
@dongjoon-hyun Ok I will improve the PR, I had in mind to test
`spark.driver.extraJavaOptions` with this anyway,but missed it. I
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-515909297
@dongjoon-hyun Ok I will improve the PR, I had in mind to test
`spark.driver.extraJavaOptions` with this anyway,but missed it. I will not
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-515909297
@dongjoon-hyun Ok I will improve the PR, I had in mind to test
`spark.driver.extraJavaOptions` with this anyway. I will not modify any tests,
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-515677267
we need to make sure its added by default and not rely to the user to pass
it. If you mean whether to pass this as a spark conf using that
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag
URL: https://github.com/apache/spark/pull/25229#issuecomment-515677267
we need to make sure its added by default and not rely to the user to pass
it. If you mean whether to pass this as a spark conf using that
31 matches
Mail list logo