[
https://issues.apache.org/jira/browse/SPARK-27927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885234#comment-16885234
]
Stavros Kontopoulos edited comment on SPARK-27927 at 7/15/19 1:40 PM:
----------------------------------------------------------------------
np :). There will be a 2.4.4 release so there is chance to fix it there. As for
the policy of maintenance releases probably you are right not sure what falls
into that category though. On one hand you are targeting K8s releases that are
way ahead and on the other you use an old client that does not support
them(check fabric8io's compatibility matrix). We had a long discussion about
what to k8s versions to support, a project with high velocity that does not
match Spark release planning.
So for good or bad we need a fix, not sure if there is a workaround like the
one with ping interval.
The jackson core thing is another important but also hard to upgrade. Actually
a customer asked this because it didnt pass security checks. That means 2.4.x
is not acceptable for some people.
Personally I was not aware of the daemon thread issue. I hope Spark 3.0.0 will
solve these two issues once and for all.
was (Author: skonto):
np :). There will be a 2.4.4 release so there is chance to fix it there. As for
the policy of maintenance releases probably you are right not sure what falls
into that category though. On one hand you are targeting K8s releases that are
way ahead and on the other you use an old client that does not support
them(check fabric8io's compatibility matrix). We had a long discussion about
what to k8s versions to support, a project with high velocity that does not
match Spark release planning.
So for good or bad we need a fix, not sure if there is a workaround like the
one with ping interval.
The jackson core thing is another important but also hard to upgrade. Actually
a customer asked this because it didnt pass security checks. That means 2.4.x
is not acceptable for some people.
Personally I was not aware of the daemon thread issue. I hope Spark 3.0.0 will
solve these two issues.
> driver pod hangs with pyspark 2.4.3 and master on kubenetes
> -----------------------------------------------------------
>
> Key: SPARK-27927
> URL: https://issues.apache.org/jira/browse/SPARK-27927
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes, PySpark
> Affects Versions: 3.0.0, 2.4.3
> Environment: k8s 1.11.9
> spark 2.4.3 and master branch.
> Reporter: Edwin Biemond
> Priority: Major
> Attachments: driver_threads.log, executor_threads.log
>
>
> When we run a simple pyspark on spark 2.4.3 or 3.0.0 the driver pods hangs
> and never calls the shutdown hook.
> {code:java}
> #!/usr/bin/env python
> from __future__ import print_function
> import os
> import os.path
> import sys
> # Are we really in Spark?
> from pyspark.sql import SparkSession
> spark = SparkSession.builder.appName('hello_world').getOrCreate()
> print('Our Spark version is {}'.format(spark.version))
> print('Spark context information: {} parallelism={} python version={}'.format(
> str(spark.sparkContext),
> spark.sparkContext.defaultParallelism,
> spark.sparkContext.pythonVer
> ))
> {code}
> When we run this on kubernetes the driver and executer are just hanging. We
> see the output of this python script.
> {noformat}
> bash-4.2# cat stdout.log
> Our Spark version is 2.4.3
> Spark context information: <SparkContext
> master=k8s://https://kubernetes.default.svc:443 appName=hello_world>
> parallelism=2 python version=3.6{noformat}
> What works
> * a simple python with a print works fine on 2.4.3 and 3.0.0
> * same setup on 2.4.0
> * 2.4.3 spark-submit with the above pyspark
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
