[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user suryag10 commented on the issue: https://github.com/apache/spark/pull/20272 > If this can get a rebase, and maybe a few sentences in the k8s docs, I'll merge. Hi, I had rebased this patch to the latest master and this patch was missing another fix to run the STS

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user deveshk0 commented on the issue: https://github.com/apache/spark/pull/20272 is this coming with spark 2.4 ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user deveshk0 commented on the issue: https://github.com/apache/spark/pull/20272 I have built spark with the same changes for thrift server. It is running fine for me. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20272 still need to run tests https://github.com/apache/spark/pull/20272#pullrequestreview-108271893 --- - To unsubscribe,

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user srowen commented on the issue: https://github.com/apache/spark/pull/20272 If this can get a rebase, and maybe a few sentences in the k8s docs, I'll merge. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/20272 @felixcheung I think yes and with https://github.com/apache/spark/pull/21748, users should be able to run the Thrift server in a pod. ---

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20272 Is this aligned with the "in cluster client"? @foxish @mccheah --- - To unsubscribe, e-mail:

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20272 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user nrchakradhar commented on the issue: https://github.com/apache/spark/pull/20272 Its working for us in our test environment. No issues so far. Only thing to take care is the clean-up of the driver manually when the thrift server is re-installed or started due to any

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user ah- commented on the issue: https://github.com/apache/spark/pull/20272 Is anyone still looking at this? It seems like it should just work? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20272 It's really not that much work to write a couple of sentences about this in the running-on-k8s docs, even if you're just pointing people to the kubernetes documentation. Is that really so

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user ozzieba commented on the issue: https://github.com/apache/spark/pull/20272 Ultimately I see this as being in the realm of Kubernetes knowledge, rather than Spark knowledge. These features are well-documented in Kubernetes documentation, and I am not sure there's a need to

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20272 Great, how about explaining that in some place more visible to Spark users than a PR on github. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user ozzieba commented on the issue: https://github.com/apache/spark/pull/20272 @vanzin In general Kubernetes makes this super easy: - The most basic workflow is to use the driver pod name (which is in the output of Spark Submit, or can be found with `kubectl get pods`),

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20272 So how are you expected to contact the thrift server after it's up, and even figure out where it started? Isn't the container unreachable from the outside world unless you do some port mapping on

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20272 >IIUC there was a issue in launching Thrift Server on YARN cluster mode, and I'm not sure whether it has been fixed (maybe @jerryshao can kindly check that?) Sorry I cannot remember the

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user foxish commented on the issue: https://github.com/apache/spark/pull/20272 Makes sense. The change LGTM. On Jan 29, 2018 10:23 AM, "Jiang Xingbo" wrote: > IIUC there was a issue in launching Thrift Server on YARN cluster mode,

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20272 IIUC there was a issue in launching Thrift Server on YARN cluster mode, and I'm not sure whether it has been fixed (maybe @jerryshao can kindly check that?) Anyway that is not a problem on

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user ozzieba commented on the issue: https://github.com/apache/spark/pull/20272 @andrewor14 @liancheng can you chime in? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user ozzieba commented on the issue: https://github.com/apache/spark/pull/20272 @foxish per SPARK-5176 and the associated [PR](https://github.com/apache/spark/pull/4137), it seems there was a technical issue with spark-internal that wouldn't allow Thrift Server to run on YARN

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user foxish commented on the issue: https://github.com/apache/spark/pull/20272 @jiangxb1987 Is there any specific owner of the thrift server that we can ping here? The testing looks good - so, all we're waiting for is confirmation from them on the original intent behind

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20272 cc @liyinan926 Do you have some time to verify this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For