[GitHub] spark pull request #21652: [SPARK-24551][K8S] Add integration tests for secr...

2018-07-04 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21652#discussion_r200135218 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala --- @@ -265,6

[GitHub] spark pull request #21652: [SPARK-24551][K8S] Add integration tests for secr...

2018-07-04 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21652#discussion_r200082423 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala --- @@ -74,10

[GitHub] spark issue #21067: [SPARK-23980][K8S] Resilient Spark driver on Kubernetes

2018-07-03 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21067 @baluchicken @foxish any update on this? HA story is pretty critical for production in many cases. --- - To unsubscribe, e-mail

[GitHub] spark pull request #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spar...

2018-07-03 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21669#discussion_r199941392 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Constants.scala --- @@ -65,11 +65,13 @@ private[spark] object

[GitHub] spark pull request #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spar...

2018-07-03 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21669#discussion_r199935864 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/OptionRequirements.scala --- @@ -0,0 +1,40

[GitHub] spark pull request #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spar...

2018-07-03 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21669#discussion_r199806583 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Constants.scala --- @@ -81,4 +83,35 @@ private[spark] object

[GitHub] spark pull request #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spar...

2018-07-03 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21669#discussion_r199803715 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -211,6 +211,51 @@ private[spark] object Config

[GitHub] spark pull request #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spar...

2018-07-03 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21669#discussion_r199801815 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -336,7 +336,7 @@ private[spark] class SparkSubmit extends Logging

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-07-03 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21669 @ifilonenko build fails due to the tags issue I guess. I fixed it in the other PR ;) --- - To unsubscribe, e-mail: reviews

[GitHub] spark issue #21697: [SPARK-24711][K8S] Fix tags for integration tests

2018-07-02 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21697 Thanks @foxish will update the PR. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #21697: [SPARK-24711][K8S] Fix tags for integration tests

2018-07-02 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21697#discussion_r199611309 --- Diff: resource-managers/kubernetes/integration-tests/pom.xml --- @@ -102,6 +103,16

[GitHub] spark pull request #21652: [SPARK-24551][K8S] Add integration tests for secr...

2018-07-02 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21652#discussion_r199610386 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala --- @@ -150,6

[GitHub] spark pull request #21652: [SPARK-24551][K8S] Add integration tests for secr...

2018-07-02 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21652#discussion_r199610464 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala --- @@ -74,10

[GitHub] spark pull request #21652: [SPARK-24551][K8S] Add integration tests for secr...

2018-07-02 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21652#discussion_r199610357 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala --- @@ -265,6

[GitHub] spark issue #21697: [SPARK-24711][K8S] Fix tags for integration tests

2018-07-02 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21697 @ssuchter @foxish @felixcheung pls review. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21697: [SPARK-24711][K8S] Fix tags for integration tests

2018-07-02 Thread skonto
GitHub user skonto opened a pull request: https://github.com/apache/spark/pull/21697 [SPARK-24711][K8S] Fix tags for integration tests ## What changes were proposed in this pull request? - disables maven surfire plugin to allow tags function properly, doc here: http

[GitHub] spark issue #21672: [SPARK-24694][K8S] Pass all app args to integration test...

2018-06-30 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21672 @felixcheung we could call a test with more than one parameters like: https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/MultiBroadcastTest.scala

[GitHub] spark pull request #21462: [SPARK-24428][K8S] Fix unused code

2018-06-29 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21462#discussion_r199307562 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala --- @@ -46,8 +46,6

[GitHub] spark issue #21672: [SPARK-24694][K8S] Pass all app args to integration test...

2018-06-29 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21672 @vanzin @ssuchter removed the condition, I think its ok now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #21672: [SPARK-24694][K8S] Pass all app args to integrati...

2018-06-29 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21672#discussion_r199297784 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesTestComponents.scala

[GitHub] spark issue #21672: [SPARK-24694][K8S] Pass all app args to integration test...

2018-06-29 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21672 @ssuchter yes if you check the jira, the old behavior does not work with more than one args. In the future might be a problem

[GitHub] spark issue #21672: [SPARK-24694][K8S] Pass all app args to integration test...

2018-06-29 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21672 @ssuchter @liyinan926 psl review. This is trivial, unless I am missing something. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #21672: [SPARK-24694][K8S] Pass all app args to integrati...

2018-06-29 Thread skonto
GitHub user skonto opened a pull request: https://github.com/apache/spark/pull/21672 [SPARK-24694][K8S] Pass all app args to integration tests ## What changes were proposed in this pull request? - Allows to pass more than one app args to tests. ## How was this patch tested

[GitHub] spark issue #21652: [SPARK-24551][K8S] Add integration tests for secrets

2018-06-29 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21652 @felixcheung @liyinan926 review pls. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark pull request #21462: [SPARK-24428][K8S] Fix unused code

2018-06-29 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21462#discussion_r199071328 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala --- @@ -46,8 +46,6

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-06-28 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21462 @foxish @felixcheung gentle ping for merge. I replied for mount paths. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #21462: [SPARK-24428][K8S] Fix unused code

2018-06-28 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21462#discussion_r198795717 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala --- @@ -46,8 +46,6

[GitHub] spark issue #21652: [SPARK-24551][K8S] Add integration tests for secrets

2018-06-28 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21652 @foxish I got this failure: "namespaces "59da2249348347a39391b4389ab418ab" not found." I don't think it relates to the PR, I haven't seen anything locally. Secret is crea

[GitHub] spark issue #21652: [SPARK-24551][K8s] Add integration tests for secrets

2018-06-27 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21652 @foxish @liyinan926 pls review. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #21652: [SPARK-24551][K8s] Add integration tests for secr...

2018-06-27 Thread skonto
GitHub user skonto opened a pull request: https://github.com/apache/spark/pull/21652 [SPARK-24551][K8s] Add integration tests for secrets ## What changes were proposed in this pull request? - Adds integration tests for env and mount secrets. ## How was this patch

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-06-26 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21462 Thank you @foxish I was also thinking of adding the following: if [ -n "$HADOOP_CONFIG_URL" ]; then echo "Setting up hadoop config files" mkdir -p /etc/had

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-06-20 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21462 @foxish gentle ping. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-06-19 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21462 @felixcheung its local. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21462: [SPARK-24428][K8S] Fix unused code

2018-06-19 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21462#discussion_r196395479 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh --- @@ -45,12 +45,10 @@ shift 1 SPARK_CLASSPATH

[GitHub] spark issue #19675: [SPARK-14540][BUILD] Support Scala 2.12 closures and Jav...

2018-06-16 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/19675 Current status: ``` ProactiveClosureSerializationSuite: - throws expected serialization exceptions on actions - mapPartitions transformations throw proactive serialization exceptions

[GitHub] spark issue #19675: [SPARK-14540][BUILD] Support Scala 2.12 closures and Jav...

2018-06-16 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/19675 @srowen If I try with `/dev/change-scala-version 2.12 and -Pscala-2.12` two tests still fail right? --- - To unsubscribe, e-mail

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-06-15 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r195745591 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocatorBlacklistTracker.scala --- @@ -0,0 +1,187

[GitHub] spark issue #20640: [SPARK-19755][Mesos] Blacklist is always active for Meso...

2018-06-15 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/20640 @IgorBerman I saw the ticket. We need to cover mesos task failures because they are also pretty common. --- - To unsubscribe, e

[GitHub] spark pull request #20640: [SPARK-19755][Mesos] Blacklist is always active f...

2018-06-15 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/20640#discussion_r195731123 --- Diff: resource-managers/mesos/src/test/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackendSuite.scala --- @@ -108,6

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use level triggering and state reconc...

2018-06-14 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21366 @mccheah thnx a lot better with the comments. +1 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-14 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195569529 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsPollingSnapshotSource.scala --- @@ -0,0 +1,65

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-14 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195565315 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsPollingSnapshotSource.scala --- @@ -0,0 +1,65

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-14 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195549746 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsPollingSnapshotSource.scala --- @@ -0,0 +1,65

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-14 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195535296 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsPollingSnapshotSource.scala --- @@ -0,0 +1,65

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-14 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195445788 --- Diff: pom.xml --- @@ -760,6 +760,12 @@ 1.10.19 test + --- End diff -- Thanx @ktoso

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-06-14 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21462 @felixcheung gentle ping. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-14 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195401593 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsPollingSnapshotSource.scala --- @@ -0,0 +1,65

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-14 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195390942 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsPollingSnapshotSource.scala --- @@ -0,0 +1,65

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-14 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195382788 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsSnapshotsStoreImpl.scala --- @@ -0,0 +1,88

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-14 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195380019 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsLifecycleManager.scala --- @@ -0,0 +1,164

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-14 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195379975 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsLifecycleManager.scala --- @@ -0,0 +1,164

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-14 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195379907 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsLifecycleManager.scala --- @@ -0,0 +1,164

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-14 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195379060 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala --- @@ -56,17 +58,44

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-14 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195351851 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsSnapshotsStoreImpl.scala --- @@ -0,0 +1,88

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-14 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195350385 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala --- @@ -0,0 +1,141

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use level triggering and state reconc...

2018-06-14 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21366 @mccheah could you add a design doc for future reference and so that new contributors can understand better the rationale behind this. There is some description in the JIRA ticket but not enough

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-13 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r195040122 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsPollingSnapshotSource.scala --- @@ -0,0 +1,65

[GitHub] spark issue #20641: [SPARK-23464][MESOS] Fix mesos cluster scheduler options...

2018-06-13 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/20641 @krcz yes it should be ok. So I verified it: Dispatcher passes the following command to mesos: ```18/06/13 10:30:45 INFO MesosClusterScheduler: Command for the driver (test):./bin

[GitHub] spark issue #21398: [SPARK-24338][SQL] Fixed Hive CREATETABLE error in Sentr...

2018-06-12 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21398 @vanzin @cloud-fan. Going back to the facts. The fix/work-around in Cloudera's distro obviously exists because the distro integrates with Sentry anyway. Removing the location did fix the Spark Job

[GitHub] spark issue #20641: [SPARK-23464][MESOS] Fix mesos cluster scheduler options...

2018-06-12 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/20641 @krcz which one? Btw both job run successfully although might not have been escaped properly. --- - To unsubscribe, e-mail

[GitHub] spark issue #20641: [SPARK-23464][MESOS] Fix mesos cluster scheduler options...

2018-06-12 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/20641 @susanxhuynh I tested this. Here is what you get: ``` ./bin/spark-submit --deploy-mode cluster --name test --master mesos://spark.marathon.mesos:13830 --driver-cores 1.0 --driver-memory

[GitHub] spark pull request #21516: [SPARK-24501][MESOS] Add Dispatcher and Driver me...

2018-06-12 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21516#discussion_r194690191 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerSource.scala --- @@ -0,0 +1,250

[GitHub] spark pull request #21516: [SPARK-24501][MESOS] Add Dispatcher and Driver me...

2018-06-12 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21516#discussion_r194690227 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerSource.scala --- @@ -0,0 +1,250

[GitHub] spark pull request #21516: [SPARK-24501][MESOS] Add Dispatcher and Driver me...

2018-06-12 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21516#discussion_r194689746 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerSource.scala --- @@ -0,0 +1,250

[GitHub] spark pull request #21516: [SPARK-24501][MESOS] Add Dispatcher and Driver me...

2018-06-12 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21516#discussion_r194688618 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -796,6 +811,38

[GitHub] spark pull request #21516: [SPARK-24501][MESOS] Add Dispatcher and Driver me...

2018-06-12 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21516#discussion_r194687575 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterSchedulerSource.scala --- @@ -17,25 +17,170

[GitHub] spark pull request #21516: [SPARK-24501][MESOS] Add Dispatcher and Driver me...

2018-06-12 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21516#discussion_r194687233 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterSchedulerSource.scala --- @@ -17,25 +17,170

[GitHub] spark pull request #21516: [SPARK-24501][MESOS] Add Dispatcher and Driver me...

2018-06-12 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21516#discussion_r194687110 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterSchedulerSource.scala --- @@ -17,25 +17,170

[GitHub] spark issue #21516: [SPARK-24501][MESOS] Add Dispatcher and Driver metrics

2018-06-12 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21516 jenkins, please test this. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21516: [SPARK-24501][MESOS] Add Dispatcher and Driver metrics

2018-06-12 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21516 @nickbp Any isntructions how to run this? Could you also provide a design doc? There are some isntructions [here](https://github.com/mesosphere/spark-build/blob

[GitHub] spark pull request #21516: [SPARK-24501][MESOS] Add Dispatcher and Driver me...

2018-06-12 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21516#discussion_r194679672 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -122,8 +122,9 @@ private[spark

[GitHub] spark pull request #21516: [SPARK-24501][MESOS] Add Dispatcher and Driver me...

2018-06-12 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21516#discussion_r19467 --- Diff: core/src/main/scala/org/apache/spark/metrics/MetricsSystem.scala --- @@ -205,7 +208,7 @@ private[spark] class MetricsSystem private

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-12 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r194670526 --- Diff: pom.xml --- @@ -760,6 +760,12 @@ 1.10.19 test + --- End diff -- > Does Akka stre

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-06-12 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21462 @foxish gentle ping. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-12 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r194664576 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsPollingSnapshotSource.scala --- @@ -0,0 +1,65

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-11 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r194560151 --- Diff: pom.xml --- @@ -760,6 +760,12 @@ 1.10.19 test + --- End diff -- @felixcheung (sorry

[GitHub] spark pull request #21366: [SPARK-24248][K8S] Use level triggering and state...

2018-06-11 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r194534077 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsSnapshotsStoreImpl.scala --- @@ -0,0 +1,95

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-06-09 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21462 @felixcheung @mccheah pls review. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #21462: [SPARK-24428][K8S] Fix unused code

2018-06-08 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21462#discussion_r193985639 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh --- @@ -45,12 +45,10 @@ shift 1 SPARK_CLASSPATH

[GitHub] spark pull request #21462: [SPARK-24428][K8S] Fix unused code

2018-06-08 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21462#discussion_r193985575 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh --- @@ -45,12 +45,10 @@ shift 1 SPARK_CLASSPATH

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-06-08 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21462 @foxish pls review. It is quite short :) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21462: [SPARK-24428][K8S] Fix unused code

2018-06-08 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21462#discussion_r193982361 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala --- @@ -46,8 +46,6

[GitHub] spark pull request #21462: [SPARK-24428][K8S] Fix unused code

2018-06-08 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21462#discussion_r193981514 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh --- @@ -46,12 +46,6 @@ shift 1 SPARK_CLASSPATH="$SPARK_CLAS

[GitHub] spark issue #21194: [SPARK-24046][SS] Fix rate source when rowsPerSecond <= ...

2018-06-06 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21194 @felixcheung Could you review pls ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #20641: [SPARK-23464][MESOS] Fix mesos cluster scheduler ...

2018-06-06 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/20641#discussion_r193437370 --- Diff: resource-managers/mesos/src/test/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterSchedulerSuite.scala --- @@ -199,6 +199,38 @@ class

[GitHub] spark issue #21398: [SPARK-24338][SQL] Fixed Hive CREATETABLE error in Sentr...

2018-06-03 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21398 @vanzin should we merge this or add a comment in the docs? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #21378: [SPARK-24326][Mesos] add support for local:// scheme for...

2018-06-02 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21378 Thnx @felixcheung ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-06-01 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/20697 @ssuchter I think minikube could run within docker with flag –vm-driver=none right? This way you dont need minikube on hosts. Just an idea havent tried might be wrong

[GitHub] spark issue #21378: [SPARK-24326][Mesos] add support for local:// scheme for...

2018-05-31 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21378 @felixcheung ready :) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #21317: [SPARK-24232][k8s] Add support for secret env var...

2018-05-31 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21317#discussion_r192204427 --- Diff: docs/running-on-kubernetes.md --- @@ -602,4 +608,20 @@ specific to Spark on Kubernetes. spark.kubernetes.executor.secrets.spark-secret

[GitHub] spark pull request #21317: [SPARK-24232][k8s] Add support for secret env var...

2018-05-31 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21317#discussion_r192189781 --- Diff: docs/running-on-kubernetes.md --- @@ -602,4 +608,20 @@ specific to Spark on Kubernetes. spark.kubernetes.executor.secrets.spark-secret

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-31 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21317 @foxish can I get this merge pls? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #21378: [SPARK-24326][Mesos] add support for local:// sch...

2018-05-31 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21378#discussion_r192061471 --- Diff: docs/running-on-mesos.md --- @@ -753,6 +753,16 @@ See the [configuration page](configuration.html) for information on Spark config

[GitHub] spark pull request #21462: [SPARK-24428][K8S] Fix unused code

2018-05-31 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21462#discussion_r192023771 --- Diff: docs/running-on-kubernetes.md --- @@ -121,8 +121,8 @@ This URI is the location of the example jar that is already in the Docker image

[GitHub] spark pull request #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21462#discussion_r191940083 --- Diff: docs/running-on-kubernetes.md --- @@ -121,8 +121,8 @@ This URI is the location of the example jar that is already in the Docker image

[GitHub] spark issue #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21462 @foxish pls review. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21462#discussion_r191775677 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh --- @@ -46,12 +46,6 @@ shift 1 SPARK_CLASSPATH="$SPARK_CLAS

[GitHub] spark pull request #21462: [SPARK-24428][K8S] Fix unused code

2018-05-30 Thread skonto
GitHub user skonto opened a pull request: https://github.com/apache/spark/pull/21462 [SPARK-24428][K8S] Fix unused code ## What changes were proposed in this pull request? Remove code that is misleading and is a leftover from previous implementation. ## How

[GitHub] spark issue #21378: [SPARK-24326][Mesos] add support for local:// scheme for...

2018-05-30 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21378 @felixcheung I think its ready now ;) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark issue #21378: [SPARK-24326][Mesos] add support for local:// scheme for...

2018-05-30 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21378 Thank you @susanxhuynh --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

<    1   2   3   4   5   6   7   8   9   10   >