[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10057 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-178180002 Merged into master. By the way I might have mentioned this before but you probably don't need 3 different issues to rename 3 configs. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-176638844 **[Test build #50352 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50352/consoleFull)** for PR 10057 at commit [`5ebaf04`](https://github.com/apache/spark/commit/5ebaf04ef29aea4c7ac8bed13b406e2de0206c7d). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-176639037 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-176639040 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/50352/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-176599484 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-176602007 **[Test build #50352 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50352/consoleFull)** for PR 10057 at commit [`5ebaf04`](https://github.com/apache/spark/commit/5ebaf04ef29aea4c7ac8bed13b406e2de0206c7d). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-176343442 **[Test build #50289 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50289/consoleFull)** for PR 10057 at commit [`4a56f6c`](https://github.com/apache/spark/commit/4a56f6c8916418627bdf2528639987587933ea52). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-176339334 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-176398448 **[Test build #50289 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50289/consoleFull)** for PR 10057 at commit [`4a56f6c`](https://github.com/apache/spark/commit/4a56f6c8916418627bdf2528639987587933ea52). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-176398929 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/50289/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-176398923 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-176526642 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-176565754 **[Test build #50333 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50333/consoleFull)** for PR 10057 at commit [`5ebaf04`](https://github.com/apache/spark/commit/5ebaf04ef29aea4c7ac8bed13b406e2de0206c7d). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-176565820 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/50333/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-176565817 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-176532276 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-176532392 **[Test build #50333 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50333/consoleFull)** for PR 10057 at commit [`5ebaf04`](https://github.com/apache/spark/commit/5ebaf04ef29aea4c7ac8bed13b406e2de0206c7d). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-176532278 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/50330/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-168780176 @andrewor14 Can you take a look at this PR sometime this week? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-166557670 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48170/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-166557669 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-166557494 **[Test build #48170 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48170/consoleFull)** for PR 10057 at commit [`4a56f6c`](https://github.com/apache/spark/commit/4a56f6c8916418627bdf2528639987587933ea52). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_:\n * `public final class LZ4BlockInputStream extends FilterInputStream `\n * `case class Range(`\n * `case class Range(`\n --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-166413509 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48121/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10057#discussion_r48191846 --- Diff: docs/spark-standalone.md --- @@ -341,23 +341,8 @@ Learn more about getting started with ZooKeeper [here](http://zookeeper.apache.o **Configuration** -In order to enable this recovery mode, you can set SPARK_DAEMON_JAVA_OPTS in spark-env using this configuration: - - - System propertyMeaning - -spark.deploy.recoveryMode -Set to ZOOKEEPER to enable standby Master recovery mode (default: NONE). - - -spark.deploy.zookeeper.url -The ZooKeeper cluster url (e.g., 192.168.1.100:2181,192.168.1.101:2181). - - -spark.deploy.zookeeper.dir -The directory in ZooKeeper to store recovery state (default: /spark). - - +In order to enable this recovery mode, you can set SPARK_DAEMON_JAVA_OPTS in spark-env by configuring `spark.deploy.recoveryMode` and related spark.deploy.zookeeper.* configurations. +For more information about these configurations please refer to the configurations (doc)[configurations.html#deploy] --- End diff -- we should also briefly mention this in the `running-on-mesos.md` docs right? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10057#discussion_r48192495
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterPersistenceEngine.scala
---
@@ -53,9 +53,12 @@ private[spark] trait MesosClusterPersistenceEngine {
* all of them reuses the same connection pool.
*/
private[spark] class ZookeeperMesosClusterPersistenceEngineFactory(conf:
SparkConf)
- extends MesosClusterPersistenceEngineFactory(conf) {
+ extends MesosClusterPersistenceEngineFactory(conf) with Logging {
- lazy val zk = SparkCuratorUtil.newClient(conf,
"spark.mesos.deploy.zookeeper.url")
+ // TODO(tnachen): Remove support for spark.mesos.deploy.zookeeper.url in
0.28.
--- End diff --
same here, no need to do this later. Just do it now. (also this is Spark so
you should use Spark versions in comments, not Mesos versions)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-166413505 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-166413261 **[Test build #48121 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48121/consoleFull)** for PR 10057 at commit [`2c29939`](https://github.com/apache/spark/commit/2c29939cd7ea3724d1bbb3defbdc1988afad85c8). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10057#discussion_r48192141
--- Diff:
core/src/main/scala/org/apache/spark/deploy/SparkCuratorUtil.scala ---
@@ -35,8 +35,11 @@ private[spark] object SparkCuratorUtil extends Logging {
def newClient(
conf: SparkConf,
zkUrlConf: String = "spark.deploy.zookeeper.url"): CuratorFramework
= {
--- End diff --
@tnachen did you miss this one? I don't think we need this param anymore
since it's always going to be the same one.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10057#discussion_r48192593
--- Diff:
core/src/main/scala/org/apache/spark/deploy/SparkCuratorUtil.scala ---
@@ -35,8 +35,11 @@ private[spark] object SparkCuratorUtil extends Logging {
def newClient(
conf: SparkConf,
zkUrlConf: String = "spark.deploy.zookeeper.url"): CuratorFramework
= {
-val ZK_URL = conf.get(zkUrlConf)
-val zk = CuratorFrameworkFactory.newClient(ZK_URL,
+newClient(conf.get(zkUrlConf))
+ }
+
+ def newClient(zkUrl: String): CuratorFramework = {
--- End diff --
actually we probably don't need this method now that we support only one
config
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10057#discussion_r48192082
--- Diff:
core/src/main/scala/org/apache/spark/deploy/mesos/MesosClusterDispatcher.scala
---
@@ -50,7 +50,10 @@ private[mesos] class MesosClusterDispatcher(
extends Logging {
private val publicAddress =
Option(conf.getenv("SPARK_PUBLIC_DNS")).getOrElse(args.host)
- private val recoveryMode = conf.get("spark.mesos.deploy.recoveryMode",
"NONE").toUpperCase()
+ // TODO(tnachen): Remove support for spark.mesos.deploy.recoverMode in
0.28.
+ private val recoveryMode =
conf.getOption("spark.mesos.deploy.recoverMode").map { mode =>
--- End diff --
I would just remove support for this right now since we're not even
documenting it. The next version will be Spark 2.0 so this is the right time to
remove support for things that we don't want to maintain.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-166388803 **[Test build #48121 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48121/consoleFull)** for PR 10057 at commit [`2c29939`](https://github.com/apache/spark/commit/2c29939cd7ea3724d1bbb3defbdc1988afad85c8). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10057#discussion_r4825
--- Diff:
core/src/main/scala/org/apache/spark/deploy/SparkCuratorUtil.scala ---
@@ -35,8 +35,11 @@ private[spark] object SparkCuratorUtil extends Logging {
def newClient(
conf: SparkConf,
zkUrlConf: String = "spark.deploy.zookeeper.url"): CuratorFramework
= {
--- End diff --
It's an unnecessary abstraction that makes the code more complicated. We
can always introduce it later if we actually need it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user tnachen commented on a diff in the pull request:
https://github.com/apache/spark/pull/10057#discussion_r48210724
--- Diff:
core/src/main/scala/org/apache/spark/deploy/SparkCuratorUtil.scala ---
@@ -35,8 +35,11 @@ private[spark] object SparkCuratorUtil extends Logging {
def newClient(
conf: SparkConf,
zkUrlConf: String = "spark.deploy.zookeeper.url"): CuratorFramework
= {
--- End diff --
You meant the param zkUrlConf? It feels like a good abstraction so that the
url configuration can at least be overriden right?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user tnachen commented on a diff in the pull request:
https://github.com/apache/spark/pull/10057#discussion_r48210912
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterPersistenceEngine.scala
---
@@ -53,9 +53,12 @@ private[spark] trait MesosClusterPersistenceEngine {
* all of them reuses the same connection pool.
*/
private[spark] class ZookeeperMesosClusterPersistenceEngineFactory(conf:
SparkConf)
- extends MesosClusterPersistenceEngineFactory(conf) {
+ extends MesosClusterPersistenceEngineFactory(conf) with Logging {
- lazy val zk = SparkCuratorUtil.newClient(conf,
"spark.mesos.deploy.zookeeper.url")
+ // TODO(tnachen): Remove support for spark.mesos.deploy.zookeeper.url in
0.28.
--- End diff --
:)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user tnachen commented on a diff in the pull request:
https://github.com/apache/spark/pull/10057#discussion_r48210760
--- Diff:
core/src/main/scala/org/apache/spark/deploy/mesos/MesosClusterDispatcher.scala
---
@@ -50,7 +50,10 @@ private[mesos] class MesosClusterDispatcher(
extends Logging {
private val publicAddress =
Option(conf.getenv("SPARK_PUBLIC_DNS")).getOrElse(args.host)
- private val recoveryMode = conf.get("spark.mesos.deploy.recoveryMode",
"NONE").toUpperCase()
+ // TODO(tnachen): Remove support for spark.mesos.deploy.recoverMode in
0.28.
+ private val recoveryMode =
conf.getOption("spark.mesos.deploy.recoverMode").map { mode =>
--- End diff --
ok I'll remove it
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-166514995 **[Test build #48157 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48157/consoleFull)** for PR 10057 at commit [`32a33ae`](https://github.com/apache/spark/commit/32a33aec87ef1a230ce328e4c7cf86f435365e50). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_:\n * `public final class LZ4BlockInputStream extends FilterInputStream `\n * `case class Range(`\n * `case class Range(`\n --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-166515129 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48157/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-166515128 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-166534060 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48166/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-166534058 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-166534398 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-16642 **[Test build #48157 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48157/consoleFull)** for PR 10057 at commit [`32a33ae`](https://github.com/apache/spark/commit/32a33aec87ef1a230ce328e4c7cf86f435365e50). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12463][SPARK-12464][SPARK-12465][SPARK-...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10057#issuecomment-166538523 **[Test build #48170 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48170/consoleFull)** for PR 10057 at commit [`4a56f6c`](https://github.com/apache/spark/commit/4a56f6c8916418627bdf2528639987587933ea52). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
