[GitHub] spark pull request: [SPARK-10427][SQL]Respect -S option for spark-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8915#issuecomment-143146765 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10427][SQL]Respect -S option for spark-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8915#issuecomment-143146792 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10427][SQL]Respect -S option for spark-...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8915#issuecomment-143147574 [Test build #43014 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43014/consoleFull) for PR 8915 at commit [`4596424`](https://github.com/apache/spark/commit/4596424b3c4a92a784918884fdb5d8e148f2c9c6). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10812][YARN][WIP] Spark hadoop util sup...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/8911#discussion_r40404934
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala
---
@@ -385,20 +385,13 @@ class SparkHadoopUtil extends Logging {
object SparkHadoopUtil {
- private val hadoop = {
-val yarnMode = java.lang.Boolean.valueOf(
-System.getProperty("SPARK_YARN_MODE",
System.getenv("SPARK_YARN_MODE")))
-if (yarnMode) {
- try {
-
Utils.classForName("org.apache.spark.deploy.yarn.YarnSparkHadoopUtil")
- .newInstance()
- .asInstanceOf[SparkHadoopUtil]
- } catch {
- case e: Exception => throw new SparkException("Unable to load YARN
support", e)
- }
-} else {
- new SparkHadoopUtil
-}
+ private lazy val hadoop = new SparkHadoopUtil
+ private lazy val yarn = try{
--- End diff --
LGTM. Only trivial things to suggest like a space after 'try'
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10812][YARN][WIP] Spark hadoop util sup...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/8911#discussion_r40404998
--- Diff:
yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtilSuite.scala
---
@@ -233,4 +233,17 @@ class YarnSparkHadoopUtilSuite extends SparkFunSuite
with Matchers with Logging
}
assert(caught.getMessage === "Can't get Master Kerberos principal for
use as renewer")
}
+
+ test("check different hadoop utils based on env variable") {
+import org.apache.spark.deploy.SparkHadoopUtil
--- End diff --
You can put this import up top I suppose, and `java.lang.*` classes never
need to be prefixed.
Rather than cast incorrectly on purpose and check an exception, just look
at the `getSimpleName` of the class and compare to "YarnSparkHadoopUtil" ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10317] [Core] Compatibility between his...
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/8758#issuecomment-143148811 Seems pretty reasonable to me. The only issue is that it will mean existing invocations of the script no longer work, right? I wonder if it's possible to parse the first non-flag argument as if it's setting the directory to use, such that it's backwards compatible? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10317] [Core] Compatibility between his...
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/8758#discussion_r40405360 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/HistoryServerArguments.scala --- @@ -61,6 +61,7 @@ private[history] class HistoryServerArguments(conf: SparkConf, args: Array[Strin |Usage: HistoryServer [options] | |Options: + | --dir DIR (-d DIR) Sets spark.history.fs.logDirectory,deprecated use conf instead --- End diff -- ",deprecated" looks funny. Maybe write "Deprecated; set spark.history.fs.logDirectory directly" on the next line? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10317] [Core] Compatibility between his...
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/8758#discussion_r40405373 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/HistoryServerArgumentsSuite.scala --- @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + *http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.spark.deploy.history + +import java.io.File + +import com.google.common.base.Charsets._ --- End diff -- You can use Java's standard `StandardCharsets` instead --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10317] [Core] Compatibility between his...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/8758#discussion_r40405402
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/HistoryServerArgumentsSuite.scala
---
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.spark.deploy.history
+
+import java.io.File
+
+import com.google.common.base.Charsets._
+import com.google.common.io.Files
+import org.apache.spark._
+import org.apache.spark.util.Utils
+
+class HistoryServerArgumentsSuite extends SparkFunSuite {
+
+ private val logDir = new File("src/test/resources/spark-events")
+ private val conf = new SparkConf()
+.set("spark.history.fs.logDirectory", logDir.getAbsolutePath)
+.set("spark.history.fs.updateInterval", "1")
+.set("spark.testing", "true")
+
+ test("No Arguments Parsing") {
+val argStrings = Array[String]()
+val hsa = new HistoryServerArguments(conf, argStrings)
+assert(conf.get("spark.history.fs.logDirectory") ===
logDir.getAbsolutePath)
+assert(conf.get("spark.history.fs.updateInterval") === "1")
+assert(conf.get("spark.testing") === "true")
+ }
+
+ test("Directory Arguments Parsing --dir or -d") {
+val argStrings = Array("--dir", "src/test/resources/spark-events1")
+val hsa = new HistoryServerArguments(conf, argStrings)
+
assert(conf.get("spark.history.fs.logDirectory").equals("src/test/resources/spark-events1"))
+ }
+
+ test("Properties File Arguments Parsing --properties-file") {
+val tmpDir = Utils.createTempDir()
+val outFile = File.createTempFile("test-load-spark-properties",
"test", tmpDir)
+val argStrings = Array("--properties-file", outFile.getAbsolutePath)
+try {
+ System.setProperty("spark.test.DefaultPropertyB", "1")
+ Files.write("spark.test.CustomPropertyA true\n" +
+"spark.test.DefaultPropertyB 3\n", outFile, UTF_8)
+ val properties = Utils.getPropertiesFromFile(outFile.getAbsolutePath)
+ properties
+.filter { case (k, v) => k.startsWith("spark.")}
+.foreach { case (k, v) => sys.props.getOrElseUpdate(k, v)}
+ val hsa = new HistoryServerArguments(conf, argStrings)
+ assert(conf.getBoolean("spark.test.CustomPropertyA", false) === true)
--- End diff --
Also entirely trivial, but the `=== true` is redundant
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10822] Move contents of spark-unsafe su...
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/8912#issuecomment-143149940 As long as use of unsafe can still be turned off and is still pretty 'private' inside core, seems like it doesn't change much. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4449][Core]Specify port range in spark
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5722#issuecomment-143150572 I'd say it slightly differently: the goal is to define, for every configurable port, a range of 1 or more port numbers that may be bound to it. It could be a single port, or a range. Yes, plumbing through a range of `(Int,Int)` seems better than `String` or `Int`, although the plumbing work will be significant. The other problem is backwards compatibility. I think that a single number like "8080" could be interpreted as the range "[8080,8080]" but then that changes behavior, as previously, this would cause retries through to port 8095. We can make a breaking change. Or, we can preserve the behavior. So 8080 means [8080,8095]. Callers would use [8080,8080] to mean "only 8080". Which is clunkier then. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9681] [ML] Support R feature interactio...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/8830#issuecomment-143150875 LGTM. Merged into master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9681] [ML] Support R feature interactio...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8830 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10694][STREAMING]Prevent Data Loss in S...
Github user dibbhatt commented on the pull request: https://github.com/apache/spark/pull/8817#issuecomment-143151201 hi @andrewor14 @tdas , kindly let me know if this PR makes sense . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10778][MLlib]Implement toString for Ass...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/8904#issuecomment-143151203 `Returns` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10790][YARN] Fix initial executor numbe...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/8910#discussion_r40406280
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -314,5 +314,28 @@ object YarnSparkHadoopUtil {
def getClassPathSeparator(): String = {
classPathSeparatorField.get(null).asInstanceOf[String]
}
+
--- End diff --
The refactoring is nice. This is a reasonable place for this code now,
though eventually it may not be YARN-specific?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10790][YARN] Fix initial executor numbe...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/8910#discussion_r40406303
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -314,5 +314,28 @@ object YarnSparkHadoopUtil {
def getClassPathSeparator(): String = {
classPathSeparatorField.get(null).asInstanceOf[String]
}
+
+ /** Get the initial target number of executors depends on dynamic
allocation is enabled or not */
+ def getTargetExecutorNumber(conf: SparkConf): Int = {
+if (Utils.isDynamicAllocationEnabled(conf)) {
+ val minNumExecutors =
conf.getInt("spark.dynamicAllocation.minExecutors", 0)
+ val initialNumExecutors =
+conf.getInt("spark.dynamicAllocation.initialExecutors",
minNumExecutors)
+ val maxNumExecutors =
conf.getInt("spark.dynamicAllocation.maxExecutors", Int.MaxValue)
+ require(initialNumExecutors >= minNumExecutors &&
initialNumExecutors <= maxNumExecutors,
+s"initial executor number $initialNumExecutors must between min
executor number" +
+ s"$minNumExecutors and max executor number $maxNumExecutors")
+
+ initialNumExecutors
+} else {
+ var targetNumExecutors = DEFAULT_NUMBER_EXECUTORS
--- End diff --
Nit: can this be a `val` assigned by an if-else?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10790][YARN] Fix initial executor numbe...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/8910#discussion_r40406320
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -314,5 +314,28 @@ object YarnSparkHadoopUtil {
def getClassPathSeparator(): String = {
classPathSeparatorField.get(null).asInstanceOf[String]
}
+
+ /** Get the initial target number of executors depends on dynamic
allocation is enabled or not */
+ def getTargetExecutorNumber(conf: SparkConf): Int = {
--- End diff --
`getInitialTargetExecutorNumber`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-143151797 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-143151781 Created https://issues.apache.org/jira/browse/SPARK-10830. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-143151996 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-143152016 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-143152365 [Test build #43015 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43015/consoleFull) for PR 8323 at commit [`66bdb2b`](https://github.com/apache/spark/commit/66bdb2b1bd1e365c0aec5fbfb88e0d06e1ea24cd). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10427][SQL]Respect -S option for spark-...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8915#issuecomment-143153353 [Test build #43011 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43011/console) for PR 8915 at commit [`b9dc8d1`](https://github.com/apache/spark/commit/b9dc8d1c12005ea8da819cf3e8af57853e96c6d9). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10427][SQL]Respect -S option for spark-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8915#issuecomment-143153462 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10427][SQL]Respect -S option for spark-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8915#issuecomment-143153463 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43011/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10829][SQL]Filter combine partition key...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8916#issuecomment-143155566 [Test build #43012 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43012/console) for PR 8916 at commit [`1c0ba50`](https://github.com/apache/spark/commit/1c0ba50edc2ea6c09af634ede4068bd9879abff0). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10829][SQL]Filter combine partition key...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8916#issuecomment-143155656 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10829][SQL]Filter combine partition key...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8916#issuecomment-143155658 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43012/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10585][SQL] only copy data once when ge...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8747#issuecomment-143155835 [Test build #43013 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43013/console) for PR 8747 at commit [`44b1799`](https://github.com/apache/spark/commit/44b1799aa90cd7255ca4f860843fdc4f308626cf). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `public class GlobalBufferHolder ` * `public class UnsafeArrayWriter ` * `public class UnsafeRowWriter ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10585][SQL] only copy data once when ge...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8747#issuecomment-143155922 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43013/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10585][SQL] only copy data once when ge...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8747#issuecomment-143155921 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-143158210 [Test build #43015 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43015/console) for PR 8323 at commit [`66bdb2b`](https://github.com/apache/spark/commit/66bdb2b1bd1e365c0aec5fbfb88e0d06e1ea24cd). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-143158438 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-143158439 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43015/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10831][SQL]Add more configuration descr...
GitHub user chenghao-intel opened a pull request: https://github.com/apache/spark/pull/8917 [SPARK-10831][SQL]Add more configuration description spark.sql.planner.sortMergeJoin spark.sql.codegen spark.sql.caseSensitive spark.sql.dialect You can merge this pull request into a Git repository by running: $ git pull https://github.com/chenghao-intel/spark sqldoc Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/8917.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #8917 commit af702e9a9fd62be390d5fe381e6245f499050a4d Author: Cheng Hao Date: 2015-09-25T08:28:28Z add more configuration description --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10831][SQL][DOC]Add more configuration ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8917#issuecomment-143160497 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10831][SQL][DOC]Add more configuration ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8917#issuecomment-143160477 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10822] Move contents of spark-unsafe su...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8912#issuecomment-143161069 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10822] Move contents of spark-unsafe su...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8912#issuecomment-143161232 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10831][SQL][DOC]Add more configuration ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8917#issuecomment-143161249 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10831][SQL][DOC]Add more configuration ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8917#issuecomment-143161230 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10822] Move contents of spark-unsafe su...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8912#issuecomment-143161248 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10831][SQL][DOC]Add more configuration ...
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/8917#discussion_r40409790 --- Diff: docs/sql-programming-guide.md --- @@ -1320,6 +1320,39 @@ Configuration of Parquet can be done using the `setConf` method on `SQLContext` Property NameDefaultMeaning + spark.sql.planner.sortMergeJoin + true + +When true, use sort merge join (as opposed to hash join) by default for large joins. + + + + spark.sql.codegen + true + +When true, code will be dynamically generated at runtime for expression evaluation in a specific query. + + + + spark.sql.caseSensitive + false or true --- End diff -- Nit, but this doesn't say much anyway -- "depends"? you describe it below. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10831][SQL][DOC]Add more configuration ...
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/8917#discussion_r40409805 --- Diff: docs/sql-programming-guide.md --- @@ -1320,6 +1320,39 @@ Configuration of Parquet can be done using the `setConf` method on `SQLContext` Property NameDefaultMeaning + spark.sql.planner.sortMergeJoin + true + +When true, use sort merge join (as opposed to hash join) by default for large joins. + + + + spark.sql.codegen + true + +When true, code will be dynamically generated at runtime for expression evaluation in a specific query. + + + + spark.sql.caseSensitive + false or true + +Whether the query analyzer should be case sensitive or not. + For HiveContext, the default value is false; + For SQLContext, the default value is true; + + + + spark.sql.dialect + sql or hive --- End diff -- Quote or `` these values to make it clear? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10831][SQL][DOC]Add more configuration ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8917#issuecomment-143161870 [Test build #43017 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43017/consoleFull) for PR 8917 at commit [`82ec136`](https://github.com/apache/spark/commit/82ec1363ddfe43a8eefee9b32c9eb07056f229a1). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10822] Move contents of spark-unsafe su...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8912#issuecomment-143162410 [Test build #43018 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43018/consoleFull) for PR 8912 at commit [`faca1ff`](https://github.com/apache/spark/commit/faca1ff0853ed6c63370f1fe2b4753276c2bb42c). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10831][SQL][DOC]Add more configuration ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8917#issuecomment-143162871 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10831][SQL][DOC]Add more configuration ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8917#issuecomment-143162873 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43016/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10831][SQL][DOC]Add more configuration ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8917#issuecomment-143164152 [Test build #43017 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43017/console) for PR 8917 at commit [`82ec136`](https://github.com/apache/spark/commit/82ec1363ddfe43a8eefee9b32c9eb07056f229a1). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10831][SQL][DOC]Add more configuration ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8917#issuecomment-143164252 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10831][SQL][DOC]Add more configuration ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8917#issuecomment-143164253 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43017/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10778][MLlib]Implement toString for Ass...
Github user y-shimizu commented on the pull request: https://github.com/apache/spark/pull/8904#issuecomment-143164580 @mengxr I fixed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10427][SQL]Respect -S option for spark-...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8915#issuecomment-143166607 [Test build #43014 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43014/console) for PR 8915 at commit [`4596424`](https://github.com/apache/spark/commit/4596424b3c4a92a784918884fdb5d8e148f2c9c6). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10427][SQL]Respect -S option for spark-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8915#issuecomment-143166731 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10427][SQL]Respect -S option for spark-...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8915#issuecomment-143166732 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43014/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10766][SPARK-SUBMIT]Add some configurat...
GitHub user SaintBacchus opened a pull request: https://github.com/apache/spark/pull/8918 [SPARK-10766][SPARK-SUBMIT]Add some configuration for the client process in cluster mode. Add this four configurations for the client only in cluster mode: * `spark.client.memory` of property and `--client-memory` of cli command * `spark.client.extraClassPath` of property and `--client-class-path` of cli command * `spark.client.extraJavaOptions` of property and `--client-java-options` of cli command * `spark.client.extraLibraryPath` of property and `--client-library-path` of cli command You can merge this pull request into a Git repository by running: $ git pull https://github.com/SaintBacchus/spark SPARK-10766 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/8918.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #8918 commit 94a707ae2fcb4d41718e97160ec905876f716193 Author: huangzhaowei Date: 2015-09-25T09:21:21Z [SPARK-10766][SPARK-SUBMIT]Add some configuration for the client process in cluster mode. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10766][SPARK-SUBMIT]Add some configurat...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8918#issuecomment-143170043 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10766][SPARK-SUBMIT]Add some configurat...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8918#issuecomment-143170062 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10766][SPARK-SUBMIT]Add some configurat...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8918#issuecomment-143170183 [Test build #43019 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43019/consoleFull) for PR 8918 at commit [`94a707a`](https://github.com/apache/spark/commit/94a707ae2fcb4d41718e97160ec905876f716193). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10755][YARN]Set driver also update the ...
Github user SaintBacchus commented on the pull request:
https://github.com/apache/spark/pull/8867#issuecomment-143173483
@tgravescs I had noticed the code of
`UserGroupInformation.loginUserFromKeytab(args.principal, args.keytab)`
After this login motivation, `yarn.Client` will change *KERBEROS* into
*TOKEN* for the purpose of setting the token for the AM.
```scala
/** Set up security tokens for launching our ApplicationMaster container.
*/
private def setupSecurityToken(amContainer: ContainerLaunchContext): Unit
= {
val dob = new DataOutputBuffer
credentials.writeTokenStorageToStream(dob)
amContainer.setTokens(ByteBuffer.wrap(dob.getData))
}
```
After this, the `Client` will use *TOKEN* in the RPC connection.
If I login again with keytab after this, the SAALClient will use *KERBEROS*
again, and this can avoid token expired exception.
I had tested the recent spark and it still will throw this exception.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9182][SQL] Cast filters are not passed ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8718#issuecomment-143174139 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9182][SQL] Cast filters are not passed ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8718#issuecomment-143174116 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10778][MLlib]Implement toString for Ass...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8904#issuecomment-143174303 [Test build #1815 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1815/consoleFull) for PR 8904 at commit [`1f9ce43`](https://github.com/apache/spark/commit/1f9ce435f44a372d8828490eca00f095b9bbfbec). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10778][MLlib]Implement toString for Ass...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8904#issuecomment-143174654 [Test build #1815 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1815/console) for PR 8904 at commit [`1f9ce43`](https://github.com/apache/spark/commit/1f9ce435f44a372d8828490eca00f095b9bbfbec). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10778][MLlib]Implement toString for Ass...
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/8904#issuecomment-143175073 We still need a return type on the method, and the line is too long and needs to be wrapped --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9182][SQL] Cast filters are not passed ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8718#issuecomment-143175194 [Test build #43020 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43020/consoleFull) for PR 8718 at commit [`e1049e7`](https://github.com/apache/spark/commit/e1049e7a686083f6d96f1fa02f254b263d154400). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9182][SQL] Cast filters are not passed ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8718#issuecomment-143178048 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8884#issuecomment-143178133 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9182][SQL] Cast filters are not passed ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8718#issuecomment-143178132 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8884#issuecomment-143178038 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8884#issuecomment-143179460 [Test build #43021 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43021/consoleFull) for PR 8884 at commit [`98be936`](https://github.com/apache/spark/commit/98be9360c5daf9df8b841d290f1f3e899a9a7617). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9182][SQL] Cast filters are not passed ...
Github user HyukjinKwon commented on the pull request: https://github.com/apache/spark/pull/8718#issuecomment-143179571 In this commit, it only deals with numbers. I removed the roundtrip in cast and It only supports comparisons among other numeric types except `Decimal` and between `Decimal` and `Decimal` Otherwise it considers incompatible. e.g. (between `Decimal` and other types except `Decimal`). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9182][SQL] Cast filters are not passed ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8718#issuecomment-143180019 [Test build #43022 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43022/consoleFull) for PR 8718 at commit [`ce96bdd`](https://github.com/apache/spark/commit/ce96bdde00082233b2a88ae8eb0fc26682db55e9). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9182][SQL] Cast filters are not passed ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8718#issuecomment-143182661 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9182][SQL] Cast filters are not passed ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8718#issuecomment-143182633 [Test build #43022 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43022/console) for PR 8718 at commit [`ce96bdd`](https://github.com/apache/spark/commit/ce96bdde00082233b2a88ae8eb0fc26682db55e9). * This patch **fails MiMa tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9182][SQL] Cast filters are not passed ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8718#issuecomment-143182663 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43022/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8884#issuecomment-143185562 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43021/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8884#issuecomment-143185558 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8884#issuecomment-143185352 [Test build #43021 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43021/console) for PR 8884 at commit [`98be936`](https://github.com/apache/spark/commit/98be9360c5daf9df8b841d290f1f3e899a9a7617). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10822] Move contents of spark-unsafe su...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8912#issuecomment-143190324 [Test build #43018 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43018/console) for PR 8912 at commit [`faca1ff`](https://github.com/apache/spark/commit/faca1ff0853ed6c63370f1fe2b4753276c2bb42c). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10822] Move contents of spark-unsafe su...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8912#issuecomment-143190405 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10822] Move contents of spark-unsafe su...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8912#issuecomment-143190406 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43018/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10766][SPARK-SUBMIT]Add some configurat...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8918#issuecomment-143192934 [Test build #43019 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43019/console) for PR 8918 at commit [`94a707a`](https://github.com/apache/spark/commit/94a707ae2fcb4d41718e97160ec905876f716193). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10766][SPARK-SUBMIT]Add some configurat...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8918#issuecomment-143192975 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10766][SPARK-SUBMIT]Add some configurat...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8918#issuecomment-143192978 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43019/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9182][SQL] Cast filters are not passed ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8718#issuecomment-143194884 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9182][SQL] Cast filters are not passed ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8718#issuecomment-143194792 [Test build #43020 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43020/console) for PR 8718 at commit [`e1049e7`](https://github.com/apache/spark/commit/e1049e7a686083f6d96f1fa02f254b263d154400). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9182][SQL] Cast filters are not passed ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8718#issuecomment-143194885 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43020/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10833] [BUILD] [WIP] Inline, organize B...
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/8919 [SPARK-10833] [BUILD] [WIP] Inline, organize BSD/MIT licenses in LICENSE *Work in progress -- don't merge yet* In the course of https://issues.apache.org/jira/browse/LEGAL-226 it came to light that the guidance at http://www.apache.org/dev/licensing-howto.html#permissive-deps means that permissively-licensed dependencies has a different interpretation than we (er, I) had been operating under. "pointer ... to the license within the source tree" specifically means a copy of the license within Spark's distribution, whereas at the moment, Spark's LICENSE has a pointer to the project's license in the other project's source tree. The remedy is simply to inline all such license references (i.e. BSD/MIT licenses) or include their text in "licenses" subdirectory and point to that. Along the way, we can also treat other BSD/MIT licenses, whose text has been inlined into LICENSE, in the same way. The LICENSE file can continue to provide a helpful list of BSD/MIT licensed projects and a pointer to their sites. This would be over and above including license text in the distro, which is the essential thing. You can merge this pull request into a Git repository by running: $ git pull https://github.com/srowen/spark SPARK-10833 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/8919.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #8919 commit b271388385c5724e7daa6c0e70a10102ce85e737 Author: Sean Owen Date: 2015-09-25T13:33:31Z First step in BSD/MIT license update: move inlined BSD/MIT licenses from LICENSE to a licenses/LICENSE-*.txt file --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10833] [BUILD] [WIP] Inline, organize B...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8919#issuecomment-143226332 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10833] [BUILD] [WIP] Inline, organize B...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8919#issuecomment-143226362 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10833] [BUILD] [WIP] Inline, organize B...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8919#issuecomment-143228111 [Test build #43023 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43023/consoleFull) for PR 8919 at commit [`b271388`](https://github.com/apache/spark/commit/b271388385c5724e7daa6c0e70a10102ce85e737). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10833] [BUILD] [WIP] Inline, organize B...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8919#issuecomment-143231258 [Test build #43023 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43023/console) for PR 8919 at commit [`b271388`](https://github.com/apache/spark/commit/b271388385c5724e7daa6c0e70a10102ce85e737). * This patch **fails MiMa tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10833] [BUILD] [WIP] Inline, organize B...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8919#issuecomment-143231292 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10833] [BUILD] [WIP] Inline, organize B...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8919#issuecomment-143231295 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43023/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10831][SQL][DOC]Add more configuration ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8917#issuecomment-143240216 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10831][SQL][DOC]Add more configuration ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8917#issuecomment-143240249 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10831][SQL][DOC]Add more configuration ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8917#issuecomment-143241460 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10831][SQL][DOC]Add more configuration ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8917#issuecomment-143241488 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
