[GitHub] spark pull request: [SPARK-5161] [HOTFIX] Fix bug in Python test f...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7112#issuecomment-116992134 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8236][SQL] misc functions: crc32
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7108#issuecomment-116991278 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8236][SQL] misc functions: crc32
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7108#issuecomment-116991275 [Test build #36095 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36095/console) for PR 7108 at commit [`0d00660`](https://github.com/apache/spark/commit/0d006601fe24589641bc9aaf822a0bbf1c1e8044). * This patch **fails to build**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class Crc32(child: Expression)` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8687][YARN]Fix bug: Executor can't fetc...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7066#issuecomment-116990929 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8687][YARN]Fix bug: Executor can't fetc...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7066#issuecomment-116990668 [Test build #36078 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36078/console) for PR 7066 at commit [`1de4f48`](https://github.com/apache/spark/commit/1de4f4831df34d1127f14fc69f8dd23c64ffb874). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8236][SQL] misc functions: crc32
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7108#issuecomment-116989966 [Test build #36095 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36095/consoleFull) for PR 7108 at commit [`0d00660`](https://github.com/apache/spark/commit/0d006601fe24589641bc9aaf822a0bbf1c1e8044). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2645] [Core] Allow SparkEnv.stop() to b...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6973#issuecomment-116989753 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2645] [Core] Allow SparkEnv.stop() to b...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6973#issuecomment-116989357 [Test build #36077 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36077/console) for PR 6973 at commit [`446b0a4`](https://github.com/apache/spark/commit/446b0a4206f07525b726413444b6354dca55fea3). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8236][SQL] misc functions: crc32
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7108#issuecomment-116987974 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8236][SQL] misc functions: crc32
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7108#issuecomment-116988040 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8236][SQL] misc functions: crc32
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7108#issuecomment-116986352 Jenkins, this is ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8199][SPARK-8184][SPARK-8183][SPARK-818...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6981#issuecomment-116983650 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8199][SPARK-8184][SPARK-8183][SPARK-818...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6981#issuecomment-116983488 [Test build #36083 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36083/console) for PR 6981 at commit [`f120415`](https://github.com/apache/spark/commit/f1204154aaa96d93fd9ef723a91b7a91ba1d3773). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `abstract class TimeFormatExpression extends UnaryExpression with ExpectsInputTypes ` * `case class Hour(child: Expression) extends TimeFormatExpression ` * `case class Minute(child: Expression) extends TimeFormatExpression ` * `case class Second(child: Expression) extends TimeFormatExpression ` * `abstract class DateFormatExpression extends UnaryExpression with ExpectsInputTypes ` * `case class Year(child: Expression) extends DateFormatExpression ` * `case class Quarter(child: Expression) extends DateFormatExpression ` * `case class Month(child: Expression) extends DateFormatExpression ` * `case class Day(child: Expression) extends UnaryExpression with ExpectsInputTypes ` * `case class WeekOfYear(child: Expression) extends UnaryExpression with ExpectsInputTypes ` * `case class DateFormatClass(left: Expression, right: Expression) extends BinaryExpression` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5989] [MLlib] Model save/load for LDA
Github user hhbyyh commented on the pull request: https://github.com/apache/spark/pull/6948#issuecomment-116983018 @MechCoder. I'm OK with the localLDAModel. And for the distributed one, we need Joseph's opinions. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-7514][MLlib] Add MinMaxScaler to featur...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6039#issuecomment-116983289 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-8703] [ML] Add CountVectorizer as a ml ...
Github user feynmanliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/7084#discussion_r33542097
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/CountVectorizer.scala ---
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.ml.feature
+
+import scala.collection.mutable
+
+import org.apache.spark.annotation.Experimental
+import org.apache.spark.ml.UnaryTransformer
+import org.apache.spark.ml.param._
+import org.apache.spark.ml.util.Identifiable
+import org.apache.spark.mllib.linalg.{Vectors, VectorUDT, Vector}
+import org.apache.spark.sql.types.{StringType, ArrayType, DataType}
+
+/**
+ * :: Experimental ::
+ * Converts a text document to a sparse vector of token counts.
+ * @param vocabulary An Array over terms. Only the terms in the vocabulary
will be counted.
+ */
+@Experimental
+class CountVectorizer (override val uid: String, vocabulary: Array[String])
+ extends UnaryTransformer[Seq[String], Vector, CountVectorizer] {
+
+ def this(vocabulary: Array[String]) =
this(Identifiable.randomUID("countVectorizer"), vocabulary)
+
+ /**
+ * Corpus-specific stop words filter. Terms with count less than the
given threshold are ignored.
+ * Default: 1
+ * @group param
+ */
+ val minTermCounts: IntParam = new IntParam(this, "minTermCounts",
+"lower bound of effective term counts (>= 0)", ParamValidators.gtEq(1))
+
+ /** @group setParam */
+ def setMinTermCounts(value: Int): this.type = set(minTermCounts, value)
+
+ /** @group getParam */
+ def getMinTermCounts: Int = $(minTermCounts)
+
+ setDefault(minTermCounts -> 1)
+
+ override protected def createTransformFunc: Seq[String] => Vector = {
+val dict = vocabulary.zipWithIndex.toMap
+document =>
+ val termCounts = mutable.HashMap.empty[Int, Double]
+ document.foreach { term =>
+val index = dict.getOrElse(term, -1)
+if (index >= 0) {
+ termCounts.put(index, termCounts.getOrElse(index, 0.0) + 1.0)
+}
+ }
+ Vectors.sparse(dict.size, termCounts.filter(_._2 >=
$(minTermCounts)).toSeq)
--- End diff --
Good point, the trailing zeros should be pretty cheap with a sparse Vector
anyways so this seems fine to me.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-7514][MLlib] Add MinMaxScaler to featur...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6039#issuecomment-116982739 [Test build #36092 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36092/console) for PR 6039 at commit [`f942e9f`](https://github.com/apache/spark/commit/f942e9fd1d84fe754aeb66b1c0b7c585952f60de). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class MinMaxScaler(override val uid: String)` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [WIP] Binary processing sort for Spark SQL
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/6444#issuecomment-116981153 In order to address the object spilling case, I'm going to need to write a failing test case first. In order to do this, I'm going to have to borrow some of the manual spill triggering test interfaces that I added for UnsafeShuffle so that I can write deterministic unit tests which hit the spill paths with small inputs. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [HOTFIX] Fix bug in Python test f...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7112#issuecomment-116980291 [Test build #36094 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36094/consoleFull) for PR 7112 at commit [`c3f2961`](https://github.com/apache/spark/commit/c3f2961fe551405e7ddc3ac9e82fd8236a2fad92). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [HOTFIX] Fix bug in Python test f...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7112#issuecomment-116978452 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [HOTFIX] Fix bug in Python test f...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7112#issuecomment-116978406 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8711] [ML] Add additional methods to Py...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7095#issuecomment-116977943 I've opened #7112 to hotfix the build issue which masked the test failure here. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8236][SQL] misc functions: crc32
Github user qiansl127 commented on the pull request: https://github.com/apache/spark/pull/7108#issuecomment-116978037 @davies Hi, could your help to check this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [HOTFIX] Fix bug in Python test f...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7112#issuecomment-116977633 To test this, try manually introducing a failure in pyspark.conf's doctests. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [HOTFIX] Fix bug in Python test f...
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/7112 [SPARK-5161] [HOTFIX] Fix bug in Python test failure reporting This patch fixes a bug introduced in #7031 which can cause Jenkins to incorrectly report a build with failed Python tests as passing if an error occurred while printing the test failure message. You can merge this pull request into a Git repository by running: $ git pull https://github.com/JoshRosen/spark python-tests-hotfix Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/7112.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #7112 commit c3f2961fe551405e7ddc3ac9e82fd8236a2fad92 Author: Josh Rosen Date: 2015-06-30T05:36:12Z Hotfix for bug in Python test failure reporting --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8650][SQL] Use the user-specified app n...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/7030 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8018][MLlib]KMeans should accept initia...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6737#issuecomment-116974163 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8018][MLlib]KMeans should accept initia...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6737#issuecomment-116973799 [Test build #36086 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36086/console) for PR 6737 at commit [`e721dfe`](https://github.com/apache/spark/commit/e721dfe2b94131db4f7f99c10ff961759107431e). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-7514][MLlib] Add MinMaxScaler to featur...
Github user hhbyyh commented on the pull request: https://github.com/apache/spark/pull/6039#issuecomment-116973685 @jkbradley Thanks. A todo item is added. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8650][SQL] Use the user-specified app n...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/7030#issuecomment-116973361 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8721][SQL] Rename ExpectsInputTypes => ...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/7109 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8705][WebUI]Don't display rects when to...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7088#issuecomment-116971300 [Test build #36093 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36093/consoleFull) for PR 7088 at commit [`9ee4ef5`](https://github.com/apache/spark/commit/9ee4ef56a3721162c579ffa0012eed74fea9f6b5). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8705][WebUI]Don't display rects when to...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7088#issuecomment-116969987 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8705][WebUI]Don't display rects when to...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7088#issuecomment-116970024 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8705][WebUI]Don't display rects when to...
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/7088#issuecomment-116970124 Addressed comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8705][WebUI]Don't display rects when to...
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/7088#discussion_r33541345
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala ---
@@ -570,6 +570,35 @@ private[ui] class StagePage(parent: StagesTab) extends
WebUIPage("stage") {
val index = taskInfo.index
val attempt = taskInfo.attempt
+
+val svgTag =
+ if (totalExecutionTime == 0) {
+""
+ }
+ else {
+ s"""
+ |
--- End diff --
Good catch
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8688][YARN]Bug fix: disable the cache f...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7069#issuecomment-116969600 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8688][YARN]Bug fix: disable the cache f...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7069#issuecomment-116969379 [Test build #36074 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36074/console) for PR 7069 at commit [`8fb9eb9`](https://github.com/apache/spark/commit/8fb9eb908bc4db5a43447a4743822fce2296df24). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8650][SQL] Use the user-specified app n...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7030#issuecomment-116968509 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8679] [PySpark] [MLlib] Default values ...
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7058#issuecomment-116968393 Hmm. Scikit-learn has a `copy` argument in most of the model constructors to avoid this. Imagine a person feeding huge amounts of data to find all of them overwritten. But that's lesser of a concern here because the input data is mostly in the form of RDD's and DataFrames. We can always fix it when it is reported as broke :P --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8650][SQL] Use the user-specified app n...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7030#issuecomment-116968397 [Test build #36088 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36088/console) for PR 7030 at commit [`51b5134`](https://github.com/apache/spark/commit/51b513430ecaaf2fada8a16f6d205a8b82693988). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8711] [ML] Add additional methods to Py...
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/7095#issuecomment-116968184
Uh oh: looks like this is actually failing some tests:
```
Exception in thread Thread-1:
Traceback (most recent call last):
File "/home/anaconda/envs/py3k/lib/python3.4/threading.py", line 920, in
_bootstrap_inner
self.run()
File "/home/anaconda/envs/py3k/lib/python3.4/threading.py", line 868, in
run
self._target(*self._args, **self._kwargs)
File "./python/run-tests.py", line 166, in process_queue
run_individual_python_test(test_goal, python_exec)
File "./python/run-tests.py", line 74, in run_individual_python_test
if not re.match('[0-9]+', line):
File "/home/anaconda/envs/py3k/lib/python3.4/re.py", line 160, in match
return _compile(pattern, flags).match(string)
TypeError: can't use a string pattern on a bytes-like object
Finished test(python3.4): pyspark.ml.recommendation (15s)
Finished test(python3.4): pyspark.ml.feature (18s)
Exception in thread Thread-3:
Traceback (most recent call last):
File "/home/anaconda/envs/py3k/lib/python3.4/threading.py", line 920, in
_bootstrap_inner
self.run()
File "/home/anaconda/envs/py3k/lib/python3.4/threading.py", line 868, in
run
self._target(*self._args, **self._kwargs)
File "./python/run-tests.py", line 166, in process_queue
run_individual_python_test(test_goal, python_exec)
File "./python/run-tests.py", line 74, in run_individual_python_test
if not re.match('[0-9]+', line):
File "/home/anaconda/envs/py3k/lib/python3.4/re.py", line 160, in match
return _compile(pattern, flags).match(string)
TypeError: can't use a string pattern on a bytes-like object
```
This is the consequence of not having good test coverage on the test
script... I'll fix this shortly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7716] [UI] Stage page hangs with many t...
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/6248#issuecomment-116967596 @andrewor14 could you reopen this one? GZip is better when accessing a remote node. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8199][SPARK-8184][SPARK-8183][SPARK-818...
Github user tarekauel commented on a diff in the pull request:
https://github.com/apache/spark/pull/6981#discussion_r33540988
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetime.scala
---
@@ -0,0 +1,387 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions
+
+import java.sql.Date
+import java.text.SimpleDateFormat
+import java.util.{Calendar, TimeZone}
+import
org.apache.spark.sql.catalyst.expressions.codegen.{GeneratedExpressionCode,
CodeGenContext}
+import org.apache.spark.sql.types._
+import org.apache.spark.unsafe.types.UTF8String
+
+abstract class TimeFormatExpression extends UnaryExpression with
ExpectsInputTypes {
+ self: Product =>
+
+ protected val format: Int
+
+ protected val cntPerInterval: Int
+
+ override def expectedChildTypes: Seq[DataType] = Seq(TimestampType)
+
+ override def eval(input: InternalRow): Any = {
+val valueLeft = child.eval(input)
+if (valueLeft == null) {
+ null
+} else {
+ val time = valueLeft.asInstanceOf[Long] / 1
+ val utcTime: Long = time + TimeZone.getDefault.getOffset(time)
+ ((utcTime / format) % cntPerInterval).toInt
+}
+ }
+
+ override def genCode(
+ ctx: CodeGenContext,
+ ev: GeneratedExpressionCode): String = {
+
+val tz = classOf[TimeZone].getName
+
+defineCodeGen(ctx, ev, (c) =>
+ s"""(${ctx.javaType(dataType)})
+((($c / 1) + $tz.getDefault().getOffset($c / 1)) /
$format % $cntPerInterval)"""
+)
+ }
+}
+
+case class Hour(child: Expression) extends TimeFormatExpression {
+
+ override protected val format: Int = 1000 * 3600
+
+ override protected val cntPerInterval: Int = 24
+
+ override def dataType: DataType = IntegerType
+
+ override def toString: String = s"Hour($child)"
+}
+
+case class Minute(child: Expression) extends TimeFormatExpression {
+
+ override protected val format: Int = 1000 * 60
+
+ override protected val cntPerInterval: Int = 60
+
+ override def dataType: DataType = IntegerType
+
+ override def toString: String = s"Minute($child)"
+}
+
+case class Second(child: Expression) extends TimeFormatExpression {
+
+ override protected val format: Int = 1000
+
+ override protected val cntPerInterval: Int = 60
+
+ override def dataType: DataType = IntegerType
+
+ override def toString: String = s"Second($child)"
+}
+
+abstract class DateFormatExpression extends UnaryExpression with
ExpectsInputTypes {
+ self: Product =>
+
+ override def dataType: DataType = IntegerType
+
+ override def expectedChildTypes: Seq[DataType] = Seq(TimestampType)
+
+ protected def defineCodeGen(
+ctx: CodeGenContext,
+ev: GeneratedExpressionCode,
+f: (String, String) => String): String = {
+
+val tz = classOf[TimeZone].getName
+
+val utcTime = ctx.freshName("utcTime")
+val dayInYear = ctx.freshName("dayInYear")
+val days = ctx.freshName("days")
+val year = ctx.freshName("year")
+
+val eval = child.gen(ctx)
+ev.isNull = eval.isNull
+eval.code + s"""
+ ${ctx.javaType(dataType)} ${ev.primitive} =
${ctx.defaultValue(dataType)};
+ if (!${ev.isNull}) {
+long $utcTime = ${eval.primitive} / 1;
+long $days = $utcTime / 1000 / 3600 / 24;
+int $year = (int) ($days / 365.24);
+int $dayInYear = (int) ($days - $year * 365.24);
+${f(dayInYear, utcTime)}
+ }
+"""
+ }
+
+ def eval(input: InternalRow, f: (Int, Long) => Int): Any = {
+val valueLeft = child.eval(input)
+if (valueLeft == null) {
+ null
+} else {
+ val utcTime: Long = valueLeft.asInstanceOf[Long] / 1
+ val
[GitHub] spark pull request: [SPARK-8711] [ML] Add additional methods to Py...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7095#issuecomment-116967343 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [WIP] Binary processing sort for Spark SQL
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/6444#issuecomment-116967403 I'm in the process of updating this to reflect @davies' recent changes to support arbitrary column types in UnsafeRow. One bit of trickiness is the fact that UnsafeExternalSorter, which manages the spilling, doesn't know anything about SQL and thus can't directly implement the logic for spilling and reading ObjectPools back during the merge. In the worst case, I guess that the merging step might need to keep a bunch of ObjectPools in memory at the same time, which could end up causing us to OOM, but in those cases we would have OOM'd before, too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8711] [ML] Add additional methods to Py...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7095#issuecomment-116967198 [Test build #36090 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36090/console) for PR 7095 at commit [`2fde31a`](https://github.com/apache/spark/commit/2fde31a112891a482e333d7c4e859a97074eb8f0). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class DecisionTreeClassificationModel(DecisionTreeModel):` * `class RandomForestClassificationModel(TreeEnsembleModels):` * `class GBTClassificationModel(TreeEnsembleModels):` * `class DecisionTreeModel(JavaModel):` * `class TreeEnsembleModels(JavaModel):` * `class DecisionTreeRegressionModel(DecisionTreeModel):` * `class RandomForestRegressionModel(TreeEnsembleModels):` * `class GBTRegressionModel(TreeEnsembleModels):` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8199][SPARK-8184][SPARK-8183][SPARK-818...
Github user tarekauel commented on a diff in the pull request:
https://github.com/apache/spark/pull/6981#discussion_r33540955
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetime.scala
---
@@ -0,0 +1,387 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions
+
+import java.sql.Date
+import java.text.SimpleDateFormat
+import java.util.{Calendar, TimeZone}
+import
org.apache.spark.sql.catalyst.expressions.codegen.{GeneratedExpressionCode,
CodeGenContext}
+import org.apache.spark.sql.types._
+import org.apache.spark.unsafe.types.UTF8String
+
+abstract class TimeFormatExpression extends UnaryExpression with
ExpectsInputTypes {
--- End diff --
Can I make this class to a private class, because it shouldn't be used
somewhere else?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8450][SQL]If column type is BigDecimal,...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7106#issuecomment-116967123 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8691][WebUI]Enable GZip by default for ...
Github user zsxwing closed the pull request at: https://github.com/apache/spark/pull/7072 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8691][WebUI]Enable GZip by default for ...
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/7072#discussion_r33540925
--- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala ---
@@ -166,8 +167,26 @@ private[spark] object JettyUtils extends Logging {
contextHandler
}
+ private def addDefaultFilters(handlers: Seq[ServletContextHandler],
conf: SparkConf): Unit = {
+val compression = conf.getBoolean("spark.ui.compression", true)
--- End diff --
Since Spark supports the user to add custom Filters, I'm concerned that
they may add a Filter that conflicts with `GZipFilter`. So I add this
configuration to disable `GZipFilter`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8691][WebUI]Enable GZip by default for ...
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/7072#issuecomment-116967047 I think @andrewor14 's PR is much easier. I'm going to close this one. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8199][SPARK-8184][SPARK-8183][SPARK-818...
Github user tarekauel commented on the pull request: https://github.com/apache/spark/pull/6981#issuecomment-116967031 @davies I improved the implementation. The time functions create still a `Date`object, as part of `getOffSet`. But It's necessary to convert the time, in order to get the correct result for `Minute` and `Hour`. It isn't necessary for `Seconds` but I guess it's simpler, if it's implemented the same way. The range for the fallback is two days big. This allows to ignore the time difference. `Day` and `WeekOfYear` uses always `Calendar`. In order to try to guarantee the functionality, I added some more tests. @rxin I am going to close it for now, please keep me in the loop. You can assign the Jira tickets to me, if it helps. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8450][SQL]If column type is BigDecimal,...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7106#issuecomment-116967111 **[Test build #36068 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36068/console)** for PR 7106 at commit [`1db8482`](https://github.com/apache/spark/commit/1db8482c6fd88f125821a5b1a89f01301543a804) after a configured wait of `175m`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8721][SQL] Rename ExpectsInputTypes => ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7109#issuecomment-116966520 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8199][SPARK-8184][SPARK-8183][SPARK-818...
Github user tarekauel closed the pull request at: https://github.com/apache/spark/pull/6981 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8721][SQL] Rename ExpectsInputTypes => ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7109#issuecomment-116966283 [Test build #36076 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36076/console) for PR 7109 at commit [`a914cc3`](https://github.com/apache/spark/commit/a914cc36d39207c2730aafb5708c3feb0cffc3df). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8434][SQL]Add a "pretty" parameter to t...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6877#issuecomment-116965385 lgtm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-7514][MLlib] Add MinMaxScaler to featur...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6039#issuecomment-116963043 [Test build #36092 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36092/consoleFull) for PR 6039 at commit [`f942e9f`](https://github.com/apache/spark/commit/f942e9fd1d84fe754aeb66b1c0b7c585952f60de). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8711] [ML] Add additional methods to Py...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7095#issuecomment-116962945 [Test build #36090 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36090/consoleFull) for PR 7095 at commit [`2fde31a`](https://github.com/apache/spark/commit/2fde31a112891a482e333d7c4e859a97074eb8f0). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8619][Streaming]Don't recover keytab an...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7008#issuecomment-116962865 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8619][Streaming]Don't recover keytab an...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7008#issuecomment-116962590 [Test build #36080 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36080/console) for PR 7008 at commit [`d50dbdf`](https://github.com/apache/spark/commit/d50dbdf81210768ac9cb0ae635b583adc315125e). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8434][SQL]Add a "pretty" parameter to t...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6877#issuecomment-116962100 [Test build #36091 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36091/consoleFull) for PR 6877 at commit [`22e28e9`](https://github.com/apache/spark/commit/22e28e996dfe004c6125b9318c84973b7c2ac7db). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8711] [ML] Add additional methods to Py...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7095#issuecomment-116961824 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8434][SQL]Add a "pretty" parameter to t...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6877#issuecomment-116961776 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-7514][MLlib] Add MinMaxScaler to featur...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6039#issuecomment-116961864 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8434][SQL]Add a "pretty" parameter to t...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6877#issuecomment-116961862 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8711] [ML] Add additional methods to Py...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7095#issuecomment-116961765 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-7514][MLlib] Add MinMaxScaler to featur...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6039#issuecomment-116961793 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8711] [ML] Add additional methods to Py...
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7095#issuecomment-116960833 @feynmanliang Thanks. I have addressed your comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6548 Adding stddev to DataFrame function...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6297#issuecomment-116959293 [Test build #36089 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36089/console) for PR 6297 at commit [`e46c964`](https://github.com/apache/spark/commit/e46c9648aeede81680ca091ee6860ff7d9766cfa). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class Stddev(child: Expression) extends PartialAggregate with trees.UnaryNode[Expression] ` * `case class ComputePartialStd(child: Expression) extends AggregateExpression ` * `case class CombinePartialStd(child: Expression) extends AggregateExpression ` * `case class ComputePartialStdFunction (` * `case class CombinePartialStdFunction(` * `case class StddevFunction(` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6548 Adding stddev to DataFrame function...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6297#issuecomment-116959304 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8650][SQL] Use the user-specified app n...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7030#issuecomment-116957364 [Test build #36088 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36088/consoleFull) for PR 7030 at commit [`51b5134`](https://github.com/apache/spark/commit/51b513430ecaaf2fada8a16f6d205a8b82693988). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6548 Adding stddev to DataFrame function...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6297#issuecomment-116957360 [Test build #36089 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36089/consoleFull) for PR 6297 at commit [`e46c964`](https://github.com/apache/spark/commit/e46c9648aeede81680ca091ee6860ff7d9766cfa). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8700][ML] Disable feature scaling in Lo...
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/7080#issuecomment-116955865 Jenkins, please test again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8650][SQL] Use the user-specified app n...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7030#issuecomment-116955514 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8650][SQL] Use the user-specified app n...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7030#issuecomment-116955521 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6548 Adding stddev to DataFrame function...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6297#issuecomment-116955519 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6548 Adding stddev to DataFrame function...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6297#issuecomment-116955534 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4352][YARN][WIP] Incorporate locality p...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6394#issuecomment-116955463 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4352][YARN][WIP] Incorporate locality p...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6394#issuecomment-116955410 [Test build #36069 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36069/console) for PR 6394 at commit [`74f79c5`](https://github.com/apache/spark/commit/74f79c546225458ce3f03f507eccdc7523aac01e). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * ` case class RequestExecutors(` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6548 Adding stddev to DataFrame function...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/6297#issuecomment-116955337 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8650][SQL] Use the user-specified app n...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/7030#issuecomment-116955302 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6785][SQL] fix DateTimeUtils for dates ...
Github user ckadner commented on the pull request: https://github.com/apache/spark/pull/6983#issuecomment-116955196 @squito - thx 4 your comments! :+1: @marmbrus - back to you -- thx --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8378][Streaming]Add the Python API for ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6830#issuecomment-116955190 [Test build #36087 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36087/consoleFull) for PR 6830 at commit [`0449723`](https://github.com/apache/spark/commit/04497231b3d0c1022ece037efd48516ccc6d1995). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8700][ML] Disable feature scaling in Lo...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7080#issuecomment-116955166 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8378][Streaming]Add the Python API for ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6830#issuecomment-116955149 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8378][Streaming]Add the Python API for ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6830#issuecomment-116955138 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8590][SQL] add code gen for ExtractValu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6982#issuecomment-116955036 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8590][SQL] add code gen for ExtractValu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6982#issuecomment-116954993 [Test build #36073 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36073/console) for PR 6982 at commit [`b245524`](https://github.com/apache/spark/commit/b245524175dc09a768283005594f21c10ac89874). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class NGram(JavaTransformer, HasInputCol, HasOutputCol):` * `trait ExtractValue ` * `abstract class ExtractValueWithStruct extends UnaryExpression with ExtractValue ` * `abstract class ExtractValueWithOrdinal extends BinaryExpression with ExtractValue ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8434][SQL]Add a "pretty" parameter to t...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6877#issuecomment-116954954 That should be fine, since "pretty" is vague, and the most important factor here is truncation. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7041] Avoid writing empty files in Bypa...
Github user JoshRosen closed the pull request at: https://github.com/apache/spark/pull/5622 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Changes to support checkpointing to BlockRDD a...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5502 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SQL] SPARK-6548: Adding stddev to DataFrame f...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5357 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-8562: Log the lost of an executor only i...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/6952 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8723][SQL] improve divide and remainder...
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/7111#issuecomment-116954786 cc @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8018][MLlib]KMeans should accept initia...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6737#issuecomment-116954815 [Test build #36086 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36086/consoleFull) for PR 6737 at commit [`e721dfe`](https://github.com/apache/spark/commit/e721dfe2b94131db4f7f99c10ff961759107431e). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8323][CORE]Remove mapOutputTracker fiel...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/6778 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4069] [YARN] When AppMaster finishes, t...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5233 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8723][SQL] improve divide and remainder...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7111#issuecomment-116954865 [Test build #36085 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36085/consoleFull) for PR 7111 at commit [`d6b12ef`](https://github.com/apache/spark/commit/d6b12ef91fec295a93efc3009c5a73b25507a475). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
