[GitHub] spark pull request: [SPARK-9973][SQL]correct buffer size
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8189 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9950][SQL] Wrong Analysis Error for gro...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8222#issuecomment-131492051 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40980/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9950][SQL] Wrong Analysis Error for gro...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8222#issuecomment-131492050 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9950][SQL] Wrong Analysis Error for gro...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8222#issuecomment-131492033 [Test build #40980 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40980/console) for PR 8222 at commit [`0bbfb1c`](https://github.com/apache/spark/commit/0bbfb1c55a198f6466a973f501be89203e09ac6c). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5754] [yarn] Spark/Yarn/Windows driver/...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8053#issuecomment-131491808 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5754] [yarn] Spark/Yarn/Windows driver/...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8053#issuecomment-131491809 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40979/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5754] [yarn] Spark/Yarn/Windows driver/...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8053#issuecomment-131491793 **[Test build #40979 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40979/console)** for PR 8053 at commit [`5be3e44`](https://github.com/apache/spark/commit/5be3e449aa0306c41398408157a7db1cd94f1aa8) after a configured wait of `175m`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8920] [MLlib] Add @since tags to mllib....
Github user MechCoder commented on a diff in the pull request:
https://github.com/apache/spark/pull/7729#discussion_r37142154
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala
---
@@ -911,6 +971,7 @@ object Matrices {
* @param vector a `Vector` that will form the values on the diagonal of
the matrix
* @return Square `Matrix` with size `values.length` x `values.length`
and `values`
* on the diagonal
+ * @since 1.3.0
--- End diff --
Should this be 1.2.0?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8920] [MLlib] Add @since tags to mllib....
Github user MechCoder commented on a diff in the pull request:
https://github.com/apache/spark/pull/7729#discussion_r37142146
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala
---
@@ -854,6 +908,7 @@ object Matrices {
* Generate a dense Identity Matrix in `Matrix` format.
* @param n number of rows and columns of the matrix
* @return `Matrix` with size `n` x `n` and values of ones on the
diagonal
+ * @since 1.3.0
--- End diff --
Should this be 1.2.0?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8920] [MLlib] Add @since tags to mllib....
Github user MechCoder commented on a diff in the pull request:
https://github.com/apache/spark/pull/7729#discussion_r37142145
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala
---
@@ -847,6 +900,7 @@ object Matrices {
* @param numRows number of rows of the matrix
* @param numCols number of columns of the matrix
* @return `Matrix` with size `numRows` x `numCols` and values of ones
+ * @since 1.3.0
--- End diff --
Should this be 1.2.0?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8920] [MLlib] Add @since tags to mllib....
Github user MechCoder commented on a diff in the pull request:
https://github.com/apache/spark/pull/7729#discussion_r37142142
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala
---
@@ -839,6 +891,7 @@ object Matrices {
* @param numRows number of rows of the matrix
* @param numCols number of columns of the matrix
* @return `Matrix` with size `numRows` x `numCols` and values of zeros
+ * @since 1.3.0
--- End diff --
Should this be 1.2.0?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8920] [MLlib] Add @since tags to mllib....
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7729#discussion_r37142109 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala --- @@ -254,6 +255,7 @@ class DenseMatrix( * @param numRows number of rows * @param numCols number of columns * @param values matrix entries in column major + * @since 1.3.0 */ def this(numRows: Int, numCols: Int, values: Array[Double]) = --- End diff -- ping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9906] [ML] User guide for LogisticRegre...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8197#issuecomment-131491134 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9906] [ML] User guide for LogisticRegre...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8197#issuecomment-131491137 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40983/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9906] [ML] User guide for LogisticRegre...
Github user feynmanliang commented on the pull request: https://github.com/apache/spark/pull/8197#issuecomment-131491133 Lgtm pending tests @mengxr --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9906] [ML] User guide for LogisticRegre...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8197#issuecomment-131491136 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9906] [ML] User guide for LogisticRegre...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8197#issuecomment-131491132 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9906] [ML] User guide for LogisticRegre...
Github user feynmanliang commented on the pull request: https://github.com/apache/spark/pull/8197#issuecomment-131491123 Jenkins test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9906] [ML] User guide for LogisticRegre...
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/8197#issuecomment-131491076 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9906] [ML] User guide for LogisticRegre...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8197#issuecomment-131491056 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9906] [ML] User guide for LogisticRegre...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8197#issuecomment-131491057 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40982/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9906] [ML] User guide for LogisticRegre...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8197#issuecomment-131490932 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9906] [ML] User guide for LogisticRegre...
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/8197#issuecomment-131490936 @feynmanliang ok, I've addressed your comments, anything else? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9906] [ML] User guide for LogisticRegre...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8197#issuecomment-131490931 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9973][SQL]correct buffer size
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/8189#issuecomment-131490723 Thanks, I'm merging this to master. @rxin Is it OK to have this one in branch-1.5 at this time? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9906] [ML] User guide for LogisticRegre...
Github user MechCoder commented on a diff in the pull request:
https://github.com/apache/spark/pull/8197#discussion_r37141832
--- Diff: docs/ml-guide.md ---
@@ -801,6 +801,141 @@ jsc.stop();
+## Examples: Summaries for LogisticRegression.
+
+Once LogisticRegression is run on data, it is useful to extract statistics
such as the
+loss per iteration which will provide an intuition on overfitting and
metrics to understand
+how well the model has performed on training and test data.
+
+LogisticRegressionTrainingSummary provides an interface to access such
relevant information
+i.e the objectiveHistory and metrics to evaluate the performance on the
training data
+directly with very less code to be rewritten by the user.
+In the future, a method would be made available in the fitted
LogisticRegressionModel to obtain
+a LogisticRegressionSummary of the test data as well.
+
+This examples illustrates the use of LogisticRegressionTrainingSummary on
some toyData.
+
+
+
+{% highlight scala %}
+import org.apache.spark.{SparkConf, SparkContext}
+import org.apache.spark.ml.classification.{LogisticRegression,
BinaryLogisticRegressionSummary}
+import org.apache.spark.mllib.regression.LabeledPoint
+import org.apache.spark.mllib.linalg.Vectors
+import org.apache.spark.sql.{Row, SQLContext}
+
+val conf = new SparkConf().setAppName("LogisticRegressionSummary")
+val sc = new SparkContext(conf)
+val sqlContext = new SQLContext(sc)
+import sqlContext.implicits._
+
+// Use some random data for demonstration.
+// Note that the RDD of LabeledPoints can be converted to a dataframe
directly.
+val data = sc.parallelize(Array(
+ LabeledPoint(0.0, Vectors.dense(0.2, 4.5, 1.6)),
+ LabeledPoint(1.0, Vectors.dense(3.1, 6.8, 3.6)),
+ LabeledPoint(0.0, Vectors.dense(2.4, 0.9, 1.9)),
+ LabeledPoint(1.0, Vectors.dense(9.1, 3.1, 3.6)),
+ LabeledPoint(0.0, Vectors.dense(2.5, 1.9, 9.1)))
+)
+val logRegDataFrame = data.toDF()
+
+// Run Logistic Regression on your toy data.
+// Since LogisticRegression is an estimator, it returns an instance of
LogisticRegressionModel
+// which is a transformer.
+val logReg = new LogisticRegression()
+logReg.setMaxIter(5)
+logReg.setRegParam(0.01)
+val logRegModel = logReg.fit(logRegDataFrame)
+
+// Extract the summary directly from the returned LogisticRegressionModel
instance.
+val trainingSummary = logRegModel.summary
+
+// Obtain the loss per iteration. This should decrease upto a certain
point and
+// then increase or show negligible change after this.
+val objectiveHistory = trainingSummary.objectiveHistory
+objectiveHistory.foreach(loss => println(loss))
+
+// Obtain the metrics useful to judge performance on test data.
+val binarySummary =
trainingSummary.asInstanceOf[BinaryLogisticRegressionSummary]
+
+// Obtain the receiver-operating characteristic as a dataframe and
areaUnderROC.
+val roc = binarySummary.roc
+val truePositiveRate = roc.select("FPR")
+val area = binarySummary.areaUnderROC
+
+// Obtain the threshold with the highest fMeasure.
+val fMeasure = binarySummary.fMeasureByThreshold
+val fScoreRDD = fMeasure.map { case Row(thresh: Double, fscore: Double) =>
(thresh, fscore) }
+val (highThresh, highFScore) = fScoreRDD.fold((0.0, 0.0))((threshFScore1,
threshFScore2) => {
+ if (threshFScore1._2 > threshFScore2._2) threshFScore1 else threshFScore2
+})
--- End diff --
OK. I wouldn't complain because it makes my job easier.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10012][ML] Missing test case for Params...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8223#issuecomment-131489901 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10012][ML] Missing test case for Params...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8223#issuecomment-131489903 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40981/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10012][ML] Missing test case for Params...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8223#issuecomment-131489891 [Test build #40981 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40981/console) for PR 8223 at commit [`5da7c34`](https://github.com/apache/spark/commit/5da7c3493fb7c6e31f974c78a2226f5ee96bfeae). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10012][ML] Missing test case for Params...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8223#issuecomment-131487366 [Test build #40981 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40981/consoleFull) for PR 8223 at commit [`5da7c34`](https://github.com/apache/spark/commit/5da7c3493fb7c6e31f974c78a2226f5ee96bfeae). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9950][SQL] Wrong Analysis Error for gro...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8222#issuecomment-131486256 [Test build #40980 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40980/consoleFull) for PR 8222 at commit [`0bbfb1c`](https://github.com/apache/spark/commit/0bbfb1c55a198f6466a973f501be89203e09ac6c). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10012][ML] Missing test case for Params...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8223#issuecomment-131485801 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10012][ML] Missing test case for Params...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8223#issuecomment-131485769 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10012][ML] Missing test case for Params...
GitHub user Lewuathe opened a pull request: https://github.com/apache/spark/pull/8223 [SPARK-10012][ML] Missing test case for Params#arrayLengthGt Currently there is no test case for `Params#arrayLengthGt`. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Lewuathe/spark SPARK-10012 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/8223.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #8223 commit 5da7c3493fb7c6e31f974c78a2226f5ee96bfeae Author: lewuathe Date: 2015-08-16T03:21:52Z Missing test case for Params#arrayLengthGt --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9950][SQL] Wrong Analysis Error for gro...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8222#issuecomment-131484549 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9950][SQL] Wrong Analysis Error for gro...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8222#issuecomment-131484586 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5754] [yarn] Spark/Yarn/Windows driver/...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8053#issuecomment-131477425 [Test build #40979 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40979/consoleFull) for PR 8053 at commit [`5be3e44`](https://github.com/apache/spark/commit/5be3e449aa0306c41398408157a7db1cd94f1aa8). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8844][SPARKR] head/collect is broken in...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7419#issuecomment-131477341 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40978/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8844][SPARKR] head/collect is broken in...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7419#issuecomment-131477340 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8844][SPARKR] head/collect is broken in...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7419#issuecomment-131477331 [Test build #40978 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40978/console) for PR 7419 at commit [`cce54aa`](https://github.com/apache/spark/commit/cce54aa42aceb6def561acf1df4a61f80b75006b). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5754] [yarn] Spark/Yarn/Windows driver/...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8053#issuecomment-131477299 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5754] [yarn] Spark/Yarn/Windows driver/...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8053#issuecomment-131477292 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5754] [yarn] Spark/Yarn/Windows driver/...
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8053#issuecomment-131477252 I can't really test this, but since the Linux tests are passing, LGTM. I'll merge this Monday morning unless someone else says something. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5754] [yarn] Spark/Yarn/Windows driver/...
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8053#issuecomment-131477241 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8844][SPARKR] head/collect is broken in...
Github user davies commented on the pull request: https://github.com/apache/spark/pull/7419#issuecomment-131477139 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8844][SPARKR] head/collect is broken in...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7419#issuecomment-131477012 [Test build #40978 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40978/consoleFull) for PR 7419 at commit [`cce54aa`](https://github.com/apache/spark/commit/cce54aa42aceb6def561acf1df4a61f80b75006b). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8844][SPARKR] head/collect is broken in...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7419#issuecomment-131476906 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8844][SPARKR] head/collect is broken in...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7419#issuecomment-131476904 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8844][SPARKR] head/collect is broken in...
Github user sun-rui commented on the pull request: https://github.com/apache/spark/pull/7419#issuecomment-131476840 @shivaram, fixed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9805] [MLLIB] [PYTHON] [STREAMING] Adde...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8087 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9924] [WEB UI] Don't schedule checkForL...
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8153#issuecomment-131474836 As usual, flaky tests in other unrelated modules. I'll just give up on jenkins and merge this Monday morning. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9805] [MLLIB] [PYTHON] [STREAMING] Adde...
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/8087#issuecomment-131474391 OK, I'll merge this with master and branch-1.5 then. Thanks for reviewing, everyone! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9924] [WEB UI] Don't schedule checkForL...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8153#issuecomment-131473753 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9924] [WEB UI] Don't schedule checkForL...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8153#issuecomment-131473754 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40975/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9924] [WEB UI] Don't schedule checkForL...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8153#issuecomment-131473745 **[Test build #40975 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40975/console)** for PR 8153 at commit [`cd1ef90`](https://github.com/apache/spark/commit/cd1ef90cd34b27d7c2cd98b54bceeb5936ce60e9) after a configured wait of `175m`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10004] [shuffle] Perform auth checks wh...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8218#issuecomment-131473374 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10004] [shuffle] Perform auth checks wh...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8218#issuecomment-131473375 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40976/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10004] [shuffle] Perform auth checks wh...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8218#issuecomment-131473352 [Test build #40976 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40976/console) for PR 8218 at commit [`c68deab`](https://github.com/apache/spark/commit/c68deabdf4f0a5070a928802c18e0330b38c3df0). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9959][MLlib] Association Rules Java Com...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8206#issuecomment-131470014 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9959][MLlib] Association Rules Java Com...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8206#issuecomment-131469998 [Test build #40977 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40977/console) for PR 8206 at commit [`f2a5f40`](https://github.com/apache/spark/commit/f2a5f40017b739fa2c40351273a26ff191be6400). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9959][MLlib] Association Rules Java Com...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8206#issuecomment-131470015 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40977/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9906] [ML] User guide for LogisticRegre...
Github user feynmanliang commented on a diff in the pull request: https://github.com/apache/spark/pull/8197#discussion_r37140446 --- Diff: docs/ml-guide.md --- @@ -801,6 +801,146 @@ jsc.stop(); +## Examples: Summaries for LogisticRegression. + +Once [`LogisticRegression`](api/scala/index.html#org.apache.spark.ml.classification.LogisticRegression) +is run on data, it is useful to extract statistics such as the +loss per iteration which will provide an intuition on overfitting and metrics to understand +how well the model has performed on training and test data. + +[`LogisticRegressionTrainingSummary`](api/scala/index.html#org.apache.spark.mllib.classification.LogisticRegressionTrainingSummary) +provides an interface to access such relevant information. i.e the objectiveHistory and metrics --- End diff -- Actually, `metrics` is not part of the public API --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5945] Spark should not retry a stage in...
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r37140442
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -818,7 +1092,7 @@ class DAGSchedulerSuite
complete(taskSets(1), Seq(
(Success, makeMapStatus("hostA", 1)),
(Success, makeMapStatus("hostB", 1
-// pretend stage 0 failed because hostA went down
+// pretend stage 2 failed because hostA went down
--- End diff --
update to test comment; again, please disregard
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9906] [ML] User guide for LogisticRegre...
Github user feynmanliang commented on a diff in the pull request: https://github.com/apache/spark/pull/8197#discussion_r37140409 --- Diff: docs/ml-guide.md --- @@ -801,6 +801,146 @@ jsc.stop(); +## Examples: Summaries for LogisticRegression. + +Once [`LogisticRegression`](api/scala/index.html#org.apache.spark.ml.classification.LogisticRegression) +is run on data, it is useful to extract statistics such as the +loss per iteration which will provide an intuition on overfitting and metrics to understand +how well the model has performed on training and test data. + +[`LogisticRegressionTrainingSummary`](api/scala/index.html#org.apache.spark.mllib.classification.LogisticRegressionTrainingSummary) +provides an interface to access such relevant information. i.e the objectiveHistory and metrics +to evaluate the performance on the training data directly with very less code to be rewritten by +the user. In the future, a method would be made available in the fitted --- End diff -- nit: I would just leave this part of the doc out in case we forget to remove it when we do settle on a good name for `evaluate`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9959][MLlib] Association Rules Java Com...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8206#issuecomment-131467589 [Test build #40977 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40977/consoleFull) for PR 8206 at commit [`f2a5f40`](https://github.com/apache/spark/commit/f2a5f40017b739fa2c40351273a26ff191be6400). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9906] [ML] User guide for LogisticRegre...
Github user feynmanliang commented on a diff in the pull request: https://github.com/apache/spark/pull/8197#discussion_r37140386 --- Diff: docs/ml-guide.md --- @@ -801,6 +801,146 @@ jsc.stop(); +## Examples: Summaries for LogisticRegression. + +Once [`LogisticRegression`](api/scala/index.html#org.apache.spark.ml.classification.LogisticRegression) +is run on data, it is useful to extract statistics such as the +loss per iteration which will provide an intuition on overfitting and metrics to understand +how well the model has performed on training and test data. + +[`LogisticRegressionTrainingSummary`](api/scala/index.html#org.apache.spark.mllib.classification.LogisticRegressionTrainingSummary) +provides an interface to access such relevant information. i.e the objectiveHistory and metrics --- End diff -- "`objectiveHistory`" and "`metric`" (surround code with backticks so docs apply correct styles to it) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9906] [ML] User guide for LogisticRegre...
Github user feynmanliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/8197#discussion_r37140382
--- Diff: docs/ml-guide.md ---
@@ -801,6 +801,141 @@ jsc.stop();
+## Examples: Summaries for LogisticRegression.
+
+Once LogisticRegression is run on data, it is useful to extract statistics
such as the
+loss per iteration which will provide an intuition on overfitting and
metrics to understand
+how well the model has performed on training and test data.
+
+LogisticRegressionTrainingSummary provides an interface to access such
relevant information
+i.e the objectiveHistory and metrics to evaluate the performance on the
training data
+directly with very less code to be rewritten by the user.
+In the future, a method would be made available in the fitted
LogisticRegressionModel to obtain
+a LogisticRegressionSummary of the test data as well.
+
+This examples illustrates the use of LogisticRegressionTrainingSummary on
some toyData.
+
+
+
+{% highlight scala %}
+import org.apache.spark.{SparkConf, SparkContext}
+import org.apache.spark.ml.classification.{LogisticRegression,
BinaryLogisticRegressionSummary}
+import org.apache.spark.mllib.regression.LabeledPoint
+import org.apache.spark.mllib.linalg.Vectors
+import org.apache.spark.sql.{Row, SQLContext}
+
+val conf = new SparkConf().setAppName("LogisticRegressionSummary")
+val sc = new SparkContext(conf)
+val sqlContext = new SQLContext(sc)
+import sqlContext.implicits._
+
+// Use some random data for demonstration.
+// Note that the RDD of LabeledPoints can be converted to a dataframe
directly.
+val data = sc.parallelize(Array(
+ LabeledPoint(0.0, Vectors.dense(0.2, 4.5, 1.6)),
+ LabeledPoint(1.0, Vectors.dense(3.1, 6.8, 3.6)),
+ LabeledPoint(0.0, Vectors.dense(2.4, 0.9, 1.9)),
+ LabeledPoint(1.0, Vectors.dense(9.1, 3.1, 3.6)),
+ LabeledPoint(0.0, Vectors.dense(2.5, 1.9, 9.1)))
+)
+val logRegDataFrame = data.toDF()
+
+// Run Logistic Regression on your toy data.
+// Since LogisticRegression is an estimator, it returns an instance of
LogisticRegressionModel
+// which is a transformer.
+val logReg = new LogisticRegression()
+logReg.setMaxIter(5)
+logReg.setRegParam(0.01)
+val logRegModel = logReg.fit(logRegDataFrame)
+
+// Extract the summary directly from the returned LogisticRegressionModel
instance.
+val trainingSummary = logRegModel.summary
+
+// Obtain the loss per iteration. This should decrease upto a certain
point and
+// then increase or show negligible change after this.
+val objectiveHistory = trainingSummary.objectiveHistory
+objectiveHistory.foreach(loss => println(loss))
+
+// Obtain the metrics useful to judge performance on test data.
+val binarySummary =
trainingSummary.asInstanceOf[BinaryLogisticRegressionSummary]
+
+// Obtain the receiver-operating characteristic as a dataframe and
areaUnderROC.
+val roc = binarySummary.roc
+val truePositiveRate = roc.select("FPR")
+val area = binarySummary.areaUnderROC
+
+// Obtain the threshold with the highest fMeasure.
+val fMeasure = binarySummary.fMeasureByThreshold
+val fScoreRDD = fMeasure.map { case Row(thresh: Double, fscore: Double) =>
(thresh, fscore) }
+val (highThresh, highFScore) = fScoreRDD.fold((0.0, 0.0))((threshFScore1,
threshFScore2) => {
+ if (threshFScore1._2 > threshFScore2._2) threshFScore1 else threshFScore2
+})
--- End diff --
Nice!
I think what you described is useful, but is outside the scope of
`LogisticRegressionSummary`. L869-L872 don't demonstrate any of the
functionality these docs are intended to describe, which is why I propose we
remove it. What do you think?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9952] Fix N^2 loop when DAGScheduler.ge...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8178#issuecomment-131467021 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40974/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9952] Fix N^2 loop when DAGScheduler.ge...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8178#issuecomment-131467018 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9952] Fix N^2 loop when DAGScheduler.ge...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8178#issuecomment-131466758 [Test build #40974 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40974/console) for PR 8178 at commit [`44a15f6`](https://github.com/apache/spark/commit/44a15f61d506c67d8435a0316d61085b84cedf13). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9959][MLlib] Association Rules Java Com...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8206#issuecomment-131465828 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9959][MLlib] Association Rules Java Com...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8206#issuecomment-131465868 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9959][MLlib] Association Rules Java Com...
Github user feynmanliang commented on the pull request: https://github.com/apache/spark/pull/8206#issuecomment-131465170 jenkins test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9924] [WEB UI] Don't schedule checkForL...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8153#issuecomment-131461043 [Test build #40975 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40975/consoleFull) for PR 8153 at commit [`cd1ef90`](https://github.com/apache/spark/commit/cd1ef90cd34b27d7c2cd98b54bceeb5936ce60e9). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10004] [shuffle] Perform auth checks wh...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8218#issuecomment-131459257 [Test build #40976 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40976/consoleFull) for PR 8218 at commit [`c68deab`](https://github.com/apache/spark/commit/c68deabdf4f0a5070a928802c18e0330b38c3df0). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9924] [WEB UI] Don't schedule checkForL...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8153#issuecomment-131459036 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10004] [shuffle] Perform auth checks wh...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8218#issuecomment-131459060 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10004] [shuffle] Perform auth checks wh...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8218#issuecomment-131459031 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9924] [WEB UI] Don't schedule checkForL...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8153#issuecomment-131459059 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9924] [WEB UI] Don't schedule checkForL...
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8153#issuecomment-131457310 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10004] [shuffle] Perform auth checks wh...
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8218#issuecomment-131457210 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9973][SQL]correct buffer size
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8189#issuecomment-131456755 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40973/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9973][SQL]correct buffer size
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8189#issuecomment-131456754 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9973][SQL]correct buffer size
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8189#issuecomment-131456727 [Test build #40973 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40973/console) for PR 8189 at commit [`6741f23`](https://github.com/apache/spark/commit/6741f239052c72663b55b3d398db3b26d93ae2e3). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10004] [shuffle] Perform auth checks wh...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8218#issuecomment-131454918 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10004] [shuffle] Perform auth checks wh...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8218#issuecomment-131454922 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40971/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10004] [shuffle] Perform auth checks wh...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8218#issuecomment-131454675 [Test build #40971 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40971/console) for PR 8218 at commit [`c68deab`](https://github.com/apache/spark/commit/c68deabdf4f0a5070a928802c18e0330b38c3df0). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9924] [WEB UI] Don't schedule checkForL...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8153#issuecomment-131454149 **[Test build #1625 timed out](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1625/console)** for PR 8153 at commit [`cd1ef90`](https://github.com/apache/spark/commit/cd1ef90cd34b27d7c2cd98b54bceeb5936ce60e9) after a configured wait of `175m`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10008] Ensure shuffle locality doesn't ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8220#issuecomment-131450288 [Test build #1626 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1626/console) for PR 8220 at commit [`a7c02dc`](https://github.com/apache/spark/commit/a7c02dc062cdc2cc59a04fa6ec4062d8a57dacb4). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class StringIndexerModel (` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5945] Spark should not retry a stage in...
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/5636#discussion_r37139517
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -818,7 +1092,7 @@ class DAGSchedulerSuite
complete(taskSets(1), Seq(
(Success, makeMapStatus("hostA", 1)),
(Success, makeMapStatus("hostB", 1
-// pretend stage 0 failed because hostA went down
+// pretend stage 2 failed because hostA went down
--- End diff --
this is a test comment for spark-prs integration; please disregard
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5945] Spark should not retry a stage in...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/5636#issuecomment-131448842 (this is a test comment to test a Jenkins + Spark-PRS integration affecting long comment threads) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9955][SQL] correct error message for ag...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8203 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9955][SQL] correct error message for ag...
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/8203#issuecomment-131448600 Thanks! Merging to master and branch-1.5 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9952] Fix N^2 loop when DAGScheduler.ge...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8178#issuecomment-131441189 [Test build #40974 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40974/consoleFull) for PR 8178 at commit [`44a15f6`](https://github.com/apache/spark/commit/44a15f61d506c67d8435a0316d61085b84cedf13). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9952] Fix N^2 loop when DAGScheduler.ge...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8178#issuecomment-131440032 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9952] Fix N^2 loop when DAGScheduler.ge...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8178#issuecomment-131439990 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9952] Fix N^2 loop when DAGScheduler.ge...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8178#issuecomment-131439275 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40972/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9952] Fix N^2 loop when DAGScheduler.ge...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8178#issuecomment-131439273 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9952] Fix N^2 loop when DAGScheduler.ge...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8178#issuecomment-131439239 [Test build #40972 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40972/console) for PR 8178 at commit [`88710c1`](https://github.com/apache/spark/commit/88710c1d8ec0efd8adb3aa23c84e47e712e3577c). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9973][SQL]correct buffer size
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8189#issuecomment-131439001 [Test build #40973 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40973/consoleFull) for PR 8189 at commit [`6741f23`](https://github.com/apache/spark/commit/6741f239052c72663b55b3d398db3b26d93ae2e3). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
