[GitHub] spark pull request: [SPARK-10125][Streaming]Fix a potential deadlo...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8326#issuecomment-132855837 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41296/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10125][Streaming]Fix a potential deadlo...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8326#issuecomment-132855370 [Test build #41296 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41296/console) for PR 8326 at commit [`7a3799b`](https://github.com/apache/spark/commit/7a3799b52737baf2a062a52b76e2552a8fb989c9). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9772] [PySpark] [ML] Add Python API for...
Github user yanboliang commented on the pull request: https://github.com/apache/spark/pull/8102#issuecomment-132854062 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9654][ML][PYSPARK] Add IndexToString to...
Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/7976#discussion_r37489806
--- Diff: python/pyspark/ml/feature.py ---
@@ -817,6 +822,64 @@ class StringIndexerModel(JavaModel):
"""
Model fitted by StringIndexer.
"""
+@property
+def labels(self):
+return self._java_obj.labels
+
+
+@inherit_doc
+class IndexToString(JavaTransformer, HasInputCol, HasOutputCol):
+"""
+.. note:: Experimental
+A [[Transformer]] that maps a column of string indices back to a new
column of corresponding
+string values using either the ML attributes of the input column, or
if provided using the
+labels supplied by the user.
+All original columns are kept during transformation.
+See L{StringIndexer} for converting strings into indices
+"""
+
+# a placeholder to make the labels show up in generated doc
+labels = Param(Params._dummy(), "labels",
+ "Optional labels to be provided by the user, if not
supplied column " +
+ "metadata is read for labels. The default value is an
empty array, " +
+ "but the empty array is ignored and column metadata
used instead.")
+
+@keyword_only
+def __init__(self, inputCol=None, outputCol=None, labels=[]):
--- End diff --
We should avoid using mutable values ```[]``` as defaults in Python, let's
use ```None```.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10048][SPARKR] Support arbitrary nested...
Github user sun-rui commented on the pull request: https://github.com/apache/spark/pull/8276#issuecomment-132852075 will add test cases. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10092] [SQL] Multi-DB support follow up...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8324#issuecomment-132851991 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10092] [SQL] Multi-DB support follow up...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8324#issuecomment-132851994 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41292/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10092] [SQL] Multi-DB support follow up...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8324#issuecomment-132851909 [Test build #41292 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41292/console) for PR 8324 at commit [`8a62b76`](https://github.com/apache/spark/commit/8a62b769c98d4ef22e578c73a01e2311ceca64e8). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10124] [MESOS] Fix removing queued driv...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8322#issuecomment-132850491 [Test build #41297 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41297/consoleFull) for PR 8322 at commit [`b801f93`](https://github.com/apache/spark/commit/b801f9345a28e1cb676d6d95f3223201f4d1488d). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10124] [MESOS] Fix removing queued driv...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8322#issuecomment-132848964 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10124] [MESOS] Fix removing queued driv...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8322#issuecomment-132848952 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10124] [MESOS] Fix removing queued driv...
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/8322#issuecomment-132848583 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10124] [MESOS] Fix removing queued driv...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8322#issuecomment-132848347 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10124] [MESOS] Fix removing queued driv...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8322#issuecomment-132848348 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41289/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10124] [MESOS] Fix removing queued driv...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8322#issuecomment-132848260 [Test build #41289 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41289/console) for PR 8322 at commit [`b801f93`](https://github.com/apache/spark/commit/b801f9345a28e1cb676d6d95f3223201f4d1488d). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class PromotePrecision(child: Expression) extends UnaryExpression ` * `case class CheckOverflow(child: Expression, dataType: DecimalType) extends UnaryExpression ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9689][SQL]Fix bug of not invalidate the...
Github user chhao01 commented on a diff in the pull request:
https://github.com/apache/spark/pull/8023#discussion_r37488201
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala ---
@@ -156,10 +165,27 @@ private[sql] class CacheManager(sqlContext:
SQLContext) extends Logging {
* function will over invalidate.
*/
private[sql] def invalidateCache(plan: LogicalPlan): Unit = writeLock {
-cachedData.foreach {
- case data if data.plan.collect { case p if p.sameResult(plan) => p
}.nonEmpty =>
-data.cachedRepresentation.recache()
- case _ =>
+var i = 0
+var locatedIdx = -1
+// find the index of the cached data, according to the specified
logical plan
+while (i < cachedData.length && locatedIdx < 0) {
+ cachedData(i) match {
+case data if data.plan.collect { case p if p.sameResult(plan) => p
}.nonEmpty =>
+ locatedIdx = i
+case _ =>
+ }
+ i += 1
+}
+
+if (locatedIdx >= 0) {
+ // if the cached data exists, remove it from the cache data list, as
we need to
+ // re-generate the spark plan, and we don't want the this to be used
during the
+ // re-generation
+ val entry = cachedData.remove(locatedIdx) // TODO do we have to use
ArrayBuffer?
+ // rebuild the cache
+ entry.recache(sqlContext)
+ // add it back to the cache data list
+ cachedData += entry
--- End diff --
And probably we can not simply use the `data.plan.find`, as the cached data
most likely will add wrapper like `LogicalPlan` etc.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9812][Streaming]Fix Python 3 compatibil...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8315 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9689][SQL]Fix bug of not invalidate the...
Github user chhao01 commented on a diff in the pull request:
https://github.com/apache/spark/pull/8023#discussion_r37488173
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala ---
@@ -156,10 +165,27 @@ private[sql] class CacheManager(sqlContext:
SQLContext) extends Logging {
* function will over invalidate.
*/
private[sql] def invalidateCache(plan: LogicalPlan): Unit = writeLock {
-cachedData.foreach {
- case data if data.plan.collect { case p if p.sameResult(plan) => p
}.nonEmpty =>
-data.cachedRepresentation.recache()
- case _ =>
+var i = 0
+var locatedIdx = -1
+// find the index of the cached data, according to the specified
logical plan
+while (i < cachedData.length && locatedIdx < 0) {
+ cachedData(i) match {
+case data if data.plan.collect { case p if p.sameResult(plan) => p
}.nonEmpty =>
+ locatedIdx = i
+case _ =>
+ }
+ i += 1
+}
+
+if (locatedIdx >= 0) {
+ // if the cached data exists, remove it from the cache data list, as
we need to
+ // re-generate the spark plan, and we don't want the this to be used
during the
+ // re-generation
+ val entry = cachedData.remove(locatedIdx) // TODO do we have to use
ArrayBuffer?
+ // rebuild the cache
+ entry.recache(sqlContext)
+ // add it back to the cache data list
+ cachedData += entry
--- End diff --
yes, that's a good catch, I will see how to fix the chained logical cached
plan.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10125][Streaming]Fix a potential deadlo...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8326#issuecomment-132845139 [Test build #41296 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41296/consoleFull) for PR 8326 at commit [`7a3799b`](https://github.com/apache/spark/commit/7a3799b52737baf2a062a52b76e2552a8fb989c9). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10125][Streaming]Fix a potential deadlo...
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/8326#issuecomment-132844994 LGTM! Will merge when tests pass. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9689][SQL]Fix bug of not invalidate the...
Github user chhao01 commented on a diff in the pull request:
https://github.com/apache/spark/pull/8023#discussion_r37487843
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelation.scala
---
@@ -160,6 +160,9 @@ private[sql] case class InsertIntoHadoopFsRelation(
logInfo("Skipping insertion into a relation that already exists.")
}
+// Invalidate the cache.
+sqlContext.cacheManager.invalidateCache(LogicalRelation(relation))
--- End diff --
Yes, true, I will update it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10126] [PROJECT INFRA] Fix typo in rele...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8325#issuecomment-132844686 [Test build #41295 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41295/consoleFull) for PR 8325 at commit [`cdcb6e5`](https://github.com/apache/spark/commit/cdcb6e52a5e402eca96bc5b39c6839397684f699). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10126] [PROJECT INFRA] Fix typo in rele...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8325#issuecomment-132844079 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10126] [PROJECT INFRA] Fix typo in rele...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8325#issuecomment-132844067 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10125][Streaming]Fix a potential deadlo...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8326#issuecomment-132844078 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10125][Streaming]Fix a potential deadlo...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8326#issuecomment-132844061 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9812][Streaming]Fix Python 3 compatibil...
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/8315#issuecomment-132844018 Thanks! merging this! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10125][Streaming]Fix a potential deadlo...
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/8326 [SPARK-10125][Streaming]Fix a potential deadlock in JobGenerator.stop Because `lazy val` uses `this` lock, if JobGenerator.stop and JobGenerator.doCheckpoint (JobGenerator.shouldCheckpoint has not yet been initialized) run at the same time, it may hang. Here are the stack traces for the deadlock: ```Java "pool-1-thread-1-ScalaTest-running-StreamingListenerSuite" #11 prio=5 os_prio=31 tid=0x7fd35d094800 nid=0x5703 in Object.wait() [0x00012ecaf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1245) - locked <0x0007b5d8d7f8> (a org.apache.spark.util.EventLoop$$anon$1) at java.lang.Thread.join(Thread.java:1319) at org.apache.spark.util.EventLoop.stop(EventLoop.scala:81) at org.apache.spark.streaming.scheduler.JobGenerator.stop(JobGenerator.scala:155) - locked <0x0007b5d8cea0> (a org.apache.spark.streaming.scheduler.JobGenerator) at org.apache.spark.streaming.scheduler.JobScheduler.stop(JobScheduler.scala:95) - locked <0x0007b5d8ced8> (a org.apache.spark.streaming.scheduler.JobScheduler) at org.apache.spark.streaming.StreamingContext.stop(StreamingContext.scala:687) "JobGenerator" #67 daemon prio=5 os_prio=31 tid=0x7fd35c3b9800 nid=0x9f03 waiting for monitor entry [0x000139e4a000] java.lang.Thread.State: BLOCKED (on object monitor) at org.apache.spark.streaming.scheduler.JobGenerator.shouldCheckpoint$lzycompute(JobGenerator.scala:63) - waiting to lock <0x0007b5d8cea0> (a org.apache.spark.streaming.scheduler.JobGenerator) at org.apache.spark.streaming.scheduler.JobGenerator.shouldCheckpoint(JobGenerator.scala:63) at org.apache.spark.streaming.scheduler.JobGenerator.doCheckpoint(JobGenerator.scala:290) at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:182) at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:83) at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:82) at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48) ``` I can use this patch to produce this deadlock: https://github.com/zsxwing/spark/commit/8a88f28d1331003a65fabef48ae3d22a7c21f05f And a timeout build in Jenkins due to this deadlock: https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1654/ This PR initializes `checkpointWriter` before `eventLoop` uses it to avoid this deadlock. You can merge this pull request into a Git repository by running: $ git pull https://github.com/zsxwing/spark SPARK-10125 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/8326.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #8326 commit 7a3799b52737baf2a062a52b76e2552a8fb989c9 Author: zsxwing Date: 2015-08-20T01:18:34Z Fix a potential deadlock in JobGenerator.stop --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10053] Export lapply and lapplyPartitio...
Github user sun-rui commented on the pull request: https://github.com/apache/spark/pull/8240#issuecomment-132843607 I think UDF() is important, hope it can be available in 1.6 release. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10126] [PROJECT INFRA] Fix typo in rele...
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/8325 [SPARK-10126] [PROJECT INFRA] Fix typo in release-build.sh which broke snapshot publishing for Scala 2.11 The current `release-build.sh` has a typo which breaks snapshot publication for Scala 2.11. We should change the Scala version to 2.11 and clean before building a 2.11 snapshot. You can merge this pull request into a Git repository by running: $ git pull https://github.com/JoshRosen/spark fix-2.11-snapshots Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/8325.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #8325 commit cdcb6e52a5e402eca96bc5b39c6839397684f699 Author: Josh Rosen Date: 2015-08-20T01:18:11Z Fix typos release-build.sh --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9689][SQL]Fix bug of not invalidate the...
Github user chhao01 commented on a diff in the pull request:
https://github.com/apache/spark/pull/8023#discussion_r37487474
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala ---
@@ -156,10 +165,27 @@ private[sql] class CacheManager(sqlContext:
SQLContext) extends Logging {
* function will over invalidate.
*/
private[sql] def invalidateCache(plan: LogicalPlan): Unit = writeLock {
-cachedData.foreach {
- case data if data.plan.collect { case p if p.sameResult(plan) => p
}.nonEmpty =>
-data.cachedRepresentation.recache()
- case _ =>
+var i = 0
+var locatedIdx = -1
+// find the index of the cached data, according to the specified
logical plan
+while (i < cachedData.length && locatedIdx < 0) {
+ cachedData(i) match {
+case data if data.plan.collect { case p if p.sameResult(plan) => p
}.nonEmpty =>
+ locatedIdx = i
+case _ =>
+ }
+ i += 1
+}
+
+if (locatedIdx >= 0) {
+ // if the cached data exists, remove it from the cache data list, as
we need to
+ // re-generate the spark plan, and we don't want the this to be used
during the
+ // re-generation
--- End diff --
In `SQLContext.QueryExecution`, we always try to load the cached
`PhysicalRDD` according to the logical plan first, if we don't remove it from
the cached list, then we will always get the stale one, which is not what we
want, right?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9400][SQL]Codegen for StringLocate
Github user yjshen commented on a diff in the pull request:
https://github.com/apache/spark/pull/8274#discussion_r37487469
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringOperations.scala
---
@@ -582,6 +582,109 @@ case class StringLocate(substr: Expression, str:
Expression, start: Expression)
}
}
+ override def genCode(ctx: CodeGenContext, ev: GeneratedExpressionCode):
String = {
+if (start.foldable) {
--- End diff --
I was following
https://github.com/apache/spark/pull/7779#discussion_r35945742, if it's not
required, I think 7779 is fine.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9689][SQL]Fix bug of not invalidate the...
Github user chhao01 commented on a diff in the pull request:
https://github.com/apache/spark/pull/8023#discussion_r37487286
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala ---
@@ -97,13 +106,13 @@ private[sql] class CacheManager(sqlContext:
SQLContext) extends Logging {
logWarning("Asked to cache already cached data.")
} else {
cachedData +=
-CachedData(
+new CachedData(
planToCache,
InMemoryRelation(
sqlContext.conf.useCompression,
sqlContext.conf.columnBatchSize,
storageLevel,
-sqlContext.executePlan(query.logicalPlan).executedPlan,
+sqlContext.executePlan(planToCache).executedPlan,
--- End diff --
This is just an optimization, and we don't want to re-analysis the logical
plan as it's done right before the calls.
When you check the full code of the function `cacheQuery`, you will see the
`planToCache` is just a local variable, not a logical plan cached by somewhere
else, it should not be the case you were talking.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6236] [core] [wip] caching for blocks o...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8320#issuecomment-132838921 [Test build #41293 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41293/consoleFull) for PR 8320 at commit [`200afdc`](https://github.com/apache/spark/commit/200afdc33f59508a04172cfefcc4a40a3bdf6169). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9400][SQL]Codegen for StringLocate
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8274#discussion_r37485964
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringOperations.scala
---
@@ -582,6 +582,109 @@ case class StringLocate(substr: Expression, str:
Expression, start: Expression)
}
}
+ override def genCode(ctx: CodeGenContext, ev: GeneratedExpressionCode):
String = {
+if (start.foldable) {
--- End diff --
I think we do not need these special cases for StringLocate, because you do
not need any prepare to call UTF8String.indexOf(substr, start)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9982] [SPARKR] SparkR DataFrame fail to...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8239#issuecomment-132838314 [Test build #41294 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41294/consoleFull) for PR 8239 at commit [`fab7042`](https://github.com/apache/spark/commit/fab7042f8b9a11cc9bf3b9b761292c2a92abaa7e). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6236] [core] [wip] caching for blocks o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8320#issuecomment-132838159 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6236] [core] [wip] caching for blocks o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8320#issuecomment-132838183 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9982] [SPARKR] SparkR DataFrame fail to...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8239#issuecomment-132838190 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9982] [SPARKR] SparkR DataFrame fail to...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8239#issuecomment-132838171 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9851] Support submitting map stages ind...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8180#issuecomment-132837778 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9851] Support submitting map stages ind...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8180#issuecomment-132837779 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41286/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9851] Support submitting map stages ind...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8180#issuecomment-132837674 [Test build #41286 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41286/console) for PR 8180 at commit [`210ca43`](https://github.com/apache/spark/commit/210ca431ec5e0d51c0e206a5eca549851c470e5e). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9982] [SPARKR] SparkR DataFrame fail to...
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/8239#issuecomment-132837650 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9242][SQL] Audit UDAF interface.
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8321 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-1267][PYSPARK] Adds pip installer for p...
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/8318#discussion_r37485263
--- Diff: python/setup.py ---
@@ -0,0 +1,19 @@
+#!/usr/bin/env python
+
+from setuptools import setup
+
+exec(compile(open("pyspark/pyspark_version.py").read(),
+ "pyspark/pyspark_version.py", 'exec'))
+VERSION = __version__
+
+setup(name='pyspark',
+version=VERSION,
+description='Apache Spark Python API',
+author='Spark Developers',
+author_email='d...@spark.apache.org',
+url='https://github.com/apache/spark/tree/master/python',
+packages=['pyspark', 'pyspark.mllib', 'pyspark.ml', 'pyspark.sql',
'pyspark.streaming'],
+data_files=[('pyspark', ['pyspark/pyspark_version.py'])],
+install_requires=['numpy>=1.7', 'py4j==0.8.2.1', 'pandas'],
+license='http://www.apache.org/licenses/LICENSE-2.0',
+)
--- End diff --
This is maybe asking for too much, but in Sparkling Pandas we install our
own assembly jar*, would it maybe make sense to do that as part of this process?
(*and getting it working has been painful, but doable).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9982] [SPARKR] SparkR DataFrame fail to...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8239#issuecomment-132836236 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9982] [SPARKR] SparkR DataFrame fail to...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8239#issuecomment-132836238 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41282/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9982] [SPARKR] SparkR DataFrame fail to...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8239#issuecomment-132836197 **[Test build #41282 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41282/console)** for PR 8239 at commit [`fab7042`](https://github.com/apache/spark/commit/fab7042f8b9a11cc9bf3b9b761292c2a92abaa7e) after a configured wait of `175m`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6236] [core] [wip] caching for blocks o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8320#issuecomment-132836139 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41283/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6236] [core] [wip] caching for blocks o...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8320#issuecomment-132836087 [Test build #41283 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41283/console) for PR 8320 at commit [`06c8ffa`](https://github.com/apache/spark/commit/06c8ffa6f576baad66b5897756eda990f114ade4). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `public class LargeByteBufferInputStream extends InputStream ` * `public class LargeByteBufferOutputStream extends OutputStream ` * `abstract class BlockSizeLimitException(msg: String, cause: BufferTooLargeException)` * `class ReplicationBlockSizeLimitException(cause: BufferTooLargeException)` * `class TachyonBlockSizeLimitException(cause: BufferTooLargeException)` * `class ShuffleBlockSizeLimitException(size: Long)` * `class ShuffleRemoteBlockSizeLimitException(cause: BufferTooLargeException)` * `public class BufferTooLargeException extends IOException ` * `public class LargeByteBufferHelper ` * `public class WrappedLargeByteBuffer implements LargeByteBuffer ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6236] [core] [wip] caching for blocks o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8320#issuecomment-132836138 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10017] [MLlib]: ML model broadcasts sho...
Github user holdenk commented on the pull request: https://github.com/apache/spark/pull/8241#issuecomment-132836131 Glad to help, maybe @jkbradley can say if this is ok to test to jenkins :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9242][SQL] Audit UDAF interface.
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8321#issuecomment-132834417 [Test build #1673 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1673/console) for PR 8321 at commit [`8d29a42`](https://github.com/apache/spark/commit/8d29a42926ec6452e0f37cbbf0c4d7b106e2ce5b). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10035][SQL] Parquet filters does not pr...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8275 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10035][SQL] Parquet filters does not pr...
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/8275#issuecomment-132830954 Thanks! Merging to master (not to branch-1.5 at this moment since this is a new feature). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10092] [SQL] Multi-DB support follow up...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8324#issuecomment-132830847 [Test build #41292 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41292/consoleFull) for PR 8324 at commit [`8a62b76`](https://github.com/apache/spark/commit/8a62b769c98d4ef22e578c73a01e2311ceca64e8). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10092] [SQL] Multi-DB support follow up...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8324#issuecomment-132830468 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10092] [SQL] Multi-DB support follow up...
GitHub user yhuai opened a pull request: https://github.com/apache/spark/pull/8324 [SPARK-10092] [SQL] Multi-DB support follow up. https://issues.apache.org/jira/browse/SPARK-10092 This pr is a follow-up one for Multi-DB support. It has the following changes: * `Hive.Context.refreshTable` now accepts `dbName.tableName`. * `Hive.Context.analyze` now accepts `dbName.tableName`. * `CreateTableUsing`, `CreateTableUsingAsSelect`, `CreateTempTableUsing`, `CreateTempTableUsingAsSelect`, `CreateMetastoreDataSource`, and `CreateMetastoreDataSourceAsSelect` all take `TableIdentifier` instead of the string representation of table name. * When you call `saveAsTable` with a specified database, the data will be saved to the correct location. * Explicitly do not allow user to create a temporary with a specified database name (users cannot do it before). You can merge this pull request into a Git repository by running: $ git pull https://github.com/yhuai/spark saveAsTableDB Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/8324.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #8324 commit 8a62b769c98d4ef22e578c73a01e2311ceca64e8 Author: Yin Huai Date: 2015-08-19T23:54:24Z Multi-db support follow up. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10092] [SQL] Multi-DB support follow up...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8324#issuecomment-132830450 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-132828228 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41291/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-132828227 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-132828223 [Test build #41291 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41291/console) for PR 8323 at commit [`d4f48ba`](https://github.com/apache/spark/commit/d4f48ba4855fbb6f21c50924475b3810df048826). * This patch **fails RAT tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class _DelegateIO(object):` * `class _TestInfo(object):` * `class _XMLTestResult(_TextTestResult):` * `class XMLTestRunner(TextTestRunner):` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-132828208 [Test build #41291 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41291/consoleFull) for PR 8323 at commit [`d4f48ba`](https://github.com/apache/spark/commit/d4f48ba4855fbb6f21c50924475b3810df048826). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user gliptak commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-132827862 I just added the headers, xmlrunner is LGPL (and only used to run the tests). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-132827574 The build failed on RAT: ``` Could not find Apache license headers in the following files: !? /home/jenkins/workspace/SparkPullRequestBuilder/dev/xmlrunner/__init__.py !? /home/jenkins/workspace/SparkPullRequestBuilder/dev/xmlrunner/xmlrunner.py [error] running /home/jenkins/workspace/SparkPullRequestBuilder/dev/check-license ; received return code 1 ``` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-132827411 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-132827392 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user gliptak commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-132825406 https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41290/console Recording test results ERROR: Publisher 'Publish JUnit test result report' failed: No test report files were found. Configuration error? Test FAILed. Can somebody comment on how is Jenkins configured to look for JUnit results (or did this build fail on RAT)? Thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9384] [core] Easier setting of executor...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7739#issuecomment-132823658 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9384] [core] Easier setting of executor...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7739#issuecomment-132823659 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41277/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9384] [core] Easier setting of executor...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7739#issuecomment-132823574 **[Test build #41277 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41277/console)** for PR 7739 at commit [`bca4fa3`](https://github.com/apache/spark/commit/bca4fa303a6de1b8b395c9fc6c2357edb9e12d5e) after a configured wait of `175m`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9384] [core] Easier setting of executor...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7739#issuecomment-132821458 Build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9384] [core] Easier setting of executor...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7739#issuecomment-132821459 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41275/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-132821390 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-132821352 [Test build #41290 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41290/consoleFull) for PR 8323 at commit [`d93e7cd`](https://github.com/apache/spark/commit/d93e7cdefda962191a3e615708f977750799422e). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-132821391 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41290/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9384] [core] Easier setting of executor...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7739#issuecomment-132821326 [Test build #41275 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41275/console) for PR 7739 at commit [`5f62d39`](https://github.com/apache/spark/commit/5f62d39aadafc1780570c368fe6ceeaa39186758). * This patch **passes all tests**. * This patch **does not merge cleanly**. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-132821383 [Test build #41290 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41290/console) for PR 8323 at commit [`d93e7cd`](https://github.com/apache/spark/commit/d93e7cdefda962191a3e615708f977750799422e). * This patch **fails RAT tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class _DelegateIO(object):` * `class _TestInfo(object):` * `class _XMLTestResult(_TextTestResult):` * `class XMLTestRunner(TextTestRunner):` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6190][core] create LargeByteBuffer for ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5400#issuecomment-132820924 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41274/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6190][core] create LargeByteBuffer for ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5400#issuecomment-132820920 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6190][core] create LargeByteBuffer for ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5400#issuecomment-132820852 **[Test build #41274 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41274/console)** for PR 5400 at commit [`80c4032`](https://github.com/apache/spark/commit/80c40324e266590a03971b0e8002d8c9ac585eea) after a configured wait of `175m`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-132819515 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10035][SQL] Parquet filters does not pr...
Github user HyukjinKwon commented on the pull request: https://github.com/apache/spark/pull/8275#issuecomment-132819486 Could you merge this :) ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-132819556 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-132819078 Jenkins, this is ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9846] [Docs] User guide for Multilayer ...
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/8262#issuecomment-132818915 The test fail seems unrelated to my commit. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9846] [Docs] User guide for Multilayer ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8262#issuecomment-132817697 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9846] [Docs] User guide for Multilayer ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8262#issuecomment-132817708 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41288/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10124] [MESOS] Fix removing queued driv...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8322#issuecomment-132816388 [Test build #41289 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41289/consoleFull) for PR 8322 at commit [`b801f93`](https://github.com/apache/spark/commit/b801f9345a28e1cb676d6d95f3223201f4d1488d). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
GitHub user gliptak opened a pull request: https://github.com/apache/spark/pull/8323 SPARK-7021 Add JUnit output for Python tests WIP You can merge this pull request into a Git repository by running: $ git pull https://github.com/gliptak/spark SPARK-7021 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/8323.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #8323 commit d93e7cdefda962191a3e615708f977750799422e Author: Gábor Lipták Date: 2015-08-18T01:20:19Z SPARK-7021 Add JUnit output for Python tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-132816243 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10124] [MESOS] Fix removing queued driv...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8322#issuecomment-132816122 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10124] [MESOS] Fix removing queued driv...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8322#issuecomment-132815499 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10124] [MESOS] Fix removing queued driv...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8322#issuecomment-132815481 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10124] [MESOS] Fix removing queued driv...
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/8322#issuecomment-132815064 @andrewor14 PTAL! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10124] [MESOS] Fix removing queued driv...
GitHub user tnachen opened a pull request: https://github.com/apache/spark/pull/8322 [SPARK-10124] [MESOS] Fix removing queued driver in mesos cluster mode. You can merge this pull request into a Git repository by running: $ git pull https://github.com/tnachen/spark fix_cluster_mode Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/8322.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #8322 commit b801f9345a28e1cb676d6d95f3223201f4d1488d Author: Timothy Chen Date: 2015-08-19T22:42:47Z Fix removing queued driver in mesos cluster mode. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9846] [Docs] User guide for Multilayer ...
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/8262#issuecomment-132814397 @mengxr Thank you for your comments! I've addressed everything except the full chop of setters for the Java trainer. It complains on `java: incompatible types: org.apache.spark.ml.classification.MultilayerPerceptronParams cannot be converted to org.apache.spark.ml.classification.MultilayerPerceptronClassifier`. I did it in two stages. Also I've left Java imports. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9895] User Guide for RFormula Feature T...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8293 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9895] User Guide for RFormula Feature T...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/8293#issuecomment-132813989 LGTM. Merged into master and branch-1.5. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
