[GitHub] spark pull request: [SPARK-11419][STREAMING] Parallel recovery for...
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/9373#discussion_r44508962
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/util/WriteAheadLogSuite.scala
---
@@ -199,6 +198,63 @@ class FileBasedWriteAheadLogSuite
import WriteAheadLogSuite._
+ test("FileBasedWriteAheadLog - parallel readAll opens at most
'numThreads' files") {
--- End diff --
This really does not test file reading, just the utility function. Make the
name only the name of the method.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11593][SQL] Replace catalyst converter ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9565#issuecomment-155697401 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11419][STREAMING] Parallel recovery for...
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/9373#discussion_r44508920
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/util/FileBasedWriteAheadLog.scala
---
@@ -251,4 +261,22 @@ private[streaming] object FileBasedWriteAheadLog {
}
}.sortBy { _.startTime }
}
+
+ /**
+ * This creates an iterator from a parallel collection, by keeping at
most `n` objects in memory
+ * at any given time, where `n` is the size of the thread pool. This is
crucial for use cases
+ * where we create `FileBasedWriteAheadLogReader`s during parallel
recovery. We don't want to
+ * open up `k` streams altogether where `k` is the size of the Seq that
we want to parallelize.
+ */
+ def parallelIteratorCreator[I, O](
+ threadpool: ThreadPoolExecutor,
+ source: Seq[I],
+ handler: I => Iterator[O]): Iterator[O] = {
+val taskSupport = new ThreadPoolTaskSupport(threadpool)
+source.grouped(threadpool.getCorePoolSize).flatMap { element =>
--- End diff --
Nvm, i see you have implemented it differently than what we discussed.
Seems to work. In fact this is more evenly load balanced i think. Just change
`element` to `group`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11593][SQL] Replace catalyst converter ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9565#issuecomment-155697390 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11564][SQL][follow-up] clean up java tu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9567#issuecomment-155697232 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11564][SQL][follow-up] clean up java tu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9567#issuecomment-155697234 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/45588/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11564][SQL][follow-up] clean up java tu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9567#issuecomment-155697178 **[Test build #45588 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45588/consoleFull)** for PR 9567 at commit [`9fc5456`](https://github.com/apache/spark/commit/9fc5456812482f52b02a2d11b8b14c5bc89534b5). * This patch **fails from timeout after a configured wait of \`250m\`**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11646] WholeTextFileRDD should return T...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9622#issuecomment-155696018 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/45610/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11646] WholeTextFileRDD should return T...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9622#issuecomment-155696013 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11646] WholeTextFileRDD should return T...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9622#issuecomment-155695852 **[Test build #45610 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45610/consoleFull)** for PR 9622 at commit [`4d412dd`](https://github.com/apache/spark/commit/4d412dde4289c6fe55233c79f38d4c3ad5ee6f21). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_:\n * `public class JavaLBFGSExample `\n * `class LDA @Since(\"1.6.0\") (`\n * ` case class Metadata(`\n * ` require(className == expectedClassName, s\"Error loading metadata: Expected class name\" +`\n * `class SlidingRDDPartition[T](val idx: Int, val prev: Partition, val tail: Seq[T], val offset: Int)`\n * `class SlidingRDD[T: ClassTag](@transient val parent: RDD[T], val windowSize: Int, val step: Int)`\n --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...
Github user CharlesYeh commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-155695451 Thanks! I'll look into this --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11647][WIP] Attempt to reduce time/flak...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9623#issuecomment-155695179 **[Test build #45615 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45615/consoleFull)** for PR 9623 at commit [`801afe7`](https://github.com/apache/spark/commit/801afe7d1b0666c3f84c1e030ede72d0dc35300c). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10500][SPARKR] sparkr.zip cannot be cre...
Github user sun-rui commented on the pull request: https://github.com/apache/spark/pull/9390#issuecomment-155695077 I tested this PR on yarn-client/yarn-cluster with/without R source packages. All work fine. I may investigate later adding test cases for YARN mode. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11647][WIP] Attempt to reduce time/flak...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9623#issuecomment-155693464 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11647][WIP] Attempt to reduce time/flak...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9623#issuecomment-155693454 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11647][WIP] Attempt to reduce time/flak...
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/9623 [SPARK-11647][WIP] Attempt to reduce time/flakiness of Thriftserver CLI and SparkSubmit tests This work-in-progress patch aims to reduce the test time and flakiness of HiveSparkSubmitSuite, SparkSubmitSuite, and CliSuite. Key changes: - Disable IO synchronization calls for Derby writes, since durability doesn't matter for tests. This was done for HiveCompatibilitySuite in #6651 and resulted in huge test speedups. - Add a few missing `--conf`s to disable various Spark UIs. The CliSuite, in particular, never disabled these UIs, leaving it prone to port-contention-related flakiness. - Fix two instances where tests defined `beforeAll()` methods which were never called because the appropriate traits were not mixed in. I updated these tests suites to extend `BeforeAndAfterEach` so that they play nicely with our `ResetSystemProperties` trait. You can merge this pull request into a Git repository by running: $ git pull https://github.com/JoshRosen/spark SPARK-11647 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/9623.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #9623 commit 801afe7d1b0666c3f84c1e030ede72d0dc35300c Author: Josh Rosen Date: 2015-11-11T07:31:16Z Misc. test conf changes aimed at reducing test time / flakiness. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SQL][minor] rename present to finish in Aggre...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9617#issuecomment-155693254 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SQL][minor] rename present to finish in Aggre...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9617#issuecomment-155693255 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/45596/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SQL][minor] rename present to finish in Aggre...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9617#issuecomment-155693125 **[Test build #45596 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45596/consoleFull)** for PR 9617 at commit [`94ac5c1`](https://github.com/apache/spark/commit/94ac5c14aed98b9ef761b0e5af37082b677faba3). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_:\n * `public class JavaLBFGSExample `\n * `class LDA @Since(\"1.6.0\") (`\n * `class SlidingRDDPartition[T](val idx: Int, val prev: Partition, val tail: Seq[T], val offset: Int)`\n * `class SlidingRDD[T: ClassTag](@transient val parent: RDD[T], val windowSize: Int, val step: Int)`\n --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11419][STREAMING] Parallel recovery for...
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/9373#discussion_r44507814
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/util/FileBasedWriteAheadLog.scala
---
@@ -251,4 +261,22 @@ private[streaming] object FileBasedWriteAheadLog {
}
}.sortBy { _.startTime }
}
+
+ /**
+ * This creates an iterator from a parallel collection, by keeping at
most `n` objects in memory
+ * at any given time, where `n` is the size of the thread pool. This is
crucial for use cases
+ * where we create `FileBasedWriteAheadLogReader`s during parallel
recovery. We don't want to
+ * open up `k` streams altogether where `k` is the size of the Seq that
we want to parallelize.
+ */
+ def parallelIteratorCreator[I, O](
--- End diff --
Rename to `seqToParIterator`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11419][STREAMING] Parallel recovery for...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9373#issuecomment-155692990 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/45614/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11419][STREAMING] Parallel recovery for...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9373#issuecomment-155692988 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11419][STREAMING] Parallel recovery for...
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/9373#discussion_r44507453
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/util/WriteAheadLogSuite.scala
---
@@ -199,6 +198,63 @@ class FileBasedWriteAheadLogSuite
import WriteAheadLogSuite._
+ test("FileBasedWriteAheadLog - parallel readAll opens at most
'numThreads' files") {
+/*
+ If the setting `closeFileAfterWrite` is enabled, we start generating
a very large number of
+ files. This causes recovery to take a very long time. In order to
make it quicker, we
+ parallelized the reading of these files. This test makes sure that we
limit the number of
+ open files to the size of the number of threads in our thread pool
rather than the size of
+ the list of files.
+ */
+val numThreads = 8
+val tpool = ThreadUtils.newDaemonFixedThreadPool(numThreads,
"wal-test-thread-pool")
+class GetMaxCounter {
+ private val value = new AtomicInteger()
+ @volatile private var max: Int = 0
+ def increment(): Unit = {
--- End diff --
I think this needs synchronization across both max and value. Otherwise
there are race conditions. Put increment and decrement in `synchronized`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11419][STREAMING] Parallel recovery for...
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/9373#discussion_r44507368
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/util/FileBasedWriteAheadLog.scala
---
@@ -251,4 +261,22 @@ private[streaming] object FileBasedWriteAheadLog {
}
}.sortBy { _.startTime }
}
+
+ /**
+ * This creates an iterator from a parallel collection, by keeping at
most `n` objects in memory
+ * at any given time, where `n` is the size of the thread pool. This is
crucial for use cases
+ * where we create `FileBasedWriteAheadLogReader`s during parallel
recovery. We don't want to
+ * open up `k` streams altogether where `k` is the size of the Seq that
we want to parallelize.
+ */
+ def parallelIteratorCreator[I, O](
+ threadpool: ThreadPoolExecutor,
+ source: Seq[I],
+ handler: I => Iterator[O]): Iterator[O] = {
+val taskSupport = new ThreadPoolTaskSupport(threadpool)
+source.grouped(threadpool.getCorePoolSize).flatMap { element =>
--- End diff --
Actually, my bad. `grouped` creates groups of n, not n groups. We want n
groups.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9552] Add force control for killExecuto...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7888#issuecomment-155690986 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/45612/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9552] Add force control for killExecuto...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7888#issuecomment-155690981 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11639][STREAMING][FLAKY-TEST] Implement...
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/9605#issuecomment-155690734 I am merging this to master and 1.6. Thanks @brkyvz for getting this done fast --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11290][STREAMING] Basic implementation ...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9256 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11419][STREAMING] Parallel recovery for...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9373#issuecomment-155690192 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11419][STREAMING] Parallel recovery for...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9373#issuecomment-155690177 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6328] [Python] Python API for Streaming...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9186#issuecomment-155690123 **[Test build #45611 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45611/consoleFull)** for PR 9186 at commit [`5415389`](https://github.com/apache/spark/commit/5415389e95dbe642dab8f00acb2c57a08d8b97b3). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9552] Add force control for killExecuto...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7888#issuecomment-155689784 **[Test build #45613 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45613/consoleFull)** for PR 7888 at commit [`01c236a`](https://github.com/apache/spark/commit/01c236ad3cb435c8b63f8be59c3f5d099b797cf3). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9552] Add force control for killExecuto...
Github user GraceH commented on the pull request: https://github.com/apache/spark/pull/7888#issuecomment-155689636 @vanzin @andrewor14 I have changed code accordingly. Please let me know your comments. Meanwhile, I will try to add unit tests. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9552] Add force control for killExecuto...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7888#issuecomment-155689576 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9552] Add force control for killExecuto...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7888#issuecomment-155689565 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2629][STREAMING] Basic implementation o...
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/9256#issuecomment-155688949 All right. I am merging this for now. I have a few more changes, refactorings and more unit tests that I am working on, but that can be done later as well. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9552] Add force control for killExecuto...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7888#issuecomment-155687798 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9552] Add force control for killExecuto...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7888#issuecomment-155687837 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11062] Thrift server does not support o...
Github user navis closed the pull request at: https://github.com/apache/spark/pull/9074 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6541 - Sort executors by ID (numeric)
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9165#issuecomment-155687000 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/45594/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6541 - Sort executors by ID (numeric)
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9165#issuecomment-155686998 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8333] [SQL] Spark failed to delete temp...
Github user navis closed the pull request at: https://github.com/apache/spark/pull/6858 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-6541 - Sort executors by ID (numeric)
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9165#issuecomment-155686965 **[Test build #45594 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45594/consoleFull)** for PR 9165 at commit [`1dc37ea`](https://github.com/apache/spark/commit/1dc37ea63bce699e0488453cb1ba6fa7e037a241). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_:\n * ` case class Metadata(`\n * ` require(className == expectedClassName, s\"Error loading metadata: Expected class name\" +`\n --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6328] [Python] Python API for Streaming...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9186#issuecomment-155686757 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6328] [Python] Python API for Streaming...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9186#issuecomment-155686739 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155686323 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155686247 **[Test build #45604 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45604/consoleFull)** for PR 9618 at commit [`dd1fe92`](https://github.com/apache/spark/commit/dd1fe927fe50f397bd6bf90e6015e8e57f06b26a). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_:\n * `public class JavaLBFGSExample `\n * `class LDA @Since(\"1.6.0\") (`\n * ` case class Metadata(`\n * ` require(className == expectedClassName, s\"Error loading metadata: Expected class name\" +`\n --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155686326 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/45604/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11419][STREAMING] Parallel recovery for...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9373#issuecomment-155685330 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/45601/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11419][STREAMING] Parallel recovery for...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9373#issuecomment-155685296 **[Test build #45601 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45601/consoleFull)** for PR 9373 at commit [`c9ea423`](https://github.com/apache/spark/commit/c9ea4238e63b579f42179b1da6c00140d9aa0c0f). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11419][STREAMING] Parallel recovery for...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9373#issuecomment-155685327 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11468][SPARKR] add stddev/variance agg ...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9489 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11463] [PYSPARK] only install signal in...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9574 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11468][SPARKR] add stddev/variance agg ...
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9489#issuecomment-155683568 Merging this into master and 1.6 branch, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11625][SQL] add java test for typed agg...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9591#issuecomment-155683387 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/45595/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Branch 1.4
Github user f7753 closed the pull request at: https://github.com/apache/spark/pull/9616 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11625][SQL] add java test for typed agg...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9591#issuecomment-155683348 **[Test build #45595 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45595/consoleFull)** for PR 9591 at commit [`ae55976`](https://github.com/apache/spark/commit/ae55976c11d7084606e39f49f0753b701f18f8bb). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_:\n * `abstract class Aggregator[-A, B, C] extends Serializable `\n --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11625][SQL] add java test for typed agg...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9591#issuecomment-155683386 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9957][ML][WIP] spark ml trees should fi...
Github user holdenk closed the pull request at: https://github.com/apache/spark/pull/8540 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11646] WholeTextFileRDD should return T...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9622#issuecomment-155681800 **[Test build #45610 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45610/consoleFull)** for PR 9622 at commit [`4d412dd`](https://github.com/apache/spark/commit/4d412dde4289c6fe55233c79f38d4c3ad5ee6f21). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11646] WholeTextFileRDD should return T...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9622#issuecomment-155681664 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11646] WholeTextFileRDD should return T...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9622#issuecomment-155681681 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11646] WholeTextFileRDD should return T...
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/9622 [SPARK-11646] WholeTextFileRDD should return Text rather than String If it returns Text, we can reuse this in Spark SQL to provide a WholeTextFile data source and directly convert the Text into UTF8String without extra string decoding and encoding. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rxin/spark SPARK-11646 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/9622.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #9622 commit 4d412dde4289c6fe55233c79f38d4c3ad5ee6f21 Author: Reynold Xin Date: 2015-11-11T06:36:38Z [SPARK-11646] WholeTextFileRDD should return Text rather than String --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11468][SPARKR] add stddev/variance agg ...
Github user sun-rui commented on the pull request: https://github.com/apache/spark/pull/9489#issuecomment-155681439 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2629][STREAMING] Basic implementation o...
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/9256#issuecomment-155681290 @tdas, look good. My comments are minor. There are some unnecessary Java APIs in this PR. But I can fix them while I'm adding the Java APIs. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6328] [Python] Python API for Streaming...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9186#issuecomment-155681209 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/45599/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6328] [Python] Python API for Streaming...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9186#issuecomment-155681206 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11564][SQL][follow-up] clean up java tu...
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/9567#discussion_r44505316
--- Diff:
sql/core/src/test/java/test/org/apache/spark/sql/JavaDatasetSuite.java ---
@@ -66,7 +65,7 @@ public void tearDown() {
@Test
public void testCollect() {
List data = Arrays.asList("hello", "world");
-Dataset ds = context.createDataset(data, e.STRING());
+Dataset ds = context.createDataset(data, Encoders.STRING());
--- End diff --
oh awesome!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6328] [Python] Python API for Streaming...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9186#issuecomment-155681175 **[Test build #45599 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45599/consoleFull)** for PR 9186 at commit [`9e4e04a`](https://github.com/apache/spark/commit/9e4e04a6c369da13c3cc8e798f5a5b0e210b24d5). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_:\n * `class StreamingListener(object):`\n * `class PythonStreamingListenerWrapper(listener: PythonStreamingListener)`\n --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11628][SQL][WIP] support column datatyp...
Github user Zhangshunyu commented on a diff in the pull request:
https://github.com/apache/spark/pull/9612#discussion_r44505240
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala
---
@@ -96,6 +96,7 @@ private[sql] object JDBCRDD extends Logging {
case java.sql.Types.TINYINT => IntegerType
case java.sql.Types.VARBINARY => BinaryType
case java.sql.Types.VARCHAR => StringType
+ case java.sql.Types.CHAR => StringType
--- End diff --
case java.sql.Types.CHAR => StringType
Repeat code between line99 and line 65, plz delete this code.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11645][SQL] Remove OpenHashSet for the ...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/9621#issuecomment-155680947 cc @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6152] Use shaded ASM5 to support closur...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9512#issuecomment-155680821 **[Test build #45609 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45609/consoleFull)** for PR 9512 at commit [`9833667`](https://github.com/apache/spark/commit/9833667c88000d895edbc90439cd2729325d6e4a). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11617] [network] Fix leak in TransportF...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9619#issuecomment-155680777 **[Test build #45608 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45608/consoleFull)** for PR 9619 at commit [`ed0c1d7`](https://github.com/apache/spark/commit/ed0c1d7e6df3357344574b3ef2dccb526ed3f9b7). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10827] replace volatile with Atomic* in...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9611#issuecomment-155680676 **[Test build #2037 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2037/consoleFull)** for PR 9611 at commit [`b083309`](https://github.com/apache/spark/commit/b0833098acc9662ec2ca182fbb271977742cc400). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155680594 **[Test build #2039 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2039/consoleFull)** for PR 9618 at commit [`dd1fe92`](https://github.com/apache/spark/commit/dd1fe927fe50f397bd6bf90e6015e8e57f06b26a). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11617] [network] Fix leak in TransportF...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9619#issuecomment-155680629 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6152] Use shaded ASM5 to support closur...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9512#issuecomment-155680628 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6152] Use shaded ASM5 to support closur...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9512#issuecomment-155680616 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11617] [network] Fix leak in TransportF...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9619#issuecomment-155680615 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6152] Use shaded ASM5 to support closur...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9512#issuecomment-155680433 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11645][SQL] Remove OpenHashSet for the ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9621#issuecomment-155680409 **[Test build #45607 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45607/consoleFull)** for PR 9621 at commit [`bfdc937`](https://github.com/apache/spark/commit/bfdc9375d62c2d05d0d360afd643730a54b97d8f). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11645][SQL] Remove OpenHashSet for the ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9621#issuecomment-155680436 **[Test build #2038 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2038/consoleFull)** for PR 9621 at commit [`bfdc937`](https://github.com/apache/spark/commit/bfdc9375d62c2d05d0d360afd643730a54b97d8f). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11617] [network] Fix leak in TransportF...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9619#issuecomment-155680397 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [HOTFIX][SPARK-10192] Fix NPE in test that was...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9620 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11584][SQL] The attribute of temporay t...
Github user l00357089 commented on the pull request: https://github.com/apache/spark/pull/9588#issuecomment-155680035 @dilipbiswal I'm a new face here and I 'd like to join you three to learn something. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10192] [core] simple test w/ failure in...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/8402#issuecomment-155679955 Verified locally that this fixes the tests, so I'm merging to master and 1.6. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155679850 **[Test build #45604 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45604/consoleFull)** for PR 9618 at commit [`dd1fe92`](https://github.com/apache/spark/commit/dd1fe927fe50f397bd6bf90e6015e8e57f06b26a). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11645][SQL] Remove OpenHashSet for the ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9621#issuecomment-155679566 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11645][SQL] Remove OpenHashSet for the ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9621#issuecomment-155679547 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [HOTFIX] Fix build-breaking NPE introduced in ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9620#issuecomment-155679448 **[Test build #45605 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45605/consoleFull)** for PR 9620 at commit [`f9fcbf3`](https://github.com/apache/spark/commit/f9fcbf37facaedf4887f926729c2e14eb56caf09). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11645][SQL] Remove all the set-related ...
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/9621 [SPARK-11645][SQL] Remove all the set-related expressions for the old aggregate. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rxin/spark SPARK-11645 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/9621.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #9621 commit bfdc9375d62c2d05d0d360afd643730a54b97d8f Author: Reynold Xin Date: 2015-11-11T06:13:52Z [SPARK-11645][SQL] Remove all the set-related expressions for the old aggregate. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10827] replace volatile with Atomic* in...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9611#issuecomment-155679128 **[Test build #2036 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2036/consoleFull)** for PR 9611 at commit [`b083309`](https://github.com/apache/spark/commit/b0833098acc9662ec2ca182fbb271977742cc400). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11102] [SQL] Uninformative exception wh...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9490#issuecomment-155679162 **[Test build #45606 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45606/consoleFull)** for PR 9490 at commit [`219db87`](https://github.com/apache/spark/commit/219db877bc4a599e76ae1a0541db94b387540a05). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11102] [SQL] Uninformative exception wh...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9490#issuecomment-155679059 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [HOTFIX] Fix build-breaking NPE introduced in ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9620#issuecomment-155679058 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [HOTFIX] Fix build-breaking NPE introduced in ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9620#issuecomment-155679047 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11102] [SQL] Uninformative exception wh...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9490#issuecomment-155679050 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11081] Shade Jersey and javax.rs.ws
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/9615#issuecomment-155678946 Another thing to keep in mind, given the recent issues with `@VisibleForTesting`: JAX-RS APIs use lots of annotations, and if you shade them, that is likely to cause problems with scalac-generated code (see #9585). The good news is that since new versions of these APIs are backwards compatible, you shouldn't need to shade the `javax.rs` stuff at all; someone who wants to override them just needs to put the newer version in front of Spark's classpath. It's not as easy as including it in your application, but it's doable. Given that, I wonder if it's even necessary to shade jersey at all; the version of jersey used in Spark is old and doesn't clash with the new package name used by newer releases of Jersey. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11617] [network] Fix leak in TransportF...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9619#issuecomment-155678833 **[Test build #45603 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45603/consoleFull)** for PR 9619 at commit [`ed0c1d7`](https://github.com/apache/spark/commit/ed0c1d7e6df3357344574b3ef2dccb526ed3f9b7). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11396] [SQL] add native implementation ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9347#issuecomment-155678770 **[Test build #45602 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45602/consoleFull)** for PR 9347 at commit [`533775f`](https://github.com/apache/spark/commit/533775f57c852911036badd0a3113f9da7c9b628). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
