[GitHub] spark pull request: [SPARK-12506][SQL]push down WHERE clause arith...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10503#issuecomment-167741641 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12506][SQL]push down WHERE clause arith...
GitHub user wilson8 opened a pull request: https://github.com/apache/spark/pull/10503 [SPARK-12506][SQL]push down WHERE clause arithmetic operator to JDBC ⦠â¦layer For arithmetic operator in WHERE clause such as select * from table where c1 + c2 > 10 Currently where c1 + c2 >10 is done at spark layer. Will push this to JDBC layer so it will be done in database. You can merge this pull request into a Git repository by running: $ git pull https://github.com/huaxingao/spark spark-12506 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/10503.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #10503 commit 09e2135d6e5abef21328941d73031109e6d4d4b6 Author: Huaxin Gao Date: 2015-12-27T19:58:12Z [SPARK-12506][SQL]push down WHERE clause arithmetic operator to JDBC layer --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-8426] [scheduler] enhance blacklist mec...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8760#issuecomment-167741271 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-8426] [scheduler] enhance blacklist mec...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8760#issuecomment-167741274 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48401/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-8426] [scheduler] enhance blacklist mec...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8760#issuecomment-167741137 **[Test build #48401 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48401/consoleFull)** for PR 8760 at commit [`d94bdd7`](https://github.com/apache/spark/commit/d94bdd7d16b77317bdfe5851e9ba297a229b4113). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12409][SPARK-12387][SPARK-12391][SQL] S...
Github user sarutak commented on a diff in the pull request:
https://github.com/apache/spark/pull/10468#discussion_r48525108
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala
---
@@ -186,8 +187,26 @@ class JDBCSuite extends SparkFunSuite
assert(stripSparkFilter(sql("SELECT * FROM foobar WHERE NAME =
'fred'")).collect().size == 1)
assert(stripSparkFilter(sql("SELECT * FROM foobar WHERE NAME >
'fred'")).collect().size == 2)
assert(stripSparkFilter(sql("SELECT * FROM foobar WHERE NAME !=
'fred'")).collect().size == 2)
+assert(stripSparkFilter(sql("SELECT * FROM foobar WHERE NAME IN
('mary', 'fred')"))
+ .collect().size == 2)
--- End diff --
Ah yes, now `Assertions` trait provides `assert` macro and the trait mixes
in the `TripleEquals` trait so we don't need to change `==` to `===`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12486] Worker should kill the executors...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10438#issuecomment-167739453 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48398/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12486] Worker should kill the executors...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10438#issuecomment-167739452 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12486] Worker should kill the executors...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10438#issuecomment-167739406 **[Test build #48398 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48398/consoleFull)** for PR 10438 at commit [`67611ac`](https://github.com/apache/spark/commit/67611acec29cb6cadadc038f27759c19578f6e21). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10502#issuecomment-167738930 **[Test build #48405 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48405/consoleFull)** for PR 10502 at commit [`2ad8182`](https://github.com/apache/spark/commit/2ad81822100229318d1d80796edb92916a10d820). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12470] [SQL] Fix size reduction calcula...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10421#issuecomment-167738673 I think @davies did some pretty big refactoring of this later. @davies can you take a quick look at this? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARKR] [SPARK-11199] Improve R context manag...
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/9185#issuecomment-167738526 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Add more exceptions to Guava relocation
Github user microhello commented on the pull request: https://github.com/apache/spark/pull/10442#issuecomment-167738304 @andrewor14 I have created a issue: https://issues.apache.org/jira/browse/SPARK-12548 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12526][SPARKR]`ifelse`, `when`, `otherw...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10481 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12409][SPARK-12387][SPARK-12391][SQL] R...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10470#issuecomment-167738141 @maropu can you review this change? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12409][SPARK-12387][SPARK-12391][SQL] R...
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10470#discussion_r48524499
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala
---
@@ -184,16 +185,38 @@ private[sql] object JDBCRDD extends Logging {
* Turns a single Filter into a String representing a SQL expression.
* Returns null for an unhandled filter.
--- End diff --
need to update this comment
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12470] [SQL] Fix size reduction calcula...
Github user robbinspg commented on the pull request: https://github.com/apache/spark/pull/10421#issuecomment-167738123 @rxin as the original author of this code could you please review the PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12409][SPARK-12387][SPARK-12391][SQL] S...
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10468#discussion_r48524447
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala
---
@@ -186,8 +187,26 @@ class JDBCSuite extends SparkFunSuite
assert(stripSparkFilter(sql("SELECT * FROM foobar WHERE NAME =
'fred'")).collect().size == 1)
assert(stripSparkFilter(sql("SELECT * FROM foobar WHERE NAME >
'fred'")).collect().size == 2)
assert(stripSparkFilter(sql("SELECT * FROM foobar WHERE NAME !=
'fred'")).collect().size == 2)
+assert(stripSparkFilter(sql("SELECT * FROM foobar WHERE NAME IN
('mary', 'fred')"))
+ .collect().size == 2)
--- End diff --
Does the newer version of scalatest library we use already uses macro so ==
and === are the same? Can you confirm? Anyway it's not that big of a deal here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10435#issuecomment-167737270 **[Test build #48404 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48404/consoleFull)** for PR 10435 at commit [`1cdb2bc`](https://github.com/apache/spark/commit/1cdb2bcc1ede58fdd9c1e98bff4b5544b8a6e74e). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10502#issuecomment-167737261 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10502#issuecomment-167737265 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48402/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10502#issuecomment-167737190 **[Test build #48402 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48402/consoleFull)** for PR 10502 at commit [`8c376af`](https://github.com/apache/spark/commit/8c376af467b939d3687e24f7d0e1ccf4e0ea5fca). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12409][SPARK-12387][SPARK-12391][SQL] S...
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10468#discussion_r48524429
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala
---
@@ -186,13 +187,19 @@ private[sql] object JDBCRDD extends Logging {
*/
private def compileFilter(f: Filter): String = f match {
case EqualTo(attr, value) => s"$attr = ${compileValue(value)}"
-case Not(EqualTo(attr, value)) => s"$attr != ${compileValue(value)}"
+case Not(f) => s"NOT (${compileFilter(f)})"
--- End diff --
Yes
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10502#issuecomment-167736500 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10502#issuecomment-167736506 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48403/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10502#issuecomment-167736372 **[Test build #48403 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48403/consoleFull)** for PR 10502 at commit [`8c376af`](https://github.com/apache/spark/commit/8c376af467b939d3687e24f7d0e1ccf4e0ea5fca). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12218] Fixes ORC conjunction predicate ...
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10377#issuecomment-167735494 Thanks for the review! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12526][SPARKR]`ifelse`, `when`, `otherw...
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/10481#issuecomment-167735437 LGTM. Thanks @saurfang and @sun-rui - Merging this --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10502#issuecomment-167734521 **[Test build #48402 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48402/consoleFull)** for PR 10502 at commit [`8c376af`](https://github.com/apache/spark/commit/8c376af467b939d3687e24f7d0e1ccf4e0ea5fca). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10502#issuecomment-167733908 **[Test build #48403 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48403/consoleFull)** for PR 10502 at commit [`8c376af`](https://github.com/apache/spark/commit/8c376af467b939d3687e24f7d0e1ccf4e0ea5fca). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12149] [Web UI] Executor UI improvement...
Github user holdenk commented on the pull request: https://github.com/apache/spark/pull/10154#issuecomment-167733524 I think green for completed makes sense, and having it always stand out doesn't seem like a particularly bad thing. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...
Github user HyukjinKwon commented on the pull request: https://github.com/apache/spark/pull/10502#issuecomment-167733162 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12526][SPARKR]`ifelse`, `when`, `otherw...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10481#issuecomment-167730405 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48395/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12526][SPARKR]`ifelse`, `when`, `otherw...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10481#issuecomment-167730404 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10435#issuecomment-167730352 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12526][SPARKR]`ifelse`, `when`, `otherw...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10481#issuecomment-167730355 **[Test build #48395 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48395/consoleFull)** for PR 10481 at commit [`052cb51`](https://github.com/apache/spark/commit/052cb518cd5b9eef412946dbb54750816187bb50). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10435#issuecomment-167730354 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48399/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10435#issuecomment-167730331 **[Test build #48399 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48399/consoleFull)** for PR 10435 at commit [`84902bb`](https://github.com/apache/spark/commit/84902bb2b9488212e0006bf359135073b8a9e496). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167729202 **[Test build #48400 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48400/consoleFull)** for PR 10451 at commit [`358d62e`](https://github.com/apache/spark/commit/358d62e7736191420f0d0a364269baa4fa54b0cb). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...
Github user HyukjinKwon commented on the pull request: https://github.com/apache/spark/pull/10502#issuecomment-167729051 I see. `UnsafeRowParquetRecordReader` at Parquet does not support filter record by record but just block. So, even with `=` operator produces the same results below. ``` +---+ | _1| +---+ | 5| | 6| | 7| | 8| | 9| +---+ ``` I think I should disable `unhandledFilters` when --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-8426] [scheduler] enhance blacklist mec...
Github user mwws commented on the pull request: https://github.com/apache/spark/pull/8760#issuecomment-167728914 @mridulm I have changed the interface a little bit and create `AdvancedSingleTaskStrategy` to support the use case you described above. With this new strategy, we enable node blacklist and scope of influence is limited in every stage. A unit test *AdvancedSingleTask strategy works* is added to show how it works. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [Spark-8426] [scheduler] enhance blacklist mec...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8760#issuecomment-167728825 **[Test build #48401 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48401/consoleFull)** for PR 8760 at commit [`d94bdd7`](https://github.com/apache/spark/commit/d94bdd7d16b77317bdfe5851e9ba297a229b4113). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10435#issuecomment-167727788 **[Test build #48399 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48399/consoleFull)** for PR 10435 at commit [`84902bb`](https://github.com/apache/spark/commit/84902bb2b9488212e0006bf359135073b8a9e496). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/10451#discussion_r48523071
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala
---
@@ -91,6 +91,11 @@ abstract class LogicalPlan extends
QueryPlan[LogicalPlan] with Logging {
}
/**
+ * Returns the limited number of rows to be returned.
--- End diff --
Fixed. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12536] [SQL] Added "Empty Seq" in Expla...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10494#issuecomment-167725381 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12536] [SQL] Added "Empty Seq" in Expla...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10494#issuecomment-167725386 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48394/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12536] [SQL] Added "Empty Seq" in Expla...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10494#issuecomment-167725034 **[Test build #48394 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48394/consoleFull)** for PR 10494 at commit [`c60d7f0`](https://github.com/apache/spark/commit/c60d7f0dec4d8d43a1accdd3f98e0e2e51c0f3a3). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * ` * Abstract class all optimizers should inherit of, contains the standard batches (extending` * `abstract class Optimizer extends RuleExecutor[LogicalPlan] ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12486] Worker should kill the executors...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10438#issuecomment-167723226 **[Test build #48398 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48398/consoleFull)** for PR 10438 at commit [`67611ac`](https://github.com/apache/spark/commit/67611acec29cb6cadadc038f27759c19578f6e21). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11394][SQL] Throw IllegalArgumentExcept...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9350#issuecomment-167722881 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48397/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11394][SQL] Throw IllegalArgumentExcept...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9350#issuecomment-167722880 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...
Github user HyukjinKwon commented on the pull request: https://github.com/apache/spark/pull/10502#issuecomment-167722257 The test is failed from wrong results from Parquet. The test result was below: ``` == Physical Plan == Scan ParquetRelation[_1#4] InputPaths: file:/private/var/folders/9j/gf_c342d7d150mwrxvkqnc18gn/T/spark-2fa8f75a-8ac1-4273-afcd-f271529a24a7, PushedFilters: [GreaterThan(_1,5)] ``` ``` == Results == !== Correct Answer - 4 == == Spark Answer - 5 == ![6][5] ![7][6] ![8][7] ![9][8] ! [9] ``` As can be seen, `GreaterThan(_1,5)` is pushed down correctly but the results includes `5`. I am looking into this deeper. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12486] Worker should kill the executors...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/10438#issuecomment-167722251 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11394][SQL] Throw IllegalArgumentExcept...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9350 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11394][SQL] Throw IllegalArgumentExcept...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9350#issuecomment-167721434 Merging to branch 1.6 and master. Thank you! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11394][SQL] Throw IllegalArgumentExcept...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9350#issuecomment-167721192 oh, it already passed the tests. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11394][SQL] Throw IllegalArgumentExcept...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9350#issuecomment-167721041 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10502#issuecomment-167720165 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10502#issuecomment-167720172 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48396/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10502#issuecomment-167720147 **[Test build #48396 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48396/consoleFull)** for PR 10502 at commit [`cf331a4`](https://github.com/apache/spark/commit/cf331a453c3f99ee40ee5ca6f5029dadee3d07f6). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10502#issuecomment-167719205 **[Test build #48396 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48396/consoleFull)** for PR 10502 at commit [`cf331a4`](https://github.com/apache/spark/commit/cf331a453c3f99ee40ee5ca6f5029dadee3d07f6). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12526][SPARKR]`ifelse`, `when`, `otherw...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10481#issuecomment-167718452 **[Test build #48395 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48395/consoleFull)** for PR 10481 at commit [`052cb51`](https://github.com/apache/spark/commit/052cb518cd5b9eef412946dbb54750816187bb50). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/10502 [SPARK-12355][SQL] Implement unhandledFilter interface for Parquet https://issues.apache.org/jira/browse/SPARK-12355 This is similar with https://github.com/apache/spark/pull/10427. As discussed here https://github.com/apache/spark/pull/10221, this PR implemented `unhandledFilter` to remove duplicated Spark-side filtering. In case of Parquet, the columns referenced in pushed down filters should be given to `org.apache.spark.sql.parquet.row.requested_schema` whereas general datasources such as JDBC do not require the columns. However, the current interface of `DataSourceStrategy` removes the columns only referenced in pushed down filters. Therefore, this PR resolved this problem by manually generating the columns referenced in pushed down filters. You can merge this pull request into a Git repository by running: $ git pull https://github.com/HyukjinKwon/spark SPARK-12355 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/10502.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #10502 commit a55ad54a820f095e5116df05979f169c0fe8e0cf Author: hyukjinkwon Date: 2015-12-29T04:34:53Z Implement unhandled filters for Parquet commit cf331a453c3f99ee40ee5ca6f5029dadee3d07f6 Author: hyukjinkwon Date: 2015-12-29T04:36:42Z Correct existing tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12547][SQL] Tighten scala style checker...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10501 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12547][SQL] Tighten scala style checker...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/10501#issuecomment-167718080 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12526][SPARKR]`ifelse`, `when`, `otherw...
Github user saurfang commented on the pull request: https://github.com/apache/spark/pull/10481#issuecomment-167717447 Thanks for the review @sun-rui. Hope that's better. Looks like `lintr`, as awesome as it is, let that slip through, which I have filed a separate issue here: https://github.com/jimhester/lintr/pull/128 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12536] [SQL] Added "Empty Seq" in Expla...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10494#issuecomment-167717428 **[Test build #48394 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48394/consoleFull)** for PR 10494 at commit [`c60d7f0`](https://github.com/apache/spark/commit/c60d7f0dec4d8d43a1accdd3f98e0e2e51c0f3a3). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12536] [SQL] Added "Empty Seq" in Expla...
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10494#discussion_r48521080 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LocalRelation.scala --- @@ -62,6 +62,10 @@ case class LocalRelation(output: Seq[Attribute], data: Seq[InternalRow] = Nil) case _ => false } + override def simpleString: String = +if (data == Seq.empty) super.simpleString + " [Empty Seq]" --- End diff -- Thank you! @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12547][SQL] Tighten scala style checker...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10501#issuecomment-167716481 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12547][SQL] Tighten scala style checker...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10501#issuecomment-167716483 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48392/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12547][SQL] Tighten scala style checker...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10501#issuecomment-167716360 **[Test build #48392 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48392/consoleFull)** for PR 10501 at commit [`5157f27`](https://github.com/apache/spark/commit/5157f276a68eef3eebf70df66ee526f1529ac354). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12547][SQL] Tighten scala style checker...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10501#issuecomment-167715541 **[Test build #48393 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48393/consoleFull)** for PR 10501 at commit [`c6d6ba0`](https://github.com/apache/spark/commit/c6d6ba066bab90b46ccfd7f7080b7c9520c8ad0a). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [DOC] Adjust coverage for partitionBy()
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10499#issuecomment-167715531 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48391/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [DOC] Adjust coverage for partitionBy()
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10499#issuecomment-167715530 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [DOC] Adjust coverage for partitionBy()
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10499#issuecomment-167715498 **[Test build #48391 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48391/consoleFull)** for PR 10499 at commit [`dff3935`](https://github.com/apache/spark/commit/dff3935b571bcbf121aa017b1cf52bc5757d04ab). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-167715428 oh sorry I haven't noticed it... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12512][SQL] support column name with do...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10500#issuecomment-167715207 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12512][SQL] support column name with do...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10500#issuecomment-167715208 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48390/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12512][SQL] support column name with do...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10500#issuecomment-167715171 **[Test build #48390 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48390/consoleFull)** for PR 10500 at commit [`6372f92`](https://github.com/apache/spark/commit/6372f92d7ce57cdf12ed98af513b11d97e613a88). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12513] [Streaming] SocketReceiver hang ...
Github user guoxu1231 commented on the pull request: https://github.com/apache/spark/pull/10464#issuecomment-167714955 @zsxwing However, it looks unlikely since it sleeps 2 seconds before calling startReceiver(). Is it possible about thread pool capability issue and 2 seconds sleep is not guaranteed? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12547][SQL] Tighten scala style checker...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10501#issuecomment-167714800 Yea that's an interesting one -- although in some files we might actually want that and I don't know whether we can still disable it (maybe we can?). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12547][SQL] Tighten scala style checker...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/10501#issuecomment-167714726 Can we add a Scalastyle rule to match the shotgun-approach `scalastyle:off` directive? :smiley: --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user viirya closed the pull request at: https://github.com/apache/spark/pull/9883 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-167714278 Looks like I need to close another pr again... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/10435#issuecomment-167713829 Actually I have already created a pr #9883 for this long time ago... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12513] [Streaming] SocketReceiver hang ...
Github user guoxu1231 commented on the pull request: https://github.com/apache/spark/pull/10464#issuecomment-167711102 @srowen, thanks for the reply. I tested it in my yarn environment, it looks more stable now. In current implementation, restart will start new socket in new thread in a fixed sleep interval and old socket resource will be released in finally block in old thread, in some rare cicumastance old socket resource may not be released properly(not sure why it happened, after 10 hours or sometimes 36 hous continous running). In this PR, I suggest to move the socket release to Receive.onStop(), it will be explictly invoked and release the socket resource before start new Socket connection, should be safer and keep the same behaviour like other Receivers and release resource in onStop() method. @zsxwing, @jerryshao --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10435#issuecomment-167710233 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48388/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10435#issuecomment-167710232 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12495][SQL] use true as default value f...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10443#issuecomment-167710193 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10435#issuecomment-167710215 **[Test build #48388 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48388/consoleFull)** for PR 10435 at commit [`6311aa7`](https://github.com/apache/spark/commit/6311aa75a7a41fee8464ee96e5949ccad3e7d7a5). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12495][SQL] use true as default value f...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10443#issuecomment-167710195 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48389/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12495][SQL] use true as default value f...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10443#issuecomment-167710116 **[Test build #48389 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48389/consoleFull)** for PR 10443 at commit [`a6b826c`](https://github.com/apache/spark/commit/a6b826c4cd55545e2ca2f1478a16c030bc0a86df). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12340][SQL]fix Int overflow in the Spar...
Github user QiangCai commented on the pull request: https://github.com/apache/spark/pull/10487#issuecomment-167709123 @sarutak I will try to add test cases. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12512][SQL] support column name with do...
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/10500#issuecomment-167705370 LGTM, pending test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12203][STREAMING] Add KafkaDirectInputD...
Github user viirya closed the pull request at: https://github.com/apache/spark/pull/10197 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12203][STREAMING] Add KafkaDirectInputD...
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/10197#issuecomment-167704274 I would like to close this now. But the latency should be a problem in real use case. You can see a [benchmarking](http://yahooeng.tumblr.com/post/135321837876/benchmarking-streaming-computation-engines-at) result. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12547][SQL] Tighten scala style checker...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10501#issuecomment-167703839 **[Test build #48392 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48392/consoleFull)** for PR 10501 at commit [`5157f27`](https://github.com/apache/spark/commit/5157f276a68eef3eebf70df66ee526f1529ac354). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARKR] [SPARK-11199] Improve R context manag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9185#issuecomment-167703848 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48386/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARKR] [SPARK-11199] Improve R context manag...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9185#issuecomment-167703846 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARKR] [SPARK-11199] Improve R context manag...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9185#issuecomment-167703809 **[Test build #48386 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48386/consoleFull)** for PR 9185 at commit [`0633a73`](https://github.com/apache/spark/commit/0633a73ddbc6a328d579434f3c3ec349765d70ef). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [DOC] Adjust coverage for partitionBy()
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10499#issuecomment-167703422 **[Test build #48391 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48391/consoleFull)** for PR 10499 at commit [`dff3935`](https://github.com/apache/spark/commit/dff3935b571bcbf121aa017b1cf52bc5757d04ab). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
