[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[maropu]

Github user maropu commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-214131211
  
Close this for now and, if needed, I'll reopen this.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[maropu]

Github user maropu closed the pull request at:

https://github.com/apache/spark/pull/10631


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[maropu]

Github user maropu commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-173866941
  
@rxin Okay and I'll wait for that.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[maropu]

Github user maropu commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-173552264
  
@rxin ping


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-173787354
  
I will add this to the review queue. But to be honest this API is super 
hard to design and I am not sure if there is an easy solution. The priority is 
also not very high so I don't know when I will get to it.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-171681629
  
**[Test build #49401 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49401/consoleFull)**
 for PR 10631 at commit 
[`9f039e6`](https://github.com/apache/spark/commit/9f039e610263ec8ac1f98639e1d618564dd54107).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-171681713
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-171681714
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49401/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-171723257
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-171722910
  
**[Test build #49402 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49402/consoleFull)**
 for PR 10631 at commit 
[`736a3ab`](https://github.com/apache/spark/commit/736a3ab9ceca9d1e00a4644cae38e3129c325541).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-171723261
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49402/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-171635825
  
**[Test build #49399 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49399/consoleFull)**
 for PR 10631 at commit 
[`c9cb213`](https://github.com/apache/spark/commit/c9cb2137b2343d86fcad7e4da00ac2903892d9eb).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-171635572
  
Build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-171637086
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49399/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-171637079
  
**[Test build #49399 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49399/consoleFull)**
 for PR 10631 at commit 
[`c9cb213`](https://github.com/apache/spark/commit/c9cb2137b2343d86fcad7e4da00ac2903892d9eb).
 * This patch **fails to build**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-171637085
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-171635576
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49398/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-171656357
  
**[Test build #49400 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49400/consoleFull)**
 for PR 10631 at commit 
[`5f6441d`](https://github.com/apache/spark/commit/5f6441dffe45722590cd72fb2182191bf4521b4b).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-171659679
  
**[Test build #49400 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49400/consoleFull)**
 for PR 10631 at commit 
[`5f6441d`](https://github.com/apache/spark/commit/5f6441dffe45722590cd72fb2182191bf4521b4b).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-171659736
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49400/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[maropu]

Github user maropu commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-171866035
  
@rxin Could you give me any suggestion on this workaround?

https://github.com/apache/spark/pull/10631/files#diff-d99813bd5bbc18277e4090475e4944cfR130


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-171659734
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-171677678
  
**[Test build #49401 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49401/consoleFull)**
 for PR 10631 at commit 
[`9f039e6`](https://github.com/apache/spark/commit/9f039e610263ec8ac1f98639e1d618564dd54107).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-171690890
  
**[Test build #49402 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49402/consoleFull)**
 for PR 10631 at commit 
[`736a3ab`](https://github.com/apache/spark/commit/736a3ab9ceca9d1e00a4644cae38e3129c325541).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[maropu]

Github user maropu commented on a diff in the pull request:

https://github.com/apache/spark/pull/10631#discussion_r49049015
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala ---
@@ -286,7 +286,7 @@ trait PrunedScan {
  */
 @DeveloperApi
 trait PrunedFilteredScan {
-  def buildScan(requiredColumns: Array[String], filters: Array[Filter]): 
RDD[Row]
+  def buildScan(requiredColumns: Seq[String], filters: Seq[Filter], 
aggregate: Aggregate): RDD[Row]
--- End diff --

Sorry and I'm not familiar this interface issue.
The fix satisfies your comment?

https://github.com/apache/spark/pull/10631/files#diff-40c347747af9101e7e9fee52fc4120b8R290


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[rxin]

Github user rxin commented on a diff in the pull request:

https://github.com/apache/spark/pull/10631#discussion_r49049257
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala ---
@@ -301,6 +301,8 @@ trait PrunedScan {
 @DeveloperApi
 trait PrunedFilteredScan {
   def buildScan(requiredColumns: Array[String], filters: Array[Filter]): 
RDD[Row]
+  def buildScan(
--- End diff --

this also breaks all existing implementations because we can't add a method 
to the interface.

Really - it is really hard to design a public interface that works for 
complicated cases (e.g. aggregation) and at the same time can be maintained in 
the years to come. If you are only interested in doing this for jdbc, I think 
we should consider some hacks to work around the public interface.





---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-169619635
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-169619162
  
**[Test build #48922 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48922/consoleFull)**
 for PR 10631 at commit 
[`42bc664`](https://github.com/apache/spark/commit/42bc664daf70c384ae3f5df2451bb34d52ba53de).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-169590863
  
**[Test build #48920 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48920/consoleFull)**
 for PR 10631 at commit 
[`aa1607c`](https://github.com/apache/spark/commit/aa1607cb5e465e26775a3336d3cd1a5a4f52f1e4).
 * This patch **fails Scala style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-169590871
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48920/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-169593029
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[maropu]

GitHub user maropu opened a pull request:

https://github.com/apache/spark/pull/10631

[SPARK-12686][SQL] Support group-by push down to data sources

This pr enables pushed-down MIN/MAX aggregation into JDBC data sources. As 
for logical plan nodes like 'Aggregate -> Project -> (Filter) -> Scan', try to 
push down partial aggregation processing into data sources that could aggregate 
their own data efficiently because Orc/Parquet could fetch the MIN/MAX value by 
using statistics data and some databases have efficient aggregation 
implementations.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/maropu/spark SupportPreAggregateInDS

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/10631.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #10631


commit ec53a07a74594c8862ee3f8a0ed8f72c5078322f
Author: Takeshi YAMAMURO 
Date:   2016-01-06T10:19:19Z

Support group-by push down to data sources




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10631#issuecomment-169565599
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48899/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12686][SQL] Support group-by push down ...

[rxin]

Github user rxin commented on a diff in the pull request:

https://github.com/apache/spark/pull/10631#discussion_r49044190
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala ---
@@ -286,7 +286,7 @@ trait PrunedScan {
  */
 @DeveloperApi
 trait PrunedFilteredScan {
-  def buildScan(requiredColumns: Array[String], filters: Array[Filter]): 
RDD[Row]
+  def buildScan(requiredColumns: Seq[String], filters: Seq[Filter], 
aggregate: Aggregate): RDD[Row]
--- End diff --

we can't just change the interface like this ...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org