[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-21 Thread maver1ck
Github user maver1ck commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-158662413 Thanks. And when I compile with hive is there a chance to do sth like this ? "select id, collect_list(table.*) as data from table group by id" ? --- If your pro

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-21 Thread nburoojy
Github user nburoojy commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-158661662 This is a wrapper around the Hive collect fns. Try compiling with `-Phive -Phive-thriftserver` --- If your project is set up for it, you can reply to this email and ha

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-21 Thread maver1ck
Github user maver1ck commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-158649521 Hi, How can I run this ? Spark 1.6.0-preview1 Compiled with: mvn -e -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean package I'm tryi

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9526 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-09 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-155219611 Hey, i just noticed this wasn't opened against master. In the future please always go against master unless creating a specific backport. We will cherry-pick when com

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-09 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-155219299 Thanks, merging to master and 1.6 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does n

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-155215327 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-155215134 **[Test build #45404 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45404/consoleFull)** for PR 9526 at commit [`07de8a2`](https://git

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-155170436 **[Test build #45404 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45404/consoleFull)** for PR 9526 at commit [`07de8a2`](https://gith

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-155167998 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-155167971 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-155150049 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-155150043 **[Test build #45385 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45385/consoleFull)** for PR 9526 at commit [`f19b466`](https://git

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-09 Thread nburoojy
Github user nburoojy commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-155149837 I've added these fns to pyspark and wrote unit tests. Which description needs to be updated? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-155147174 **[Test build #45385 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45385/consoleFull)** for PR 9526 at commit [`f19b466`](https://gith

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-155145018 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-155144985 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-154531316 Build finished. No test results found. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-154531335 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45240/ ---

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-154531344 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45242/ ---

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-154531326 Build finished. No test results found. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-154516185 **[Test build #45242 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45242/consoleFull)** for PR 9526 at commit [`b3d3551`](https://gith

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-154515995 **[Test build #45240 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45240/consoleFull)** for PR 9526 at commit [`289ace5`](https://gith

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-154514967 Build triggered. sha1 is merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project do

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-154515001 Build started sha1 is merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-154513816 Build triggered. sha1 is merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project do

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-154513845 Build started sha1 is merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-06 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-154513363 Please also add them to functions.py and add unit tests for both languages. Thanks! Also, when you are done update the description as that becomes the commit

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-06 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/9526#discussion_r44179146 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -175,6 +175,26 @@ object functions { def avg(columnName: String): Column

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-06 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-154512918 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature en

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-154507902 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-06 Thread nburoojy
GitHub user nburoojy opened a pull request: https://github.com/apache/spark/pull/9526 [SPARK-9301] [SQL] Add collect_set and collect_list aggregate functions For now they are thin wrappers around the corresponding Hive UDAFs. One limitation with these in Hive 0.13.0 is they