[GitHub] spark pull request #16933: [SPARK-19601] [SQL] Fix CollapseRepartition rule ...

2017-02-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16933#discussion_r101911156 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/CollapseRepartitionSuite.scala --- @@ -43,15 +55,44 @@ class

[GitHub] spark pull request #16988: [SPARK-19658] [SQL] Set NumPartitions of Repartit...

2017-02-18 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/16988 [SPARK-19658] [SQL] Set NumPartitions of RepartitionByExpression In Analyzer ### What changes were proposed in this pull request? Currently, if `NumPartitions` is not set in

[GitHub] spark issue #16987: [WIP][SPARK-][SS] FileSource read from FileSink

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16987 **[Test build #73124 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73124/testReport)** for PR 16987 at commit

[GitHub] spark pull request #16987: [WIP][SPARK-][SS] FileSource read from FileSink

2017-02-18 Thread lw-lin
GitHub user lw-lin opened a pull request: https://github.com/apache/spark/pull/16987 [WIP][SPARK-][SS] FileSource read from FileSink ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested?

[GitHub] spark pull request #16928: [SPARK-18699][SQL] Put malformed tokens into a ne...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16928#discussion_r101910249 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVFileFormat.scala --- @@ -101,6 +101,11 @@ class CSVFileFormat

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16944 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73123/ Test FAILed. ---

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16944 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16944 **[Test build #73123 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73123/testReport)** for PR 16944 at commit

[GitHub] spark pull request #16981: [SPARK-19637][SQL] Add from_json/to_json in Funct...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16981#discussion_r101909627 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonUtils.scala --- @@ -55,4 +60,26 @@ object JacksonUtils {

[GitHub] spark pull request #16981: [SPARK-19637][SQL] Add from_json/to_json in Funct...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16981#discussion_r101909598 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonUtils.scala --- @@ -55,4 +60,26 @@ object JacksonUtils {

[GitHub] spark pull request #16981: [SPARK-19637][SQL] Add from_json/to_json in Funct...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16981#discussion_r101909847 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonUtils.scala --- @@ -55,4 +60,26 @@ object JacksonUtils {

[GitHub] spark pull request #16981: [SPARK-19637][SQL] Add from_json/to_json in Funct...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16981#discussion_r101909916 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -482,6 +482,15 @@ case class

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16978 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73122/ Test PASSed. ---

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16978 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16978 **[Test build #73122 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73122/testReport)** for PR 16978 at commit

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16944 **[Test build #73123 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73123/testReport)** for PR 16944 at commit

[GitHub] spark issue #16938: [SPARK-19583][SQL]CTAS for data source table with a crea...

2017-02-18 Thread windpiger
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/16938 @gatorsmile I have test all the cases above updated. The result shows that spark for datasource table with HiveExternalCatalog and InMemoryCatatlog have the same actions. spark for hive

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-18 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r101908155 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSchemaInferenceSuite.scala --- @@ -0,0 +1,192 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-18 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r101908105 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSchemaInferenceSuite.scala --- @@ -0,0 +1,192 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16978 **[Test build #73122 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73122/testReport)** for PR 16978 at commit

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16985 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73121/ Test PASSed. ---

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16985 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16985 **[Test build #73121 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73121/testReport)** for PR 16985 at commit

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16978 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16978 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73120/ Test FAILed. ---

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16978 **[Test build #73120 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73120/testReport)** for PR 16978 at commit

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16985 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73119/ Test PASSed. ---

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16985 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16985 **[Test build #73119 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73119/testReport)** for PR 16985 at commit

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-02-18 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/16985 @hvanhovell : This is as per our discussion in the jira : https://issues.apache.org/jira/browse/SPARK-19122 --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16395 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73118/ Test PASSed. ---

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16395 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added...

2017-02-18 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16985#discussion_r101905768 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala --- @@ -33,8 +33,8 @@ import

[GitHub] spark issue #16979: [SPARK-19617][SS]Fix the race condition when starting an...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16979 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73117/ Test PASSed. ---

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16395 **[Test build #73118 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73118/testReport)** for PR 16395 at commit

[GitHub] spark issue #16979: [SPARK-19617][SS]Fix the race condition when starting an...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16979 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16979: [SPARK-19617][SS]Fix the race condition when starting an...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16979 **[Test build #73117 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73117/testReport)** for PR 16979 at commit

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16985 **[Test build #73121 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73121/testReport)** for PR 16985 at commit

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16978 **[Test build #73120 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73120/testReport)** for PR 16978 at commit

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16978 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16985: [WIP] [SPARK-19122][SQL] Unnecessary shuffle+sort added ...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16985 **[Test build #73119 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73119/testReport)** for PR 16985 at commit

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16395 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16395 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73116/ Test FAILed. ---

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16395 **[Test build #73116 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73116/testReport)** for PR 16395 at commit

[GitHub] spark issue #16903: [MLlib][TYPO] Replace LeastSquaresAggregator with Logist...

2017-02-18 Thread moutai
Github user moutai commented on the issue: https://github.com/apache/spark/pull/16903 Thanks, Yes I think the rest of the messages look fine on this file. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16395 **[Test build #73118 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73118/testReport)** for PR 16395 at commit

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16978 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16978 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73115/ Test FAILed. ---

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16978 **[Test build #73115 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73115/testReport)** for PR 16978 at commit

[GitHub] spark issue #16979: [SPARK-19617][SS]Fix the race condition when starting an...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16979 **[Test build #73117 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73117/testReport)** for PR 16979 at commit

[GitHub] spark issue #16979: [SPARK-19617][SS]Fix the race condition when starting an...

2017-02-18 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16979 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16395 **[Test build #73116 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73116/testReport)** for PR 16395 at commit

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16929 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73114/ Test PASSed. ---

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16929 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16929 **[Test build #73114 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73114/testReport)** for PR 16929 at commit

[GitHub] spark issue #16985: [WIP] [SPARK-19122][SQL] Unnecessary shuffle+sort added ...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16985 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16985: [WIP] [SPARK-19122][SQL] Unnecessary shuffle+sort added ...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16985 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73112/ Test PASSed. ---

[GitHub] spark issue #16985: [WIP] [SPARK-19122][SQL] Unnecessary shuffle+sort added ...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16985 **[Test build #73112 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73112/testReport)** for PR 16985 at commit

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16929 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16929 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73111/ Test PASSed. ---

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16929 **[Test build #73111 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73111/testReport)** for PR 16929 at commit

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16978 **[Test build #73115 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73115/testReport)** for PR 16978 at commit

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16929 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73113/ Test FAILed. ---

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16929 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16929 **[Test build #73113 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73113/testReport)** for PR 16929 at commit

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16929 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73110/ Test PASSed. ---

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16929 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16929 **[Test build #73110 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73110/testReport)** for PR 16929 at commit

[GitHub] spark issue #16983: [SPARK-19550][BUILD][WIP] Addendum: select Java 1.7 for ...

2017-02-18 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/16983 LGTM if the build finishes --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16929 **[Test build #73114 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73114/testReport)** for PR 16929 at commit

[GitHub] spark issue #16986: [SPARK-18891][SQL] Support for Map collection types

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16986 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16929 **[Test build #73113 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73113/testReport)** for PR 16929 at commit

[GitHub] spark pull request #16986: [SPARK-18891][SQL] Support for Map collection typ...

2017-02-18 Thread michalsenkyr
GitHub user michalsenkyr opened a pull request: https://github.com/apache/spark/pull/16986 [SPARK-18891][SQL] Support for Map collection types ## What changes were proposed in this pull request? Add support for arbitrary Scala `Map` types in deserialization as well as a

[GitHub] spark issue #16845: [SPARK-19505][Python] AttributeError on Exception.messag...

2017-02-18 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16845 Let's see if I can ask Jenkins to test or if I need to get another list updates for that. Jenkins ok to test. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #16845: [SPARK-19505][Python] AttributeError on Exception.messag...

2017-02-18 Thread dgingrich
Github user dgingrich commented on the issue: https://github.com/apache/spark/pull/16845 Another alternate is to use [`six`](https://pythonhosted.org/six/). In the standard library `text_type`) works: ```python # Python2.6 >>> import six >>>

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r101899726 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -0,0 +1,539 @@ +/*

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r101899620 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -0,0 +1,539 @@ +/*

[GitHub] spark issue #16985: [WIP] [SPARK-19122][SQL] Unnecessary shuffle+sort added ...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16985 **[Test build #73112 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73112/testReport)** for PR 16985 at commit

[GitHub] spark pull request #16985: [WIP] [SPARK-19122][SQL] Unnecessary shuffle+sort...

2017-02-18 Thread tejasapatil
GitHub user tejasapatil opened a pull request: https://github.com/apache/spark/pull/16985 [WIP] [SPARK-19122][SQL] Unnecessary shuffle+sort added if join predicates ordering differ from bucketing and sorting order ## What changes were proposed in this pull request? WIP

[GitHub] spark issue #16985: [WIP] [SPARK-19122][SQL] Unnecessary shuffle+sort added ...

2017-02-18 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/16985 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16929 **[Test build #73111 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73111/testReport)** for PR 16929 at commit

[GitHub] spark issue #16623: [SPARK-19066][SPARKR][Backport-2.1]:LDA doesn't set opti...

2017-02-18 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/16623 Thanks for cc'ing me on this - I think @jkbradley has a good point that we should be a bit more explicit in discussing when / why we backport changes in SparkR. While we have not declared it a

[GitHub] spark issue #16928: [SPARK-18699][SQL] Put malformed tokens into a new field...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16928 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16928: [SPARK-18699][SQL] Put malformed tokens into a new field...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16928 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73109/ Test PASSed. ---

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16929 @hvanhovell, @zsxwing and @marmbrus, I just updated and rebased. Could you take another look please? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #16928: [SPARK-18699][SQL] Put malformed tokens into a new field...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16928 **[Test build #73109 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73109/testReport)** for PR 16928 at commit

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16929 **[Test build #73110 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73110/testReport)** for PR 16929 at commit

[GitHub] spark issue #16845: [SPARK-19505][Python] AttributeError on Exception.messag...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16845 Yea, I think she is meant to decide this :). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16845: [SPARK-19505][Python] AttributeError on Exception.messag...

2017-02-18 Thread dgingrich
Github user dgingrich commented on the issue: https://github.com/apache/spark/pull/16845 IMO the helper is better since the Exception message is controlled by the user. I thought Holden was +1'ing the problem (not handling unicode) but re-reading he might have been +1'ing that it

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add from_json/to_json in FunctionRegi...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16981 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73107/ Test PASSed. ---

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add from_json/to_json in FunctionRegi...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16981 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add from_json/to_json in FunctionRegi...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16981 **[Test build #73107 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73107/testReport)** for PR 16981 at commit

[GitHub] spark issue #16928: [SPARK-18699][SQL] Put malformed tokens into a new field...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16928 **[Test build #73109 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73109/testReport)** for PR 16928 at commit

[GitHub] spark issue #8384: [SPARK-8510] [CORE] [PYSPARK] NumPy matrices as values in...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/8384 Thank you @paberline. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16171: [SPARK-18739][ML][PYSPARK] Classification and reg...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16171#discussion_r101896246 --- Diff: python/pyspark/ml/wrapper.py --- @@ -20,12 +20,14 @@ if sys.version >= '3': xrange = range +from pyspark import since,

[GitHub] spark pull request #8384: [SPARK-8510] [CORE] [PYSPARK] NumPy matrices as va...

2017-02-18 Thread paberline
Github user paberline closed the pull request at: https://github.com/apache/spark/pull/8384 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #8384: [SPARK-8510] [CORE] [PYSPARK] NumPy matrices as values in...

2017-02-18 Thread paberline
Github user paberline commented on the issue: https://github.com/apache/spark/pull/8384 Closing as requested due to this feature being too narrow a use case for core Spark. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request #16609: [SPARK-8480] [CORE] [PYSPARK] [SPARKR] Add setNam...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16609#discussion_r101895951 --- Diff: python/pyspark/rdd.py --- @@ -2169,9 +2169,7 @@ def name(self): """ Return the name of this RDD. """

[GitHub] spark issue #16824: [SPARK-18069][PYTHON] Make PySpark doctests for SQL self...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16824 @davies and @holdenk, what do you think about this change? I am fine if we can't decide it is worth for now. I can close this if any of you is not confident of this change. --- If your

[GitHub] spark issue #16845: [SPARK-19505][Python] AttributeError on Exception.messag...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16845 BTW, could anyone say "ok to test"? Anyhow it is true that `Exception` does not have `message` in Python 3 and we need a fix. --- If your project is set up for it, you can reply to this email

  1   2   >