[GitHub] spark pull request: [SPARK-12537] [SQL] Add option to accept quoti...

2015-12-29 Thread Cazen
Github user Cazen commented on a diff in the pull request: https://github.com/apache/spark/pull/10497#discussion_r48590825 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JSONOptions.scala --- @@ -59,6 +62,8 @@ object JSONOptions { allowN

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167954059 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167954060 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...

2015-12-29 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/10502#discussion_r48590599 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -208,11 +210,30 @@ private[sql] object

[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...

2015-12-29 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/10502#discussion_r48590538 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -208,11 +210,30 @@ private[sql] object

[GitHub] spark pull request: [SPARK-12562][SQL] DataFrame.write.format(text...

2015-12-29 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/10515#discussion_r48590491 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala --- @@ -58,6 +58,17 @@ class TextSuite extends QueryTest with

[GitHub] spark pull request: [SPARK-12537] [SQL] Add option to accept quoti...

2015-12-29 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/10497#discussion_r48590477 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JSONOptions.scala --- @@ -59,6 +62,8 @@ object JSONOptions { allowNu

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48590465 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -91,6 +91,11 @@ abstract class LogicalPlan ext

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-29 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/10451#issuecomment-167952117 The latest version covers both cases: 1) If the children' `maxRows` has a smaller number, add the extra limit. 2) If the children' `maxRows` is `None`, add the

[GitHub] spark pull request: [SPARK-12564] [SQL] Improve missing column Ana...

2015-12-29 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/10518#issuecomment-167951717 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature en

[GitHub] spark pull request: [SPARK-12286] [SPARK-12290] [SPARK-12294] [SPA...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10511#issuecomment-167951626 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12286] [SPARK-12290] [SPARK-12294] [SPA...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10511#issuecomment-167951627 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12286] [SPARK-12290] [SPARK-12294] [SPA...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10511#issuecomment-167951622 **[Test build #48454 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48454/consoleFull)** for PR 10511 at commit [`6bc518c`](https://g

[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...

2015-12-29 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/10502#discussion_r48590244 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -208,11 +210,30 @@ private[sql] object Parqu

[GitHub] spark pull request: [SPARK-12537] [SQL] Add option to accept quoti...

2015-12-29 Thread Cazen
Github user Cazen commented on a diff in the pull request: https://github.com/apache/spark/pull/10497#discussion_r48590209 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JSONOptions.scala --- @@ -59,6 +62,8 @@ object JSONOptions { allowN

[GitHub] spark pull request: [SPARK-12562][SQL] DataFrame.write.format(text...

2015-12-29 Thread xguo27
Github user xguo27 commented on a diff in the pull request: https://github.com/apache/spark/pull/10515#discussion_r48590105 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala --- @@ -58,6 +58,17 @@ class TextSuite extends QueryTest wi

[GitHub] spark pull request: [SPARK-12286] [SPARK-12290] [SPARK-12294] [SQL...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10511#issuecomment-167951333 **[Test build #48454 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48454/consoleFull)** for PR 10511 at commit [`6bc518c`](https://gi

[GitHub] spark pull request: [SPARK-12541] [SQL] support cube/rollup as fun...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10522#issuecomment-167951289 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12541] [SQL] support cube/rollup as fun...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10522#issuecomment-167951290 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12541] [SQL] support cube/rollup as fun...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10522#issuecomment-167951284 **[Test build #48453 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48453/consoleFull)** for PR 10522 at commit [`bf6603f`](https://g

[GitHub] spark pull request: [SPARK-12562][SQL] DataFrame.write.format(text...

2015-12-29 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/10515#discussion_r48590012 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala --- @@ -58,6 +58,17 @@ class TextSuite extends QueryTest with

[GitHub] spark pull request: [SPARK-12355][SQL] Implement unhandledFilter i...

2015-12-29 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/10502#discussion_r48589974 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -208,11 +210,30 @@ private[sql] object Parqu

[GitHub] spark pull request: [SPARK-12541] [SQL] support cube/rollup as fun...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10522#issuecomment-167950725 **[Test build #48453 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48453/consoleFull)** for PR 10522 at commit [`bf6603f`](https://gi

[GitHub] spark pull request: [SPARK-10359] Enumerate dependencies in a file...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10461#issuecomment-167950665 **[Test build #48452 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48452/consoleFull)** for PR 10461 at commit [`9932c80`](https://gi

[GitHub] spark pull request: [SPARK-12562][SQL] DataFrame.write.format(text...

2015-12-29 Thread xguo27
Github user xguo27 commented on the pull request: https://github.com/apache/spark/pull/10515#issuecomment-167950674 Thanks @viirya ! I have updated the comment and added unit test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub a

[GitHub] spark pull request: [SPARK-2827][GraphX] Add collectDegreeDist to ...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10521#issuecomment-167950112 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-2827][GraphX] Add collectDegreeDist to ...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10521#issuecomment-167950109 **[Test build #48451 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48451/consoleFull)** for PR 10521 at commit [`99776c9`](https://g

[GitHub] spark pull request: [SPARK-2827][GraphX] Add collectDegreeDist to ...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10521#issuecomment-167950114 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12537] [SQL] Add option to accept quoti...

2015-12-29 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/10497#discussion_r48589687 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JSONOptions.scala --- @@ -59,6 +62,8 @@ object JSONOptions { allowNu

[GitHub] spark pull request: [SPARK-12286] [SPARK-12290] [SPARK-12294] [SQL...

2015-12-29 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/10511#issuecomment-167949764 @davies Thanks for replying my comments. It is good to know what it is going on. Now it does feel better. It is ok for the closed PR. I just got confusing why after seve

[GitHub] spark pull request: [SPARK-2827][GraphX] Add collectDegreeDist to ...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10521#issuecomment-167949758 **[Test build #48451 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48451/consoleFull)** for PR 10521 at commit [`99776c9`](https://gi

[GitHub] spark pull request: [SPARK-10359] Enumerate dependencies in a file...

2015-12-29 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/10461#issuecomment-167949707 Let me try that dummy change again... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-12541] [SQL] support cube/rollup as fun...

2015-12-29 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/10522 [SPARK-12541] [SQL] support cube/rollup as function This PR enable cube/rollup as function, so they can be used as this: ``` select a, b, sum(c) from t group by rollup(a, b) ``` You can

[GitHub] spark pull request: [SPARK-12537] [SQL] Add option to accept quoti...

2015-12-29 Thread Cazen
Github user Cazen commented on a diff in the pull request: https://github.com/apache/spark/pull/10497#discussion_r48589607 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JSONOptions.scala --- @@ -59,6 +62,8 @@ object JSONOptions { allowN

[GitHub] spark pull request: [SPARK-10180] [SQL] JDBC datasource are not pr...

2015-12-29 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/8743#issuecomment-167948607 @HyukjinKwon +1, I think it'd better to add tests in `MySQLIntegrationSuite` and `PostgresIntegrationSuite`. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-12286] [SPARK-12290] [SPARK-12294] [SQL...

2015-12-29 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/10511#issuecomment-167947271 @viirya I'm very sorry for this. Originally, I thought this could be done step by step, then finally do this clean up. After reviewed some of the PRs, realized that tryi

[GitHub] spark pull request: [SPARK-4819] Remove Guava's "Optional" from pu...

2015-12-29 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10512#issuecomment-167947156 Going to close this one since I was only posting it for demo. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as wel

[GitHub] spark pull request: [SPARK-4819] Remove Guava's "Optional" from pu...

2015-12-29 Thread rxin
Github user rxin closed the pull request at: https://github.com/apache/spark/pull/10512 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] spark pull request: [SPARK-2827][GraphX] Add collectDegreeDist to ...

2015-12-29 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/10521#issuecomment-167947021 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...

2015-12-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10435#discussion_r48589312 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala --- @@ -176,3 +179,229 @@ case class Crc32(child: Expression) e

[GitHub] spark pull request: [SPARK-12286] [SPARK-12290] [SPARK-12294] [SQL...

2015-12-29 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/10511#issuecomment-167946717 @nongli InsertIntoHiveTable also does not support UnsafeRow now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub a

[GitHub] spark pull request: [SPARK-12539][SQL][WIP] support writing bucket...

2015-12-29 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10498#discussion_r48588812 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/sources/BucketedWriteSuite.scala --- @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-12539][SQL][WIP] support writing bucket...

2015-12-29 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10498#discussion_r48588795 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/sources/BucketedWriteSuite.scala --- @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-12564] [SQL] Improve missing column Ana...

2015-12-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10518 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [DOC] Adjust coverage for partitionBy()

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10499#issuecomment-167945033 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [DOC] Adjust coverage for partitionBy()

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10499#issuecomment-167945034 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12564] [SQL] Improve missing column Ana...

2015-12-29 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10518#issuecomment-167945003 I've merged this. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: [DOC] Adjust coverage for partitionBy()

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10499#issuecomment-167945008 **[Test build #48443 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48443/consoleFull)** for PR 10499 at commit [`a021725`](https://g

[GitHub] spark pull request: [SPARK-12564] [SQL] Improve missing column Ana...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10518#issuecomment-167944933 **[Test build #48441 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48441/consoleFull)** for PR 10518 at commit [`bc57b3a`](https://g

[GitHub] spark pull request: [SPARK-12564] [SQL] Improve missing column Ana...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10518#issuecomment-167944953 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12564] [SQL] Improve missing column Ana...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10518#issuecomment-167944952 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12564] [SQL] Improve missing column Ana...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10518#issuecomment-167944795 **[Test build #2261 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2261/consoleFull)** for PR 10518 at commit [`bc57b3a`](https://

[GitHub] spark pull request: [SPARK-12539][SQL][WIP] support writing bucket...

2015-12-29 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10498#discussion_r48588647 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/WriterContainer.scala --- @@ -317,14 +318,39 @@ private[sql] class DynamicPartit

[GitHub] spark pull request: [SPARK-12539][SQL][WIP] support writing bucket...

2015-12-29 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10498#discussion_r48588632 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/WriterContainer.scala --- @@ -317,14 +318,39 @@ private[sql] class DynamicPartit

[GitHub] spark pull request: [SPARK-12558][SQL][WIP] AnalysisException when...

2015-12-29 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/10520#discussion_r48588613 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala --- @@ -233,6 +233,18 @@ private[hive] case class HiveGenericUDF(funcWrapper:

[GitHub] spark pull request: [SPARK-12558][SQL][WIP] AnalysisException when...

2015-12-29 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/10520#discussion_r48588602 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala --- @@ -233,6 +233,18 @@ private[hive] case class HiveGenericUDF(funcWrapper:

[GitHub] spark pull request: [SPARK-12539][SQL][WIP] support writing bucket...

2015-12-29 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/10498#discussion_r48588590 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -129,6 +129,19 @@ final class DataFrameWriter private[sql](df: DataFrame) {

[GitHub] spark pull request: [SPARK-12409][SPARK-12387][SPARK-12391][SQL] R...

2015-12-29 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/10470#discussion_r48588506 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala --- @@ -182,18 +183,40 @@ private[sql] object JDBCRDD extends

[GitHub] spark pull request: [SPARK-12539][SQL][WIP] support writing bucket...

2015-12-29 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10498#discussion_r48588458 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -129,6 +129,19 @@ final class DataFrameWriter private[sql](df: DataFrame)

[GitHub] spark pull request: [SPARK-12513] [Streaming] SocketReceiver hang ...

2015-12-29 Thread guoxu1231
Github user guoxu1231 commented on a diff in the pull request: https://github.com/apache/spark/pull/10464#discussion_r48588447 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/SocketInputDStream.scala --- @@ -52,46 +52,60 @@ class SocketReceiver[T: ClassTag](

[GitHub] spark pull request: [SPARK-12539][SQL][WIP] support writing bucket...

2015-12-29 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10498#discussion_r48588450 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -189,13 +205,44 @@ final class DataFrameWriter private[sql](df: DataFrame

[GitHub] spark pull request: [SPARK-12539][SQL][WIP] support writing bucket...

2015-12-29 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10498#discussion_r48588391 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -129,6 +129,19 @@ final class DataFrameWriter private[sql](df: DataFrame)

[GitHub] spark pull request: [SPARK-12539][SQL][WIP] support writing bucket...

2015-12-29 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10498#discussion_r48588334 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -189,13 +205,44 @@ final class DataFrameWriter private[sql](df: DataFrame

[GitHub] spark pull request: [SPARK-12409][SPARK-12387][SPARK-12391][SQL] R...

2015-12-29 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/10470#discussion_r48588333 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala --- @@ -182,18 +183,40 @@ private[sql] object JDBCRDD extends

[GitHub] spark pull request: [SPARK-12539][SQL][WIP] support writing bucket...

2015-12-29 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10498#discussion_r48588321 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -129,6 +129,19 @@ final class DataFrameWriter private[sql](df: DataFrame)

[GitHub] spark pull request: [SPARK-12539][SQL][WIP] support writing bucket...

2015-12-29 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10498#discussion_r48588277 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -129,6 +129,19 @@ final class DataFrameWriter private[sql](df: DataFrame)

[GitHub] spark pull request: [SPARK-12513] [Streaming] SocketReceiver hang ...

2015-12-29 Thread guoxu1231
Github user guoxu1231 commented on a diff in the pull request: https://github.com/apache/spark/pull/10464#discussion_r48588257 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/SocketInputDStream.scala --- @@ -52,46 +52,60 @@ class SocketReceiver[T: ClassTag](

[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...

2015-12-29 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10435#discussion_r48588221 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala --- @@ -176,3 +179,229 @@ case class Crc32(child: Expression) exte

[GitHub] spark pull request: [SPARK-10180] [SQL] JDBC datasource are not pr...

2015-12-29 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/8743#issuecomment-167943712 @zsxwing can you review this one? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-12409][SPARK-12387][SPARK-12391][SQL] R...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10470#issuecomment-167943198 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12409][SPARK-12387][SPARK-12391][SQL] R...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10470#issuecomment-167943199 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12409][SPARK-12387][SPARK-12391][SQL] R...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10470#issuecomment-167943160 **[Test build #48449 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48449/consoleFull)** for PR 10470 at commit [`a7ef79e`](https://g

[GitHub] spark pull request: [SPARK-12537] [SQL] Add option to accept quoti...

2015-12-29 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/10497#discussion_r48587932 --- Diff: python/pyspark/sql/readwriter.py --- @@ -160,6 +160,8 @@ def json(self, path, schema=None): quotes * ``allowNumer

[GitHub] spark pull request: [SPARK-12409][SPARK-12387][SPARK-12391][SQL] S...

2015-12-29 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/10468#issuecomment-167941926 @rxin Good for me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...

2015-12-29 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10435#discussion_r48587902 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala --- @@ -176,3 +179,229 @@ case class Crc32(child: Expression) exte

[GitHub] spark pull request: [SPARK-12537] [SQL] Add option to accept quoti...

2015-12-29 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/10497#discussion_r48587837 --- Diff: python/pyspark/sql/readwriter.py --- @@ -160,6 +160,8 @@ def json(self, path, schema=None): quotes * ``allowNumer

[GitHub] spark pull request: [SPARK-12558][SQL][WIP] AnalysisException when...

2015-12-29 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10520#discussion_r48587483 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala --- @@ -233,6 +233,18 @@ private[hive] case class HiveGenericUDF(funcWrapper: Hive

[GitHub] spark pull request: [SPARK-12558][SQL][WIP] AnalysisException when...

2015-12-29 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10520#discussion_r48587428 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala --- @@ -233,6 +233,18 @@ private[hive] case class HiveGenericUDF(funcWrapper: Hive

[GitHub] spark pull request: [SPARK-12286] [SPARK-12290] [SPARK-12294] [SQL...

2015-12-29 Thread nongli
Github user nongli commented on the pull request: https://github.com/apache/spark/pull/10511#issuecomment-167939468 The only heavy operator that didn't support unsaferow was just sortbased agg? Is that right? This seems reasonable to me given how much code this removes. --- If your

[GitHub] spark pull request: [SPARK-12562][SQL] DataFrame.write.format(text...

2015-12-29 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/10515#issuecomment-167939136 Can you add some tests for this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-12562][SQL] DataFrame.write.format(text...

2015-12-29 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/10515#discussion_r48587173 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/text/DefaultSource.scala --- @@ -77,8 +78,8 @@ private[sql] class TextRelation(

[GitHub] spark pull request: [SPARK-12537] [SQL] Add option to accept quoti...

2015-12-29 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10496#issuecomment-167939054 It's fine to use https://github.com/apache/spark/pull/10497 Just update it there. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-12363] [SQL] Inline Hive parser into sp...

2015-12-29 Thread nongli
Github user nongli closed the pull request at: https://github.com/apache/spark/pull/10420 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-12579][SQL] Force user-specified JDBC d...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10519#issuecomment-167938479 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12579][SQL] Force user-specified JDBC d...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10519#issuecomment-167938480 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12579][SQL] Force user-specified JDBC d...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10519#issuecomment-167938457 **[Test build #48446 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48446/consoleFull)** for PR 10519 at commit [`3554d68`](https://g

[GitHub] spark pull request: [SPARK-12537] [SQL] Add option to accept quoti...

2015-12-29 Thread Cazen
Github user Cazen commented on the pull request: https://github.com/apache/spark/pull/10496#issuecomment-167938272 Hi Xin Thank you for review I've created PR(11496, this PR) but it doesn't connect with jira(SPARK-12537) so I've closed. After that, I recreated PR(1149

[GitHub] spark pull request: [SPARK-12537] [SQL] Add option to accept quoti...

2015-12-29 Thread Cazen
Github user Cazen commented on a diff in the pull request: https://github.com/apache/spark/pull/10496#discussion_r48587091 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonParsingOptionsSuite.scala --- @@ -111,4 +111,21 @@ class JsonParsingOpti

[GitHub] spark pull request: [SPARK-11507] [MLlib] add compact in Matrices ...

2015-12-29 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/9520#discussion_r48586886 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala --- @@ -879,6 +879,7 @@ object Matrices { case dm: BDM[Double] =>

[GitHub] spark pull request: [SPARK-12409][SPARK-12387][SPARK-12391][SQL] R...

2015-12-29 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/10470#discussion_r48586845 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala --- @@ -182,18 +183,40 @@ private[sql] object JDBCRDD ex

[GitHub] spark pull request: [SPARK-10963] [Streaming] [Kafka] make KafkaCl...

2015-12-29 Thread yhilem
Github user yhilem commented on the pull request: https://github.com/apache/spark/pull/9007#issuecomment-167937170 For which spark version the merge of this pull request is planned? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-10963] [Streaming] [Kafka] make KafkaCl...

2015-12-29 Thread yhilem
Github user yhilem commented on the pull request: https://github.com/apache/spark/pull/9007#issuecomment-167936993 Which spark version the merge of this pull request is planned? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as w

[GitHub] spark pull request: [SPARK-12564] [SQL] Improve missing column Ana...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10518#issuecomment-167936950 **[Test build #2261 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2261/consoleFull)** for PR 10518 at commit [`bc57b3a`](https://g

[GitHub] spark pull request: [SPARK-10963] [Streaming] [Kafka] make KafkaCl...

2015-12-29 Thread yhilem
Github user yhilem commented on the pull request: https://github.com/apache/spark/pull/9007#issuecomment-167936934 Which version of the spark the merge is planned? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your p

[GitHub] spark pull request: [SPARK-10359] Enumerate dependencies in a file...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10461#issuecomment-167936783 **[Test build #2260 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2260/consoleFull)** for PR 10461 at commit [`63e4109`](https://

[GitHub] spark pull request: [SPARK-12409][SPARK-12387][SPARK-12391][SQL] S...

2015-12-29 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10468#issuecomment-167936763 Looks like you also need to update the test case. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12409][SPARK-12387][SPARK-12391][SQL] R...

2015-12-29 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/10470#discussion_r48586764 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala --- @@ -182,18 +183,40 @@ private[sql] object JDBCRDD extends

[GitHub] spark pull request: [SPARK-10359] Enumerate dependencies in a file...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10461#issuecomment-167936653 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12537] [SQL] Add option to accept quoti...

2015-12-29 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10496#issuecomment-167936614 @Cazen how come you closed the pull request? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-10359] Enumerate dependencies in a file...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10461#issuecomment-167936654 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

  1   2   3   4   5   >