[GitHub] spark issue #19165: [SPARKR][BACKPORT-2.1] backporting package and test chan...

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19165 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19165: [SPARKR][BACKPORT-2.1] backporting package and test chan...

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19165 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81572/ Test FAILed. ---

[GitHub] spark issue #19165: [SPARKR][BACKPORT-2.1] backporting package and test chan...

2017-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19165 **[Test build #81572 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81572/testReport)** for PR 19165 at commit

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-09-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18975 Looks pretty well, left few minor comments. Thanks for working on this. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137919207 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveDirCommand.scala --- @@ -0,0 +1,131 @@ +/* + * Licensed to the

[GitHub] spark issue #19118: [SPARK-21882][CORE] OutputMetrics doesn't count written ...

2017-09-08 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19118 This should be a valid fix. Could you add a test case for this? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137919154 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -1512,4 +1512,81 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18975 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81571/ Test PASSed. ---

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18975 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18975 **[Test build #81571 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81571/testReport)** for PR 18975 at commit

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137919040 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala --- @@ -0,0 +1,82 @@ +/* + * Licensed

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137919007 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -1512,4 +1512,81 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark pull request #19167: [SPARK-21954][SQL] JacksonUtils should verify Map...

2017-09-08 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19167#discussion_r137918960 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala --- @@ -257,6 +257,18 @@ class JsonFunctionsSuite extends QueryTest

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137918931 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -360,6 +360,31 @@ case class

[GitHub] spark issue #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet state sho...

2017-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18982 **[Test build #81577 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81577/testReport)** for PR 18982 at commit

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137918839 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -192,12 +245,23 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137918845 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -192,12 +245,23 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137918792 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -192,12 +245,23 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137918751 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -192,12 +245,23 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark pull request #16422: [SPARK-17642] [SQL] support DESC EXTENDED/FORMATT...

2017-09-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16422#discussion_r137918748 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -626,6 +624,73 @@ case class DescribeTableCommand( }

[GitHub] spark issue #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet state sho...

2017-09-08 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/18982 @holdenk , the error was because I was calling `setDefault` with a separate `Param` and value. So for something like a `seed` param, the Long value would get converted to an Int in the exchange

[GitHub] spark pull request #16422: [SPARK-17642] [SQL] support DESC EXTENDED/FORMATT...

2017-09-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16422#discussion_r137918691 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -626,6 +624,73 @@ case class DescribeTableCommand( }

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18975 **[Test build #81576 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81576/testReport)** for PR 18975 at commit

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-08 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137918600 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveDirCommand.scala --- @@ -0,0 +1,131 @@ +/* + * Licensed to

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137918571 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveDirCommand.scala --- @@ -0,0 +1,131 @@ +/* + * Licensed to the

[GitHub] spark issue #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet state sho...

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18982 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81574/ Test FAILed. ---

[GitHub] spark issue #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet state sho...

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18982 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet state sho...

2017-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18982 **[Test build #81574 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81574/testReport)** for PR 18982 at commit

[GitHub] spark issue #19167: [SPARK-21954][SQL] JacksonUtils should verify MapType's ...

2017-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19167 **[Test build #81575 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81575/testReport)** for PR 19167 at commit

[GitHub] spark issue #19167: [SPARK-21954][SQL] JacksonUtils should verify MapType's ...

2017-09-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19167 @HyukjinKwon Thanks for review. I simplified the test cases. Please take a look when you are available. --- - To unsubscribe,

[GitHub] spark issue #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet state sho...

2017-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18982 **[Test build #81574 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81574/testReport)** for PR 18982 at commit

[GitHub] spark issue #19167: [SPARK-21954][SQL] JacksonUtils should verify MapType's ...

2017-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19167 **[Test build #81573 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81573/testReport)** for PR 19167 at commit

[GitHub] spark pull request #18865: [SPARK-21610][SQL] Corrupt records are not handle...

2017-09-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18865#discussion_r137917977 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2034,4 +2034,25 @@ class JsonSuite extends

[GitHub] spark pull request #18865: [SPARK-21610][SQL] Corrupt records are not handle...

2017-09-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18865#discussion_r137917938 --- Diff: docs/sql-programming-guide.md --- @@ -1542,6 +1542,10 @@ options. # Migration Guide +## Upgrading From Spark SQL 2.2 to

[GitHub] spark pull request #19171: [SPARK-21902][CORE] Print root cause for BlockMan...

2017-09-08 Thread caneGuy
Github user caneGuy commented on a diff in the pull request: https://github.com/apache/spark/pull/19171#discussion_r137917675 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -988,6 +988,12 @@ private[spark] class BlockManager(

[GitHub] spark issue #19165: [SPARKR][BACKPORT-2.1] backporting package and test chan...

2017-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19165 **[Test build #81572 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81572/testReport)** for PR 19165 at commit

[GitHub] spark issue #16422: [SPARK-17642] [SQL] support DESC EXTENDED/FORMATTED tabl...

2017-09-08 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/16422 @gatorsmile @cloud-fan Sorry for the late update. I changed the output form as @cloud-fan previously suggested: ``` col_name abc data_type int max3 ``` ---

[GitHub] spark issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18339 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18339 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81570/ Test PASSed. ---

[GitHub] spark issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2017-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18339 **[Test build #81570 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81570/testReport)** for PR 18339 at commit

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18975 **[Test build #81571 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81571/testReport)** for PR 18975 at commit

[GitHub] spark pull request #19171: [SPARK-21902][CORE] Print root cause for BlockMan...

2017-09-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19171#discussion_r137916689 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -988,6 +988,12 @@ private[spark] class BlockManager(

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-08 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137916528 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -534,4 +534,176 @@ class InsertIntoHiveTableSuite extends QueryTest

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-08 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137916510 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -360,6 +360,31 @@ case class

[GitHub] spark issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2017-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18339 **[Test build #81570 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81570/testReport)** for PR 18339 at commit

[GitHub] spark issue #18953: [SPARK-20682][SQL] Update ORC data source based on Apach...

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18953 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18953: [SPARK-20682][SQL] Update ORC data source based on Apach...

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18953 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81569/ Test PASSed. ---

[GitHub] spark issue #18953: [SPARK-20682][SQL] Update ORC data source based on Apach...

2017-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18953 **[Test build #81569 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81569/testReport)** for PR 18953 at commit

[GitHub] spark issue #19168: [SPARK-21956][CORE] Fetch up to max bytes when buf reall...

2017-09-08 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19168 Yes you are right @kiszk Benchmark on the way.I will put here later.Thanks for your time @kiszk --- - To unsubscribe, e-mail:

[GitHub] spark issue #19171: [SPARK-21902][CORE] Print root cause for BlockManager#do...

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19171 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19171: [SPARK-21902][CORE] Print root cause for BlockManager#do...

2017-09-08 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19171 @kiszk I updated and split this from [PR-19133](https://github.com/apache/spark/pull/19133) --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19171: [SPARK-21902][CORE] Print root cause for BlockMan...

2017-09-08 Thread caneGuy
GitHub user caneGuy opened a pull request: https://github.com/apache/spark/pull/19171 [SPARK-21902][CORE] Print root cause for BlockManager#doPut ## What changes were proposed in this pull request? As logging below, actually exception will be hidden when

[GitHub] spark issue #19161: [SPARK-21726][SQL][Follow-UP] Check for structural integ...

2017-09-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19161 @gatorsmile Yeah...sorry not notice this too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #18592: [SPARK-21368][SQL] TPCDSQueryBenchmark can't refe...

2017-09-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18592#discussion_r137912400 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/TPCDSQueryBenchmark.scala --- @@ -99,6 +95,20 @@ object TPCDSQueryBenchmark

[GitHub] spark issue #17383: [SPARK-3165][MLlib] DecisionTree use sparsity in data

2017-09-08 Thread facaiy
Github user facaiy commented on the issue: https://github.com/apache/spark/pull/17383 Sure, @WeichenXu123 , perhaps one or two weeks later, is it OK? By the way, I think using sparse representation can only reduce memory usage, and it is in the cost of compute performance.

[GitHub] spark issue #19170: [SPARK-21961][Core] Filter out BlockStatuses Accumulator...

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19170 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19170: [SPARK-21961][Core] Filter out BlockStatuses Accu...

2017-09-08 Thread zhouyejoe
GitHub user zhouyejoe opened a pull request: https://github.com/apache/spark/pull/19170 [SPARK-21961][Core] Filter out BlockStatuses Accumulators during replaying history logs in Spark History Server ## What changes were proposed in this pull request? Filter out

[GitHub] spark issue #18953: [SPARK-20682][SQL] Update ORC data source based on Apach...

2017-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18953 **[Test build #81569 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81569/testReport)** for PR 18953 at commit

[GitHub] spark issue #18953: [SPARK-20682][SQL] Update ORC data source based on Apach...

2017-09-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18953 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-08 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/16578 > @mallman (just pure curiosity :) ) how is it possible that this NPE was not found by automated tests? There's no single test case in entire spark suite which verifies scenarios like that

[GitHub] spark issue #18460: [SPARK-21247][SQL] Type comparision should respect case-...

2017-09-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18460 Ping, @cloud-fan ! Could you review this PR again? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #18460: [SPARK-21247][SQL] Type comparision should respect case-...

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18460 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18460: [SPARK-21247][SQL] Type comparision should respect case-...

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18460 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81568/ Test PASSed. ---

[GitHub] spark issue #18460: [SPARK-21247][SQL] Type comparision should respect case-...

2017-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18460 **[Test build #81568 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81568/testReport)** for PR 18460 at commit

[GitHub] spark issue #19160: [SPARK-21934][CORE] Expose Shuffle Netty memory usage to...

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19160 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81567/ Test PASSed. ---

[GitHub] spark issue #19160: [SPARK-21934][CORE] Expose Shuffle Netty memory usage to...

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19160 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19160: [SPARK-21934][CORE] Expose Shuffle Netty memory usage to...

2017-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19160 **[Test build #81567 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81567/testReport)** for PR 19160 at commit

[GitHub] spark pull request #16722: [SPARK-19591][ML][MLlib] Add sample weights to de...

2017-09-08 Thread XXXShao
Github user XXXShao commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r137897201 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala --- @@ -1002,9 +1018,9 @@ private[spark] object RandomForest extends

[GitHub] spark issue #18592: [SPARK-21368][SQL] TPCDSQueryBenchmark can't refer query...

2017-09-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18592 Will review it today. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19164: [SPARK-21953] Show both memory and disk bytes spilled if...

2017-09-08 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/19164 nice catch, LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #17943: [SPARK-20682][SQL] Implement new ORC data source based o...

2017-09-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/17943 Please refer the superset PR #17980 . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #17943: [SPARK-20682][SQL] Implement new ORC data source ...

2017-09-08 Thread dongjoon-hyun
Github user dongjoon-hyun closed the pull request at: https://github.com/apache/spark/pull/17943 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #17924: [SPARK-20682][SQL] Support a new faster ORC data ...

2017-09-08 Thread dongjoon-hyun
Github user dongjoon-hyun closed the pull request at: https://github.com/apache/spark/pull/17924 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #17924: [SPARK-20682][SQL] Support a new faster ORC data source ...

2017-09-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/17924 Please refer the superset in #17980 . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19169: [SPARK-21957][SQL] Add current_user function

2017-09-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/19169 The Python side looks reasonable, maybe @marmbrus can share his view on adding this function in general? In the mean time Jenkins OK to test. ---

[GitHub] spark issue #17435: [SPARK-20098][PYSPARK] dataType's typeName fix

2017-09-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17435 @szalai1 I think @HyukjinKwon wanted the space on the previous line/sentence. --- - To unsubscribe, e-mail:

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18875 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18875 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81566/ Test PASSed. ---

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18875 **[Test build #81566 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81566/testReport)** for PR 18875 at commit

[GitHub] spark issue #19131: [MINOR][SQL]remove unuse import class

2017-09-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19131 I didn't think scalastyle could do that? didn't see a rule at http://www.scalastyle.org/rules-dev.html --- - To unsubscribe,

[GitHub] spark issue #18460: [SPARK-21247][SQL] Type comparision should respect case-...

2017-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18460 **[Test build #81568 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81568/testReport)** for PR 18460 at commit

[GitHub] spark issue #18460: [SPARK-21247][SQL] Type comparision should respect case-...

2017-09-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18460 Rebased to the master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19131: [MINOR][SQL]remove unuse import class

2017-09-08 Thread ash211
Github user ash211 commented on the issue: https://github.com/apache/spark/pull/19131 A check for unused imports should be added to scalastyle to prevent these from creeping back in. If this PR was accompanied with that check (failing before, now passing) I think the merge conflicts

[GitHub] spark issue #17451: [SPARK-19866][ML][PySpark] Add local version of Word2Vec...

2017-09-08 Thread keypointt
Github user keypointt commented on the issue: https://github.com/apache/spark/pull/17451 sorry being slow on this simple fix (my first pyspark/py4j commit :P) thank you very much for help all the way through 😄 ---

[GitHub] spark pull request #17451: [SPARK-19866][ML][PySpark] Add local version of W...

2017-09-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17451 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #17451: [SPARK-19866][ML][PySpark] Add local version of Word2Vec...

2017-09-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17451 LGTM merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19134: [SPARK-21893][BUILD][STREAMING][WIP] Put Kafka 0....

2017-09-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19134#discussion_r137867121 --- Diff: python/pyspark/streaming/tests.py --- @@ -1051,6 +1062,9 @@ def test_kafka_stream(self): def test_kafka_direct_stream(self):

[GitHub] spark pull request #19134: [SPARK-21893][BUILD][STREAMING][WIP] Put Kafka 0....

2017-09-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19134#discussion_r137867212 --- Diff: python/pyspark/streaming/tests.py --- @@ -1003,6 +1003,11 @@ def check_output(n): self.ssc.stop(True, True) +# Must be

[GitHub] spark issue #17096: [SPARK-15243][ML][SQL][PYTHON] Add missing support for u...

2017-09-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17096 Thank you @holdenk, @viirya and @ueshin. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2017-09-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/18339 Let's get some extra eyes on this, maybe @davies or @HyukjinKwon want to take a quick look? I think it makes sense as an advanced developer API but I'm open to other ideas. ---

[GitHub] spark pull request #17096: [SPARK-15243][ML][SQL][PYTHON] Add missing suppor...

2017-09-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17096 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #17451: [SPARK-19866][ML][PySpark] Add local version of Word2Vec...

2017-09-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17451 LGTM, thank you for all your hard work @keypointt -- the Py4J integration can be a little rough. --- - To unsubscribe, e-mail:

[GitHub] spark issue #17096: [SPARK-15243][ML][SQL][PYTHON] Add missing support for u...

2017-09-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17096 Thanks, merged to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet state sho...

2017-09-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/18982 Cool, let me know if you want a hand reproing the error. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19134: [SPARK-21893][BUILD][STREAMING][WIP] Put Kafka 0.8 behin...

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19134 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19134: [SPARK-21893][BUILD][STREAMING][WIP] Put Kafka 0.8 behin...

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19134 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81564/ Test PASSed. ---

[GitHub] spark issue #19134: [SPARK-21893][BUILD][STREAMING][WIP] Put Kafka 0.8 behin...

2017-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19134 **[Test build #81564 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81564/testReport)** for PR 19134 at commit

[GitHub] spark issue #19165: [SPARKR][BACKPORT-2.1] backporting package and test chan...

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19165 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19165: [SPARKR][BACKPORT-2.1] backporting package and test chan...

2017-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19165 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81565/ Test FAILed. ---

[GitHub] spark issue #19160: [SPARK-21934][CORE] Expose Shuffle Netty memory usage to...

2017-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19160 **[Test build #81567 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81567/testReport)** for PR 19160 at commit

  1   2   3   4   >