[GitHub] spark pull request #19112: [SPARK-21901][SS] Define toString for StateOperat...

2017-09-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19112#discussion_r136726519 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala --- @@ -200,7 +202,7 @@ class SourceProgress protected[sql]( */

[GitHub] spark pull request #19112: [SPARK-21901][SS] Define toString for StateOperat...

2017-09-03 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/19112#discussion_r136726445 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala --- @@ -177,11 +179,11 @@ class SourceProgress protected[sql](

[GitHub] spark pull request #19112: [SPARK-21901][SS] Define toString for StateOperat...

2017-09-03 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/19112#discussion_r136726289 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala --- @@ -200,7 +202,7 @@ class SourceProgress protected[sql](

[GitHub] spark issue #19112: [SPARK-21901][SS] Define toString for StateOperatorProgr...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19112 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81363/ Test PASSed. ---

[GitHub] spark issue #19112: [SPARK-21901][SS] Define toString for StateOperatorProgr...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19112 **[Test build #81363 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81363/testReport)** for PR 19112 at commit

[GitHub] spark issue #19112: [SPARK-21901][SS] Define toString for StateOperatorProgr...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19112 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136725354 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -178,11 +179,50 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark pull request #19086: [SPARK-21874][SQL] Support changing database when...

2017-09-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19086#discussion_r136725094 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -573,28 +573,29 @@ class SessionCatalog(

[GitHub] spark pull request #19086: [SPARK-21874][SQL] Support changing database when...

2017-09-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19086#discussion_r136725012 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -573,28 +573,29 @@ class SessionCatalog(

[GitHub] spark pull request #19086: [SPARK-21874][SQL] Support changing database when...

2017-09-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19086#discussion_r136724918 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -418,6 +436,39 @@ abstract class

[GitHub] spark pull request #19086: [SPARK-21874][SQL] Support changing database when...

2017-09-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19086#discussion_r136724986 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -357,25 +357,43 @@ abstract class

[GitHub] spark pull request #19086: [SPARK-21874][SQL] Support changing database when...

2017-09-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19086#discussion_r136724969 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -502,17 +502,16 @@ private[spark] class

[GitHub] spark pull request #19086: [SPARK-21874][SQL] Support changing database when...

2017-09-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19086#discussion_r136724919 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -418,6 +436,39 @@ abstract class

[GitHub] spark pull request #19086: [SPARK-21874][SQL] Support changing database when...

2017-09-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19086#discussion_r136723572 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -573,28 +573,29 @@ class SessionCatalog(

[GitHub] spark pull request #19086: [SPARK-21874][SQL] Support changing database when...

2017-09-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19086#discussion_r136724965 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -418,6 +436,39 @@ abstract class

[GitHub] spark pull request #19086: [SPARK-21874][SQL] Support changing database when...

2017-09-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19086#discussion_r136724927 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -418,6 +436,39 @@ abstract class

[GitHub] spark pull request #19086: [SPARK-21874][SQL] Support changing database when...

2017-09-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19086#discussion_r136723544 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -573,28 +573,29 @@ class SessionCatalog(

[GitHub] spark issue #19111: [SPARK-21801][SPARKR][TEST][WIP] set random seed for pre...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19111 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81364/ Test PASSed. ---

[GitHub] spark issue #19111: [SPARK-21801][SPARKR][TEST][WIP] set random seed for pre...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19111 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19111: [SPARK-21801][SPARKR][TEST][WIP] set random seed for pre...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19111 **[Test build #81364 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81364/testReport)** for PR 19111 at commit

[GitHub] spark pull request #19112: [SPARK-21901][SS] Define toString for StateOperat...

2017-09-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19112#discussion_r136724364 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala --- @@ -177,11 +179,11 @@ class SourceProgress protected[sql](

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18975 **[Test build #81366 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81366/testReport)** for PR 18975 at commit

[GitHub] spark pull request #19112: [SPARK-21901][SS] Define toString for StateOperat...

2017-09-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19112#discussion_r136724317 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala --- @@ -200,7 +202,7 @@ class SourceProgress protected[sql]( */

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-03 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136724258 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -178,11 +179,50 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark issue #19060: [WIP][SQL] Add DataSourceSuite validating data sources l...

2017-09-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19060 Hi, @gatorsmile . I believe I understand your advice correctly at this time. Could you take a look at this `data source` verification PR? --- If your project is set up for it, you can

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-03 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r136724143 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -178,11 +179,50 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark issue #19060: [WIP][SQL] Add DataSourceSuite validating data sources l...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19060 **[Test build #81365 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81365/testReport)** for PR 19060 at commit

[GitHub] spark issue #19111: [SPARK-21801][SPARKR][TEST][WIP] set random seed for pre...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19111 **[Test build #81364 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81364/testReport)** for PR 19111 at commit

[GitHub] spark issue #19112: [SPARK-21901][SS] Define toString for StateOperatorProgr...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19112 **[Test build #81363 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81363/testReport)** for PR 19112 at commit

[GitHub] spark pull request #19112: [SPARK-21901][SS] Define toString for StateOperat...

2017-09-03 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/19112 [SPARK-21901][SS] Define toString for StateOperatorProgress ## What changes were proposed in this pull request? Just `StateOperatorProgress.toString` + few formatting fixes

[GitHub] spark issue #18953: [SPARK-20682][SQL] Update ORC data source based on Apach...

2017-09-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18953 Hi, All. Although ORC seems not to be a prefered storage format in Apache Spark, ORC is very important to me. Could anyone review this again? --- If your project is set up for it, you

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18869 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81362/ Test PASSed. ---

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18869 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18869 **[Test build #81362 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81362/testReport)** for PR 18869 at commit

[GitHub] spark pull request #19109: Update index.md

2017-09-03 Thread demoonism
Github user demoonism closed the pull request at: https://github.com/apache/spark/pull/19109 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #19109: Update index.md

2017-09-03 Thread demoonism
Github user demoonism commented on the issue: https://github.com/apache/spark/pull/19109 Closing as the change is subtle. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #19086: [SPARK-21874][SQL] Support changing database when rename...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19086 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81361/ Test FAILed. ---

[GitHub] spark issue #19086: [SPARK-21874][SQL] Support changing database when rename...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19086 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19086: [SPARK-21874][SQL] Support changing database when rename...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19086 **[Test build #81361 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81361/testReport)** for PR 19086 at commit

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18865 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81360/ Test PASSed. ---

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18865 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18865 **[Test build #81360 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81360/testReport)** for PR 18865 at commit

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18865 `requiredSchema.length == 1 && requiredSchema.head.name == parsedOptions.columnNameOfCorruptRecord` What is the usage scenario if we do not block the above scenario? --- If your

[GitHub] spark pull request #16774: [SPARK-19357][ML] Adding parallel model evaluatio...

2017-09-03 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/16774#discussion_r136719561 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/TrainValidationSplit.scala --- @@ -87,37 +91,63 @@ class TrainValidationSplit

[GitHub] spark pull request #16774: [SPARK-19357][ML] Adding parallel model evaluatio...

2017-09-03 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/16774#discussion_r136719485 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala --- @@ -100,31 +113,53 @@ class CrossValidator @Since("1.2.0")

[GitHub] spark pull request #16774: [SPARK-19357][ML] Adding parallel model evaluatio...

2017-09-03 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/16774#discussion_r136719383 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala --- @@ -100,31 +113,53 @@ class CrossValidator @Since("1.2.0")

[GitHub] spark pull request #19086: [SPARK-21874][SQL] Support changing database when...

2017-09-03 Thread jinxing64
Github user jinxing64 commented on a diff in the pull request: https://github.com/apache/spark/pull/19086#discussion_r136719337 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -502,17 +502,16 @@ private[spark] class

[GitHub] spark issue #19086: [SPARK-21874][SQL] Support changing database when rename...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19086 **[Test build #81361 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81361/testReport)** for PR 19086 at commit

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18869 **[Test build #81362 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81362/testReport)** for PR 18869 at commit

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18869 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81359/ Test FAILed. ---

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-09-03 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18869 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #17924: [SPARK-20682][SQL] Support a new faster ORC data source ...

2017-09-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/17924 BTW, the latest version is maintained in #17980. Recently, Spark Vector format is changed. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #17924: [SPARK-20682][SQL] Support a new faster ORC data source ...

2017-09-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/17924 Hi, I didn't try that, but that's not a concept of Spark data source table. Please don't expect that. :) --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18869 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18869 **[Test build #81359 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81359/testReport)** for PR 18869 at commit

[GitHub] spark issue #17924: [SPARK-20682][SQL] Support a new faster ORC data source ...

2017-09-03 Thread cenyuhai
Github user cenyuhai commented on the issue: https://github.com/apache/spark/pull/17924 @dongjoon-hyun I have a question: does this orc data sources reader support a table contains multiple file format for example: table/ day=2017-09-01 RCFile

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18865 **[Test build #81360 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81360/testReport)** for PR 18865 at commit

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18865 @gatorsmile and @cloud-fan, do you guys prefer throwing an exception or printing a log? I think I like logging one more conservatively. BTW, I am sorry for raising this issue late now.

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18865 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81357/ Test PASSed. ---

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18865 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18865 **[Test build #81357 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81357/testReport)** for PR 18865 at commit

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18869 **[Test build #81359 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81359/testReport)** for PR 18869 at commit

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-03 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18865 @HyukjinKwon Thanks for the comment. I think the current behavior confuses users in some ways, as it can have weird query results shown in previous discussion. The previous fix that

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-09-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18869 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18029 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18029 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81358/ Test PASSed. ---

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18029 **[Test build #81358 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81358/testReport)** for PR 18029 at commit

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18869 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18869 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81356/ Test FAILed. ---

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18869 **[Test build #81356 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81356/testReport)** for PR 18869 at commit

[GitHub] spark pull request #18865: [SPARK-21610][SQL] Corrupt records are not handle...

2017-09-03 Thread jmchung
Github user jmchung commented on a diff in the pull request: https://github.com/apache/spark/pull/18865#discussion_r136715367 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonFileFormat.scala --- @@ -113,6 +113,18 @@ class JsonFileFormat

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18029 **[Test build #81358 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81358/testReport)** for PR 18029 at commit

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-09-03 Thread yssharma
Github user yssharma commented on the issue: https://github.com/apache/spark/pull/18029 Resolved conflict introduced by other code commits. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #19105: [SPARK-21897][PYTHON][R] Add unionByName API to DataFram...

2017-09-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19105 I am hitting a network issue. Will update JIRA tomorrow. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #19105: [SPARK-21897][PYTHON][R] Add unionByName API to D...

2017-09-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19105 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #19105: [SPARK-21897][PYTHON][R] Add unionByName API to DataFram...

2017-09-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19105 Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #19105: [SPARK-21897][PYTHON][R] Add unionByName API to DataFram...

2017-09-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19105 Thank you @viirya and @felixcheung. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18865: [SPARK-21610][SQL] Corrupt records are not handle...

2017-09-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18865#discussion_r136714872 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonFileFormat.scala --- @@ -113,6 +113,18 @@ class JsonFileFormat

[GitHub] spark pull request #18865: [SPARK-21610][SQL] Corrupt records are not handle...

2017-09-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18865#discussion_r136714892 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2034,4 +2034,26 @@ class JsonSuite extends

[GitHub] spark pull request #18865: [SPARK-21610][SQL] Corrupt records are not handle...

2017-09-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18865#discussion_r136714668 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonFileFormat.scala --- @@ -113,6 +113,18 @@ class JsonFileFormat

[GitHub] spark pull request #18865: [SPARK-21610][SQL] Corrupt records are not handle...

2017-09-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18865#discussion_r136714700 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonFileFormat.scala --- @@ -113,6 +113,18 @@ class JsonFileFormat

[GitHub] spark pull request #18865: [SPARK-21610][SQL] Corrupt records are not handle...

2017-09-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18865#discussion_r136714509 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonFileFormat.scala --- @@ -113,6 +113,18 @@ class JsonFileFormat

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-03 Thread jmchung
Github user jmchung commented on the issue: https://github.com/apache/spark/pull/18865 To @viirya and @gatorsmile, I made some modifications as follows: 1. move the check of `_corrupt_record` out of the function block to get fast fail in driver instead of executor side. 2.

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18865 **[Test build #81357 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81357/testReport)** for PR 18865 at commit

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18869 **[Test build #81356 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81356/testReport)** for PR 18869 at commit

[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...

2017-09-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19082 @viirya @rednaxelafx @kiszk okay, could you check again? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19082 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19082 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81354/ Test PASSed. ---

[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...

2017-09-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19082 **[Test build #81354 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81354/testReport)** for PR 19082 at commit

[GitHub] spark pull request #19109: Update index.md

2017-09-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19109#discussion_r136711660 --- Diff: docs/index.md --- @@ -24,7 +24,7 @@ source, visit [Building Spark](building-spark.html). Spark runs on both Windows and UNIX-like

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2017-09-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r136711099 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SortExec.scala --- @@ -177,6 +177,8 @@ case class SortExec( """.stripMargin.trim

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2017-09-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r136710928 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,14 +149,146 @@ trait CodegenSupport extends

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2017-09-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r136710824 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,14 +149,146 @@ trait CodegenSupport extends

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2017-09-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r136710234 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,14 +149,146 @@ trait CodegenSupport extends

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2017-09-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r136710741 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,14 +149,146 @@ trait CodegenSupport extends

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2017-09-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r136710667 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,14 +149,146 @@ trait CodegenSupport extends

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2017-09-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r136710909 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,14 +149,146 @@ trait CodegenSupport extends

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2017-09-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r136711183 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExpandExec.scala --- @@ -89,6 +89,8 @@ case class ExpandExec(

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2017-09-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r136710872 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,14 +149,146 @@ trait CodegenSupport extends

[GitHub] spark issue #19111: [SPARK-21801][SPARKR][TEST][WIP] set random seed for pre...

2017-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19111 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81355/ Test FAILed. ---

<    1   2   3   >