[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18865 Parsing all the columns is another option when users just select the `_corrupt_record`. However, users might not need all the columns, especially for the semi-structured formats (CSV and JSON).

[GitHub] spark issue #19126: [SPARK-21915][ML][PySpark]Model 1 and Model 2 ParamMaps ...

2017-09-05 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/19126 So I'm wondering if @BryanCutler's change for copying the params means that this should only be applied to old branches (2.2)? ---

[GitHub] spark issue #19056: [SPARK-21765] Check that optimization doesn't affect isS...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19056 **[Test build #81434 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81434/testReport)** for PR 19056 at commit

[GitHub] spark pull request #19056: [SPARK-21765] Check that optimization doesn't aff...

2017-09-05 Thread joseph-torres
Github user joseph-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/19056#discussion_r137143373 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/socket.scala --- @@ -128,8 +128,9 @@ class TextSocketSource(host: String,

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137143172 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -130,10 +130,12 @@ case class

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137143065 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -848,4 +851,24 @@ object DDLUtils { } }

[GitHub] spark pull request #19117: [SPARK-21904] [SQL] Rename tempTables to tempView...

2017-09-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19117#discussion_r137142990 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -812,7 +812,7 @@ abstract class DDLSuite extends QueryTest

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19124 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19124 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81430/ Test PASSed. ---

[GitHub] spark pull request #19050: [SPARK-21835][SQL] RewritePredicateSubquery shoul...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19050#discussion_r137142974 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala --- @@ -49,6 +49,30 @@ object RewritePredicateSubquery

[GitHub] spark pull request #19117: [SPARK-21904] [SQL] Rename tempTables to tempView...

2017-09-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19117#discussion_r137142830 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -812,7 +812,7 @@ abstract class DDLSuite extends QueryTest

[GitHub] spark pull request #19117: [SPARK-21904] [SQL] Rename tempTables to tempView...

2017-09-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19117#discussion_r137142923 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -860,7 +860,7 @@ abstract class DDLSuite extends QueryTest

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19124 **[Test build #81430 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81430/testReport)** for PR 19124 at commit

[GitHub] spark pull request #19050: [SPARK-21835][SQL] RewritePredicateSubquery shoul...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19050#discussion_r137142729 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala --- @@ -98,6 +122,7 @@ object RewritePredicateSubquery

[GitHub] spark pull request #19050: [SPARK-21835][SQL] RewritePredicateSubquery shoul...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19050#discussion_r137142788 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SubquerySuite.scala --- @@ -875,4 +876,71 @@ class SubquerySuite extends QueryTest with

[GitHub] spark pull request #19050: [SPARK-21835][SQL] RewritePredicateSubquery shoul...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19050#discussion_r137142674 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala --- @@ -49,6 +49,30 @@ object RewritePredicateSubquery

[GitHub] spark pull request #19050: [SPARK-21835][SQL] RewritePredicateSubquery shoul...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19050#discussion_r137142666 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala --- @@ -49,6 +49,30 @@ object RewritePredicateSubquery

[GitHub] spark issue #19117: [SPARK-21904] [SQL] Rename tempTables to tempViews in Se...

2017-09-05 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19117 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18865 In that case, the current fix does not cover when selecting few column together - https://github.com/apache/spark/pull/18865#issuecomment-326865161. Why wouldn't we block all cases, or document

[GitHub] spark pull request #19117: [SPARK-21904] [SQL] Rename tempTables to tempView...

2017-09-05 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19117#discussion_r137142277 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -90,13 +87,13 @@ class SessionCatalog(

[GitHub] spark pull request #19056: [SPARK-21765] Check that optimization doesn't aff...

2017-09-05 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/19056#discussion_r137142093 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/socket.scala --- @@ -128,8 +128,9 @@ class TextSocketSource(host: String, port:

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137141966 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -130,10 +130,12 @@ case class

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137141677 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -848,4 +851,24 @@ object DDLUtils { } }

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137141720 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala --- @@ -0,0 +1,42 @@ +/* + * Licensed

[GitHub] spark issue #18950: [SPARK-20589][Core][Scheduler] Allow limiting task concu...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18950 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81427/ Test PASSed. ---

[GitHub] spark issue #18950: [SPARK-20589][Core][Scheduler] Allow limiting task concu...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18950 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18950: [SPARK-20589][Core][Scheduler] Allow limiting task concu...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18950 **[Test build #81427 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81427/testReport)** for PR 18950 at commit

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18865 Based on the original PR https://github.com/apache/spark/pull/2680 and the current behavior, we do not think the current behavior is by design. The usage cases you mentioned above are rare. When

[GitHub] spark issue #19050: [SPARK-21835][SQL] RewritePredicateSubquery should not p...

2017-09-05 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19050 @cloud-fan and @hvanhovell seems too busy these days. maybe @gatorsmile can also help review this. Thanks. --- - To unsubscribe,

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19129 Wow! Thank you, @HyukjinKwon ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18865 Could you share the offline discussion and why - @gatorsmile and @sameeragarwal? --- - To unsubscribe, e-mail:

[GitHub] spark issue #19117: [SPARK-21904] [SQL] Rename tempTables to tempViews in Se...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19117 **[Test build #81433 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81433/testReport)** for PR 19117 at commit

[GitHub] spark issue #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cacheMetadat...

2017-09-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19129 The last code looks removed in https://github.com/apache/spark/commit/678b96e77bf77a64b8df14b19db5a3bb18febfe3 and this option looks introduced in

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19124 Please also fix `AlterTableAddColumnsCommand`? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137132365 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala --- @@ -0,0 +1,42 @@ +/* + * Licensed to

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137132286 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -130,10 +130,12 @@ case class

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137132157 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -848,4 +851,24 @@ object DDLUtils { } }

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18865 After an offline discussion with @sameeragarwal , we think we should block it with a reasonable error message. @jmchung Please update the error message and also add it to the migration

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19124 **[Test build #81432 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81432/testReport)** for PR 19124 at commit

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137130122 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -145,15 +146,27 @@ class DetermineTableStats(session:

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19124 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19124 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81424/ Test PASSed. ---

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19124 **[Test build #81424 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81424/testReport)** for PR 19124 at commit

[GitHub] spark issue #19139: [MINOR][DOC] Update `Partition Discovery` section to enu...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19139 Thank you, @gatorsmile ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137122638 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -145,15 +146,27 @@ class DetermineTableStats(session:

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137121898 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -145,15 +146,27 @@ class DetermineTableStats(session:

[GitHub] spark pull request #19139: [MINOR][DOC] Update `Partition Discovery` section...

2017-09-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19139 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19139: [MINOR][DOC] Update `Partition Discovery` section to enu...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19139 Thanks! Merged to master/2.2 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137120663 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -145,15 +146,27 @@ class DetermineTableStats(session:

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19140 **[Test build #81431 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81431/testReport)** for PR 19140 at commit

[GitHub] spark issue #19139: [MINOR][DOC] Update `Partition Discovery` section to enu...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19139 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19139: [MINOR][DOC] Update `Partition Discovery` section to enu...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19139 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81429/ Test PASSed. ---

[GitHub] spark issue #19139: [MINOR][DOC] Update `Partition Discovery` section to enu...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19139 **[Test build #81429 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81429/testReport)** for PR 19139 at commit

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-05 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137116856 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19124 **[Test build #81430 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81430/testReport)** for PR 19124 at commit

[GitHub] spark issue #19139: [MINOR][DOC] Update `Partition Discovery` section to enu...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19139 **[Test build #81429 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81429/testReport)** for PR 19139 at commit

[GitHub] spark pull request #19139: [MINOR][DOC] Update `Partition Discovery` section...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19139#discussion_r137115627 --- Diff: docs/sql-programming-guide.md --- @@ -733,8 +733,9 @@ SELECT * FROM parquetTable Table partitioning is a common optimization

[GitHub] spark issue #18253: [SPARK-18838][CORE] Introduce multiple queues in LiveLis...

2017-09-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18253 I really dislike `WithListenerBus` - both as a name and as a concept. There's already a `ListenerBus` trait; if it's not enough or is broken in some way, it should be fixed, instead of being patched

[GitHub] spark issue #18935: [SPARK-9104][CORE] Expose Netty memory metrics in Spark

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18935 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81420/ Test PASSed. ---

[GitHub] spark issue #18935: [SPARK-9104][CORE] Expose Netty memory metrics in Spark

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18935 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18935: [SPARK-9104][CORE] Expose Netty memory metrics in Spark

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18935 **[Test build #81420 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81420/testReport)** for PR 18935 at commit

[GitHub] spark pull request #19139: [MINOR][DOC] Update `Partition Discovery` section...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19139#discussion_r137112679 --- Diff: docs/sql-programming-guide.md --- @@ -733,8 +733,9 @@ SELECT * FROM parquetTable Table partitioning is a common optimization

[GitHub] spark issue #19139: [MINOR][DOC] Update `Partition Discovery` section to enu...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19139 LGTM except a minor comment --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-05 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137111624 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

[GitHub] spark pull request #18692: [SPARK-21417][SQL] Infer join conditions using pr...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18692#discussion_r137111457 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala --- @@ -152,3 +152,71 @@ object EliminateOuterJoin extends

[GitHub] spark pull request #18692: [SPARK-21417][SQL] Infer join conditions using pr...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18692#discussion_r137110438 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala --- @@ -152,3 +152,71 @@ object EliminateOuterJoin extends

[GitHub] spark issue #19139: [MINOR][DOC] Update `Partition Discovery` section to enu...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19139 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81428/ Test PASSed. ---

[GitHub] spark pull request #18692: [SPARK-21417][SQL] Infer join conditions using pr...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18692#discussion_r137110370 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala --- @@ -152,3 +152,71 @@ object EliminateOuterJoin extends

[GitHub] spark issue #19139: [MINOR][DOC] Update `Partition Discovery` section to enu...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19139 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19139: [MINOR][DOC] Update `Partition Discovery` section to enu...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19139 **[Test build #81428 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81428/testReport)** for PR 19139 at commit

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-05 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137108881 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-05 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137108805 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-05 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137108746 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-05 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137108608 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137107658 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFileFormat.scala --- @@ -83,6 +83,8 @@ class OrcFileFormat extends FileFormat with

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137107283 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala --- @@ -85,6 +88,14 @@ case class

[GitHub] spark issue #19139: [MINOR][DOC] Update `Partition Discovery` section to enu...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19139 **[Test build #81428 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81428/testReport)** for PR 19139 at commit

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137107152 --- Diff: sql/core/src/test/resources/sql-tests/results/show_columns.sql.out --- @@ -19,23 +19,23 @@ struct<> -- !query 2

[GitHub] spark pull request #18975: [SPARK-4131] Support "Writing data into the files...

2017-09-05 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18975#discussion_r137106989 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -241,11 +241,21 @@ query : ctes? queryNoWith

[GitHub] spark issue #19139: [MINOR][DOC] Update `Partition Discovery` section to enu...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19139 I added `(including TEXT/CSV/JSON/ORC/Parquet)` and updated the PR title, too. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19139: [MINOR][DOC] Add ORC in `Partition Discovery` section.

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19139 BTW, please update the PR title. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137103061 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFileFormat.scala --- @@ -83,6 +83,8 @@ class OrcFileFormat extends FileFormat with

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137102745 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala --- @@ -85,6 +88,14 @@ case class

[GitHub] spark issue #18950: [SPARK-20589][Core][Scheduler] Allow limiting task concu...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18950 **[Test build #81427 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81427/testReport)** for PR 18950 at commit

[GitHub] spark issue #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasource table...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19124 That is normal. When we find a bug, it normally means we ignore it in more than one place. Thus, we need to check all the other code paths that could trigger it. ---

[GitHub] spark pull request #19099: [SPARK-21652][SQL] Fix rule confliction between I...

2017-09-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19099 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19124: [SPARK-21912][SQL] Creating ORC/Parquet datasourc...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19124#discussion_r137101814 --- Diff: sql/core/src/test/resources/sql-tests/results/show_columns.sql.out --- @@ -19,23 +19,23 @@ struct<> -- !query 2 -CREATE

[GitHub] spark pull request #19139: [MINOR][DOC] Add ORC in `Partition Discovery` sec...

2017-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19139#discussion_r137101509 --- Diff: docs/sql-programming-guide.md --- @@ -733,7 +733,7 @@ SELECT * FROM parquetTable Table partitioning is a common optimization

[GitHub] spark pull request #19138: [SPARK-21925] Update trigger interval documentati...

2017-09-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19138 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19099: [SPARK-21652][SQL] Fix rule confliction between InferFil...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19099 Thanks! Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19099: [SPARK-21652][SQL] Fix rule confliction between InferFil...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19099 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19139: [MINOR][DOC] Add ORC in `Partition Discovery` sec...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19139#discussion_r137101003 --- Diff: docs/sql-programming-guide.md --- @@ -733,7 +733,7 @@ SELECT * FROM parquetTable Table partitioning is a common optimization

[GitHub] spark issue #19099: [SPARK-21652][SQL] Fix rule confliction between InferFil...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19099 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81418/ Test PASSed. ---

[GitHub] spark issue #19099: [SPARK-21652][SQL] Fix rule confliction between InferFil...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19099 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-05 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/19132 Though this may fix the problem it is 100% the wrong way to do it for multiple reasons: 1. The UI should not touch the FS, FS access it abstracted out so the UI is can work with multiple FS

[GitHub] spark issue #19099: [SPARK-21652][SQL] Fix rule confliction between InferFil...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19099 **[Test build #81418 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81418/testReport)** for PR 19099 at commit

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19140 **[Test build #81426 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81426/testReport)** for PR 19140 at commit

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19140 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81426/ Test FAILed. ---

[GitHub] spark issue #19140: [SPARK-21890] Credentials not being passed to add the to...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19140 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r137100467 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,319 @@ case class

<    1   2   3   4   5   6   >