[GitHub] spark pull request #21771: [SPARK-24807][CORE] Adding files/jars twice: outp...

2018-07-14 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21771#discussion_r202531999 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1555,6 +1559,9 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark issue #21771: [SPARK-24807][CORE] Adding files/jars twice: output a wa...

2018-07-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21771 We could have updated the doc for `spark.files.overwrite` too since the confusion probably with this configuration. --- -

[GitHub] spark pull request #21771: [SPARK-24807][CORE] Adding files/jars twice: outp...

2018-07-14 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21771#discussion_r202531920 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1849,6 +1858,9 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #21771: [SPARK-24807][CORE] Adding files/jars twice: outp...

2018-07-14 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21771#discussion_r202531879 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1555,6 +1559,9 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21770 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21770 **[Test build #93013 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93013/testReport)** for PR 21770 at commit

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21770 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93013/ Test FAILed. ---

[GitHub] spark pull request #21740: [SPARK-18230][MLLib]Throw a better exception, if ...

2018-07-14 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21740#discussion_r202531693 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -75,10 +75,22 @@ class MatrixFactorizationModel

[GitHub] spark pull request #21740: [SPARK-18230][MLLib]Throw a better exception, if ...

2018-07-14 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21740#discussion_r202531686 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModelSuite.scala --- @@ -72,6 +72,22 @@ class

[GitHub] spark issue #21556: [SPARK-24549][SQL] Support Decimal type push down to the...

2018-07-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21556 **[Test build #93014 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93014/testReport)** for PR 21556 at commit

[GitHub] spark issue #21556: [SPARK-24549][SQL] Support Decimal type push down to the...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21556 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/960/

[GitHub] spark issue #21556: [SPARK-24549][SQL] Support Decimal type push down to the...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21556 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21771: [SPARK-24807][CORE] Adding files/jars twice: outp...

2018-07-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21771 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-14 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21764 ok, thx for the kind explanation. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #21771: [SPARK-24807][CORE] Adding files/jars twice: outp...

2018-07-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21771#discussion_r202531555 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1555,6 +1559,9 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark issue #21771: [SPARK-24807][CORE] Adding files/jars twice: output a wa...

2018-07-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21771 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21771: [SPARK-24807][CORE] Adding files/jars twice: output a wa...

2018-07-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21771 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21771: [SPARK-24807][CORE] Adding files/jars twice: outp...

2018-07-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21771#discussion_r202531553 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1555,6 +1559,9 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21770 **[Test build #93013 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93013/testReport)** for PR 21770 at commit

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21770 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21770 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/959/

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21770 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21764 @maropu This is for advanced end users or Spark developers. External conf looks fine, but I have to admit this might be rarely used. BTW, after having this conf, we can deprecate a few internal

[GitHub] spark pull request #21768: [SPARK-24776][SQL]Avro unit test: deduplicate cod...

2018-07-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21768 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21711: [SPARK-24681][SQL] Verify nested column names in ...

2018-07-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21711#discussion_r202531134 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -138,17 +138,35 @@ private[spark] class

[GitHub] spark pull request #21711: [SPARK-24681][SQL] Verify nested column names in ...

2018-07-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21711#discussion_r202531122 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -138,17 +138,35 @@ private[spark] class

[GitHub] spark pull request #21711: [SPARK-24681][SQL] Verify nested column names in ...

2018-07-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21711#discussion_r202531090 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -2005,6 +2005,24 @@ class SQLQuerySuite extends

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21770 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93012/ Test FAILed. ---

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21770 **[Test build #93012 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93012/testReport)** for PR 21770 at commit

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21770 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21771: [SPARK-24807][CORE] Adding files/jars twice: outp...

2018-07-14 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21771#discussion_r202530515 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1555,6 +1559,9 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark issue #21771: [SPARK-24807][CORE] Adding files/jars twice: output a wa...

2018-07-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21771 Seems fine to me. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21771: [SPARK-24807][CORE] Adding files/jars twice: outp...

2018-07-14 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21771#discussion_r202530492 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1555,6 +1559,9 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #21771: [SPARK-24807][CORE] Adding files/jars twice: outp...

2018-07-14 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21771#discussion_r202530444 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1555,6 +1559,9 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #21771: [SPARK-24807][CORE] Adding files/jars twice: outp...

2018-07-14 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21771#discussion_r202530411 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1555,6 +1559,9 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21770 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/958/

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21770 **[Test build #93012 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93012/testReport)** for PR 21770 at commit

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21770 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21741: [SPARK-24718][SQL] Timestamp support pushdown to ...

2018-07-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21741 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21770 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21741: [SPARK-24718][SQL] Timestamp support pushdown to parquet...

2018-07-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21741 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21657 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93010/ Test PASSed. ---

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21657 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21657 **[Test build #93010 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93010/testReport)** for PR 21657 at commit

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21770 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93011/ Test FAILed. ---

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21770 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21770 **[Test build #93011 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93011/testReport)** for PR 21770 at commit

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-14 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21764 @gatorsmile aha, ok. We need to make this option not `internal` but `external`? BTW, the interfaces to add/delete optimizer rules (addition via `ExperimentalMethods` and deletion via

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21770 Also, `SparkException` needs to extend `RuntimeException` instead of `Exception` because some generated codes do not have code to catch the exception. But, the change causes Mima test failures, so

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21770 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21770 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/957/

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21770 **[Test build #93011 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93011/testReport)** for PR 21770 at commit

[GitHub] spark issue #21766: [SPARK-24803][SQL] add support for numeric

2018-07-14 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21766 aha, I didn't know that and postgresql also uses them interchangeably, too. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-07-14 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21657#discussion_r202528977 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -197,15 +202,21 @@ class UnivocityParser(

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-07-14 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21657#discussion_r202528933 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -38,24 +38,29 @@ class UnivocityParser(

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-07-14 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21657#discussion_r202528931 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -223,8 +234,8 @@ class UnivocityParser(

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-07-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21657#discussion_r202528391 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -197,15 +202,21 @@ class UnivocityParser(

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-07-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21657#discussion_r202528267 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -38,24 +38,29 @@ class UnivocityParser(

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-07-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21657#discussion_r202528372 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -223,8 +234,8 @@ class UnivocityParser(

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-07-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21657#discussion_r202527982 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1579,4 +1579,20 @@ class CSVSuite extends

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-07-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21657#discussion_r202528247 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -38,24 +38,29 @@ class UnivocityParser(

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-07-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21657#discussion_r202527784 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -38,24 +38,29 @@ class UnivocityParser(

[GitHub] spark pull request #20611: [SPARK-23425][SQL]Support wildcard in HDFS path f...

2018-07-14 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20611#discussion_r202528097 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -303,94 +303,44 @@ case class LoadDataCommand(

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21657 **[Test build #93010 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93010/testReport)** for PR 21657 at commit

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21657 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/956/

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21657 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21769: [SPARK-24805][SQL] Do not ignore avro files without exte...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21769 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93009/ Test PASSed. ---

[GitHub] spark issue #21769: [SPARK-24805][SQL] Do not ignore avro files without exte...

2018-07-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21769 **[Test build #93009 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93009/testReport)** for PR 21769 at commit

[GitHub] spark issue #21769: [SPARK-24805][SQL] Do not ignore avro files without exte...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21769 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21657 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21657 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21766: [SPARK-24803][SQL] add support for numeric

2018-07-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21766 They are not exactly the same in ANSI SQL, although MS SQL Server looks like use them interchangeably.

[GitHub] spark issue #21769: [SPARK-24805][SQL] Do not ignore avro files without exte...

2018-07-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21769 **[Test build #93009 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93009/testReport)** for PR 21769 at commit

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21770 cc @rednaxelafx --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21769: [SPARK-24805][SQL] Do not ignore avro files without exte...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21769 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21769: [SPARK-24805][SQL] Do not ignore avro files without exte...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21769 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93008/ Test FAILed. ---

[GitHub] spark issue #21769: [SPARK-24805][SQL] Do not ignore avro files without exte...

2018-07-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21769 **[Test build #93008 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93008/testReport)** for PR 21769 at commit

[GitHub] spark issue #21769: [SPARK-24805][SQL] Do not ignore avro files without exte...

2018-07-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21769 **[Test build #93008 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93008/testReport)** for PR 21769 at commit

[GitHub] spark pull request #21769: [SPARK-24805][SQL] Do not ignore avro files witho...

2018-07-14 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21769#discussion_r202526423 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroFileFormat.scala --- @@ -64,7 +64,7 @@ private[avro] class AvroFileFormat extends

[GitHub] spark issue #21698: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-07-14 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/21698 Taking a step back and analyzing the solution for the problem at hand. There are three main issues with the proposal: * It does not solve the problem in a general manner. * I

[GitHub] spark pull request #21750: [SPARK-24754][ML] Minhash integer overflow

2018-07-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21750 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21750: [SPARK-24754][ML] Minhash integer overflow

2018-07-14 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21750 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21769: [SPARK-24805][SQL] Do not ignore avro files witho...

2018-07-14 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21769#discussion_r202525034 --- Diff: external/avro/src/test/scala/org/apache/spark/sql/avro/AvroSuite.scala --- @@ -623,7 +624,7 @@ class AvroSuite extends SparkFunSuite {

[GitHub] spark pull request #21769: [SPARK-24805][SQL] Do not ignore avro files witho...

2018-07-14 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21769#discussion_r202524884 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroFileFormat.scala --- @@ -64,7 +64,7 @@ private[avro] class AvroFileFormat extends

[GitHub] spark pull request #21769: [SPARK-24805][SQL] Do not ignore avro files witho...

2018-07-14 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21769#discussion_r202524696 --- Diff: external/avro/src/test/scala/org/apache/spark/sql/avro/AvroSuite.scala --- @@ -809,4 +810,16 @@ class AvroSuite extends SparkFunSuite {

[GitHub] spark issue #21556: [SPARK-24549][SQL] Support Decimal type push down to the...

2018-07-14 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21556 I misunderstood how it was safe as well. It was Yuming's clarification that helped. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21769: [SPARK-24805][SQL] Do not ignore avro files witho...

2018-07-14 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21769#discussion_r202524552 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroFileFormat.scala --- @@ -64,7 +64,7 @@ private[avro] class AvroFileFormat extends

[GitHub] spark issue #21771: [SPARK-24807][CORE] Adding files/jars twice: output a wa...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21771 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93007/ Test PASSed. ---

[GitHub] spark issue #21771: [SPARK-24807][CORE] Adding files/jars twice: output a wa...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21771 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21771: [SPARK-24807][CORE] Adding files/jars twice: output a wa...

2018-07-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21771 **[Test build #93007 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93007/testReport)** for PR 21771 at commit

[GitHub] spark issue #21771: [SPARK-24807][CORE] Adding files/jars twice: output a wa...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21771 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93005/ Test PASSed. ---

[GitHub] spark issue #21771: [SPARK-24807][CORE] Adding files/jars twice: output a wa...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21771 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21771: [SPARK-24807][CORE] Adding files/jars twice: output a wa...

2018-07-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21771 **[Test build #93005 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93005/testReport)** for PR 21771 at commit

[GitHub] spark issue #21741: [SPARK-24718][SQL] Timestamp support pushdown to parquet...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21741 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21741: [SPARK-24718][SQL] Timestamp support pushdown to parquet...

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21741 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93006/ Test PASSed. ---

[GitHub] spark issue #21741: [SPARK-24718][SQL] Timestamp support pushdown to parquet...

2018-07-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21741 **[Test build #93006 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93006/testReport)** for PR 21741 at commit

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21102 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93002/ Test PASSed. ---

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21102 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21102 **[Test build #93002 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93002/testReport)** for PR 21102 at commit

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21103 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

  1   2   >