[GitHub] spark issue #21769: [SPARK-24805][SQL] Do not ignore avro files without exte...

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21769 **[Test build #93034 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93034/testReport)** for PR 21769 at commit

[GitHub] spark issue #21775: [SPARK-24812][SQL] Last Access Time in the table descrip...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21775 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21775: [SPARK-24812][SQL] Last Access Time in the table descrip...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21775 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21775: [SPARK-24812][SQL] Last Access Time in the table descrip...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21775 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21775: [SPARK-24812][SQL] Last Access Time in the table ...

2018-07-15 Thread sujith71955
GitHub user sujith71955 opened a pull request: https://github.com/apache/spark/pull/21775 [SPARK-24812][SQL] Last Access Time in the table description is not valid ## What changes were proposed in this pull request? Last Access Time will always displayed wrong date Wed Dec 31

[GitHub] spark issue #21740: [SPARK-18230][MLLib]Throw a better exception, if the use...

2018-07-15 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/21740 Hi @srowen. The build has passed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21773: [SPARK-24810][SQL] Fix paths to test files in AvroSuite

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21773 **[Test build #93033 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93033/testReport)** for PR 21773 at commit

[GitHub] spark issue #21774: [WIP][SPARK-24811][SQL]Avro: add new function from_avro ...

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21774 **[Test build #93032 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93032/testReport)** for PR 21774 at commit

[GitHub] spark issue #21774: [WIP][SPARK-24811][SQL]Avro: add new function from_avro ...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21774 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21774: [WIP][SPARK-24811][SQL]Avro: add new function from_avro ...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21774 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93032/ Test PASSed. ---

[GitHub] spark issue #21773: [SPARK-24810][SQL] Fix paths to test files in AvroSuite

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21773 **[Test build #4212 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4212/testReport)** for PR 21773 at commit

[GitHub] spark issue #21774: [WIP][SPARK-24811][SQL]Avro: add new function from_avro ...

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21774 **[Test build #93032 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93032/testReport)** for PR 21774 at commit

[GitHub] spark issue #21774: [WIP][SPARK-24811][SQL]Avro: add new function from_avro ...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21774 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/973/

[GitHub] spark issue #21774: [WIP][SPARK-24811][SQL]Avro: add new function from_avro ...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21774 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21774: [WIP][SPARK-24811][SQL]Avro: add new function fro...

2018-07-15 Thread gengliangwang
GitHub user gengliangwang opened a pull request: https://github.com/apache/spark/pull/21774 [WIP][SPARK-24811][SQL]Avro: add new function from_avro and to_avro ## What changes were proposed in this pull request? Add a new function from_avro for parsing a binary column of

[GitHub] spark issue #21740: [SPARK-18230][MLLib]Throw a better exception, if the use...

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21740 **[Test build #4211 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4211/testReport)** for PR 21740 at commit

[GitHub] spark pull request #21773: [SPARK-24810][SQL] Fix paths to test files in Avr...

2018-07-15 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/21773#discussion_r202550099 --- Diff: external/avro/src/test/scala/org/apache/spark/sql/avro/AvroSuite.scala --- @@ -37,27 +37,27 @@ import

[GitHub] spark issue #21589: [SPARK-24591][CORE] Number of cores and executors in the...

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21589 **[Test build #93031 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93031/testReport)** for PR 21589 at commit

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21770 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21770 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93030/ Test FAILed. ---

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21770 **[Test build #93030 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93030/testReport)** for PR 21770 at commit

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21770 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21770 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93029/ Test FAILed. ---

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21770 **[Test build #93029 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93029/testReport)** for PR 21770 at commit

[GitHub] spark issue #21608: [SPARK-24626] [SQL] Improve location size calculation in...

2018-07-15 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/21608 Maybe @ericl has some cycles? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21565: [SPARK-24558][Core]wrong Idle Timeout value is used in c...

2018-07-15 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21565 @cloud-fan did this merge? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21505: [SPARK-24457][SQL] Improving performance of strin...

2018-07-15 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21505#discussion_r202548671 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/StringToTimestampBenchmark.scala --- @@ -0,0 +1,53 @@ +/* --- End diff -- Does

[GitHub] spark issue #21755: Doc fix: The Imputer is an Estimator

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21755 **[Test build #4214 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4214/testReport)** for PR 21755 at commit

[GitHub] spark issue #21765: [MINOR][CORE] Add test cases for RDD.cartesian

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21765 **[Test build #4213 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4213/testReport)** for PR 21765 at commit

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-15 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21770 PS I don't think SparkException should be a RuntimeException even if it were possible. It's possible to get the Scala code to declare SparkException in the byte code if that's what you need it to do

[GitHub] spark pull request #21770: [SPARK-24806][SQL] Brush up generated code so tha...

2018-07-15 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21770#discussion_r202548522 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala --- @@ -890,7 +890,7 @@ case class Cast(child: Expression,

[GitHub] spark pull request #21770: [SPARK-24806][SQL] Brush up generated code so tha...

2018-07-15 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21770#discussion_r202548556 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1588,6 +1588,15 @@ object CodeGenerator

[GitHub] spark issue #21773: [SPARK-24810][SQL] Fix paths to test files in AvroSuite

2018-07-15 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/21773 jenkins, retest this, please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21773: [SPARK-24810][SQL] Fix paths to test files in AvroSuite

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21773 **[Test build #4212 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4212/testReport)** for PR 21773 at commit

[GitHub] spark issue #21740: [SPARK-18230][MLLib]Throw a better exception, if the use...

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21740 **[Test build #4211 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4211/testReport)** for PR 21740 at commit

[GitHub] spark issue #21773: [SPARK-24810][SQL] Fix paths to test files in AvroSuite

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21773 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93028/ Test FAILed. ---

[GitHub] spark issue #21773: [SPARK-24810][SQL] Fix paths to test files in AvroSuite

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21773 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21773: [SPARK-24810][SQL] Fix paths to test files in AvroSuite

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21773 **[Test build #93028 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93028/testReport)** for PR 21773 at commit

[GitHub] spark pull request #21740: [SPARK-18230][MLLib]Throw a better exception, if ...

2018-07-15 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/21740#discussion_r202547781 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -75,11 +75,29 @@ class

[GitHub] spark issue #18714: [SPARK-20236][SQL] dynamic partition overwrite

2018-07-15 Thread koertkuipers
Github user koertkuipers commented on the issue: https://github.com/apache/spark/pull/18714 should this be exposed per write instead of as a global variable? e.g. dataframe.write.csv.partitionMode(Dynamic).partitionBy(...).save(...) ---

[GitHub] spark pull request #21740: [SPARK-18230][MLLib]Throw a better exception, if ...

2018-07-15 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/21740#discussion_r202547705 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -75,11 +75,29 @@ class

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21770 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21770 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/972/

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21770 **[Test build #93030 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93030/testReport)** for PR 21770 at commit

[GitHub] spark pull request #21770: [SPARK-24806][SQL] Brush up generated code so tha...

2018-07-15 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21770#discussion_r202547003 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3758,7 +3758,14 @@ case class

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21770 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/971/

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21770 **[Test build #93029 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93029/testReport)** for PR 21770 at commit

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21770 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21589: [SPARK-24591][CORE] Number of cores and executors in the...

2018-07-15 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/21589 > @felixcheung I am not sure that our users are so interested in getting a list of cores per executors and calculate total numbers cores by summurizing the list. It will just complicate API and

[GitHub] spark pull request #21589: [SPARK-24591][CORE] Number of cores and executors...

2018-07-15 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/21589#discussion_r202545679 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskScheduler.scala --- @@ -67,6 +67,10 @@ private[spark] trait TaskScheduler { // Get

[GitHub] spark issue #21589: [SPARK-24591][CORE] Number of cores and executors in the...

2018-07-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21589 Yarn can use dynamic allocation as well. That's why I said "in general". To address @felixcheung's concern, I guess it's good to mention like see the configuration section and details can be

[GitHub] spark pull request #21740: [SPARK-18230][MLLib]Throw a better exception, if ...

2018-07-15 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21740#discussion_r202545740 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -165,7 +183,7 @@ class MatrixFactorizationModel

[GitHub] spark pull request #21740: [SPARK-18230][MLLib]Throw a better exception, if ...

2018-07-15 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21740#discussion_r202545589 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -75,11 +75,29 @@ class MatrixFactorizationModel

[GitHub] spark issue #21664: [SPARK-24687][CORE] NoClassDefFoundError will not be cat...

2018-07-15 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/21664 I agree we shall fall the job instead of let the job stay hanging. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21729: [SPARK-24755][Core] Executor loss can cause task ...

2018-07-15 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/21729#discussion_r202545576 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -87,7 +87,7 @@ private[spark] class TaskSetManager( // Set

[GitHub] spark issue #21773: [SPARK-24810][SQL] Fix paths to test files in AvroSuite

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21773 **[Test build #93028 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93028/testReport)** for PR 21773 at commit

[GitHub] spark issue #21698: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-07-15 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/21698 Actually I think @mridulm have a point here - if we only retry all the tasks for repartition/zip*, it's still possible that some tasks in a succeeding stage may have finished before retry, and

[GitHub] spark issue #21773: [SPARK-24810][SQL] Fix paths to test files in AvroSuite

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21773 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93027/ Test FAILed. ---

[GitHub] spark issue #21773: [SPARK-24810][SQL] Fix paths to test files in AvroSuite

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21773 **[Test build #93027 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93027/testReport)** for PR 21773 at commit

[GitHub] spark issue #21773: [SPARK-24810][SQL] Fix paths to test files in AvroSuite

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21773 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21773: [SPARK-24810][SQL] Fix paths to test files in AvroSuite

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21773 **[Test build #93027 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93027/testReport)** for PR 21773 at commit

[GitHub] spark issue #21773: [SPARK-24810][SQL] Fix paths to test files in AvroSuite

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21773 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21773: [SPARK-24810][SQL] Fix paths to test files in AvroSuite

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21773 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21773: [SPARK-24810][SQL] Fix paths to test files in Avr...

2018-07-15 Thread MaxGekk
GitHub user MaxGekk opened a pull request: https://github.com/apache/spark/pull/21773 [SPARK-24810][SQL] Fix paths to test files in AvroSuite ## What changes were proposed in this pull request? In the PR, I propose to move `testFile()` to the common trait

[GitHub] spark issue #21698: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-07-15 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21698 I think all we discussed here is about promise and expectation, and your zip example seems is not an issue: according to the promise that Spark does not guarantee output order of unsorted

[GitHub] spark pull request #21762: [SPARK-24800][SQL] Refactor Avro Serializer and D...

2018-07-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21762 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21762: [SPARK-24800][SQL] Refactor Avro Serializer and Deserial...

2018-07-15 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21762 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21771: [SPARK-24807][CORE] Adding files/jars twice: outp...

2018-07-15 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21771#discussion_r202543543 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1555,6 +1559,9 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #21770: [SPARK-24806][SQL] Brush up generated code so tha...

2018-07-15 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21770#discussion_r202543135 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -1585,6 +1585,9 @@ case class

[GitHub] spark pull request #21770: [SPARK-24806][SQL] Brush up generated code so tha...

2018-07-15 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21770#discussion_r202542918 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -318,7 +318,8 @@ case class SampleExec( v

[GitHub] spark pull request #21740: [SPARK-18230][MLLib]Throw a better exception, if ...

2018-07-15 Thread dmateusp
Github user dmateusp commented on a diff in the pull request: https://github.com/apache/spark/pull/21740#discussion_r202542318 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -165,7 +183,7 @@ class

[GitHub] spark issue #21762: [SPARK-24800][SQL] Refactor Avro Serializer and Deserial...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21762 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21762: [SPARK-24800][SQL] Refactor Avro Serializer and Deserial...

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21762 **[Test build #93026 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93026/testReport)** for PR 21762 at commit

[GitHub] spark issue #21762: [SPARK-24800][SQL] Refactor Avro Serializer and Deserial...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21762 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93026/ Test PASSed. ---

[GitHub] spark issue #21711: [SPARK-24681][SQL] Verify nested column names in Hive me...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21711 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93024/ Test PASSed. ---

[GitHub] spark issue #21711: [SPARK-24681][SQL] Verify nested column names in Hive me...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21711 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21711: [SPARK-24681][SQL] Verify nested column names in Hive me...

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21711 **[Test build #93024 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93024/testReport)** for PR 21711 at commit

[GitHub] spark issue #21762: [SPARK-24800][SQL] Refactor Avro Serializer and Deserial...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21762 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/970/

[GitHub] spark issue #21762: [SPARK-24800][SQL] Refactor Avro Serializer and Deserial...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21762 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21556: [SPARK-24549][SQL] Support Decimal type push down to the...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21556 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93017/ Test PASSed. ---

[GitHub] spark issue #21556: [SPARK-24549][SQL] Support Decimal type push down to the...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21556 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21556: [SPARK-24549][SQL] Support Decimal type push down to the...

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21556 **[Test build #93017 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93017/testReport)** for PR 21556 at commit

[GitHub] spark issue #21762: [SPARK-24800][SQL] Refactor Avro Serializer and Deserial...

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21762 **[Test build #93026 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93026/testReport)** for PR 21762 at commit

[GitHub] spark issue #21711: [SPARK-24681][SQL] Verify nested column names in Hive me...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21711 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21711: [SPARK-24681][SQL] Verify nested column names in Hive me...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21711 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93021/ Test PASSed. ---

[GitHub] spark issue #21711: [SPARK-24681][SQL] Verify nested column names in Hive me...

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21711 **[Test build #93021 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93021/testReport)** for PR 21711 at commit

[GitHub] spark issue #21711: [SPARK-24681][SQL] Verify nested column names in Hive me...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21711 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21711: [SPARK-24681][SQL] Verify nested column names in Hive me...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21711 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93020/ Test PASSed. ---

[GitHub] spark issue #21711: [SPARK-24681][SQL] Verify nested column names in Hive me...

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21711 **[Test build #93020 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93020/testReport)** for PR 21711 at commit

[GitHub] spark pull request #21769: [SPARK-24805][SQL] Do not ignore avro files witho...

2018-07-15 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/21769#discussion_r202541358 --- Diff: external/avro/src/test/scala/org/apache/spark/sql/avro/AvroSuite.scala --- @@ -680,12 +689,22 @@ class AvroSuite extends QueryTest with

[GitHub] spark issue #21772: [SPARK-24809] [SQL] Serializing LongHashedRelation in ex...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21772 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21772: [SPARK-24809] [SQL] Serializing LongHashedRelation in ex...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21772 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21772: [SPARK-24809] [SQL] Serializing LongHashedRelation in ex...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21772 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21711: [SPARK-24681][SQL] Verify nested column names in Hive me...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21711 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21711: [SPARK-24681][SQL] Verify nested column names in Hive me...

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21711 **[Test build #93025 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93025/testReport)** for PR 21711 at commit

[GitHub] spark issue #21711: [SPARK-24681][SQL] Verify nested column names in Hive me...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21711 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93025/ Test FAILed. ---

[GitHub] spark pull request #21772: [SPARK-24809] [SQL] Serializing LongHashedRelatio...

2018-07-15 Thread liutang123
GitHub user liutang123 opened a pull request: https://github.com/apache/spark/pull/21772 [SPARK-24809] [SQL] Serializing LongHashedRelation in executor may result in data error When join key is long or int in broadcast join, Spark will use LongHashedRelation as the broadcast

[GitHub] spark issue #21589: [SPARK-24591][CORE] Number of cores and executors in the...

2018-07-15 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/21589 > AFAIK, we always have num of executor ... Not in all cases, Databricks clients can create auto-scaling clusters:

[GitHub] spark issue #21711: [SPARK-24681][SQL] Verify nested column names in Hive me...

2018-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21711 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93022/ Test FAILed. ---

[GitHub] spark issue #21711: [SPARK-24681][SQL] Verify nested column names in Hive me...

2018-07-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21711 **[Test build #93022 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93022/testReport)** for PR 21711 at commit

<    1   2   3   4   5   >