[GitHub] [spark] MaxGekk commented on a change in pull request #27617: [SPARK-30865][SQL] Refactor DateTimeUtils

2020-06-18 Thread GitBox
MaxGekk commented on a change in pull request #27617: URL: https://github.com/apache/spark/pull/27617#discussion_r442417064 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala ## @@ -810,36 +793,36 @@ object DateTimeUtils { }

[GitHub] [spark] viirya edited a comment on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

2020-06-18 Thread GitBox
viirya edited a comment on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-646213264 It needs to wait for it to complete, this is how AQE does. As you said, we need to know the size of exchange.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28864: [SPARK-32004][ALL] Drop references to slave

2020-06-18 Thread GitBox
AmplabJenkins removed a comment on pull request #28864: URL: https://github.com/apache/spark/pull/28864#issuecomment-646222363 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] holdenk removed a comment on pull request #28864: [SPARK-32004][ALL] Drop references to slave

2020-06-18 Thread GitBox
holdenk removed a comment on pull request #28864: URL: https://github.com/apache/spark/pull/28864#issuecomment-646222315 cc @vish This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] holdenk commented on pull request #28864: [SPARK-32004][ALL] Drop references to slave

2020-06-18 Thread GitBox
holdenk commented on pull request #28864: URL: https://github.com/apache/spark/pull/28864#issuecomment-646222315 cc @vish This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [spark] AmplabJenkins commented on pull request #28864: [SPARK-32004][ALL] Drop references to slave

2020-06-18 Thread GitBox
AmplabJenkins commented on pull request #28864: URL: https://github.com/apache/spark/pull/28864#issuecomment-646222363 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] holdenk closed pull request #28864: [SPARK-32004][ALL] Drop references to slave

2020-06-18 Thread GitBox
holdenk closed pull request #28864: URL: https://github.com/apache/spark/pull/28864 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] holdenk opened a new pull request #28864: [SPARK-32004][ALL] Drop references to slave

2020-06-18 Thread GitBox
holdenk opened a new pull request #28864: URL: https://github.com/apache/spark/pull/28864 ### What changes were proposed in this pull request? This change replaces the world slave with alternatives matching the context. ### Why are the changes needed? There is no need to

[GitHub] [spark] MaxGekk commented on a change in pull request #27617: [SPARK-30865][SQL] Refactor DateTimeUtils

2020-06-18 Thread GitBox
MaxGekk commented on a change in pull request #27617: URL: https://github.com/apache/spark/pull/27617#discussion_r442408161 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala ## @@ -178,59 +171,56 @@ object DateTimeUtils { *

[GitHub] [spark] MaxGekk commented on a change in pull request #27617: [SPARK-30865][SQL] Refactor DateTimeUtils

2020-06-18 Thread GitBox
MaxGekk commented on a change in pull request #27617: URL: https://github.com/apache/spark/pull/27617#discussion_r442405985 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala ## @@ -178,59 +171,56 @@ object DateTimeUtils { *

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

2020-06-18 Thread GitBox
AmplabJenkins removed a comment on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-646215176 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

2020-06-18 Thread GitBox
AmplabJenkins commented on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-646215176 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

2020-06-18 Thread GitBox
SparkQA commented on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-646214625 **[Test build #124229 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124229/testReport)** for PR 28123 at commit [`38c7d6e`](https://github.com

[GitHub] [spark] imback82 commented on a change in pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

2020-06-18 Thread GitBox
imback82 commented on a change in pull request #28123: URL: https://github.com/apache/spark/pull/28123#discussion_r442399880 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -356,7 +360,8 @@ case class FileSourceScanExec(

[GitHub] [spark] imback82 commented on a change in pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

2020-06-18 Thread GitBox
imback82 commented on a change in pull request #28123: URL: https://github.com/apache/spark/pull/28123#discussion_r442400050 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -544,8 +549,19 @@ case class FileSourceScanExec(

[GitHub] [spark] viirya commented on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

2020-06-18 Thread GitBox
viirya commented on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-646213264 It needs to wait for it to complete, this is how AQE does. This is an automated message from the Apache Git Servi

[GitHub] [spark] holdenk commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

2020-06-18 Thread GitBox
holdenk commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-646209154 I will merge early next week unless anyone has any outstanding issues. This is an automated message from the Apa

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28835: [SPARK-31926][SQL][TESTS][FOLLOWUP] Fix concurrency issue for ThriftCLIService to getPortNumber

2020-06-18 Thread GitBox
AmplabJenkins removed a comment on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-646201131 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28835: [SPARK-31926][SQL][TESTS][FOLLOWUP] Fix concurrency issue for ThriftCLIService to getPortNumber

2020-06-18 Thread GitBox
AmplabJenkins commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-646201131 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28835: [SPARK-31926][SQL][TESTS][FOLLOWUP] Fix concurrency issue for ThriftCLIService to getPortNumber

2020-06-18 Thread GitBox
SparkQA commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-646199933 **[Test build #124228 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124228/testReport)** for PR 28835 at commit [`e409f9a`](https://github.com

[GitHub] [spark] yaooqinn commented on pull request #28835: [SPARK-31926][SQL][TESTS][FOLLOWUP] Fix concurrency issue for ThriftCLIService to getPortNumber

2020-06-18 Thread GitBox
yaooqinn commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-646198981 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] MaxGekk commented on a change in pull request #27617: [SPARK-30865][SQL] Refactor DateTimeUtils

2020-06-18 Thread GitBox
MaxGekk commented on a change in pull request #27617: URL: https://github.com/apache/spark/pull/27617#discussion_r442386606 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala ## @@ -42,49 +42,42 @@ import org.apache.spark.unsafe.ty

[GitHub] [spark] cloud-fan commented on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

2020-06-18 Thread GitBox
cloud-fan commented on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-646191835 > It creates and materializes Do you mean to trigger the materialization or wait for it to complete? Th

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

2020-06-18 Thread GitBox
AmplabJenkins removed a comment on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-646187111 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

2020-06-18 Thread GitBox
AmplabJenkins commented on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-646187111 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

2020-06-18 Thread GitBox
SparkQA commented on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-646185707 **[Test build #124227 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124227/testReport)** for PR 28710 at commit [`ead6da2`](https://github.com

[GitHub] [spark] huaxingao commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

2020-06-18 Thread GitBox
huaxingao commented on a change in pull request #28710: URL: https://github.com/apache/spark/pull/28710#discussion_r442378549 ## File path: python/pyspark/ml/classification.py ## @@ -990,241 +1218,26 @@ def featuresCol(self): """ return self._call_java("featur

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

2020-06-18 Thread GitBox
AmplabJenkins removed a comment on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-646174885 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

2020-06-18 Thread GitBox
AmplabJenkins commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-646174885 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] viirya edited a comment on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

2020-06-18 Thread GitBox
viirya edited a comment on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-646172235 I updated previous comment. "It creates and materializes one stage first..." You can see the query plan in previous comment, it optimizes the join to broadcast join.

[GitHub] [spark] SparkQA commented on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

2020-06-18 Thread GitBox
SparkQA commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-646173660 **[Test build #124226 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124226/testReport)** for PR 28848 at commit [`4b18369`](https://github.com

[GitHub] [spark] viirya commented on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

2020-06-18 Thread GitBox
viirya commented on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-646172235 I updated previous comment. "It creates and materializes one stage first..." This is an automated message from th

[GitHub] [spark] viirya edited a comment on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

2020-06-18 Thread GitBox
viirya edited a comment on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-646089689 This change does not create and materialize all query stages of the join in a batch. It creates and materializes one stage first and then re-optimize the join. So once it

[GitHub] [spark] viirya edited a comment on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

2020-06-18 Thread GitBox
viirya edited a comment on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-646089689 This change does not create and materialize all query stages of the join in a batch. It creates and materialize one stage first and then re-optimize the join. So once it m

[GitHub] [spark] wypoon commented on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

2020-06-18 Thread GitBox
wypoon commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-646170970 retest this please This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

2020-06-18 Thread GitBox
AmplabJenkins removed a comment on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-646169373 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124

[GitHub] [spark] SparkQA removed a comment on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

2020-06-18 Thread GitBox
SparkQA removed a comment on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-646166931 **[Test build #124225 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124225/testReport)** for PR 28848 at commit [`4b18369`](https://gi

[GitHub] [spark] wypoon commented on a change in pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

2020-06-18 Thread GitBox
wypoon commented on a change in pull request #28848: URL: https://github.com/apache/spark/pull/28848#discussion_r442369811 ## File path: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ## @@ -540,6 +540,43 @@ class DAGSchedulerSuite extends SparkFunSuite

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

2020-06-18 Thread GitBox
AmplabJenkins removed a comment on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-646169358 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins commented on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

2020-06-18 Thread GitBox
AmplabJenkins commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-646169358 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

2020-06-18 Thread GitBox
SparkQA commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-646169326 **[Test build #124225 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124225/testReport)** for PR 28848 at commit [`4b18369`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

2020-06-18 Thread GitBox
AmplabJenkins removed a comment on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-646167955 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

2020-06-18 Thread GitBox
AmplabJenkins commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-646167955 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

2020-06-18 Thread GitBox
SparkQA commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-646166931 **[Test build #124225 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124225/testReport)** for PR 28848 at commit [`4b18369`](https://github.com

[GitHub] [spark] MaxGekk commented on a change in pull request #28856: [SPARK-31982][SQL]Function sequence doesn't handle date increments that cross DST

2020-06-18 Thread GitBox
MaxGekk commented on a change in pull request #28856: URL: https://github.com/apache/spark/pull/28856#discussion_r442366706 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala ## @@ -2623,8 +2623,16 @@ object Sequence

[GitHub] [spark] imback82 commented on a change in pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

2020-06-18 Thread GitBox
imback82 commented on a change in pull request #28123: URL: https://github.com/apache/spark/pull/28123#discussion_r442365966 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/bucketing/CoalesceBucketsInSortMergeJoin.scala ## @@ -0,0 +1,132 @@ +/* + * License

[GitHub] [spark] sap1ens commented on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

2020-06-18 Thread GitBox
sap1ens commented on pull request #28852: URL: https://github.com/apache/spark/pull/28852#issuecomment-646160221 @maropu regarding testing, I don't see a dedicated test suite for the `FileStatusCache`. It's implicitly tested in `HiveSchemaInferenceSuite` though. Do you think I should

[GitHub] [spark] gatorsmile commented on pull request #28861: [MINOR] update dev/create-release/known_translations

2020-06-18 Thread GitBox
gatorsmile commented on pull request #28861: URL: https://github.com/apache/spark/pull/28861#issuecomment-646155636 After the release note is published, we still can update the contributor names. Please provide your full name. Thanks! --

[GitHub] [spark] imback82 commented on a change in pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

2020-06-18 Thread GitBox
imback82 commented on a change in pull request #28123: URL: https://github.com/apache/spark/pull/28123#discussion_r442360453 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/bucketing/CoalesceBucketsInSortMergeJoin.scala ## @@ -0,0 +1,132 @@ +/* + * License

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

2020-06-18 Thread GitBox
AmplabJenkins removed a comment on pull request #28852: URL: https://github.com/apache/spark/pull/28852#issuecomment-646153796 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

2020-06-18 Thread GitBox
AmplabJenkins commented on pull request #28852: URL: https://github.com/apache/spark/pull/28852#issuecomment-646153796 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

2020-06-18 Thread GitBox
SparkQA commented on pull request #28852: URL: https://github.com/apache/spark/pull/28852#issuecomment-646152933 **[Test build #124224 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124224/testReport)** for PR 28852 at commit [`f409366`](https://github.com

[GitHub] [spark] sap1ens commented on a change in pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

2020-06-18 Thread GitBox
sap1ens commented on a change in pull request #28852: URL: https://github.com/apache/spark/pull/28852#discussion_r442357968 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -835,6 +835,15 @@ object SQLConf { .longConf .c

[GitHub] [spark] cxzl25 commented on a change in pull request #28861: [MINOR] update dev/create-release/known_translations

2020-06-18 Thread GitBox
cxzl25 commented on a change in pull request #28861: URL: https://github.com/apache/spark/pull/28861#discussion_r442358417 ## File path: dev/create-release/known_translations ## @@ -261,3 +261,151 @@ yanlin-Lynn - Yanlin Wang yucai - Yucai Yu zhengruifeng - Ruifeng Zheng zuo

[GitHub] [spark] MaxGekk commented on a change in pull request #28856: [SPARK-31982][SQL]Function sequence doesn't handle date increments that cross DST

2020-06-18 Thread GitBox
MaxGekk commented on a change in pull request #28856: URL: https://github.com/apache/spark/pull/28856#discussion_r442357548 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollectionExpressionsSuite.scala ## @@ -1836,4 +1836,16 @@ class Coll

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28858: [WIP][SPARK-7101][SQL] support java.sql.Time

2020-06-18 Thread GitBox
AmplabJenkins removed a comment on pull request #28858: URL: https://github.com/apache/spark/pull/28858#issuecomment-646148381 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124

[GitHub] [spark] SparkQA removed a comment on pull request #28858: [WIP][SPARK-7101][SQL] support java.sql.Time

2020-06-18 Thread GitBox
SparkQA removed a comment on pull request #28858: URL: https://github.com/apache/spark/pull/28858#issuecomment-646147245 **[Test build #124223 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124223/testReport)** for PR 28858 at commit [`d172dd4`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28858: [WIP][SPARK-7101][SQL] support java.sql.Time

2020-06-18 Thread GitBox
AmplabJenkins removed a comment on pull request #28858: URL: https://github.com/apache/spark/pull/28858#issuecomment-646148139 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28858: [WIP][SPARK-7101][SQL] support java.sql.Time

2020-06-18 Thread GitBox
SparkQA commented on pull request #28858: URL: https://github.com/apache/spark/pull/28858#issuecomment-646148335 **[Test build #124223 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124223/testReport)** for PR 28858 at commit [`d172dd4`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #28858: [WIP][SPARK-7101][SQL] support java.sql.Time

2020-06-18 Thread GitBox
AmplabJenkins commented on pull request #28858: URL: https://github.com/apache/spark/pull/28858#issuecomment-646148139 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28858: [WIP][SPARK-7101][SQL] support java.sql.Time

2020-06-18 Thread GitBox
AmplabJenkins commented on pull request #28858: URL: https://github.com/apache/spark/pull/28858#issuecomment-646148364 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28858: [WIP][SPARK-7101][SQL] support java.sql.Time

2020-06-18 Thread GitBox
SparkQA commented on pull request #28858: URL: https://github.com/apache/spark/pull/28858#issuecomment-646147245 **[Test build #124223 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124223/testReport)** for PR 28858 at commit [`d172dd4`](https://github.com

[GitHub] [spark] roland1982 commented on a change in pull request #28861: [MINOR] update dev/create-release/known_translations

2020-06-18 Thread GitBox
roland1982 commented on a change in pull request #28861: URL: https://github.com/apache/spark/pull/28861#discussion_r442354078 ## File path: dev/create-release/known_translations ## @@ -261,3 +261,151 @@ yanlin-Lynn - Yanlin Wang yucai - Yucai Yu zhengruifeng - Ruifeng Zheng

[GitHub] [spark] roland-ondeviceresearch commented on a change in pull request #28861: [MINOR] update dev/create-release/known_translations

2020-06-18 Thread GitBox
roland-ondeviceresearch commented on a change in pull request #28861: URL: https://github.com/apache/spark/pull/28861#discussion_r442353088 ## File path: dev/create-release/known_translations ## @@ -261,3 +261,151 @@ yanlin-Lynn - Yanlin Wang yucai - Yucai Yu zhengruifeng - R

[GitHub] [spark] sap1ens commented on a change in pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

2020-06-18 Thread GitBox
sap1ens commented on a change in pull request #28852: URL: https://github.com/apache/spark/pull/28852#discussion_r442351399 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -835,6 +835,15 @@ object SQLConf { .longConf .c

[GitHub] [spark] cloud-fan commented on a change in pull request #28861: [MINOR] update dev/create-release/known_translations

2020-06-18 Thread GitBox
cloud-fan commented on a change in pull request #28861: URL: https://github.com/apache/spark/pull/28861#discussion_r442347665 ## File path: dev/create-release/known_translations ## @@ -261,3 +261,151 @@ yanlin-Lynn - Yanlin Wang yucai - Yucai Yu zhengruifeng - Ruifeng Zheng

[GitHub] [spark] cloud-fan commented on a change in pull request #28861: [MINOR] update dev/create-release/known_translations

2020-06-18 Thread GitBox
cloud-fan commented on a change in pull request #28861: URL: https://github.com/apache/spark/pull/28861#discussion_r442347168 ## File path: dev/create-release/known_translations ## @@ -261,3 +261,151 @@ yanlin-Lynn - Yanlin Wang yucai - Yucai Yu zhengruifeng - Ruifeng Zheng

[GitHub] [spark] cloud-fan commented on a change in pull request #28861: [MINOR] update dev/create-release/known_translations

2020-06-18 Thread GitBox
cloud-fan commented on a change in pull request #28861: URL: https://github.com/apache/spark/pull/28861#discussion_r442347042 ## File path: dev/create-release/known_translations ## @@ -261,3 +261,151 @@ yanlin-Lynn - Yanlin Wang yucai - Yucai Yu zhengruifeng - Ruifeng Zheng

[GitHub] [spark] cloud-fan commented on a change in pull request #28861: [MINOR] update dev/create-release/known_translations

2020-06-18 Thread GitBox
cloud-fan commented on a change in pull request #28861: URL: https://github.com/apache/spark/pull/28861#discussion_r442346780 ## File path: dev/create-release/known_translations ## @@ -261,3 +261,151 @@ yanlin-Lynn - Yanlin Wang yucai - Yucai Yu zhengruifeng - Ruifeng Zheng

[GitHub] [spark] cloud-fan commented on a change in pull request #28861: [MINOR] update dev/create-release/known_translations

2020-06-18 Thread GitBox
cloud-fan commented on a change in pull request #28861: URL: https://github.com/apache/spark/pull/28861#discussion_r442346192 ## File path: dev/create-release/known_translations ## @@ -261,3 +261,151 @@ yanlin-Lynn - Yanlin Wang yucai - Yucai Yu zhengruifeng - Ruifeng Zheng

[GitHub] [spark] cloud-fan commented on a change in pull request #28861: [MINOR] update dev/create-release/known_translations

2020-06-18 Thread GitBox
cloud-fan commented on a change in pull request #28861: URL: https://github.com/apache/spark/pull/28861#discussion_r442346505 ## File path: dev/create-release/known_translations ## @@ -261,3 +261,151 @@ yanlin-Lynn - Yanlin Wang yucai - Yucai Yu zhengruifeng - Ruifeng Zheng

[GitHub] [spark] cloud-fan commented on a change in pull request #28861: [MINOR] update dev/create-release/known_translations

2020-06-18 Thread GitBox
cloud-fan commented on a change in pull request #28861: URL: https://github.com/apache/spark/pull/28861#discussion_r442346339 ## File path: dev/create-release/known_translations ## @@ -261,3 +261,151 @@ yanlin-Lynn - Yanlin Wang yucai - Yucai Yu zhengruifeng - Ruifeng Zheng

[GitHub] [spark] gaborgsomogyi commented on a change in pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

2020-06-18 Thread GitBox
gaborgsomogyi commented on a change in pull request #28863: URL: https://github.com/apache/spark/pull/28863#discussion_r442327252 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/connection/OracleConnectionProvider.scala ## @@ -0,0 +1,60 @@

[GitHub] [spark] cloud-fan commented on a change in pull request #28861: [MINOR] update dev/create-release/known_translations

2020-06-18 Thread GitBox
cloud-fan commented on a change in pull request #28861: URL: https://github.com/apache/spark/pull/28861#discussion_r442346636 ## File path: dev/create-release/known_translations ## @@ -261,3 +261,151 @@ yanlin-Lynn - Yanlin Wang yucai - Yucai Yu zhengruifeng - Ruifeng Zheng

[GitHub] [spark] cloud-fan commented on a change in pull request #28861: [MINOR] update dev/create-release/known_translations

2020-06-18 Thread GitBox
cloud-fan commented on a change in pull request #28861: URL: https://github.com/apache/spark/pull/28861#discussion_r442345720 ## File path: dev/create-release/known_translations ## @@ -261,3 +261,151 @@ yanlin-Lynn - Yanlin Wang yucai - Yucai Yu zhengruifeng - Ruifeng Zheng

[GitHub] [spark] cloud-fan commented on a change in pull request #28861: [MINOR] update dev/create-release/known_translations

2020-06-18 Thread GitBox
cloud-fan commented on a change in pull request #28861: URL: https://github.com/apache/spark/pull/28861#discussion_r442345899 ## File path: dev/create-release/known_translations ## @@ -261,3 +261,151 @@ yanlin-Lynn - Yanlin Wang yucai - Yucai Yu zhengruifeng - Ruifeng Zheng

[GitHub] [spark] cloud-fan commented on a change in pull request #28861: [MINOR] update dev/create-release/known_translations

2020-06-18 Thread GitBox
cloud-fan commented on a change in pull request #28861: URL: https://github.com/apache/spark/pull/28861#discussion_r442345567 ## File path: dev/create-release/known_translations ## @@ -261,3 +261,151 @@ yanlin-Lynn - Yanlin Wang yucai - Yucai Yu zhengruifeng - Ruifeng Zheng

[GitHub] [spark] gaborgsomogyi commented on a change in pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

2020-06-18 Thread GitBox
gaborgsomogyi commented on a change in pull request #28863: URL: https://github.com/apache/spark/pull/28863#discussion_r442333450 ## File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleIntegrationSuite.scala ## @@ -32,35 +32,29 @@ import

[GitHub] [spark] younggyuchun commented on pull request #28858: [WIP][SPARK-7101][SQL] support java.sql.Time

2020-06-18 Thread GitBox
younggyuchun commented on pull request #28858: URL: https://github.com/apache/spark/pull/28858#issuecomment-646128615 @dongjoon-hyun I don't expect that someone will see WIP merge request :). Yeah, of course, I will fix the merge conflicts and issues. Thank you ---

[GitHub] [spark] cloud-fan commented on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

2020-06-18 Thread GitBox
cloud-fan commented on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-646127629 > It creates one stage first and then re-optimize the join. This is the confusing part. Creating a stage is not enough, we must wait for it to complete, then we cam know

[GitHub] [spark] cloud-fan edited a comment on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

2020-06-18 Thread GitBox
cloud-fan edited a comment on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-646127629 > It creates one stage first and then re-optimize the join. This is the confusing part. Creating a stage is not enough, we must wait for it to complete, then we c

[GitHub] [spark] gaborgsomogyi commented on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

2020-06-18 Thread GitBox
gaborgsomogyi commented on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-646127287 retest it please This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [spark] gatorsmile commented on a change in pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

2020-06-18 Thread GitBox
gatorsmile commented on a change in pull request #28852: URL: https://github.com/apache/spark/pull/28852#discussion_r442339018 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -835,6 +835,15 @@ object SQLConf { .longConf

[GitHub] [spark] gaborgsomogyi commented on a change in pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

2020-06-18 Thread GitBox
gaborgsomogyi commented on a change in pull request #28863: URL: https://github.com/apache/spark/pull/28863#discussion_r442333450 ## File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleIntegrationSuite.scala ## @@ -32,35 +32,29 @@ import

[GitHub] [spark] viirya commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

2020-06-18 Thread GitBox
viirya commented on a change in pull request #28710: URL: https://github.com/apache/spark/pull/28710#discussion_r442330970 ## File path: project/MimaExcludes.scala ## @@ -39,18 +39,43 @@ object MimaExcludes { // [SPARK-31077] Remove ChiSqSelector dependency on mllib.ChiSq

[GitHub] [spark] dongjoon-hyun commented on pull request #28858: [WIP][SPARK-7101][SQL] support java.sql.Time

2020-06-18 Thread GitBox
dongjoon-hyun commented on pull request #28858: URL: https://github.com/apache/spark/pull/28858#issuecomment-646119450 Hi, @younggyuchun . Could you resolve conflicts? This is an automated message from the Apache Git Service.

[GitHub] [spark] viirya commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

2020-06-18 Thread GitBox
viirya commented on a change in pull request #28710: URL: https://github.com/apache/spark/pull/28710#discussion_r442330970 ## File path: project/MimaExcludes.scala ## @@ -39,18 +39,43 @@ object MimaExcludes { // [SPARK-31077] Remove ChiSqSelector dependency on mllib.ChiSq

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28858: [WIP][SPARK-7101][SQL] support java.sql.Time

2020-06-18 Thread GitBox
dongjoon-hyun commented on a change in pull request #28858: URL: https://github.com/apache/spark/pull/28858#discussion_r442330815 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/RowEncoderSuite.scala ## @@ -83,20 +83,22 @@ class RowEncoderSuite

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28858: [WIP][SPARK-7101][SQL] support java.sql.Time

2020-06-18 Thread GitBox
AmplabJenkins removed a comment on pull request #28858: URL: https://github.com/apache/spark/pull/28858#issuecomment-646116306 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28858: [WIP][SPARK-7101][SQL] support java.sql.Time

2020-06-18 Thread GitBox
dongjoon-hyun commented on a change in pull request #28858: URL: https://github.com/apache/spark/pull/28858#discussion_r442330503 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala ## @@ -191,6 +191,22 @@ object RandomDataGenerator {

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28858: [WIP][SPARK-7101][SQL] support java.sql.Time

2020-06-18 Thread GitBox
AmplabJenkins removed a comment on pull request #28858: URL: https://github.com/apache/spark/pull/28858#issuecomment-646116294 Build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] AmplabJenkins commented on pull request #28858: [WIP][SPARK-7101][SQL] support java.sql.Time

2020-06-18 Thread GitBox
AmplabJenkins commented on pull request #28858: URL: https://github.com/apache/spark/pull/28858#issuecomment-646116294 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28858: [WIP][SPARK-7101][SQL] support java.sql.Time

2020-06-18 Thread GitBox
SparkQA removed a comment on pull request #28858: URL: https://github.com/apache/spark/pull/28858#issuecomment-646018900 **[Test build #124216 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124216/testReport)** for PR 28858 at commit [`be53ff6`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

2020-06-18 Thread GitBox
AmplabJenkins removed a comment on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-646115416 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28858: [WIP][SPARK-7101][SQL] support java.sql.Time

2020-06-18 Thread GitBox
dongjoon-hyun commented on a change in pull request #28858: URL: https://github.com/apache/spark/pull/28858#discussion_r442329380 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/TimeType.scala ## @@ -0,0 +1,63 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28858: [WIP][SPARK-7101][SQL] support java.sql.Time

2020-06-18 Thread GitBox
dongjoon-hyun commented on a change in pull request #28858: URL: https://github.com/apache/spark/pull/28858#discussion_r442328745 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/TimeType.scala ## @@ -0,0 +1,63 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [spark] viirya commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

2020-06-18 Thread GitBox
viirya commented on a change in pull request #28710: URL: https://github.com/apache/spark/pull/28710#discussion_r442328666 ## File path: python/pyspark/ml/classification.py ## @@ -990,241 +1218,26 @@ def featuresCol(self): """ return self._call_java("featuresC

[GitHub] [spark] SparkQA removed a comment on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

2020-06-18 Thread GitBox
SparkQA removed a comment on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-646113845 **[Test build #124222 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124222/testReport)** for PR 28863 at commit [`5cc3d0a`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28858: [WIP][SPARK-7101][SQL] support java.sql.Time

2020-06-18 Thread GitBox
SparkQA commented on pull request #28858: URL: https://github.com/apache/spark/pull/28858#issuecomment-646115593 **[Test build #124216 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124216/testReport)** for PR 28858 at commit [`be53ff6`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

2020-06-18 Thread GitBox
AmplabJenkins commented on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-646115406 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

<    1   2   3   4   5   6   7   8   >