[GitHub] [spark] iRakson commented on pull request #28512: [SPARK-31688][WEBUI] Refactor Pagination framework

iRakson commented on pull request #28512: URL: https://github.com/apache/spark/pull/28512#issuecomment-632258679 Thank You.  @srowen @sarutak This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] srowen closed pull request #28512: [SPARK-31688][WEBUI] Refactor Pagination framework

srowen closed pull request #28512: URL: https://github.com/apache/spark/pull/28512 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] srowen commented on pull request #28512: [SPARK-31688][WEBUI] Refactor Pagination framework

srowen commented on pull request #28512: URL: https://github.com/apache/spark/pull/28512#issuecomment-632255678 Merged to master This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] huaxingao commented on pull request #28553: [SPARK-31734][ML][PySpark] Add weight support in ClusteringEvaluator

huaxingao commented on pull request #28553: URL: https://github.com/apache/spark/pull/28553#issuecomment-632252469 cc @srowen This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] huaxingao commented on pull request #28595: [SPARK-31781][ML][PySpark] Move param k (number of clusters) to shared params

huaxingao commented on pull request #28595: URL: https://github.com/apache/spark/pull/28595#issuecomment-632252188 cc @srowen This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] huaxingao commented on pull request #28595: [SPARK-31781][ML][PySpark] Move param k (number of clusters) to shared params

huaxingao commented on pull request #28595: URL: https://github.com/apache/spark/pull/28595#issuecomment-632251801 Actually the param in ALS is called ```rank``` instead of ```k```. I am a little hesitant to reuse ```k``` in RankingEvaluator because it has a totally different meaning.

[GitHub] [spark] huaxingao commented on pull request #28590: [SPARK-31768][ML] add getMetrics in Evaluators

huaxingao commented on pull request #28590: URL: https://github.com/apache/spark/pull/28590#issuecomment-632246256 > Is there any way to recompute another metric in evaluator without involving mllib.metrics? It might be better to create a ml wrapper for mllib.metrics instead of

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28576: [SPARK-31755][SQL] allow missing year/hour when parsing date/timestamp string

AmplabJenkins removed a comment on pull request #28576: URL: https://github.com/apache/spark/pull/28576#issuecomment-632244984 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28576: [SPARK-31755][SQL] allow missing year/hour when parsing date/timestamp string

AmplabJenkins commented on pull request #28576: URL: https://github.com/apache/spark/pull/28576#issuecomment-632244984 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

AmplabJenkins removed a comment on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632244007 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27366: [SPARK-30648][SQL] Support filters pushdown in JSON datasource

AmplabJenkins removed a comment on pull request #27366: URL: https://github.com/apache/spark/pull/27366#issuecomment-632244056 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27617: [SPARK-30865][SQL] Refactor DateTimeUtils

AmplabJenkins removed a comment on pull request #27617: URL: https://github.com/apache/spark/pull/27617#issuecomment-632244083 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #28576: [SPARK-31755][SQL] allow missing year/hour when parsing date/timestamp string

SparkQA removed a comment on pull request #28576: URL: https://github.com/apache/spark/pull/28576#issuecomment-632068610 **[Test build #122932 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122932/testReport)** for PR 28576 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #27366: [SPARK-30648][SQL] Support filters pushdown in JSON datasource

AmplabJenkins commented on pull request #27366: URL: https://github.com/apache/spark/pull/27366#issuecomment-632244056 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

AmplabJenkins commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632244007 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28576: [SPARK-31755][SQL] allow missing year/hour when parsing date/timestamp string

SparkQA commented on pull request #28576: URL: https://github.com/apache/spark/pull/28576#issuecomment-632243655 **[Test build #122932 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122932/testReport)** for PR 28576 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #27617: [SPARK-30865][SQL] Refactor DateTimeUtils

AmplabJenkins commented on pull request #27617: URL: https://github.com/apache/spark/pull/27617#issuecomment-632244075 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

SparkQA commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632243456 **[Test build #122938 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122938/testReport)** for PR 28592 at commit

[GitHub] [spark] SparkQA commented on pull request #27617: [SPARK-30865][SQL] Refactor DateTimeUtils

SparkQA commented on pull request #27617: URL: https://github.com/apache/spark/pull/27617#issuecomment-632243446 **[Test build #122939 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122939/testReport)** for PR 27617 at commit

[GitHub] [spark] SparkQA commented on pull request #27366: [SPARK-30648][SQL] Support filters pushdown in JSON datasource

SparkQA commented on pull request #27366: URL: https://github.com/apache/spark/pull/27366#issuecomment-632243504 **[Test build #122940 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122940/testReport)** for PR 27366 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

AmplabJenkins removed a comment on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632239959 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

AmplabJenkins commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632239959 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

SparkQA commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632239244 **[Test build #122937 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122937/testReport)** for PR 28592 at commit

[GitHub] [spark] cloud-fan commented on pull request #28523: [SPARK-31706][SQL] add back the support of streaming update mode

cloud-fan commented on pull request #28523: URL: https://github.com/apache/spark/pull/28523#issuecomment-632230688 This PR (including code diff and PR description) is verbatim as it was when created. I thought you misread this PR, because your comment right after the veto comment actually

[GitHub] [spark] SparkQA commented on pull request #28598: [SPARK-31785][SQL][TESTS] Add a helper function to test all parquet readers

SparkQA commented on pull request #28598: URL: https://github.com/apache/spark/pull/28598#issuecomment-63711 **[Test build #122936 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122936/testReport)** for PR 28598 at commit

[GitHub] [spark] MaxGekk commented on a change in pull request #28598: [SPARK-31785][SQL][TESTS] Add a helper function to test all parquet readers

MaxGekk commented on a change in pull request #28598: URL: https://github.com/apache/spark/pull/28598#discussion_r428779913 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetTest.scala ## @@ -162,4 +162,36 @@ private[sql] trait

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28598: [SPARK-31785][SQL][TESTS] Add a helper function to test all parquet readers

AmplabJenkins removed a comment on pull request #28598: URL: https://github.com/apache/spark/pull/28598#issuecomment-632219275 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28598: [SPARK-31785][SQL][TESTS] Add a helper function to test all parquet readers

AmplabJenkins commented on pull request #28598: URL: https://github.com/apache/spark/pull/28598#issuecomment-632219275 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28598: [SPARK-31785][SQL][TESTS] Add a helper function to test all parquet readers

AmplabJenkins removed a comment on pull request #28598: URL: https://github.com/apache/spark/pull/28598#issuecomment-632188410 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28598: [SPARK-31785][SQL][TESTS] Add a helper function to test all parquet readers

AmplabJenkins commented on pull request #28598: URL: https://github.com/apache/spark/pull/28598#issuecomment-632188410 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28598: [SPARK-31785][SQL][TESTS] Add a helper function to test all parquet readers

SparkQA removed a comment on pull request #28598: URL: https://github.com/apache/spark/pull/28598#issuecomment-632045706 **[Test build #122930 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122930/testReport)** for PR 28598 at commit

[GitHub] [spark] SparkQA commented on pull request #28598: [SPARK-31785][SQL][TESTS] Add a helper function to test all parquet readers

SparkQA commented on pull request #28598: URL: https://github.com/apache/spark/pull/28598#issuecomment-632187258 **[Test build #122930 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122930/testReport)** for PR 28598 at commit

[GitHub] [spark] rdblue commented on pull request #28523: [SPARK-31706][SQL] add back the support of streaming update mode

rdblue commented on pull request #28523: URL: https://github.com/apache/spark/pull/28523#issuecomment-632185172 This PR wasn't ready to review until late Monday where I am, and was merged less than a day later. I had also assumed that the fix would be in a separate PR, which is why I

[GitHub] [spark] cloud-fan commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

cloud-fan commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r428759386 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeFormatterHelper.scala ## @@ -86,6 +86,25 @@ trait

[GitHub] [spark] cloud-fan commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

cloud-fan commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r428758919 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateFormatter.scala ## @@ -40,7 +41,11 @@ class Iso8601DateFormatter(

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28128: [SPARK-31354] SparkContext only register one SparkSession ApplicationEnd listener

AmplabJenkins removed a comment on pull request #28128: URL: https://github.com/apache/spark/pull/28128#issuecomment-632180091 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28128: [SPARK-31354] SparkContext only register one SparkSession ApplicationEnd listener

AmplabJenkins commented on pull request #28128: URL: https://github.com/apache/spark/pull/28128#issuecomment-632180091 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] cloud-fan commented on pull request #28128: [SPARK-31354] SparkContext only register one SparkSession ApplicationEnd listener

cloud-fan commented on pull request #28128: URL: https://github.com/apache/spark/pull/28128#issuecomment-632179830 thanks, merging to master/3.0! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] cloud-fan closed pull request #28128: [SPARK-31354] SparkContext only register one SparkSession ApplicationEnd listener

cloud-fan closed pull request #28128: URL: https://github.com/apache/spark/pull/28128 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] SparkQA removed a comment on pull request #28128: [SPARK-31354] SparkContext only register one SparkSession ApplicationEnd listener

SparkQA removed a comment on pull request #28128: URL: https://github.com/apache/spark/pull/28128#issuecomment-632038999 **[Test build #122929 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122929/testReport)** for PR 28128 at commit

[GitHub] [spark] SparkQA commented on pull request #28128: [SPARK-31354] SparkContext only register one SparkSession ApplicationEnd listener

SparkQA commented on pull request #28128: URL: https://github.com/apache/spark/pull/28128#issuecomment-632178495 **[Test build #122929 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122929/testReport)** for PR 28128 at commit

[GitHub] [spark] cloud-fan commented on pull request #27627: [WIP][SPARK-28067][SQL] Fix incorrect results for decimal aggregate sum by returning null on decimal overflow

cloud-fan commented on pull request #27627: URL: https://github.com/apache/spark/pull/27627#issuecomment-632166970 Hi @skambha , I've sent you a PR to only check overflow in `evaluateExpression`, and pass the tests. Can you take a look? https://github.com/skambha/spark/pull/1

[GitHub] [spark] Ngone51 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

Ngone51 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428729898 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if

[GitHub] [spark] akshatb1 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

akshatb1 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428727134 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

AmplabJenkins removed a comment on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632147078 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

AmplabJenkins commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632147078 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

SparkQA commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632146298 **[Test build #122935 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122935/testReport)** for PR 28592 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28525: [SPARK-27562][Shuffle] Complete the verification mechanism for shuffle transmitted data

AmplabJenkins removed a comment on pull request #28525: URL: https://github.com/apache/spark/pull/28525#issuecomment-632142075 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28525: [SPARK-27562][Shuffle] Complete the verification mechanism for shuffle transmitted data

AmplabJenkins commented on pull request #28525: URL: https://github.com/apache/spark/pull/28525#issuecomment-632142075 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28525: [SPARK-27562][Shuffle] Complete the verification mechanism for shuffle transmitted data

SparkQA removed a comment on pull request #28525: URL: https://github.com/apache/spark/pull/28525#issuecomment-632058436 **[Test build #122931 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122931/testReport)** for PR 28525 at commit

[GitHub] [spark] SparkQA commented on pull request #28525: [SPARK-27562][Shuffle] Complete the verification mechanism for shuffle transmitted data

SparkQA commented on pull request #28525: URL: https://github.com/apache/spark/pull/28525#issuecomment-632140854 **[Test build #122931 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122931/testReport)** for PR 28525 at commit

[GitHub] [spark] Ngone51 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

Ngone51 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428704776 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if

[GitHub] [spark] akshatb1 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

akshatb1 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428699825 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if

[GitHub] [spark] akshatb1 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

akshatb1 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428699825 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if

[GitHub] [spark] Ngone51 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

Ngone51 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428693738 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if

[GitHub] [spark] Ngone51 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

Ngone51 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428693738 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if

[GitHub] [spark] akshatb1 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

akshatb1 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428691523 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if

[GitHub] [spark] SparkQA commented on pull request #25965: [SPARK-26425][SS] Add more constraint checks in file streaming source to avoid checkpoint corruption

SparkQA commented on pull request #25965: URL: https://github.com/apache/spark/pull/25965#issuecomment-632108254 **[Test build #122934 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122934/testReport)** for PR 25965 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27664: [SPARK-30915][SS] CompactibleFileStreamLog: Avoid reading the metadata log file when finding the latest batch ID

AmplabJenkins removed a comment on pull request #27664: URL: https://github.com/apache/spark/pull/27664#issuecomment-632104886 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #27664: [SPARK-30915][SS] CompactibleFileStreamLog: Avoid reading the metadata log file when finding the latest batch ID

AmplabJenkins commented on pull request #27664: URL: https://github.com/apache/spark/pull/27664#issuecomment-632104886 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #27664: [SPARK-30915][SS] CompactibleFileStreamLog: Avoid reading the metadata log file when finding the latest batch ID

SparkQA commented on pull request #27664: URL: https://github.com/apache/spark/pull/27664#issuecomment-632104207 **[Test build #122933 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122933/testReport)** for PR 27664 at commit

[GitHub] [spark] HeartSaVioR commented on a change in pull request #27664: [SPARK-30915][SS] CompactibleFileStreamLog: Avoid reading the metadata log file when finding the latest batch ID

HeartSaVioR commented on a change in pull request #27664: URL: https://github.com/apache/spark/pull/27664#discussion_r428668027 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala ## @@ -182,19 +182,34 @@ class

[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r428667113 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateFormatter.scala ## @@ -40,7 +43,23 @@ class Iso8601DateFormatter(

[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r428667113 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateFormatter.scala ## @@ -40,7 +43,23 @@ class Iso8601DateFormatter(

[GitHub] [spark] maropu commented on a change in pull request #28560: [SPARK-27217][SQL] Nested column aliasing for more operators which can prune nested column

maropu commented on a change in pull request #28560: URL: https://github.com/apache/spark/pull/28560#discussion_r428666366 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -341,6 +341,78 @@ class

[GitHub] [spark] maropu commented on a change in pull request #28560: [SPARK-27217][SQL] Nested column aliasing for more operators which can prune nested column

maropu commented on a change in pull request #28560: URL: https://github.com/apache/spark/pull/28560#discussion_r428662926 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -48,7 +53,10 @@ object

[GitHub] [spark] xuanyuanking commented on a change in pull request #27664: [SPARK-30915][SS] CompactibleFileStreamLog: Avoid reading the metadata log file when finding the latest batch ID

xuanyuanking commented on a change in pull request #27664: URL: https://github.com/apache/spark/pull/27664#discussion_r428661867 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala ## @@ -182,19 +182,34 @@ class

[GitHub] [spark] Ngone51 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

Ngone51 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428661567 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if

[GitHub] [spark] akshatb1 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

akshatb1 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428657865 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if

[GitHub] [spark] cloud-fan commented on pull request #27066: [SPARK-31317][SQL] Add withFields method to Column

cloud-fan commented on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-632087480 I went through the discussion in the JIRA, there is one interesting use case ``` data.withColumn("newArray", transform('array, structElem =>

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28556: [SPARK-31736][SQL] Nested column aliasing for RepartitionByExpression/Join

AmplabJenkins removed a comment on pull request #28556: URL: https://github.com/apache/spark/pull/28556#issuecomment-632082073 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] GuoPhilipse commented on a change in pull request #28593: [SPARK-31710][SQL] Add two compatibility flag to cast long to timestamp

GuoPhilipse commented on a change in pull request #28593: URL: https://github.com/apache/spark/pull/28593#discussion_r428646486 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -2586,6 +2586,22 @@ object SQLConf {

[GitHub] [spark] AmplabJenkins commented on pull request #28556: [SPARK-31736][SQL] Nested column aliasing for RepartitionByExpression/Join

AmplabJenkins commented on pull request #28556: URL: https://github.com/apache/spark/pull/28556#issuecomment-632082073 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] Ngone51 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

Ngone51 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428645814 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if

[GitHub] [spark] SparkQA removed a comment on pull request #28556: [SPARK-31736][SQL] Nested column aliasing for RepartitionByExpression/Join

SparkQA removed a comment on pull request #28556: URL: https://github.com/apache/spark/pull/28556#issuecomment-631965191 **[Test build #122922 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122922/testReport)** for PR 28556 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28583: [SPARK-31764][CORE] JsonProtocol doesn't write RDDInfo#isBarrier

AmplabJenkins removed a comment on pull request #28583: URL: https://github.com/apache/spark/pull/28583#issuecomment-632081247 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28556: [SPARK-31736][SQL] Nested column aliasing for RepartitionByExpression/Join

SparkQA commented on pull request #28556: URL: https://github.com/apache/spark/pull/28556#issuecomment-632081113 **[Test build #122922 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122922/testReport)** for PR 28556 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28583: [SPARK-31764][CORE] JsonProtocol doesn't write RDDInfo#isBarrier

AmplabJenkins commented on pull request #28583: URL: https://github.com/apache/spark/pull/28583#issuecomment-632081247 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28583: [SPARK-31764][CORE] JsonProtocol doesn't write RDDInfo#isBarrier

SparkQA removed a comment on pull request #28583: URL: https://github.com/apache/spark/pull/28583#issuecomment-632016717 **[Test build #122927 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122927/testReport)** for PR 28583 at commit

[GitHub] [spark] SparkQA commented on pull request #28583: [SPARK-31764][CORE] JsonProtocol doesn't write RDDInfo#isBarrier

SparkQA commented on pull request #28583: URL: https://github.com/apache/spark/pull/28583#issuecomment-632080202 **[Test build #122927 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122927/testReport)** for PR 28583 at commit

[GitHub] [spark] Ngone51 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

Ngone51 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428643339 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -123,16 +134,24 @@ private class ClientEndpoint(

[GitHub] [spark] Ngone51 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

Ngone51 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428643084 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -123,16 +134,24 @@ private class ClientEndpoint(

[GitHub] [spark] maropu commented on pull request #28556: [SPARK-31736][SQL] Nested column aliasing for RepartitionByExpression/Join

maropu commented on pull request #28556: URL: https://github.com/apache/spark/pull/28556#issuecomment-632075943 I left some minor comments though, it looks okay. cc: @dongjoon-hyun @dbtsai This is an automated message from

[GitHub] [spark] maropu commented on a change in pull request #28556: [SPARK-31736][SQL] Nested column aliasing for RepartitionByExpression/Join

maropu commented on a change in pull request #28556: URL: https://github.com/apache/spark/pull/28556#discussion_r428635965 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -144,7 +144,6 @@ class

[GitHub] [spark] cloud-fan commented on pull request #27066: [SPARK-31317][SQL] Add withFields method to Column

cloud-fan commented on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-632075377 I haven't looked at the patch yet, but looking at the proposed API ``` val result = data.withColumn("a", 'a.withFields( ("b", $"a.b".withFields( ("a",

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28576: [SPARK-31755][SQL] allow missing year/hour when parsing date/timestamp string

AmplabJenkins removed a comment on pull request #28576: URL: https://github.com/apache/spark/pull/28576#issuecomment-632069072 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28576: [SPARK-31755][SQL] allow missing year/hour when parsing date/timestamp string

AmplabJenkins commented on pull request #28576: URL: https://github.com/apache/spark/pull/28576#issuecomment-632069072 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28576: [SPARK-31755][SQL] allow missing year/hour when parsing date/timestamp string

SparkQA commented on pull request #28576: URL: https://github.com/apache/spark/pull/28576#issuecomment-632068610 **[Test build #122932 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122932/testReport)** for PR 28576 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

AmplabJenkins removed a comment on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632065812 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

SparkQA removed a comment on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632032488 **[Test build #122928 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122928/testReport)** for PR 28592 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

AmplabJenkins removed a comment on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632065807 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

SparkQA commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632065738 **[Test build #122928 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122928/testReport)** for PR 28592 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

AmplabJenkins commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632065807 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

AmplabJenkins removed a comment on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632059081 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28525: [SPARK-27562][Shuffle] Complete the verification mechanism for shuffle transmitted data

AmplabJenkins removed a comment on pull request #28525: URL: https://github.com/apache/spark/pull/28525#issuecomment-632059000 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

AmplabJenkins removed a comment on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632059071 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

SparkQA removed a comment on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-631955288 **[Test build #122921 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122921/testReport)** for PR 28534 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

AmplabJenkins commented on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632059071 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28525: [SPARK-27562][Shuffle] Complete the verification mechanism for shuffle transmitted data

AmplabJenkins commented on pull request #28525: URL: https://github.com/apache/spark/pull/28525#issuecomment-632059000 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

SparkQA commented on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632058682 **[Test build #122921 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122921/testReport)** for PR 28534 at commit

<    1   2   3   4   5   6   >