[GitHub] [spark] viirya commented on a change in pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

viirya commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r445321841 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -113,6 +113,11 @@ object

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

AmplabJenkins removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649234400 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

AmplabJenkins commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649234400 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

SparkQA removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649153764 **[Test build #124507 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124507/testReport)** for PR 28898 at commit

[GitHub] [spark] SparkQA commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

SparkQA commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649233655 **[Test build #124507 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124507/testReport)** for PR 28898 at commit

[GitHub] [spark] maropu commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

maropu commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r445319707 ## File path: sql/core/benchmarks/ExternalAppendOnlyUnsafeRowArrayBenchmark-results.txt ## @@ -42,4 +42,8 @@ Spilling with 1 rows:

[GitHub] [spark] TJX2014 edited a comment on pull request #28926: [SPARK-31982][SQL][FOLLOWUP]Function sequence doesn't handle date increments that cross DST

TJX2014 edited a comment on pull request #28926: URL: https://github.com/apache/spark/pull/28926#issuecomment-649231790 Follow: https://github.com/apache/spark/pull/28856 cc @cloud-fan @MaxGekk This is an automated

[GitHub] [spark] TJX2014 edited a comment on pull request #28926: [SPARK-31982][SQL][FOLLOWUP]Function sequence doesn't handle date increments that cross DST

TJX2014 edited a comment on pull request #28926: URL: https://github.com/apache/spark/pull/28926#issuecomment-649231790 https://github.com/apache/spark/pull/28856 cc @cloud-fan @MaxGekk This is an automated message from

[GitHub] [spark] TJX2014 edited a comment on pull request #28926: [SPARK-31982][SQL][FOLLOWUP]Function sequence doesn't handle date increments that cross DST

TJX2014 edited a comment on pull request #28926: URL: https://github.com/apache/spark/pull/28926#issuecomment-649231790 https://github.com/apache/spark/pull/28856 This is an automated message from the Apache Git Service. To

[GitHub] [spark] TJX2014 commented on pull request #28926: [SPARK-31982][SQL][FOLLOWUP]Function sequence doesn't handle date increments that cross DST

TJX2014 commented on pull request #28926: URL: https://github.com/apache/spark/pull/28926#issuecomment-649231790 #https://github.com/apache/spark/pull/28856 This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28926: [SPARK-31982][SQL][FOLLOWUP]Function sequence doesn't handle date increments that cross DST

AmplabJenkins removed a comment on pull request #28926: URL: https://github.com/apache/spark/pull/28926#issuecomment-649230613 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins commented on pull request #28926: [SPARK-31982][SQL][FOLLOWUP]Function sequence doesn't handle date increments that cross DST

AmplabJenkins commented on pull request #28926: URL: https://github.com/apache/spark/pull/28926#issuecomment-649230871 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] TJX2014 commented on a change in pull request #28926: [SPARK-31982][SQL][FOLLOWUP]Function sequence doesn't handle date increments that cross DST

TJX2014 commented on a change in pull request #28926: URL: https://github.com/apache/spark/pull/28926#discussion_r445317708 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala ## @@ -2612,6 +2614,9 @@ object Sequence

[GitHub] [spark] TJX2014 opened a new pull request #28926: [SPARK-31982][SQL][FOLLOWUP]Function sequence doesn't handle date increments that cross DST

TJX2014 opened a new pull request #28926: URL: https://github.com/apache/spark/pull/28926 ### What changes were proposed in this pull request? Add a unit test. Bug fix in `org.apache.spark.sql.catalyst.expressions.Sequence.TemporalSequenceImpl` Add `sequence step must be a day

[GitHub] [spark] AmplabJenkins commented on pull request #28926: [SPARK-31982][SQL][FOLLOWUP]Function sequence doesn't handle date increments that cross DST

AmplabJenkins commented on pull request #28926: URL: https://github.com/apache/spark/pull/28926#issuecomment-649230613 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] gatorsmile commented on pull request #28902: [SPARK-31801][API][SHUFFLE][TESTS] Tests for registering map output metadata

gatorsmile commented on pull request #28902: URL: https://github.com/apache/spark/pull/28902#issuecomment-649226867 cc @Ngone51 @jiangxb1987 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] gatorsmile commented on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

gatorsmile commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-649226614 retest this please This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] frankyin-factual commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

frankyin-factual commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649224393 > btw, could you follow the PR template? https://github.com/apache/spark/blob/master/.github/PULL_REQUEST_TEMPLATE Just updated the PR.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

AmplabJenkins removed a comment on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-649221849 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

AmplabJenkins commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-649221849 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] frankyin-factual commented on a change in pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

frankyin-factual commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r445309526 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -113,6 +113,11 @@ object

[GitHub] [spark] SparkQA removed a comment on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

SparkQA removed a comment on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-649138531 **[Test build #124505 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124505/testReport)** for PR 28804 at commit

[GitHub] [spark] SparkQA commented on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

SparkQA commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-649221158 **[Test build #124505 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124505/testReport)** for PR 28804 at commit

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r445307223 ## File path: sql/core/benchmarks/ExternalAppendOnlyUnsafeRowArrayBenchmark-results.txt ## @@ -42,4 +42,8 @@ Spilling with 1 rows:

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

AmplabJenkins removed a comment on pull request #28761: URL: https://github.com/apache/spark/pull/28761#issuecomment-649217406 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r445306120 ## File path: sql/core/benchmarks/ExternalAppendOnlyUnsafeRowArrayBenchmark-results.txt ## @@ -42,4 +42,8 @@ Spilling with 1 rows:

[GitHub] [spark] AmplabJenkins commented on pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

AmplabJenkins commented on pull request #28761: URL: https://github.com/apache/spark/pull/28761#issuecomment-649217406 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

SparkQA removed a comment on pull request #28761: URL: https://github.com/apache/spark/pull/28761#issuecomment-649140417 **[Test build #124506 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124506/testReport)** for PR 28761 at commit

[GitHub] [spark] SparkQA commented on pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

SparkQA commented on pull request #28761: URL: https://github.com/apache/spark/pull/28761#issuecomment-649216688 **[Test build #124506 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124506/testReport)** for PR 28761 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28896: [SPARK-32025][SQL] Csv schema inference problems with different types in the same column

AmplabJenkins commented on pull request #28896: URL: https://github.com/apache/spark/pull/28896#issuecomment-649215908 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28896: [SPARK-32025][SQL] Csv schema inference problems with different types in the same column

AmplabJenkins removed a comment on pull request #28896: URL: https://github.com/apache/spark/pull/28896#issuecomment-649215908 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28896: [SPARK-32025][SQL] Csv schema inference problems with different types in the same column

HyukjinKwon commented on a change in pull request #28896: URL: https://github.com/apache/spark/pull/28896#discussion_r444833442 ## File path: sql/core/src/test/resources/test-data/mixed-types1.csv ## @@ -0,0 +1,4 @@ +col_mixed_types +2012 +1997 +True Review comment:

[GitHub] [spark] SparkQA commented on pull request #28896: [SPARK-32025][SQL] Csv schema inference problems with different types in the same column

SparkQA commented on pull request #28896: URL: https://github.com/apache/spark/pull/28896#issuecomment-649215558 **[Test build #124510 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124510/testReport)** for PR 28896 at commit

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r445304792 ## File path: sql/core/benchmarks/ExternalAppendOnlyUnsafeRowArrayBenchmark-results.txt ## @@ -42,4 +42,8 @@ Spilling with 1 rows:

[GitHub] [spark] HyukjinKwon removed a comment on pull request #28896: [SPARK-32025][SQL] Csv schema inference problems with different types in the same column

HyukjinKwon removed a comment on pull request #28896: URL: https://github.com/apache/spark/pull/28896#issuecomment-649214971 retest this please This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28896: [SPARK-32025][SQL] Csv schema inference problems with different types in the same column

AmplabJenkins removed a comment on pull request #28896: URL: https://github.com/apache/spark/pull/28896#issuecomment-647758404 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] HyukjinKwon commented on pull request #28896: [SPARK-32025][SQL] Csv schema inference problems with different types in the same column

HyukjinKwon commented on pull request #28896: URL: https://github.com/apache/spark/pull/28896#issuecomment-649215341 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] HyukjinKwon commented on pull request #28896: [SPARK-32025][SQL] Csv schema inference problems with different types in the same column

HyukjinKwon commented on pull request #28896: URL: https://github.com/apache/spark/pull/28896#issuecomment-649214971 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HyukjinKwon commented on pull request #28919: [SPARK-32038][SQL][FOLLOWUP] Make the alias name pretty after float/double normalization

HyukjinKwon commented on pull request #28919: URL: https://github.com/apache/spark/pull/28919#issuecomment-649213796 @cloud-fan, shall we add a short comment https://github.com/apache/spark/pull/28919#discussion_r445274348?

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r445299048 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArrayBenchmark.scala ## @@ -182,6 +182,47 @@ object

[GitHub] [spark] viirya commented on a change in pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

viirya commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r445294440 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -113,6 +113,11 @@ object

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r445291555 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -113,6 +113,11 @@ object

[GitHub] [spark] maropu commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

maropu commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649199639 btw, could you follow the PR template? https://github.com/apache/spark/blob/master/.github/PULL_REQUEST_TEMPLATE

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r445291555 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -113,6 +113,11 @@ object

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r445291555 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -113,6 +113,11 @@ object

[GitHub] [spark] Hellsen83 commented on pull request #23877: [SPARK-26449][PYTHON] Add transform method to DataFrame API

Hellsen83 commented on pull request #23877: URL: https://github.com/apache/spark/pull/23877#issuecomment-649198577 Hello @MrPowers , you are right, this is in fact motivated by your excellent blog post - thank you so much for that! From my experience - i.e. bringing this style of

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

AmplabJenkins removed a comment on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-649188105 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

AmplabJenkins removed a comment on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-649188097 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

SparkQA commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-649187983 **[Test build #124509 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124509/testReport)** for PR 28912 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

SparkQA removed a comment on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-649179138 **[Test build #124509 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124509/testReport)** for PR 28912 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

AmplabJenkins commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-649188097 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

AmplabJenkins removed a comment on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-649179488 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

AmplabJenkins commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-649179488 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

SparkQA commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-649179138 **[Test build #124509 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124509/testReport)** for PR 28912 at commit

[GitHub] [spark] HyukjinKwon commented on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

HyukjinKwon commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-649178851 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28864: [SPARK-32004][ALL] Drop references to slave

AmplabJenkins removed a comment on pull request #28864: URL: https://github.com/apache/spark/pull/28864#issuecomment-649177945 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28864: [SPARK-32004][ALL] Drop references to slave

AmplabJenkins commented on pull request #28864: URL: https://github.com/apache/spark/pull/28864#issuecomment-649177945 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28864: [SPARK-32004][ALL] Drop references to slave

SparkQA removed a comment on pull request #28864: URL: https://github.com/apache/spark/pull/28864#issuecomment-649120767 **[Test build #124501 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124501/testReport)** for PR 28864 at commit

[GitHub] [spark] SparkQA commented on pull request #28864: [SPARK-32004][ALL] Drop references to slave

SparkQA commented on pull request #28864: URL: https://github.com/apache/spark/pull/28864#issuecomment-649177297 **[Test build #124501 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124501/testReport)** for PR 28864 at commit

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28919: [SPARK-32038][SQL][FOLLOWUP] Make the alias name pretty after float/double normalization

HyukjinKwon commented on a change in pull request #28919: URL: https://github.com/apache/spark/pull/28919#discussion_r445274348 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala ## @@ -460,7 +460,12 @@ abstract class SparkStrategies

[GitHub] [spark] HyukjinKwon commented on a change in pull request #27331: [SPARK-29157][SQL][PYSPARK] Add DataFrameWriterV2 to Python API

HyukjinKwon commented on a change in pull request #27331: URL: https://github.com/apache/spark/pull/27331#discussion_r445270934 ## File path: python/pyspark/sql/readwriter.py ## @@ -1048,6 +1048,128 @@ def jdbc(self, url, table, mode=None, properties=None):

[GitHub] [spark] HyukjinKwon commented on a change in pull request #27331: [SPARK-29157][SQL][PYSPARK] Add DataFrameWriterV2 to Python API

HyukjinKwon commented on a change in pull request #27331: URL: https://github.com/apache/spark/pull/27331#discussion_r445268946 ## File path: python/pyspark/sql/readwriter.py ## @@ -1048,6 +1048,128 @@ def jdbc(self, url, table, mode=None, properties=None):

[GitHub] [spark] HyukjinKwon commented on a change in pull request #27331: [SPARK-29157][SQL][PYSPARK] Add DataFrameWriterV2 to Python API

HyukjinKwon commented on a change in pull request #27331: URL: https://github.com/apache/spark/pull/27331#discussion_r445269268 ## File path: python/pyspark/sql/functions.py ## @@ -3300,6 +3300,88 @@ def map_zip_with(col1, col2, f): return

[GitHub] [spark] HyukjinKwon commented on a change in pull request #27331: [SPARK-29157][SQL][PYSPARK] Add DataFrameWriterV2 to Python API

HyukjinKwon commented on a change in pull request #27331: URL: https://github.com/apache/spark/pull/27331#discussion_r445267780 ## File path: python/pyspark/sql/tests/test_readwriter.py ## @@ -163,6 +163,43 @@ def test_insert_into(self): self.assertEqual(6,

[GitHub] [spark] HyukjinKwon commented on pull request #27331: [SPARK-29157][SQL][PYSPARK] Add DataFrameWriterV2 to Python API

HyukjinKwon commented on pull request #27331: URL: https://github.com/apache/spark/pull/27331#issuecomment-649168859 > I don't think that maintenance is a huge issue here. Just saying... That's probably you're used to Python side .. For people who don't know Python, reading itself

[GitHub] [spark] HyukjinKwon commented on pull request #27849: [SPARK-31081][UI][SQL] Make the display of stageId/stageAttemptId/taskId of sql metrics configurable in UI

HyukjinKwon commented on pull request #27849: URL: https://github.com/apache/spark/pull/27849#issuecomment-649167755 Closing in favour of #27927 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] HyukjinKwon closed pull request #27849: [SPARK-31081][UI][SQL] Make the display of stageId/stageAttemptId/taskId of sql metrics configurable in UI

HyukjinKwon closed pull request #27849: URL: https://github.com/apache/spark/pull/27849 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

AmplabJenkins removed a comment on pull request #28852: URL: https://github.com/apache/spark/pull/28852#issuecomment-649163956 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

AmplabJenkins removed a comment on pull request #28852: URL: https://github.com/apache/spark/pull/28852#issuecomment-649163945 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

AmplabJenkins commented on pull request #28852: URL: https://github.com/apache/spark/pull/28852#issuecomment-649163945 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

SparkQA removed a comment on pull request #28852: URL: https://github.com/apache/spark/pull/28852#issuecomment-649125587 **[Test build #124504 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124504/testReport)** for PR 28852 at commit

[GitHub] [spark] SparkQA commented on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

SparkQA commented on pull request #28852: URL: https://github.com/apache/spark/pull/28852#issuecomment-649163590 **[Test build #124504 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124504/testReport)** for PR 28852 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

AmplabJenkins removed a comment on pull request #28852: URL: https://github.com/apache/spark/pull/28852#issuecomment-649162543 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

AmplabJenkins removed a comment on pull request #28852: URL: https://github.com/apache/spark/pull/28852#issuecomment-649162539 Build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] SparkQA removed a comment on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

SparkQA removed a comment on pull request #28852: URL: https://github.com/apache/spark/pull/28852#issuecomment-649123085 **[Test build #124502 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124502/testReport)** for PR 28852 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

AmplabJenkins commented on pull request #28852: URL: https://github.com/apache/spark/pull/28852#issuecomment-649162539 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

SparkQA commented on pull request #28852: URL: https://github.com/apache/spark/pull/28852#issuecomment-649162330 **[Test build #124502 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124502/testReport)** for PR 28852 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

AmplabJenkins removed a comment on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-649160040 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

AmplabJenkins commented on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-649160040 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

SparkQA commented on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-649159672 **[Test build #124508 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124508/testReport)** for PR 27690 at commit

[GitHub] [spark] maropu commented on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

maropu commented on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-649158798 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] maropu commented on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

maropu commented on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-649158887 @HyukjinKwon @viirya no more comment? This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] maropu commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

maropu commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r445255528 ## File path: sql/core/benchmarks/ExternalAppendOnlyUnsafeRowArrayBenchmark-results.txt ## @@ -42,4 +42,8 @@ Spilling with 1 rows:

[GitHub] [spark] maropu commented on pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

maropu commented on pull request #27246: URL: https://github.com/apache/spark/pull/27246#issuecomment-649158069 Looks okay to me. Anyone could check this? @cloud-fan @dongjoon-hyun @JoshRosen @jiangxb1987 This is an

[GitHub] [spark] maropu commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

maropu commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r445256544 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArrayBenchmark.scala ## @@ -182,6 +182,47 @@ object

[GitHub] [spark] maropu commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

maropu commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r445256446 ## File path: sql/core/benchmarks/ExternalAppendOnlyUnsafeRowArrayBenchmark-results.txt ## @@ -42,4 +42,8 @@ Spilling with 1 rows:

[GitHub] [spark] maropu commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

maropu commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r445255528 ## File path: sql/core/benchmarks/ExternalAppendOnlyUnsafeRowArrayBenchmark-results.txt ## @@ -42,4 +42,8 @@ Spilling with 1 rows:

[GitHub] [spark] maropu commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

maropu commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r445255528 ## File path: sql/core/benchmarks/ExternalAppendOnlyUnsafeRowArrayBenchmark-results.txt ## @@ -42,4 +42,8 @@ Spilling with 1 rows:

[GitHub] [spark] frankyin-factual commented on a change in pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

frankyin-factual commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r445255141 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -113,6 +113,11 @@ object

[GitHub] [spark] maropu commented on pull request #28923: [SPARK-32090][SQL] UserDefinedType.equal() should be symmetrical

maropu commented on pull request #28923: URL: https://github.com/apache/spark/pull/28923#issuecomment-649155603 LGTM except for the @cloud-fan comment. This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] maropu commented on a change in pull request #28923: [SPARK-32090][SQL] UserDefinedType.equal() should be symmetrical

maropu commented on a change in pull request #28923: URL: https://github.com/apache/spark/pull/28923#discussion_r445254621 ## File path: sql/core/src/test/scala/org/apache/spark/sql/UserDefinedTypeSuite.scala ## @@ -134,6 +134,17 @@ class UserDefinedTypeSuite extends

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

AmplabJenkins removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649153978 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

AmplabJenkins commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649153978 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] ulysses-you commented on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand

ulysses-you commented on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-649153781 Hive use it's own way to copy properties, all useful properties which need to copy are declared at serde class annotation so that Hive needn't copy redundant properties.

[GitHub] [spark] SparkQA commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

SparkQA commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649153764 **[Test build #124507 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124507/testReport)** for PR 28898 at commit

[GitHub] [spark] maropu commented on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

maropu commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-649152732 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

AmplabJenkins removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-647849466 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow schema pruning thru window functions

maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r445251813 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -113,6 +113,11 @@ object

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28905: [SPARK-32071][SQL] Benchmark `make_interval`

AmplabJenkins removed a comment on pull request #28905: URL: https://github.com/apache/spark/pull/28905#issuecomment-649147733 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28905: [SPARK-32071][SQL] Benchmark `make_interval`

AmplabJenkins commented on pull request #28905: URL: https://github.com/apache/spark/pull/28905#issuecomment-649147733 This is an automated message from the Apache Git Service. To respond to the message, please log on to

  1   2   3   4   5   6   7   >