[GitHub] [spark] bmarcott commented on issue #28188: [SPARK-18886][CORE][FOLLOWUP] allow follow up locality resets even if no task was launched

bmarcott commented on issue #28188: URL: https://github.com/apache/spark/pull/28188#issuecomment-617587316 thanks. I agree my terminology was confusing. I reversed the terms increase/decrease and provided a graph showing what I am referring to. ---

[GitHub] [spark] AmplabJenkins commented on issue #28279: [SPARK-31497][ML][PYSPARK] Fix Pyspark CrossValidator/TrainValidationSplit with pipeline estimator cannot save and load model

AmplabJenkins commented on issue #28279: URL: https://github.com/apache/spark/pull/28279#issuecomment-617584116 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] AmplabJenkins removed a comment on issue #28279: [SPARK-31497][ML][PYSPARK] Fix Pyspark CrossValidator/TrainValidationSplit with pipeline estimator cannot save and load model

AmplabJenkins removed a comment on issue #28279: URL: https://github.com/apache/spark/pull/28279#issuecomment-617584116 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] SparkQA removed a comment on issue #28279: [SPARK-31497][ML][PYSPARK] Fix Pyspark CrossValidator/TrainValidationSplit with pipeline estimator cannot save and load model

SparkQA removed a comment on issue #28279: URL: https://github.com/apache/spark/pull/28279#issuecomment-617576279 **[Test build #121609 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121609/testReport)** for PR 28279 at commit [`9b31c3a`](https://github.co

[GitHub] [spark] SparkQA commented on issue #28279: [SPARK-31497][ML][PYSPARK] Fix Pyspark CrossValidator/TrainValidationSplit with pipeline estimator cannot save and load model

SparkQA commented on issue #28279: URL: https://github.com/apache/spark/pull/28279#issuecomment-617583784 **[Test build #121609 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121609/testReport)** for PR 28279 at commit [`9b31c3a`](https://github.com/apach

[GitHub] [spark] AmplabJenkins commented on issue #27066: [SPARK-31317][SQL] Add withField method to Column class

AmplabJenkins commented on issue #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-617583546 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] AmplabJenkins removed a comment on issue #27066: [SPARK-31317][SQL] Add withField method to Column class

AmplabJenkins removed a comment on issue #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-617583546 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] SparkQA removed a comment on issue #27066: [SPARK-31317][SQL] Add withField method to Column class

SparkQA removed a comment on issue #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-617507065 **[Test build #121602 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121602/testReport)** for PR 27066 at commit [`fba3bd8`](https://github.co

[GitHub] [spark] SparkQA commented on issue #27066: [SPARK-31317][SQL] Add withField method to Column class

SparkQA commented on issue #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-617582823 **[Test build #121602 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121602/testReport)** for PR 27066 at commit [`fba3bd8`](https://github.com/apach

[GitHub] [spark] AmplabJenkins removed a comment on issue #28260: [SPARK-31487][CORE] Move slots check of barrier job from DAGScheduler to TaskSchedulerImpl

AmplabJenkins removed a comment on issue #28260: URL: https://github.com/apache/spark/pull/28260#issuecomment-617582036 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] AmplabJenkins commented on issue #28260: [SPARK-31487][CORE] Move slots check of barrier job from DAGScheduler to TaskSchedulerImpl

AmplabJenkins commented on issue #28260: URL: https://github.com/apache/spark/pull/28260#issuecomment-617582036 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] SparkQA commented on issue #28260: [SPARK-31487][CORE] Move slots check of barrier job from DAGScheduler to TaskSchedulerImpl

SparkQA commented on issue #28260: URL: https://github.com/apache/spark/pull/28260#issuecomment-617581587 **[Test build #121611 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121611/testReport)** for PR 28260 at commit [`fdb7840`](https://github.com/apache

[GitHub] [spark] AmplabJenkins removed a comment on issue #28271: [SPARK-31495][SQL] Support formatted explain for AQE

AmplabJenkins removed a comment on issue #28271: URL: https://github.com/apache/spark/pull/28271#issuecomment-617578240 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] Ngone51 commented on issue #28271: [SPARK-31495][SQL] Support formatted explain for AQE

Ngone51 commented on issue #28271: URL: https://github.com/apache/spark/pull/28271#issuecomment-617578374 cc @cloud-fan @maryannxue This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] AmplabJenkins commented on issue #28271: [SPARK-31495][SQL] Support formatted explain for AQE

AmplabJenkins commented on issue #28271: URL: https://github.com/apache/spark/pull/28271#issuecomment-617578240 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] SparkQA removed a comment on issue #28271: [SPARK-31495][SQL] Support formatted explain for AQE

SparkQA removed a comment on issue #28271: URL: https://github.com/apache/spark/pull/28271#issuecomment-617500658 **[Test build #121600 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121600/testReport)** for PR 28271 at commit [`1228d2e`](https://github.co

[GitHub] [spark] SparkQA commented on issue #28271: [SPARK-31495][SQL] Support formatted explain for AQE

SparkQA commented on issue #28271: URL: https://github.com/apache/spark/pull/28271#issuecomment-617577525 **[Test build #121600 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121600/testReport)** for PR 28271 at commit [`1228d2e`](https://github.com/apach

[GitHub] [spark] AmplabJenkins removed a comment on issue #28109: [SPARK-31253][SQL][followup] Add metric to the split task number for skew optimization

AmplabJenkins removed a comment on issue #28109: URL: https://github.com/apache/spark/pull/28109#issuecomment-617576708 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] AmplabJenkins commented on issue #28279: [SPARK-31497][ML][PYSPARK] Fix Pyspark CrossValidator/TrainValidationSplit with pipeline estimator cannot save and load model

AmplabJenkins commented on issue #28279: URL: https://github.com/apache/spark/pull/28279#issuecomment-617576662 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] AmplabJenkins removed a comment on issue #28279: [SPARK-31497][ML][PYSPARK] Fix Pyspark CrossValidator/TrainValidationSplit with pipeline estimator cannot save and load model

AmplabJenkins removed a comment on issue #28279: URL: https://github.com/apache/spark/pull/28279#issuecomment-617576662 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] AmplabJenkins commented on issue #28109: [SPARK-31253][SQL][followup] Add metric to the split task number for skew optimization

AmplabJenkins commented on issue #28109: URL: https://github.com/apache/spark/pull/28109#issuecomment-617576708 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] SparkQA commented on issue #28109: [SPARK-31253][SQL][followup] Add metric to the split task number for skew optimization

SparkQA commented on issue #28109: URL: https://github.com/apache/spark/pull/28109#issuecomment-617576281 **[Test build #121610 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121610/testReport)** for PR 28109 at commit [`5f39225`](https://github.com/apache

[GitHub] [spark] SparkQA commented on issue #28279: [SPARK-31497][ML][PYSPARK] Fix Pyspark CrossValidator/TrainValidationSplit with pipeline estimator cannot save and load model

SparkQA commented on issue #28279: URL: https://github.com/apache/spark/pull/28279#issuecomment-617576279 **[Test build #121609 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121609/testReport)** for PR 28279 at commit [`9b31c3a`](https://github.com/apache

[GitHub] [spark] SparkQA removed a comment on issue #28279: [SPARK-31497][ML][PYSPARK] Fix Pyspark CrossValidator/TrainValidationSplit with pipeline estimator cannot save and load model

SparkQA removed a comment on issue #28279: URL: https://github.com/apache/spark/pull/28279#issuecomment-617574024 **[Test build #121608 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121608/testReport)** for PR 28279 at commit [`85ebf53`](https://github.c

[GitHub] [spark] AmplabJenkins removed a comment on issue #28279: [SPARK-31497][ML][PYSPARK] Fix Pyspark CrossValidator/TrainValidationSplit with pipeline estimator cannot save and load model

AmplabJenkins removed a comment on issue #28279: URL: https://github.com/apache/spark/pull/28279#issuecomment-617574038 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121608/

[GitHub] [spark] SparkQA removed a comment on issue #28279: [SPARK-31497][ML][PYSPARK] Fix Pyspark CrossValidator/TrainValidationSplit with pipeline estimator cannot save and load model

SparkQA removed a comment on issue #28279: URL: https://github.com/apache/spark/pull/28279#issuecomment-617573706 **[Test build #121608 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121608/testReport)** for PR 28279 at commit [`85ebf53`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on issue #28109: [SPARK-31253][SQL][followup] Add metric to the split task number for skew optimization

AmplabJenkins removed a comment on issue #28109: URL: https://github.com/apache/spark/pull/28109#issuecomment-617574129 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] AmplabJenkins removed a comment on issue #28279: [SPARK-31497][ML][PYSPARK] Fix Pyspark CrossValidator/TrainValidationSplit with pipeline estimator cannot save and load model

AmplabJenkins removed a comment on issue #28279: URL: https://github.com/apache/spark/pull/28279#issuecomment-617574033 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] WeichenXu123 commented on a change in pull request #28279: [SPARK-31497][ML][PYSPARK] Fix Pyspark CrossValidator/TrainValidationSplit with pipeline estimator cannot save and load mode

WeichenXu123 commented on a change in pull request #28279: URL: https://github.com/apache/spark/pull/28279#discussion_r412699397 ## File path: python/pyspark/ml/pipeline.py ## @@ -174,6 +174,55 @@ def _to_java(self): return _java_obj +def _make_java_param_pair(

[GitHub] [spark] AmplabJenkins removed a comment on issue #28288: [SPARK-31515][SQL] Canonicalize Cast should consider the value of needTimeZone

AmplabJenkins removed a comment on issue #28288: URL: https://github.com/apache/spark/pull/28288#issuecomment-617573841 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121605/

[GitHub] [spark] SparkQA commented on issue #28279: [SPARK-31497][ML][PYSPARK] Fix Pyspark CrossValidator/TrainValidationSplit with pipeline estimator cannot save and load model

SparkQA commented on issue #28279: URL: https://github.com/apache/spark/pull/28279#issuecomment-617574024 **[Test build #121608 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121608/testReport)** for PR 28279 at commit [`85ebf53`](https://github.com/apach

[GitHub] [spark] AmplabJenkins commented on issue #28279: [SPARK-31497][ML][PYSPARK] Fix Pyspark CrossValidator/TrainValidationSplit with pipeline estimator cannot save and load model

AmplabJenkins commented on issue #28279: URL: https://github.com/apache/spark/pull/28279#issuecomment-617574033 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] SparkQA removed a comment on issue #28288: [SPARK-31515][SQL] Canonicalize Cast should consider the value of needTimeZone

SparkQA removed a comment on issue #28288: URL: https://github.com/apache/spark/pull/28288#issuecomment-617540272 **[Test build #121605 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121605/testReport)** for PR 28288 at commit [`4961eb3`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on issue #28288: [SPARK-31515][SQL] Canonicalize Cast should consider the value of needTimeZone

AmplabJenkins removed a comment on issue #28288: URL: https://github.com/apache/spark/pull/28288#issuecomment-617573835 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] AmplabJenkins commented on issue #28109: [SPARK-31253][SQL][followup] Add metric to the split task number for skew optimization

AmplabJenkins commented on issue #28109: URL: https://github.com/apache/spark/pull/28109#issuecomment-617574129 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] SparkQA commented on issue #28279: [SPARK-31497][ML][PYSPARK] Fix Pyspark CrossValidator/TrainValidationSplit with pipeline estimator cannot save and load model

SparkQA commented on issue #28279: URL: https://github.com/apache/spark/pull/28279#issuecomment-617573706 **[Test build #121608 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121608/testReport)** for PR 28279 at commit [`85ebf53`](https://github.com/apache

[GitHub] [spark] AmplabJenkins commented on issue #28288: [SPARK-31515][SQL] Canonicalize Cast should consider the value of needTimeZone

AmplabJenkins commented on issue #28288: URL: https://github.com/apache/spark/pull/28288#issuecomment-617573835 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] SparkQA commented on issue #28288: [SPARK-31515][SQL] Canonicalize Cast should consider the value of needTimeZone

SparkQA commented on issue #28288: URL: https://github.com/apache/spark/pull/28288#issuecomment-617573595 **[Test build #121605 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121605/testReport)** for PR 28288 at commit [`4961eb3`](https://github.com/apach

[GitHub] [spark] WeichenXu123 commented on a change in pull request #28279: [SPARK-31497][ML][PYSPARK] Fix Pyspark CrossValidator/TrainValidationSplit with pipeline estimator cannot save and load mode

WeichenXu123 commented on a change in pull request #28279: URL: https://github.com/apache/spark/pull/28279#discussion_r412697172 ## File path: python/pyspark/ml/pipeline.py ## @@ -174,6 +174,55 @@ def _to_java(self): return _java_obj +def _make_java_param_pair(

[GitHub] [spark] AmplabJenkins removed a comment on issue #28085: [SPARK-29641][PYTHON][CORE] Stage Level Sched: Add python api's and tests

AmplabJenkins removed a comment on issue #28085: URL: https://github.com/apache/spark/pull/28085#issuecomment-617571038 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] AmplabJenkins commented on issue #28085: [SPARK-29641][PYTHON][CORE] Stage Level Sched: Add python api's and tests

AmplabJenkins commented on issue #28085: URL: https://github.com/apache/spark/pull/28085#issuecomment-617571038 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] SparkQA commented on issue #28085: [SPARK-29641][PYTHON][CORE] Stage Level Sched: Add python api's and tests

SparkQA commented on issue #28085: URL: https://github.com/apache/spark/pull/28085#issuecomment-617570393 **[Test build #121604 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121604/testReport)** for PR 28085 at commit [`528094c`](https://github.com/apach

[GitHub] [spark] SparkQA removed a comment on issue #28085: [SPARK-29641][PYTHON][CORE] Stage Level Sched: Add python api's and tests

SparkQA removed a comment on issue #28085: URL: https://github.com/apache/spark/pull/28085#issuecomment-617528793 **[Test build #121604 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121604/testReport)** for PR 28085 at commit [`528094c`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on issue #28291: [SPARK-31400][ML,MLlib]The catalogString doesn't distinguish Vectors in ml and mllib

AmplabJenkins removed a comment on issue #28291: URL: https://github.com/apache/spark/pull/28291#issuecomment-617568524 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To res

[GitHub] [spark] AmplabJenkins commented on issue #28291: [SPARK-31400][ML,MLlib]The catalogString doesn't distinguish Vectors in ml and mllib

AmplabJenkins commented on issue #28291: URL: https://github.com/apache/spark/pull/28291#issuecomment-617568949 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] AmplabJenkins commented on issue #28291: [SPARK-31400][ML,MLlib]The catalogString doesn't distinguish Vectors in ml and mllib

AmplabJenkins commented on issue #28291: URL: https://github.com/apache/spark/pull/28291#issuecomment-617568524 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] TJX2014 opened a new pull request #28291: [SPARK-31400][ML,MLlib]The catalogString doesn't distinguish Vectors in ml and mllib

TJX2014 opened a new pull request #28291: URL: https://github.com/apache/spark/pull/28291 ### What changes were proposed in this pull request? add package check in org.apache.spark.ml.util.SchemaUtils#checkColumnType and add unit test in org.scalatest.FunSuiteLike#test

[GitHub] [spark] MaxGekk commented on issue #28272: [SPARK-31489][SPARK-31488][SQL] Translate date values of pushed down filters to `java.sql.Date`

MaxGekk commented on issue #28272: URL: https://github.com/apache/spark/pull/28272#issuecomment-617564224 @cloud-fan @HyukjinKwon @dongjoon-hyun Please, take a look at the PR if you have time. This is an automated message fr

[GitHub] [spark] SparkQA commented on issue #28290: [MINOR][SQL][DOCS] Add a paragraph for scalar function in sql getting started

SparkQA commented on issue #28290: URL: https://github.com/apache/spark/pull/28290#issuecomment-617560789 **[Test build #121607 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121607/testReport)** for PR 28290 at commit [`e5e69e2`](https://github.com/apach

[GitHub] [spark] SparkQA removed a comment on issue #28290: [MINOR][SQL][DOCS] Add a paragraph for scalar function in sql getting started

SparkQA removed a comment on issue #28290: URL: https://github.com/apache/spark/pull/28290#issuecomment-617557202 **[Test build #121607 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121607/testReport)** for PR 28290 at commit [`e5e69e2`](https://github.co

[GitHub] [spark] AmplabJenkins commented on issue #28290: [MINOR][SQL][DOCS] Add a paragraph for scalar function in sql getting started

AmplabJenkins commented on issue #28290: URL: https://github.com/apache/spark/pull/28290#issuecomment-617560892 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] AmplabJenkins removed a comment on issue #28290: [MINOR][SQL][DOCS] Add a paragraph for scalar function in sql getting started

AmplabJenkins removed a comment on issue #28290: URL: https://github.com/apache/spark/pull/28290#issuecomment-617560892 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] huaxingao commented on a change in pull request #28290: [MINOR][SQL][DOCS] Add a paragraph for scalar function in sql getting started

huaxingao commented on a change in pull request #28290: URL: https://github.com/apache/spark/pull/28290#discussion_r412681949 ## File path: docs/sql-getting-started.md ## @@ -347,7 +347,8 @@ For example: ## Scalar Functions -(to be filled soon) + +Scalar functions are func

[GitHub] [spark] maropu commented on a change in pull request #28290: [MINOR][SQL][DOCS] Add a paragraph for scalar function in sql getting started

maropu commented on a change in pull request #28290: URL: https://github.com/apache/spark/pull/28290#discussion_r412678792 ## File path: docs/sql-getting-started.md ## @@ -347,7 +347,8 @@ For example: ## Scalar Functions -(to be filled soon) + +Scalar functions are functio

[GitHub] [spark] AmplabJenkins removed a comment on issue #28290: [MINOR][SQL][DOCS] Add a paragraph for scalar function in sql getting started

AmplabJenkins removed a comment on issue #28290: URL: https://github.com/apache/spark/pull/28290#issuecomment-617555485 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] SparkQA commented on issue #28290: [MINOR][SQL][DOCS] Add a paragraph for scalar function in sql getting started

SparkQA commented on issue #28290: URL: https://github.com/apache/spark/pull/28290#issuecomment-617557202 **[Test build #121607 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121607/testReport)** for PR 28290 at commit [`e5e69e2`](https://github.com/apache

[GitHub] [spark] huaxingao commented on issue #28290: [MINOR][SQL][DOCS] Add a paragraph for scalar function in sql getting started

huaxingao commented on issue #28290: URL: https://github.com/apache/spark/pull/28290#issuecomment-617556913 @HyukjinKwon @maropu I quickly wrote a paragraph for scalar function. It has to be scalar function because the following paragraph is for aggregation. The definition for scalar f

[GitHub] [spark] AmplabJenkins commented on issue #28290: [MINOR][SQL][DOCS] Add a paragraph for scalar function in sql getting started

AmplabJenkins commented on issue #28290: URL: https://github.com/apache/spark/pull/28290#issuecomment-617555485 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] huaxingao opened a new pull request #28290: [MINOR][SQL][DOCS] Add a paragraph for scalar function in sql getting started

huaxingao opened a new pull request #28290: URL: https://github.com/apache/spark/pull/28290 ### What changes were proposed in this pull request? Add a paragraph for scalar function in sql getting started ### Why are the changes needed? To make 3.0 doc complete.

[GitHub] [spark] AmplabJenkins removed a comment on issue #28289: [SPARK-31484][Core][Flollowup] Use taskAttemptId in checkpoint filename

AmplabJenkins removed a comment on issue #28289: URL: https://github.com/apache/spark/pull/28289#issuecomment-617544598 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] AmplabJenkins commented on issue #28289: [SPARK-31484][Core][Flollowup] Use taskAttemptId in checkpoint filename

AmplabJenkins commented on issue #28289: URL: https://github.com/apache/spark/pull/28289#issuecomment-617544598 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] yaooqinn commented on a change in pull request #28284: [SPARK-31507][SQL] Remove uncommon fields support and update some fields with meaningful names for extract function

yaooqinn commented on a change in pull request #28284: URL: https://github.com/apache/spark/pull/28284#discussion_r412663151 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -2033,108 +1991,26 @@ case class Mak

[GitHub] [spark] SparkQA commented on issue #28289: [SPARK-31484][Core][Flollowup] Use taskAttemptId in checkpoint filename

SparkQA commented on issue #28289: URL: https://github.com/apache/spark/pull/28289#issuecomment-617544275 **[Test build #121606 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121606/testReport)** for PR 28289 at commit [`59e99f6`](https://github.com/apache

[GitHub] [spark] viirya commented on issue #28289: [SPARK-31484][Core][Flollowup] Use taskAttemptId in checkpoint filename

viirya commented on issue #28289: URL: https://github.com/apache/spark/pull/28289#issuecomment-617543743 cc @cloud-fan @xuanyuanking @dongjoon-hyun This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] viirya opened a new pull request #28289: [SPARK-31484][Core][Flollowup] Use taskAttemptId in checkpoint filename

viirya opened a new pull request #28289: URL: https://github.com/apache/spark/pull/28289 ### What changes were proposed in this pull request? As suggested by https://github.com/apache/spark/pull/28255#discussion_r412619438, this patch proposes to use taskAttemptId in che

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28085: [SPARK-29641][PYTHON][CORE] Stage Level Sched: Add python api's and tests

HyukjinKwon commented on a change in pull request #28085: URL: https://github.com/apache/spark/pull/28085#discussion_r412644686 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -1135,6 +1136,27 @@ private[spark] class DAGScheduler( }

[GitHub] [spark] viirya commented on a change in pull request #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception

viirya commented on a change in pull request #28255: URL: https://github.com/apache/spark/pull/28255#discussion_r412659291 ## File path: core/src/main/scala/org/apache/spark/rdd/ReliableCheckpointRDD.scala ## @@ -199,8 +199,8 @@ private[spark] object ReliableCheckpointRDD exten

[GitHub] [spark] SparkQA commented on issue #28288: [SPARK-31515][SQL] Canonicalize Cast should consider the value of needTimeZone

SparkQA commented on issue #28288: URL: https://github.com/apache/spark/pull/28288#issuecomment-617540272 **[Test build #121605 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121605/testReport)** for PR 28288 at commit [`4961eb3`](https://github.com/apache

[GitHub] [spark] AmplabJenkins removed a comment on issue #28288: [SPARK-31515][SQL] Canonicalize Cast should consider the value of needTimeZone

AmplabJenkins removed a comment on issue #28288: URL: https://github.com/apache/spark/pull/28288#issuecomment-617538573 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] AmplabJenkins removed a comment on issue #28260: [SPARK-31487][CORE] Move slots check of barrier job from DAGScheduler to TaskSchedulerImpl

AmplabJenkins removed a comment on issue #28260: URL: https://github.com/apache/spark/pull/28260#issuecomment-617538904 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121601/

[GitHub] [spark] AmplabJenkins removed a comment on issue #28260: [SPARK-31487][CORE] Move slots check of barrier job from DAGScheduler to TaskSchedulerImpl

AmplabJenkins removed a comment on issue #28260: URL: https://github.com/apache/spark/pull/28260#issuecomment-617538900 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] AmplabJenkins commented on issue #28260: [SPARK-31487][CORE] Move slots check of barrier job from DAGScheduler to TaskSchedulerImpl

AmplabJenkins commented on issue #28260: URL: https://github.com/apache/spark/pull/28260#issuecomment-617538904 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] SparkQA removed a comment on issue #28260: [SPARK-31487][CORE] Move slots check of barrier job from DAGScheduler to TaskSchedulerImpl

SparkQA removed a comment on issue #28260: URL: https://github.com/apache/spark/pull/28260#issuecomment-617500691 **[Test build #121601 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121601/testReport)** for PR 28260 at commit [`3b98b86`](https://github.co

[GitHub] [spark] SparkQA commented on issue #28260: [SPARK-31487][CORE] Move slots check of barrier job from DAGScheduler to TaskSchedulerImpl

SparkQA commented on issue #28260: URL: https://github.com/apache/spark/pull/28260#issuecomment-617538600 **[Test build #121601 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121601/testReport)** for PR 28260 at commit [`3b98b86`](https://github.com/apach

[GitHub] [spark] AmplabJenkins commented on issue #28288: [SPARK-31515][SQL] Canonicalize Cast should consider the value of needTimeZone

AmplabJenkins commented on issue #28288: URL: https://github.com/apache/spark/pull/28288#issuecomment-617538573 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] xuanyuanking opened a new pull request #28288: [SPARK-31515][SQL] Canonicalize Cast should consider the value of needTimeZone

xuanyuanking opened a new pull request #28288: URL: https://github.com/apache/spark/pull/28288 ### What changes were proposed in this pull request? Override the canonicalized fields with respect to the result of `needsTimeZone`. ### Why are the changes needed? The current a

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28284: [SPARK-31507][SQL] Remove uncommon fields support and update some fields with meaningful names for extract function

dongjoon-hyun commented on a change in pull request #28284: URL: https://github.com/apache/spark/pull/28284#discussion_r412650911 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -2033,108 +1991,26 @@ case clas

[GitHub] [spark] yaooqinn commented on a change in pull request #28284: [SPARK-31507][SQL] Remove uncommon fields support and update some fields with meaningful names for extract function

yaooqinn commented on a change in pull request #28284: URL: https://github.com/apache/spark/pull/28284#discussion_r412648326 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -2033,108 +1991,26 @@ case class Mak

[GitHub] [spark] yaooqinn commented on a change in pull request #28284: [SPARK-31507][SQL] Remove uncommon fields support and update some fields with meaningful names for extract function

yaooqinn commented on a change in pull request #28284: URL: https://github.com/apache/spark/pull/28284#discussion_r412648326 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -2033,108 +1991,26 @@ case class Mak

[GitHub] [spark] mridulm commented on a change in pull request #28257: [SPARK-31485][CORE] Avoid application hang if only partial barrier tasks launched

mridulm commented on a change in pull request #28257: URL: https://github.com/apache/spark/pull/28257#discussion_r412639758 ## File path: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ## @@ -401,9 +401,7 @@ private[spark] class TaskSchedulerImpl(

[GitHub] [spark] AmplabJenkins removed a comment on issue #28285: [SPARK-31510][R][BUILD] Set setwd in R documentation build

AmplabJenkins removed a comment on issue #28285: URL: https://github.com/apache/spark/pull/28285#issuecomment-617529054 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] AmplabJenkins removed a comment on issue #28085: [SPARK-29641][PYTHON][CORE] Stage Level Sched: Add python api's and tests

AmplabJenkins removed a comment on issue #28085: URL: https://github.com/apache/spark/pull/28085#issuecomment-617529211 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] AmplabJenkins commented on issue #28285: [SPARK-31510][R][BUILD] Set setwd in R documentation build

AmplabJenkins commented on issue #28285: URL: https://github.com/apache/spark/pull/28285#issuecomment-617529054 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] AmplabJenkins commented on issue #28085: [SPARK-29641][PYTHON][CORE] Stage Level Sched: Add python api's and tests

AmplabJenkins commented on issue #28085: URL: https://github.com/apache/spark/pull/28085#issuecomment-617529211 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] SparkQA commented on issue #28285: [SPARK-31510][R][BUILD] Set setwd in R documentation build

SparkQA commented on issue #28285: URL: https://github.com/apache/spark/pull/28285#issuecomment-617528964 **[Test build #121603 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121603/testReport)** for PR 28285 at commit [`a1a81f9`](https://github.com/apach

[GitHub] [spark] SparkQA commented on issue #28085: [SPARK-29641][PYTHON][CORE] Stage Level Sched: Add python api's and tests

SparkQA commented on issue #28085: URL: https://github.com/apache/spark/pull/28085#issuecomment-617528793 **[Test build #121604 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121604/testReport)** for PR 28085 at commit [`528094c`](https://github.com/apache

[GitHub] [spark] SparkQA removed a comment on issue #28285: [SPARK-31510][R][BUILD] Set setwd in R documentation build

SparkQA removed a comment on issue #28285: URL: https://github.com/apache/spark/pull/28285#issuecomment-617520487 **[Test build #121603 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121603/testReport)** for PR 28285 at commit [`a1a81f9`](https://github.co

[GitHub] [spark] HyukjinKwon commented on issue #28085: [SPARK-29641][PYTHON][CORE] Stage Level Sched: Add python api's and tests

HyukjinKwon commented on issue #28085: URL: https://github.com/apache/spark/pull/28085#issuecomment-617528646 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] AmplabJenkins removed a comment on issue #28208: [SPARK-31440][SQL] Improve SQL Rest API

AmplabJenkins removed a comment on issue #28208: URL: https://github.com/apache/spark/pull/28208#issuecomment-617525552 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] AmplabJenkins commented on issue #28208: [SPARK-31440][SQL] Improve SQL Rest API

AmplabJenkins commented on issue #28208: URL: https://github.com/apache/spark/pull/28208#issuecomment-617525552 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] yaooqinn commented on a change in pull request #28284: [SPARK-31507][SQL] Remove uncommon fields support and update some fields with meaningful names for extract function

yaooqinn commented on a change in pull request #28284: URL: https://github.com/apache/spark/pull/28284#discussion_r412621357 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -2033,108 +1991,26 @@ case class Mak

[GitHub] [spark] SparkQA removed a comment on issue #28208: [SPARK-31440][SQL] Improve SQL Rest API

SparkQA removed a comment on issue #28208: URL: https://github.com/apache/spark/pull/28208#issuecomment-617454217 **[Test build #121598 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121598/testReport)** for PR 28208 at commit [`da88978`](https://github.co

[GitHub] [spark] SparkQA commented on issue #28208: [SPARK-31440][SQL] Improve SQL Rest API

SparkQA commented on issue #28208: URL: https://github.com/apache/spark/pull/28208#issuecomment-617525009 **[Test build #121598 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121598/testReport)** for PR 28208 at commit [`da88978`](https://github.com/apach

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28284: [SPARK-31507][SQL] Remove uncommon fields support and update some fields with meaningful names for extract function

dongjoon-hyun commented on a change in pull request #28284: URL: https://github.com/apache/spark/pull/28284#discussion_r412638011 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -2033,108 +1991,26 @@ case clas

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28284: [SPARK-31507][SQL] Remove uncommon fields support and update some fields with meaningful names for extract function

dongjoon-hyun commented on a change in pull request #28284: URL: https://github.com/apache/spark/pull/28284#discussion_r412638011 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -2033,108 +1991,26 @@ case clas

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28284: [SPARK-31507][SQL] Remove uncommon fields support and update some fields with meaningful names for extract function

dongjoon-hyun commented on a change in pull request #28284: URL: https://github.com/apache/spark/pull/28284#discussion_r412638011 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -2033,108 +1991,26 @@ case clas

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28284: [SPARK-31507][SQL] Remove uncommon fields support and update some fields with meaningful names for extract function

dongjoon-hyun commented on a change in pull request #28284: URL: https://github.com/apache/spark/pull/28284#discussion_r412638011 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -2033,108 +1991,26 @@ case clas

[GitHub] [spark] HeartSaVioR commented on issue #28215: [SPARK-31272][SQL] Support DB2 Kerberos login in JDBC connector

HeartSaVioR commented on issue #28215: URL: https://github.com/apache/spark/pull/28215#issuecomment-617524078 Oh, nice catch. That compilation error seems to only come with JDK11. That said, Spark PR builder may need to run at least two different environments; that's lucky we found t

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28284: [SPARK-31507][SQL] Remove uncommon fields support and update some fields with meaningful names for extract function

dongjoon-hyun commented on a change in pull request #28284: URL: https://github.com/apache/spark/pull/28284#discussion_r412638011 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -2033,108 +1991,26 @@ case clas

[GitHub] [spark] xuanyuanking commented on a change in pull request #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception

xuanyuanking commented on a change in pull request #28255: URL: https://github.com/apache/spark/pull/28255#discussion_r412636713 ## File path: core/src/main/scala/org/apache/spark/rdd/ReliableCheckpointRDD.scala ## @@ -199,8 +199,8 @@ private[spark] object ReliableCheckpointRDD

  1   2   3   4   5   6   >