[jira] [Updated] (SPARK-36026) Upgrade Kubernetes Client Version to 5.5.0

[ https://issues.apache.org/jira/browse/SPARK-36026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-36026: --- Summary: Upgrade Kubernetes Client Version to 5.5.0 (was: Bump Kubernetes Client

[jira] [Created] (SPARK-36026) Bump Kubernetes Client Version to 5.5.0

Attila Zsolt Piros created SPARK-36026: -- Summary: Bump Kubernetes Client Version to 5.5.0 Key: SPARK-36026 URL: https://issues.apache.org/jira/browse/SPARK-36026 Project: Spark Issue

[jira] [Updated] (SPARK-36052) Introduce pending pod limit for Spark on K8s

[ https://issues.apache.org/jira/browse/SPARK-36052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-36052: --- Description: Introduce a new configuration to limit the number of pending PODs for

[jira] [Updated] (SPARK-36052) Introduce pending pod limit for Spark on K8s

[ https://issues.apache.org/jira/browse/SPARK-36052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-36052: --- Description: Introduce pending POD limit for Spark on K8S as K8S scheduler could be

[jira] [Created] (SPARK-36052) Introduce pending pod limit for Spark on K8s

Attila Zsolt Piros created SPARK-36052: -- Summary: Introduce pending pod limit for Spark on K8s Key: SPARK-36052 URL: https://issues.apache.org/jira/browse/SPARK-36052 Project: Spark

[jira] [Assigned] (SPARK-35234) Reserve format of the stage failureMessage

[ https://issues.apache.org/jira/browse/SPARK-35234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros reassigned SPARK-35234: -- Assignee: wuyi > Reserve format of the stage failureMessage >

[jira] [Resolved] (SPARK-35234) Reserve format of the stage failureMessage

[ https://issues.apache.org/jira/browse/SPARK-35234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros resolved SPARK-35234. Fix Version/s: 3.2.0 Resolution: Fixed > Reserve format of the stage

[jira] [Resolved] (SPARK-35009) Avoid creating multiple Monitor threads for reused python workers for the same TaskContext

[ https://issues.apache.org/jira/browse/SPARK-35009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros resolved SPARK-35009. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request

[jira] [Commented] (SPARK-34389) Spark job on Kubernetes scheduled For Zero or less than minimum number of executors and Wait indefinitely under resource starvation

[ https://issues.apache.org/jira/browse/SPARK-34389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281134#comment-17281134 ] Attila Zsolt Piros commented on SPARK-34389: [~ranju] I understand your concern but Spark

[jira] [Updated] (SPARK-34370) Supporting Avro schema evolution for partitioned Hive tables using "avro.schema.url"

[ https://issues.apache.org/jira/browse/SPARK-34370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-34370: --- Description: This came up in

[jira] [Updated] (SPARK-34370) Supporting Avro schema evolution for partitioned Hive tables using "avro.schema.url"

[ https://issues.apache.org/jira/browse/SPARK-34370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-34370: --- Description: This came up in

[jira] [Updated] (SPARK-34370) Supporting Avro schema evolution for partitioned Hive tables using "avro.schema.url"

[ https://issues.apache.org/jira/browse/SPARK-34370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-34370: --- Description: This came up in

[jira] [Updated] (SPARK-34370) Supporting Avro schema evolution for partitioned Hive tables using "avro.schema.url"

[ https://issues.apache.org/jira/browse/SPARK-34370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-34370: --- Description: This came up in

[jira] [Created] (SPARK-34361) Dynamic allocation on K8s kills executors with running tasks

Attila Zsolt Piros created SPARK-34361: -- Summary: Dynamic allocation on K8s kills executors with running tasks Key: SPARK-34361 URL: https://issues.apache.org/jira/browse/SPARK-34361 Project:

[jira] [Commented] (SPARK-34361) Dynamic allocation on K8s kills executors with running tasks

[ https://issues.apache.org/jira/browse/SPARK-34361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17278932#comment-17278932 ] Attila Zsolt Piros commented on SPARK-34361: I am working on this. > Dynamic allocation on

[jira] [Commented] (SPARK-34389) Spark job on Kubernetes scheduled For Zero or less than minimum number of executors and Wait indefinitely under resource starvation

[ https://issues.apache.org/jira/browse/SPARK-34389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281645#comment-17281645 ] Attila Zsolt Piros commented on SPARK-34389: > is it possible to get the available resources

[jira] [Commented] (SPARK-34372) Speculation results in broken CSV files in Amazon S3

[ https://issues.apache.org/jira/browse/SPARK-34372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281815#comment-17281815 ] Attila Zsolt Piros commented on SPARK-34372: A Direct output committer with speculation

[jira] [Comment Edited] (SPARK-33763) Add metrics for better tracking of dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-33763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282201#comment-17282201 ] Attila Zsolt Piros edited comment on SPARK-33763 at 2/10/21, 3:30 AM:

[jira] [Commented] (SPARK-33763) Add metrics for better tracking of dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-33763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282201#comment-17282201 ] Attila Zsolt Piros commented on SPARK-33763: I am not positive about the "stage re-submitted

[jira] [Commented] (SPARK-34389) Spark job on Kubernetes scheduled For Zero or less than minimum number of executors and Wait indefinitely under resource starvation

[ https://issues.apache.org/jira/browse/SPARK-34389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282614#comment-17282614 ] Attila Zsolt Piros commented on SPARK-34389: [~ranju] the description of this config is a

[jira] [Commented] (SPARK-34389) Spark job on Kubernetes scheduled For Zero or less than minimum number of executors and Wait indefinitely under resource starvation

[ https://issues.apache.org/jira/browse/SPARK-34389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282871#comment-17282871 ] Attila Zsolt Piros commented on SPARK-34389: @Ranju from the logs you attached it is clear

[jira] [Commented] (SPARK-34389) Spark job on Kubernetes scheduled For Zero or less than minimum number of executors and Wait indefinitely under resource starvation

[ https://issues.apache.org/jira/browse/SPARK-34389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282994#comment-17282994 ] Attila Zsolt Piros commented on SPARK-34389: [~ranju] as I see your question is more about

[jira] [Commented] (SPARK-34285) Implement Parquet StringEndsWith、StringContains Filter

[ https://issues.apache.org/jira/browse/SPARK-34285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17274458#comment-17274458 ] Attila Zsolt Piros commented on SPARK-34285: [~Xudingyu] predicate pushdown is extremely

[jira] [Commented] (SPARK-34126) SQL running error, spark does not exit, resulting in data quality problems

[ https://issues.apache.org/jira/browse/SPARK-34126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17273761#comment-17273761 ] Attila Zsolt Piros commented on SPARK-34126: [~shikui] are you using the " -f " argument of

[jira] [Commented] (SPARK-29391) Default year-month units

[ https://issues.apache.org/jira/browse/SPARK-29391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17275926#comment-17275926 ] Attila Zsolt Piros commented on SPARK-29391: I am working on this and within a few days a PR

[jira] [Commented] (SPARK-34684) Hadoop config could not be successfully serilized from driver pods to executor pods

[ https://issues.apache.org/jira/browse/SPARK-34684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17307710#comment-17307710 ] Attila Zsolt Piros commented on SPARK-34684: I did not say there is no error but by

[jira] [Commented] (SPARK-34684) Hadoop config could not be successfully serilized from driver pods to executor pods

[ https://issues.apache.org/jira/browse/SPARK-34684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17306895#comment-17306895 ] Attila Zsolt Piros commented on SPARK-34684: > spark pi example job keeps failing where

[jira] [Updated] (SPARK-34869) Extend k8s "EXTRA LOGS FOR THE FAILED TEST" section with pod describe output

[ https://issues.apache.org/jira/browse/SPARK-34869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-34869: --- Description: As came up in

[jira] [Assigned] (SPARK-34869) Extend k8s "EXTRA LOGS FOR THE FAILED TEST" section with pod describe output

[ https://issues.apache.org/jira/browse/SPARK-34869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros reassigned SPARK-34869: -- Assignee: Attila Zsolt Piros > Extend k8s "EXTRA LOGS FOR THE FAILED TEST"

[jira] [Commented] (SPARK-34869) Extend k8s "EXTRA LOGS FOR THE FAILED TEST" section with pod describe output

[ https://issues.apache.org/jira/browse/SPARK-34869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308679#comment-17308679 ] Attila Zsolt Piros commented on SPARK-34869: I am working on this. > Extend k8s "EXTRA LOGS

[jira] [Created] (SPARK-34869) Extend k8s "EXTRA LOGS FOR THE FAILED TEST" section with pod describe output

Attila Zsolt Piros created SPARK-34869: -- Summary: Extend k8s "EXTRA LOGS FOR THE FAILED TEST" section with pod describe output Key: SPARK-34869 URL: https://issues.apache.org/jira/browse/SPARK-34869

[jira] [Updated] (SPARK-34869) Extend k8s "EXTRA LOGS FOR THE FAILED TEST" section with pod describe output

[ https://issues.apache.org/jira/browse/SPARK-34869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-34869: --- Description: As came up in

[jira] [Updated] (SPARK-34869) Extend k8s "EXTRA LOGS FOR THE FAILED TEST" section with describe pods output

[ https://issues.apache.org/jira/browse/SPARK-34869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-34869: --- Description: As came up in

[jira] [Updated] (SPARK-34869) Extend k8s "EXTRA LOGS FOR THE FAILED TEST" section with describe pods output

[ https://issues.apache.org/jira/browse/SPARK-34869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-34869: --- Summary: Extend k8s "EXTRA LOGS FOR THE FAILED TEST" section with describe pods

[jira] [Commented] (SPARK-34738) Upgrade Minikube and kubernetes cluster version on Jenkins

[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17313144#comment-17313144 ] Attila Zsolt Piros commented on SPARK-34738: So I wanted to check whether I can help next

[jira] [Resolved] (SPARK-34779) ExecutorMetricsPoller should keep stage entry in stageTCMP until a heartbeat occurs

[ https://issues.apache.org/jira/browse/SPARK-34779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros resolved SPARK-34779. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-34779) ExecutorMetricsPoller should keep stage entry in stageTCMP until a heartbeat occurs

[ https://issues.apache.org/jira/browse/SPARK-34779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros reassigned SPARK-34779: -- Assignee: Baohe Zhang > ExecutorMetricsPoller should keep stage entry in

[jira] [Commented] (SPARK-34738) Upgrade Minikube and kubernetes cluster version on Jenkins

[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17312851#comment-17312851 ] Attila Zsolt Piros commented on SPARK-34738: Shane, as Docker is available on every platform

[jira] [Commented] (SPARK-34684) Hadoop config could not be successfully serilized from driver pods to executor pods

[ https://issues.apache.org/jira/browse/SPARK-34684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17307392#comment-17307392 ] Attila Zsolt Piros commented on SPARK-34684: > Have you tried to create a POD from a simple

[jira] [Created] (SPARK-34732) The logForFailedTest throws an exception when driver is not started

Attila Zsolt Piros created SPARK-34732: -- Summary: The logForFailedTest throws an exception when driver is not started Key: SPARK-34732 URL: https://issues.apache.org/jira/browse/SPARK-34732

[jira] [Created] (SPARK-34789) Introduce Jetty based construct for integration tests where HTTP(S) is used

Attila Zsolt Piros created SPARK-34789: -- Summary: Introduce Jetty based construct for integration tests where HTTP(S) is used Key: SPARK-34789 URL: https://issues.apache.org/jira/browse/SPARK-34789

[jira] [Commented] (SPARK-34789) Introduce Jetty based construct for integration tests where HTTP(S) is used

[ https://issues.apache.org/jira/browse/SPARK-34789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17304245#comment-17304245 ] Attila Zsolt Piros commented on SPARK-34789: I am working on this > Introduce Jetty based

[jira] [Updated] (SPARK-34789) Introduce Jetty based construct for integration tests where HTTP(S) is used

[ https://issues.apache.org/jira/browse/SPARK-34789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-34789: --- Description: This came up during

[jira] [Commented] (SPARK-34738) Upgrade Minikube and kubernetes cluster version on Jenkins

[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17303287#comment-17303287 ] Attila Zsolt Piros commented on SPARK-34738: Thanks Shane! This sounds great! > Upgrade

[jira] [Commented] (SPARK-34736) Kubernetes and Minikube version upgrade for integration tests

[ https://issues.apache.org/jira/browse/SPARK-34736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301046#comment-17301046 ] Attila Zsolt Piros commented on SPARK-34736: Working on this > Kubernetes and Minikube

[jira] [Created] (SPARK-34736) Kubernetes and Minikube version upgrade for integration tests

Attila Zsolt Piros created SPARK-34736: -- Summary: Kubernetes and Minikube version upgrade for integration tests Key: SPARK-34736 URL: https://issues.apache.org/jira/browse/SPARK-34736 Project:

[jira] [Created] (SPARK-34738) Upgrade Minikube and kubernetes cluster version on Jenkins

Attila Zsolt Piros created SPARK-34738: -- Summary: Upgrade Minikube and kubernetes cluster version on Jenkins Key: SPARK-34738 URL: https://issues.apache.org/jira/browse/SPARK-34738 Project: Spark

[jira] [Commented] (SPARK-34509) Make dynamic allocation upscaling more progressive on K8S

[ https://issues.apache.org/jira/browse/SPARK-34509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17289158#comment-17289158 ] Attila Zsolt Piros commented on SPARK-34509: I am working on this. > Make dynamic

[jira] [Created] (SPARK-34509) Make dynamic allocation upscaling more progressive on K8S

Attila Zsolt Piros created SPARK-34509: -- Summary: Make dynamic allocation upscaling more progressive on K8S Key: SPARK-34509 URL: https://issues.apache.org/jira/browse/SPARK-34509 Project: Spark

[jira] [Commented] (SPARK-34738) Upgrade Minikube and kubernetes cluster version on Jenkins

[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17317381#comment-17317381 ] Attila Zsolt Piros commented on SPARK-34738: No worries. I have guess. Check

[jira] [Comment Edited] (SPARK-34738) Upgrade Minikube and kubernetes cluster version on Jenkins

[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17317381#comment-17317381 ] Attila Zsolt Piros edited comment on SPARK-34738 at 4/8/21, 6:02 PM: -

[jira] [Commented] (SPARK-34738) Upgrade Minikube and kubernetes cluster version on Jenkins

[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17317426#comment-17317426 ] Attila Zsolt Piros commented on SPARK-34738: [~shaneknapp] the operator is IN: {noformat}

[jira] [Commented] (SPARK-34738) Upgrade Minikube and kubernetes cluster version on Jenkins

[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17317391#comment-17317391 ] Attila Zsolt Piros commented on SPARK-34738: >From the log we can see the mount is not ok:

[jira] [Commented] (SPARK-34738) Upgrade Minikube and kubernetes cluster version on Jenkins

[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17317779#comment-17317779 ] Attila Zsolt Piros commented on SPARK-34738: For sure `withValues` is needed. This is

[jira] [Commented] (SPARK-34738) Upgrade Minikube and kubernetes and move to docker 'virtualization' layer

[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324167#comment-17324167 ] Attila Zsolt Piros commented on SPARK-34738: My new commit:

[jira] [Comment Edited] (SPARK-34738) Upgrade Minikube and kubernetes and move to docker 'virtualization' layer

[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324165#comment-17324165 ] Attila Zsolt Piros edited comment on SPARK-34738 at 4/17/21, 3:56 AM:

[jira] [Commented] (SPARK-34738) Upgrade Minikube and kubernetes and move to docker 'virtualization' layer

[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324165#comment-17324165 ] Attila Zsolt Piros commented on SPARK-34738: @shane I can modify my PR to skip the PV tests

[jira] [Comment Edited] (SPARK-34738) Upgrade Minikube and kubernetes and move to docker 'virtualization' layer

[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324167#comment-17324167 ] Attila Zsolt Piros edited comment on SPARK-34738 at 4/17/21, 4:27 AM:

[jira] [Commented] (SPARK-34738) Upgrade Minikube and kubernetes cluster version on Jenkins

[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17317034#comment-17317034 ] Attila Zsolt Piros commented on SPARK-34738: hi [~shaneknapp]! Can you please share with

[jira] [Commented] (SPARK-35009) Avoid creating multiple Monitor threads for reused python workers for the same TaskContext

[ https://issues.apache.org/jira/browse/SPARK-35009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17318097#comment-17318097 ] Attila Zsolt Piros commented on SPARK-35009: I am working on this > Avoid creating multiple

[jira] [Created] (SPARK-35009) Avoid creating multiple Monitor threads for reused python workers for the same TaskContext

Attila Zsolt Piros created SPARK-35009: -- Summary: Avoid creating multiple Monitor threads for reused python workers for the same TaskContext Key: SPARK-35009 URL:

[jira] [Commented] (SPARK-34370) Supporting Avro schema evolution for partitioned Hive tables using "avro.schema.url"

[ https://issues.apache.org/jira/browse/SPARK-34370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279156#comment-17279156 ] Attila Zsolt Piros commented on SPARK-34370: I am working on this. > Supporting Avro schema

[jira] [Created] (SPARK-34370) Supporting Avro schema evolution for partitioned Hive tables using "avro.schema.url"

Attila Zsolt Piros created SPARK-34370: -- Summary: Supporting Avro schema evolution for partitioned Hive tables using "avro.schema.url" Key: SPARK-34370 URL: https://issues.apache.org/jira/browse/SPARK-34370

[jira] [Updated] (SPARK-34370) Supporting Avro schema evolution for partitioned Hive tables using "avro.schema.url"

[ https://issues.apache.org/jira/browse/SPARK-34370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-34370: --- Description: This came up in

[jira] [Commented] (SPARK-34426) Add driver and executors POD logs to integration-tests log when the test fails

[ https://issues.apache.org/jira/browse/SPARK-34426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17283974#comment-17283974 ] Attila Zsolt Piros commented on SPARK-34426: I am working on this. > Add driver and

[jira] [Created] (SPARK-34426) Add driver and executors POD logs to integration-tests log when the test fails

Attila Zsolt Piros created SPARK-34426: -- Summary: Add driver and executors POD logs to integration-tests log when the test fails Key: SPARK-34426 URL: https://issues.apache.org/jira/browse/SPARK-34426

[jira] [Updated] (SPARK-34426) Add driver and executors POD logs to integration tests log when the test fails

[ https://issues.apache.org/jira/browse/SPARK-34426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-34426: --- Summary: Add driver and executors POD logs to integration tests log when the test

[jira] [Commented] (SPARK-34433) Lock jekyll version by Gemfile and Bundler

[ https://issues.apache.org/jira/browse/SPARK-34433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17284287#comment-17284287 ] Attila Zsolt Piros commented on SPARK-34433: I am working on this. > Lock jekyll version by

[jira] [Created] (SPARK-34433) Lock jekyll version by Gemfile and Bundler

Attila Zsolt Piros created SPARK-34433: -- Summary: Lock jekyll version by Gemfile and Bundler Key: SPARK-34433 URL: https://issues.apache.org/jira/browse/SPARK-34433 Project: Spark Issue

[jira] [Comment Edited] (SPARK-34389) Spark job on Kubernetes scheduled For Zero or less than minimum number of executors and Wait indefinitely under resource starvation

[ https://issues.apache.org/jira/browse/SPARK-34389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282614#comment-17282614 ] Attila Zsolt Piros edited comment on SPARK-34389 at 2/12/21, 8:36 AM:

[jira] [Created] (SPARK-36719) Supporting Netty Logging at the network layer

Attila Zsolt Piros created SPARK-36719: -- Summary: Supporting Netty Logging at the network layer Key: SPARK-36719 URL: https://issues.apache.org/jira/browse/SPARK-36719 Project: Spark

[jira] [Commented] (SPARK-36719) Supporting Netty Logging at the network layer

[ https://issues.apache.org/jira/browse/SPARK-36719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17413167#comment-17413167 ] Attila Zsolt Piros commented on SPARK-36719: I am working on this and soon I will open a PR.

[jira] [Resolved] (SPARK-35334) Spark should be more resilient to intermittent K8s flakiness

[ https://issues.apache.org/jira/browse/SPARK-35334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros resolved SPARK-35334. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Commented] (SPARK-35334) Spark should be more resilient to intermittent K8s flakiness

[ https://issues.apache.org/jira/browse/SPARK-35334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380390#comment-17380390 ] Attila Zsolt Piros commented on SPARK-35334: [~dongjoon] oh sorry this is feature I just

[jira] [Updated] (SPARK-35334) Spark should be more resilient to intermittent K8s flakiness

[ https://issues.apache.org/jira/browse/SPARK-35334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-35334: --- Issue Type: New Feature (was: Bug) > Spark should be more resilient to

[jira] [Created] (SPARK-36358) Upgrade Kubernetes Client Version to 5.6.0

Attila Zsolt Piros created SPARK-36358: -- Summary: Upgrade Kubernetes Client Version to 5.6.0 Key: SPARK-36358 URL: https://issues.apache.org/jira/browse/SPARK-36358 Project: Spark Issue

[jira] [Updated] (SPARK-36358) Upgrade Kubernetes Client Version to 5.6.0

[ https://issues.apache.org/jira/browse/SPARK-36358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-36358: --- Description: This way [Retry HTTP operation in case IOException too (exponential

[jira] [Updated] (SPARK-36965) Extend python test runner by logging out the temp output files

[ https://issues.apache.org/jira/browse/SPARK-36965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-36965: --- Description: I was running a python test which was extremely slow and I was

[jira] [Updated] (SPARK-36965) Extend python test runner by logging out the temp output files

[ https://issues.apache.org/jira/browse/SPARK-36965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-36965: --- Description: I was running a python test which was extremely slow and I was

[jira] [Created] (SPARK-36965) Extend python test runner by logging out the temp output files

Attila Zsolt Piros created SPARK-36965: -- Summary: Extend python test runner by logging out the temp output files Key: SPARK-36965 URL: https://issues.apache.org/jira/browse/SPARK-36965 Project:

[jira] [Updated] (SPARK-36965) Extend python test runner by logging out the temp output files

[ https://issues.apache.org/jira/browse/SPARK-36965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-36965: --- Priority: Minor (was: Major) > Extend python test runner by logging out the temp

[jira] [Updated] (SPARK-36965) Extend python test runner by logging out the temp output files

[ https://issues.apache.org/jira/browse/SPARK-36965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-36965: --- Description: I was running a python test which was extremely slow and I was

[jira] [Resolved] (SPARK-36965) Extend python test runner by logging out the temp output files

[ https://issues.apache.org/jira/browse/SPARK-36965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros resolved SPARK-36965. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Resolved] (SPARK-36406) No longer do file truncate operation before delete a write failed file held by DiskBlockObjectWriter

[ https://issues.apache.org/jira/browse/SPARK-36406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros resolved SPARK-36406. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-36406) No longer do file truncate operation before delete a write failed file held by DiskBlockObjectWriter

[ https://issues.apache.org/jira/browse/SPARK-36406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros reassigned SPARK-36406: -- Assignee: Yang Jie > No longer do file truncate operation before delete a

[jira] [Created] (SPARK-37373) Collect LocalSparkContext worker logs in case of test failure

Attila Zsolt Piros created SPARK-37373: -- Summary: Collect LocalSparkContext worker logs in case of test failure Key: SPARK-37373 URL: https://issues.apache.org/jira/browse/SPARK-37373 Project:

[jira] [Updated] (SPARK-37373) Collect LocalSparkContext worker logs in case of test failure

[ https://issues.apache.org/jira/browse/SPARK-37373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-37373: --- Affects Version/s: 3.3.0 (was: 3.2.0) > Collect

[jira] [Resolved] (SPARK-35672) Spark fails to launch executors with very large user classpath lists on YARN

[ https://issues.apache.org/jira/browse/SPARK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros resolved SPARK-35672. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-36967) Report accurate shuffle block size if its skewed

[ https://issues.apache.org/jira/browse/SPARK-36967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros reassigned SPARK-36967: -- Assignee: Wan Kun (was: Apache Spark) > Report accurate shuffle block size

[jira] [Resolved] (SPARK-36967) Report accurate shuffle block size if its skewed

[ https://issues.apache.org/jira/browse/SPARK-36967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros resolved SPARK-36967. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Created] (SPARK-37881) Cleanup ShuffleBlockResolver from polluted methods to create a developer API

Attila Zsolt Piros created SPARK-37881: -- Summary: Cleanup ShuffleBlockResolver from polluted methods to create a developer API Key: SPARK-37881 URL: https://issues.apache.org/jira/browse/SPARK-37881

[jira] [Commented] (SPARK-33206) Spark Shuffle Index Cache calculates memory usage wrong

[ https://issues.apache.org/jira/browse/SPARK-33206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493745#comment-17493745 ] Attila Zsolt Piros commented on SPARK-33206: I am working on this soon a PR will be opened.

[jira] [Commented] (SPARK-34372) Speculation results in broken CSV files in Amazon S3

[ https://issues.apache.org/jira/browse/SPARK-34372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17482041#comment-17482041 ] Attila Zsolt Piros commented on SPARK-34372: hi [~daeheh]! Please look around here:

[jira] [Created] (SPARK-45265) Support Hive 4.0 metastore

Attila Zsolt Piros created SPARK-45265: -- Summary: Support Hive 4.0 metastore Key: SPARK-45265 URL: https://issues.apache.org/jira/browse/SPARK-45265 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-39473) NullPointerException in SparkRackResolver

[ https://issues.apache.org/jira/browse/SPARK-39473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros resolved SPARK-39473. Resolution: Invalid This is because of Scala 2.11 on latter version *asScala* is

[jira] [Created] (SPARK-39473) NullPointerException in SparkRackResolver

Attila Zsolt Piros created SPARK-39473: -- Summary: NullPointerException in SparkRackResolver Key: SPARK-39473 URL: https://issues.apache.org/jira/browse/SPARK-39473 Project: Spark Issue

[jira] [Created] (SPARK-39152) StreamCorruptedException cause job failure for disk persisted RDD

Attila Zsolt Piros created SPARK-39152: -- Summary: StreamCorruptedException cause job failure for disk persisted RDD Key: SPARK-39152 URL: https://issues.apache.org/jira/browse/SPARK-39152

[jira] [Commented] (SPARK-39152) StreamCorruptedException cause job failure for disk persisted RDD

[ https://issues.apache.org/jira/browse/SPARK-39152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17535044#comment-17535044 ] Attila Zsolt Piros commented on SPARK-39152: I am working on this >

[jira] [Created] (SPARK-40458) Bump Kubernetes Client Version to 6.1.1

Attila Zsolt Piros created SPARK-40458: -- Summary: Bump Kubernetes Client Version to 6.1.1 Key: SPARK-40458 URL: https://issues.apache.org/jira/browse/SPARK-40458 Project: Spark Issue

[jira] [Created] (SPARK-40617) Assertion failed in ExecutorMetricsPoller "task count shouldn't below 0"

Attila Zsolt Piros created SPARK-40617: -- Summary: Assertion failed in ExecutorMetricsPoller "task count shouldn't below 0" Key: SPARK-40617 URL: https://issues.apache.org/jira/browse/SPARK-40617

<    1   2   3   4   >