[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383529#comment-17383529 ] Thomas Graves commented on SPARK-25075: --- Just wanted to check the plans for scala 2.13 in 3.2.  It

[jira] [Commented] (SPARK-32333) Drop references to Master

[ https://issues.apache.org/jira/browse/SPARK-32333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17378093#comment-17378093 ] Thomas Graves commented on SPARK-32333: --- Getting back to this, now that spark 3.2 branch is cut,

[jira] [Commented] (SPARK-33031) scheduler with blacklisting doesn't appear to pick up new executor added

[ https://issues.apache.org/jira/browse/SPARK-33031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17371592#comment-17371592 ] Thomas Graves commented on SPARK-33031: --- I do not think its resolved but haven't tried it lately.  

[jira] [Resolved] (SPARK-35672) Spark fails to launch executors with very large user classpath lists on YARN

[ https://issues.apache.org/jira/browse/SPARK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-35672. --- Fix Version/s: 3.2.0 Assignee: Erik Krogen Resolution: Fixed > Spark fails

[jira] [Assigned] (SPARK-35391) Memory leak in ExecutorAllocationListener breaks dynamic allocation under high load

[ https://issues.apache.org/jira/browse/SPARK-35391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-35391: - Assignee: Vasily Kolpakov > Memory leak in ExecutorAllocationListener breaks dynamic

[jira] [Resolved] (SPARK-35391) Memory leak in ExecutorAllocationListener breaks dynamic allocation under high load

[ https://issues.apache.org/jira/browse/SPARK-35391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-35391. --- Fix Version/s: 3.1.3 3.2.0 Resolution: Fixed > Memory leak in

[jira] [Assigned] (SPARK-35074) spark.jars.xxx configs should be moved to config/package.scala

[ https://issues.apache.org/jira/browse/SPARK-35074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-35074: - Assignee: dc-heros > spark.jars.xxx configs should be moved to config/package.scala >

[jira] [Resolved] (SPARK-35074) spark.jars.xxx configs should be moved to config/package.scala

[ https://issues.apache.org/jira/browse/SPARK-35074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-35074. --- Fix Version/s: 3.2.0 Resolution: Fixed > spark.jars.xxx configs should be moved to

[jira] [Resolved] (SPARK-35093) AQE columnar mismatch on exchange reuse

[ https://issues.apache.org/jira/browse/SPARK-35093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-35093. --- Fix Version/s: 3.2.0 3.1.2 3.0.3 Assignee:

[jira] [Resolved] (SPARK-34472) SparkContext.addJar with an ivy path fails in cluster mode with a custom ivySettings file

[ https://issues.apache.org/jira/browse/SPARK-34472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-34472. --- Fix Version/s: 3.2.0 Assignee: Shardul Mahadik Resolution: Fixed >

[jira] [Resolved] (SPARK-34877) Add Spark AM Log link in case of master as yarn and deploy mode as client

[ https://issues.apache.org/jira/browse/SPARK-34877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-34877. --- Fix Version/s: 3.2.0 Assignee: Saurabh Chawla Resolution: Fixed > Add Spark

[jira] [Commented] (SPARK-35108) Pickle produces incorrect key labels for GenericRowWithSchema (data corruption)

[ https://issues.apache.org/jira/browse/SPARK-35108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17325252#comment-17325252 ] Thomas Graves commented on SPARK-35108: --- seems like a correctness issue in some cases so marking

[jira] [Updated] (SPARK-35108) Pickle produces incorrect key labels for GenericRowWithSchema (data corruption)

[ https://issues.apache.org/jira/browse/SPARK-35108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-35108: -- Labels: correctness (was: ) > Pickle produces incorrect key labels for GenericRowWithSchema

[jira] [Updated] (SPARK-35108) Pickle produces incorrect key labels for GenericRowWithSchema (data corruption)

[ https://issues.apache.org/jira/browse/SPARK-35108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-35108: -- Priority: Blocker (was: Major) > Pickle produces incorrect key labels for

[jira] [Commented] (SPARK-34989) Improve the performance of mapChildren and withNewChildren methods

[ https://issues.apache.org/jira/browse/SPARK-34989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17319679#comment-17319679 ] Thomas Graves commented on SPARK-34989: --- I just saw this go, all the performance numbers here are

[jira] [Resolved] (SPARK-34828) YARN Shuffle Service: Support configurability of aux service name and service-specific config overrides

[ https://issues.apache.org/jira/browse/SPARK-34828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-34828. --- Fix Version/s: 3.2.0 Assignee: Erik Krogen Resolution: Fixed > YARN Shuffle

[jira] [Commented] (SPARK-34645) [K8S] Driver pod stuck in Running state after job completes

[ https://issues.apache.org/jira/browse/SPARK-34645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17296340#comment-17296340 ] Thomas Graves commented on SPARK-34645: --- [~dongjoon] [~holden]. have you seen this at all? >

[jira] [Commented] (SPARK-33288) Support k8s cluster manager with stage level scheduling

[ https://issues.apache.org/jira/browse/SPARK-33288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17296060#comment-17296060 ] Thomas Graves commented on SPARK-33288: --- It looks like the version you are trying to launch

[jira] [Resolved] (SPARK-27495) SPIP: Support Stage level resource configuration and scheduling

[ https://issues.apache.org/jira/browse/SPARK-27495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-27495. --- Fix Version/s: 3.1.1 Target Version/s: (was: 3.2.0) Resolution: Fixed

[jira] [Commented] (SPARK-29329) maven incremental builds not working

[ https://issues.apache.org/jira/browse/SPARK-29329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17293005#comment-17293005 ] Thomas Graves commented on SPARK-29329: --- Yeah I know 4.3.0 made things better. Its still

[jira] [Commented] (SPARK-34461) Collect API feedback and maybe revise some APIs

[ https://issues.apache.org/jira/browse/SPARK-34461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286517#comment-17286517 ] Thomas Graves commented on SPARK-34461: --- So the discussions in the original SPIP and prs went back

[jira] [Resolved] (SPARK-33739) Jobs committed through the S3A Magic committer don't report the bytes written

[ https://issues.apache.org/jira/browse/SPARK-33739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-33739. --- Fix Version/s: 3.2.0 Assignee: Steve Loughran Resolution: Fixed > Jobs

[jira] [Commented] (SPARK-27658) Catalog API to load functions

[ https://issues.apache.org/jira/browse/SPARK-27658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281984#comment-17281984 ] Thomas Graves commented on SPARK-27658: --- can we link the SPIP to this  > Catalog API to load

[jira] [Commented] (SPARK-34167) Reading parquet with Decimal(8,2) written as a Decimal64 blows up

[ https://issues.apache.org/jira/browse/SPARK-34167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270199#comment-17270199 ] Thomas Graves commented on SPARK-34167: --- [~razajafri] can you finish detailing your finding as to

[jira] [Resolved] (SPARK-33741) Add minimum threshold speculation config

[ https://issues.apache.org/jira/browse/SPARK-33741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-33741. --- Fix Version/s: 3.2.0 Assignee: Sanket Chintapalli Resolution: Fixed > Add

[jira] [Commented] (SPARK-33741) Add minimum threshold speculation config

[ https://issues.apache.org/jira/browse/SPARK-33741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260852#comment-17260852 ] Thomas Graves commented on SPARK-33741: --- probably dup of 

[jira] [Commented] (SPARK-32165) SessionState leaks SparkListener with multiple SparkSession

[ https://issues.apache.org/jira/browse/SPARK-32165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259815#comment-17259815 ] Thomas Graves commented on SPARK-32165: --- In my opinion, I'd rather not have to go trolling through

[jira] [Commented] (SPARK-32165) SessionState leaks SparkListener with multiple SparkSession

[ https://issues.apache.org/jira/browse/SPARK-32165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259753#comment-17259753 ] Thomas Graves commented on SPARK-32165: --- could we add a bit more detail here as to what problem is

[jira] [Commented] (SPARK-33031) scheduler with blacklisting doesn't appear to pick up new executor added

[ https://issues.apache.org/jira/browse/SPARK-33031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258498#comment-17258498 ] Thomas Graves commented on SPARK-33031: --- ah that could be the case, but if that is true we

[jira] [Commented] (SPARK-33614) Fix the constant folding rule to skip it if the expression fails to execute

[ https://issues.apache.org/jira/browse/SPARK-33614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246678#comment-17246678 ] Thomas Graves commented on SPARK-33614: --- can you add a description here with more details.

[jira] [Created] (SPARK-33715) Error about cores being limiting resource confusing

Thomas Graves created SPARK-33715: - Summary: Error about cores being limiting resource confusing Key: SPARK-33715 URL: https://issues.apache.org/jira/browse/SPARK-33715 Project: Spark Issue

[jira] [Commented] (SPARK-27495) SPIP: Support Stage level resource configuration and scheduling

[ https://issues.apache.org/jira/browse/SPARK-27495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245951#comment-17245951 ] Thomas Graves commented on SPARK-27495: --- the main functionality is all in, it probably makes more

[jira] [Updated] (SPARK-33504) The application log in the Spark history server contains sensitive attributes such as password that should be redated instead of plain text

[ https://issues.apache.org/jira/browse/SPARK-33504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-33504: -- Fix Version/s: (was: 3.0.2) > The application log in the Spark history server contains

[jira] [Resolved] (SPARK-33504) The application log in the Spark history server contains sensitive attributes such as password that should be redated instead of plain text

[ https://issues.apache.org/jira/browse/SPARK-33504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-33504. --- Fix Version/s: 3.1.0 3.0.2 Assignee: akiyamaneko

[jira] [Commented] (SPARK-33544) explode should not filter when used with CreateArray

[ https://issues.apache.org/jira/browse/SPARK-33544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17238371#comment-17238371 ] Thomas Graves commented on SPARK-33544: --- I'm working on a patch for this. > explode should not

[jira] [Created] (SPARK-33544) explode should not filter when used with CreateArray

Thomas Graves created SPARK-33544: - Summary: explode should not filter when used with CreateArray Key: SPARK-33544 URL: https://issues.apache.org/jira/browse/SPARK-33544 Project: Spark Issue

[jira] [Created] (SPARK-33462) ResourceProfile use Int for memory in ExecutorResourcesOrDefaults

Thomas Graves created SPARK-33462: - Summary: ResourceProfile use Int for memory in ExecutorResourcesOrDefaults Key: SPARK-33462 URL: https://issues.apache.org/jira/browse/SPARK-33462 Project: Spark

[jira] [Resolved] (SPARK-33288) Support k8s cluster manager with stage level scheduling

[ https://issues.apache.org/jira/browse/SPARK-33288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-33288. --- Fix Version/s: 3.1.0 Assignee: Thomas Graves Resolution: Fixed > Support

[jira] [Created] (SPARK-33447) Stage level scheduling, allow specifying other spark configs via ResourceProfile

Thomas Graves created SPARK-33447: - Summary: Stage level scheduling, allow specifying other spark configs via ResourceProfile Key: SPARK-33447 URL: https://issues.apache.org/jira/browse/SPARK-33447

[jira] [Created] (SPARK-33406) k8s pyspark shell doesn't honor pyspark memory setting

Thomas Graves created SPARK-33406: - Summary: k8s pyspark shell doesn't honor pyspark memory setting Key: SPARK-33406 URL: https://issues.apache.org/jira/browse/SPARK-33406 Project: Spark

[jira] [Resolved] (SPARK-31711) Register the executor source with the metrics system when running in local mode.

[ https://issues.apache.org/jira/browse/SPARK-31711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-31711. --- Fix Version/s: 3.1.0 Resolution: Fixed > Register the executor source with the

[jira] [Assigned] (SPARK-31711) Register the executor source with the metrics system when running in local mode.

[ https://issues.apache.org/jira/browse/SPARK-31711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-31711: - Assignee: Luca Canali > Register the executor source with the metrics system when

[jira] [Resolved] (SPARK-32037) Rename blacklisting feature to avoid language with racist connotation

[ https://issues.apache.org/jira/browse/SPARK-32037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-32037. --- Fix Version/s: 3.1.0 Resolution: Fixed > Rename blacklisting feature to avoid

[jira] [Assigned] (SPARK-32037) Rename blacklisting feature to avoid language with racist connotation

[ https://issues.apache.org/jira/browse/SPARK-32037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-32037: - Assignee: Thomas Graves > Rename blacklisting feature to avoid language with racist

[jira] [Updated] (SPARK-32591) Add better api docs for stage level scheduling Resources

[ https://issues.apache.org/jira/browse/SPARK-32591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-32591: -- Description: A question came up when we added offheap memory to be able to set in a

[jira] [Commented] (SPARK-33288) Support k8s cluster manager with stage level scheduling

[ https://issues.apache.org/jira/browse/SPARK-33288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222932#comment-17222932 ] Thomas Graves commented on SPARK-33288: --- yes. Really when I say hanging it means k8s won't be able

[jira] [Commented] (SPARK-33288) Support k8s cluster manager with stage level scheduling

[ https://issues.apache.org/jira/browse/SPARK-33288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222917#comment-17222917 ] Thomas Graves commented on SPARK-33288: --- note there will be a couple of caveats here in that if

[jira] [Created] (SPARK-33288) Support k8s cluster manager with stage level scheduling

Thomas Graves created SPARK-33288: - Summary: Support k8s cluster manager with stage level scheduling Key: SPARK-33288 URL: https://issues.apache.org/jira/browse/SPARK-33288 Project: Spark

[jira] [Assigned] (SPARK-33088) Enhance ExecutorPlugin API to include methods for task start and end events

[ https://issues.apache.org/jira/browse/SPARK-33088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-33088: - Assignee: Samuel Souza > Enhance ExecutorPlugin API to include methods for task start

[jira] [Commented] (SPARK-33031) scheduler with blacklisting doesn't appear to pick up new executor added

[ https://issues.apache.org/jira/browse/SPARK-33031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204265#comment-17204265 ] Thomas Graves commented on SPARK-33031: --- I tried this again on yarn and now it seems to be working

[jira] [Updated] (SPARK-33031) scheduler with blacklisting doesn't appear to pick up new executor added

[ https://issues.apache.org/jira/browse/SPARK-33031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-33031: -- Description: I was running a test with blacklisting  standalone mode and all the executors

[jira] [Updated] (SPARK-33031) scheduler with blacklisting doesn't appear to pick up new executor added

[ https://issues.apache.org/jira/browse/SPARK-33031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-33031: -- Description: I was running a test with blacklisting on yarn (and standalone mode) and all

[jira] [Updated] (SPARK-33031) scheduler with blacklisting doesn't appear to pick up new executor added

[ https://issues.apache.org/jira/browse/SPARK-33031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-33031: -- Affects Version/s: 3.0.0 > scheduler with blacklisting doesn't appear to pick up new executor

[jira] [Updated] (SPARK-33031) scheduler with blacklisting doesn't appear to pick up new executor added

[ https://issues.apache.org/jira/browse/SPARK-33031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-33031: -- Affects Version/s: (was: 3.0.0) 3.1.0 > scheduler with

[jira] [Commented] (SPARK-33029) Standalone mode blacklist executors page UI marks driver as blacklisted

[ https://issues.apache.org/jira/browse/SPARK-33029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204259#comment-17204259 ] Thomas Graves commented on SPARK-33029: --- NOte I filed

[jira] [Created] (SPARK-33031) scheduler with blacklisting doesn't appear to pick up new executor added

Thomas Graves created SPARK-33031: - Summary: scheduler with blacklisting doesn't appear to pick up new executor added Key: SPARK-33031 URL: https://issues.apache.org/jira/browse/SPARK-33031 Project:

[jira] [Updated] (SPARK-33029) Standalone mode blacklist executors page UI marks driver as blacklisted

[ https://issues.apache.org/jira/browse/SPARK-33029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-33029: -- Description: I am running a spark shell on a 1 node standalone cluster.  I noticed that the

[jira] [Updated] (SPARK-33029) Standalone mode blacklist executors page UI marks driver as blacklisted

[ https://issues.apache.org/jira/browse/SPARK-33029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-33029: -- Attachment: Screen Shot 2020-09-29 at 1.53.37 PM.png > Standalone mode blacklist executors

[jira] [Updated] (SPARK-33029) Standalone mode blacklist executors page UI marks driver as blacklisted

[ https://issues.apache.org/jira/browse/SPARK-33029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-33029: -- Attachment: Screen Shot 2020-09-29 at 1.52.09 PM.png > Standalone mode blacklist executors

[jira] [Created] (SPARK-33029) Standalone mode blacklist executors page UI marks driver as blacklisted

Thomas Graves created SPARK-33029: - Summary: Standalone mode blacklist executors page UI marks driver as blacklisted Key: SPARK-33029 URL: https://issues.apache.org/jira/browse/SPARK-33029 Project:

[jira] [Commented] (SPARK-32935) File source V2: support bucketing

[ https://issues.apache.org/jira/browse/SPARK-32935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17198368#comment-17198368 ] Thomas Graves commented on SPARK-32935: --- sorry looks like my update went in same time as yours and

[jira] [Updated] (SPARK-32935) File source V2: support bucketing

[ https://issues.apache.org/jira/browse/SPARK-32935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-32935: -- Description: Datasource V2 does not currently support bucketed reads or writes similar to

[jira] [Updated] (SPARK-32935) File source V2: support bucketing

[ https://issues.apache.org/jira/browse/SPARK-32935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-32935: -- Description: Datasource V2 does not currently support bucketed reads similar to Datasource

[jira] [Updated] (SPARK-32935) File source V2: support bucketing

[ https://issues.apache.org/jira/browse/SPARK-32935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-32935: -- Description: Datasource V2 does not currently support bucketed reads similar to Datasource

[jira] [Commented] (SPARK-27589) Spark file source V2

[ https://issues.apache.org/jira/browse/SPARK-27589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17198339#comment-17198339 ] Thomas Graves commented on SPARK-27589: --- thanks for confirming and filing the jira, wanted to make

[jira] [Commented] (SPARK-27589) Spark file source V2

[ https://issues.apache.org/jira/browse/SPARK-27589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197669#comment-17197669 ] Thomas Graves commented on SPARK-27589: --- I'm guessing my question got missed - does it currently

[jira] [Commented] (SPARK-27589) Spark file source V2

[ https://issues.apache.org/jira/browse/SPARK-27589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197143#comment-17197143 ] Thomas Graves commented on SPARK-27589: --- somewhat related, I was looking through the v2 code for

[jira] [Commented] (SPARK-32898) totalExecutorRunTimeMs is too big

[ https://issues.apache.org/jira/browse/SPARK-32898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17196972#comment-17196972 ] Thomas Graves commented on SPARK-32898: --- [~linhongliu-db] can you please provide more of a

[jira] [Updated] (SPARK-32894) Timestamp cast in exernal orc table

[ https://issues.apache.org/jira/browse/SPARK-32894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-32894: -- Summary: Timestamp cast in exernal orc table (was: Timestamp cast in exernal ocr table) >

[jira] [Updated] (SPARK-32635) When pyspark.sql.functions.lit() function is used with dataframe cache, it returns wrong result

[ https://issues.apache.org/jira/browse/SPARK-32635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-32635: -- Labels: correct (was: ) > When pyspark.sql.functions.lit() function is used with dataframe

[jira] [Updated] (SPARK-32635) When pyspark.sql.functions.lit() function is used with dataframe cache, it returns wrong result

[ https://issues.apache.org/jira/browse/SPARK-32635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-32635: -- Labels: correctness (was: correct) > When pyspark.sql.functions.lit() function is used with

[jira] [Updated] (SPARK-32635) When pyspark.sql.functions.lit() function is used with dataframe cache, it returns wrong result

[ https://issues.apache.org/jira/browse/SPARK-32635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-32635: -- Priority: Blocker (was: Major) > When pyspark.sql.functions.lit() function is used with

[jira] [Commented] (SPARK-32037) Rename blacklisting feature to avoid language with racist connotation

[ https://issues.apache.org/jira/browse/SPARK-32037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17195531#comment-17195531 ] Thomas Graves commented on SPARK-32037: --- it is a good point about blocklist being typoed (but I

[jira] [Commented] (SPARK-32411) GPU Cluster Fail

[ https://issues.apache.org/jira/browse/SPARK-32411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194485#comment-17194485 ] Thomas Graves commented on SPARK-32411: --- [~chitralverma] if you are still having an issue please

[jira] [Created] (SPARK-32824) The error is confusing when resource .amount not provided

Thomas Graves created SPARK-32824: - Summary: The error is confusing when resource .amount not provided Key: SPARK-32824 URL: https://issues.apache.org/jira/browse/SPARK-32824 Project: Spark

[jira] [Created] (SPARK-32823) Standalone Master UI resources in use wrong

Thomas Graves created SPARK-32823: - Summary: Standalone Master UI resources in use wrong Key: SPARK-32823 URL: https://issues.apache.org/jira/browse/SPARK-32823 Project: Spark Issue Type:

[jira] [Commented] (SPARK-32823) Standalone Master UI resources in use wrong

[ https://issues.apache.org/jira/browse/SPARK-32823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192256#comment-17192256 ] Thomas Graves commented on SPARK-32823: --- I'm looking into this. > Standalone Master UI resources

[jira] [Commented] (SPARK-32037) Rename blacklisting feature to avoid language with racist connotation

[ https://issues.apache.org/jira/browse/SPARK-32037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17184059#comment-17184059 ] Thomas Graves commented on SPARK-32037: --- I started a thread on dev to get feedback: 

[jira] [Commented] (SPARK-32333) Drop references to Master

[ https://issues.apache.org/jira/browse/SPARK-32333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17184051#comment-17184051 ] Thomas Graves commented on SPARK-32333: --- I send email to the dev list to get feedback, some other

[jira] [Commented] (SPARK-32672) Data corruption in some cached compressed boolean columns

[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181867#comment-17181867 ] Thomas Graves commented on SPARK-32672: --- [~cltlfcjin]  Please do not be changing priority just

[jira] [Commented] (SPARK-32672) Data corruption in some cached compressed boolean columns

[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181468#comment-17181468 ] Thomas Graves commented on SPARK-32672: --- [~cloud_fan] [~ruifengz] > Data corruption in some

[jira] [Updated] (SPARK-32672) Data corruption in some cached compressed boolean columns

[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-32672: -- Affects Version/s: 3.0.1 > Data corruption in some cached compressed boolean columns >

[jira] [Updated] (SPARK-32672) Data corruption in some cached compressed boolean columns

[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-32672: -- Labels: correctness (was: ) > Data corruption in some cached compressed boolean columns >

[jira] [Comment Edited] (SPARK-32640) Spark 3.1 log(NaN) returns null instead of NaN

[ https://issues.apache.org/jira/browse/SPARK-32640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17180772#comment-17180772 ] Thomas Graves edited comment on SPARK-32640 at 8/19/20, 7:14 PM: - hmm,

[jira] [Comment Edited] (SPARK-32640) Spark 3.1 log(NaN) returns null instead of NaN

[ https://issues.apache.org/jira/browse/SPARK-32640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17180772#comment-17180772 ] Thomas Graves edited comment on SPARK-32640 at 8/19/20, 7:14 PM: - hmm,

[jira] [Commented] (SPARK-32640) Spark 3.1 log(NaN) returns null instead of NaN

[ https://issues.apache.org/jira/browse/SPARK-32640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17180772#comment-17180772 ] Thomas Graves commented on SPARK-32640: --- hmm, interesting, this is how my test was reproducing

[jira] [Commented] (SPARK-32640) Spark 3.1 log(NaN) returns null instead of NaN

[ https://issues.apache.org/jira/browse/SPARK-32640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17180533#comment-17180533 ] Thomas Graves commented on SPARK-32640: --- The problem is with 3.1.0 not, 3.0.0. The description

[jira] [Updated] (SPARK-32640) Spark 3.1 log(NaN) returns null instead of NaN

[ https://issues.apache.org/jira/browse/SPARK-32640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-32640: -- Description: I was testing Spark 3.1.0 and I noticed that if you take the log(NaN) it now

[jira] [Created] (SPARK-32640) Spark 3.1 log(NaN) returns null instead of NaN

Thomas Graves created SPARK-32640: - Summary: Spark 3.1 log(NaN) returns null instead of NaN Key: SPARK-32640 URL: https://issues.apache.org/jira/browse/SPARK-32640 Project: Spark Issue Type:

[jira] [Created] (SPARK-32591) Add better api docs for stage level scheduling Resources

Thomas Graves created SPARK-32591: - Summary: Add better api docs for stage level scheduling Resources Key: SPARK-32591 URL: https://issues.apache.org/jira/browse/SPARK-32591 Project: Spark

[jira] [Commented] (SPARK-32037) Rename blacklisting feature to avoid language with racist connotation

[ https://issues.apache.org/jira/browse/SPARK-32037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17170817#comment-17170817 ] Thomas Graves commented on SPARK-32037: --- allowlist and blocklist have been used by others. Seems

[jira] [Commented] (SPARK-32333) Drop references to Master

[ https://issues.apache.org/jira/browse/SPARK-32333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17170451#comment-17170451 ] Thomas Graves commented on SPARK-32333: --- What about renaming Master to be ApplicationManager or

[jira] [Updated] (SPARK-32332) AQE doesn't adequately allow for Columnar Processing extension

[ https://issues.apache.org/jira/browse/SPARK-32332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-32332: -- Fix Version/s: 3.0.1 > AQE doesn't adequately allow for Columnar Processing extension >

[jira] [Commented] (SPARK-27589) Spark file source V2

[ https://issues.apache.org/jira/browse/SPARK-27589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168133#comment-17168133 ] Thomas Graves commented on SPARK-27589: --- Hey, I see most of the sources are still in the 

[jira] [Resolved] (SPARK-32332) AQE doesn't adequately allow for Columnar Processing extension

[ https://issues.apache.org/jira/browse/SPARK-32332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-32332. --- Fix Version/s: 3.1.0 Assignee: Wenchen Fan Resolution: Fixed > AQE doesn't

[jira] [Resolved] (SPARK-30322) Add stage level scheduling docs

[ https://issues.apache.org/jira/browse/SPARK-30322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-30322. --- Fix Version/s: 3.1.0 Assignee: Thomas Graves Resolution: Fixed > Add stage

[jira] [Updated] (SPARK-30322) Add stage level scheduling docs

[ https://issues.apache.org/jira/browse/SPARK-30322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-30322: -- Description: Add stage level scheduling docs. > Add stage level scheduling docs >

[jira] [Commented] (SPARK-32470) Remove task result size check for shuffle map stage

[ https://issues.apache.org/jira/browse/SPARK-32470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167233#comment-17167233 ] Thomas Graves commented on SPARK-32470: --- Please add a description to the Jira as to why > Remove

[jira] [Resolved] (SPARK-32175) Fix the order between initialization for ExecutorPlugin and starting heartbeat thread

[ https://issues.apache.org/jira/browse/SPARK-32175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-32175. --- Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed > Fix the order

[jira] [Comment Edited] (SPARK-32429) Standalone Mode allow setting CUDA_VISIBLE_DEVICES on executor launch

[ https://issues.apache.org/jira/browse/SPARK-32429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166627#comment-17166627 ] Thomas Graves edited comment on SPARK-32429 at 7/28/20, 6:47 PM: - Yes so

<    1   2   3   4   5   6   7   8   9   10   >