[jira] [Commented] (SPARK-45002) Avoid uncaught exception from state store maintenance task thread on error

[ https://issues.apache.org/jira/browse/SPARK-45002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759829#comment-17759829 ] Anish Shrigondekar commented on SPARK-45002: PR here:

[jira] [Created] (SPARK-45003) Refine docstring of `asc/desc`

Yang Jie created SPARK-45003: Summary: Refine docstring of `asc/desc` Key: SPARK-45003 URL: https://issues.apache.org/jira/browse/SPARK-45003 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-45002) Avoid uncaught exception from state store maintenance task thread on error

Anish Shrigondekar created SPARK-45002: -- Summary: Avoid uncaught exception from state store maintenance task thread on error Key: SPARK-45002 URL: https://issues.apache.org/jira/browse/SPARK-45002

[jira] [Resolved] (SPARK-44996) VolcanoFeatureStep should not create `DefaultVolcanoClient` if not needed

[ https://issues.apache.org/jira/browse/SPARK-44996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44996. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42710

[jira] [Assigned] (SPARK-44996) VolcanoFeatureStep should not create `DefaultVolcanoClient` if not needed

[ https://issues.apache.org/jira/browse/SPARK-44996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44996: - Assignee: Dongjoon Hyun > VolcanoFeatureStep should not create `DefaultVolcanoClient`

[jira] [Updated] (SPARK-44999) Refactor ExternalSorter to reduce checks on shouldPartition when calling getPartition

[ https://issues.apache.org/jira/browse/SPARK-44999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-44999: - Summary: Refactor ExternalSorter to reduce checks on shouldPartition when calling getPartition (was:

[jira] [Commented] (SPARK-44900) Cached DataFrame keeps growing

[ https://issues.apache.org/jira/browse/SPARK-44900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759821#comment-17759821 ] Yauheni Audzeichyk commented on SPARK-44900: [~yxzhang] looks like it is just disk usage

[jira] [Updated] (SPARK-44999) Refactor ExternalSorter#getPartition to reduce checks on shouldPartition

[ https://issues.apache.org/jira/browse/SPARK-44999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-44999: - Summary: Refactor ExternalSorter#getPartition to reduce checks on shouldPartition (was: Refactor

[jira] [Commented] (SPARK-44900) Cached DataFrame keeps growing

[ https://issues.apache.org/jira/browse/SPARK-44900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759818#comment-17759818 ] Yuexin Zhang commented on SPARK-44900: -- Hi [~varun2807] [~yaud] did you check the actual cached

[jira] [Created] (SPARK-45001) Implement DataFrame.foreachPartition

Hyukjin Kwon created SPARK-45001: Summary: Implement DataFrame.foreachPartition Key: SPARK-45001 URL: https://issues.apache.org/jira/browse/SPARK-45001 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-41279) Feature parity: DataFrame API in Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-41279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759803#comment-17759803 ] Hyukjin Kwon commented on SPARK-41279: -- See also https://github.com/apache/spark/pull/42714 >

[jira] [Commented] (SPARK-45000) Implement DataFrame.foreach

[ https://issues.apache.org/jira/browse/SPARK-45000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759802#comment-17759802 ] Snoot.io commented on SPARK-45000: -- User 'HyukjinKwon' has created a pull request for this issue:

[jira] [Created] (SPARK-45000) Implement DataFrame.foreach

Hyukjin Kwon created SPARK-45000: Summary: Implement DataFrame.foreach Key: SPARK-45000 URL: https://issues.apache.org/jira/browse/SPARK-45000 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-44999) Refactor `ExternalSorter#getPartition` to reduce the number of i`f else` judgments

[ https://issues.apache.org/jira/browse/SPARK-44999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-44999: - Description: {code:java}   private def getPartition(key: K): Int = {     if (shouldPartition)

[jira] [Commented] (SPARK-44999) Refactor `ExternalSorter#getPartition` to reduce the number of i`f else` judgments

[ https://issues.apache.org/jira/browse/SPARK-44999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759799#comment-17759799 ] Snoot.io commented on SPARK-44999: -- User 'LuciferYang' has created a pull request for this issue:

[jira] [Commented] (SPARK-44999) Refactor `ExternalSorter#getPartition` to reduce the number of i`f else` judgments

[ https://issues.apache.org/jira/browse/SPARK-44999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759798#comment-17759798 ] Snoot.io commented on SPARK-44999: -- User 'LuciferYang' has created a pull request for this issue:

[jira] [Updated] (SPARK-44999) Refactor `ExternalSorter#getPartition` to reduce the number of i`f else` judgments

[ https://issues.apache.org/jira/browse/SPARK-44999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-44999: - Description: {code:java}   private def getPartition(key: K): Int = {     if (shouldPartition)

[jira] [Created] (SPARK-44999) Refactor `ExternalSorter#getPartition` to reduce the number of i`f else` judgments

Yang Jie created SPARK-44999: Summary: Refactor `ExternalSorter#getPartition` to reduce the number of i`f else` judgments Key: SPARK-44999 URL: https://issues.apache.org/jira/browse/SPARK-44999 Project:

[jira] [Commented] (SPARK-44997) Align example order (Python -> Scala/Java -> R) in all Spark Doc Content

[ https://issues.apache.org/jira/browse/SPARK-44997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759794#comment-17759794 ] Snoot.io commented on SPARK-44997: -- User 'panbingkun' has created a pull request for this issue:

[jira] [Commented] (SPARK-41279) Feature parity: DataFrame API in Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-41279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759789#comment-17759789 ] Hyukjin Kwon commented on SPARK-41279: -- You can run {code} def wrapped(itr): for pandas_df in

[jira] [Assigned] (SPARK-43646) Make `connect` module daily test pass

[ https://issues.apache.org/jira/browse/SPARK-43646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-43646: Assignee: Yang Jie > Make `connect` module daily test pass >

[jira] [Resolved] (SPARK-43646) Make `connect` module daily test pass

[ https://issues.apache.org/jira/browse/SPARK-43646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-43646. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Updated] (SPARK-44998) No need to retry parsing event log path again when FileNotFoundException occurs

[ https://issues.apache.org/jira/browse/SPARK-44998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-44998: -- Description: I found a lot of retry parsing inprogress event log records in history server log. The

[jira] [Updated] (SPARK-44998) No need to retry parsing event log path again when FileNotFoundException occurs

[ https://issues.apache.org/jira/browse/SPARK-44998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-44998: -- Attachment: image-2023-08-29-10-47-08-027.png > No need to retry parsing event log path again when

[jira] [Updated] (SPARK-44998) No need to retry parsing event log path again when FileNotFoundException occurs

[ https://issues.apache.org/jira/browse/SPARK-44998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-44998: -- Attachment: image-2023-08-29-10-47-43-567.png > No need to retry parsing event log path again when

[jira] [Created] (SPARK-44998) No need to retry parsing event log path again when FileNotFoundException occurs

Zhen Wang created SPARK-44998: - Summary: No need to retry parsing event log path again when FileNotFoundException occurs Key: SPARK-44998 URL: https://issues.apache.org/jira/browse/SPARK-44998 Project:

[jira] [Resolved] (SPARK-44860) Implement SESSION_USER function

[ https://issues.apache.org/jira/browse/SPARK-44860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-44860. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42549

[jira] [Assigned] (SPARK-44965) Hide internal functions/variables from `pyspark.sql.functions`

[ https://issues.apache.org/jira/browse/SPARK-44965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-44965: - Assignee: Ruifeng Zheng > Hide internal functions/variables from

[jira] [Resolved] (SPARK-44965) Hide internal functions/variables from `pyspark.sql.functions`

[ https://issues.apache.org/jira/browse/SPARK-44965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-44965. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42680

[jira] [Created] (SPARK-44997) Align example order (Python -> Scala/Java -> R) in all Spark Doc Content

BingKun Pan created SPARK-44997: --- Summary: Align example order (Python -> Scala/Java -> R) in all Spark Doc Content Key: SPARK-44997 URL: https://issues.apache.org/jira/browse/SPARK-44997 Project:

[jira] [Assigned] (SPARK-44995) Promote SparkKubernetesClientFactory to DeveloperApi

[ https://issues.apache.org/jira/browse/SPARK-44995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44995: - Assignee: Dongjoon Hyun > Promote SparkKubernetesClientFactory to DeveloperApi >

[jira] [Resolved] (SPARK-44995) Promote SparkKubernetesClientFactory to DeveloperApi

[ https://issues.apache.org/jira/browse/SPARK-44995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44995. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42709

[jira] [Assigned] (SPARK-44993) Add ShuffleChecksumUtils.compareChecksums by reusing ShuffleChecksumTestHelp.compareChecksums

[ https://issues.apache.org/jira/browse/SPARK-44993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44993: - Assignee: Dongjoon Hyun > Add ShuffleChecksumUtils.compareChecksums by reusing >

[jira] [Resolved] (SPARK-44993) Add ShuffleChecksumUtils.compareChecksums by reusing ShuffleChecksumTestHelp.compareChecksums

[ https://issues.apache.org/jira/browse/SPARK-44993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44993. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42707

[jira] [Updated] (SPARK-44996) VolcanoFeatureStep should not create `DefaultVolcanoClient` if not needed

[ https://issues.apache.org/jira/browse/SPARK-44996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44996: -- Description: Since `VolcanoFeatureStep` creates `DefaultVolcanoClient` always, the unit test

[jira] [Updated] (SPARK-44996) VolcanoFeatureStep should not create `DefaultVolcanoClient` if not needed

[ https://issues.apache.org/jira/browse/SPARK-44996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44996: -- Description: Since `VolcanoFeatureStep` creates `DefaultVolcanoClient` always, the unit test

[jira] [Created] (SPARK-44996) VolcanoFeatureStep should not create `DefaultVolcanoClient` if not needed

Dongjoon Hyun created SPARK-44996: - Summary: VolcanoFeatureStep should not create `DefaultVolcanoClient` if not needed Key: SPARK-44996 URL: https://issues.apache.org/jira/browse/SPARK-44996 Project:

[jira] [Created] (SPARK-44995) Promote SparkKubernetesClientFactory to DeveloperApi

Dongjoon Hyun created SPARK-44995: - Summary: Promote SparkKubernetesClientFactory to DeveloperApi Key: SPARK-44995 URL: https://issues.apache.org/jira/browse/SPARK-44995 Project: Spark Issue

[jira] [Updated] (SPARK-44994) Refine docstring of `DataFrame.filter`

[ https://issues.apache.org/jira/browse/SPARK-44994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-44994: - Summary: Refine docstring of `DataFrame.filter` (was: Refine docstring for `DataFrame.filter`)

[jira] [Updated] (SPARK-44993) Add ShuffleChecksumUtils.compareChecksums by reusing ShuffleChecksumTestHelp.compareChecksums

[ https://issues.apache.org/jira/browse/SPARK-44993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44993: -- Summary: Add ShuffleChecksumUtils.compareChecksums by reusing

[jira] [Created] (SPARK-44994) Refine the docstring of `DataFrame.filter`

Allison Wang created SPARK-44994: Summary: Refine the docstring of `DataFrame.filter` Key: SPARK-44994 URL: https://issues.apache.org/jira/browse/SPARK-44994 Project: Spark Issue Type:

[jira] [Updated] (SPARK-44994) Refine docstring for `DataFrame.filter`

[ https://issues.apache.org/jira/browse/SPARK-44994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-44994: - Summary: Refine docstring for `DataFrame.filter` (was: Refine the docstring of

[jira] [Updated] (SPARK-44993) Move compareChecksums from ShuffleChecksumTestHelpe to ShuffleChecksumUtils

[ https://issues.apache.org/jira/browse/SPARK-44993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44993: -- Summary: Move compareChecksums from ShuffleChecksumTestHelpe to ShuffleChecksumUtils (was:

[jira] [Created] (SPARK-44993) Move compareChecksums from ShuffleChecksumTestHelpe to ShuffleChecksumUtils and move compareChecksums

Dongjoon Hyun created SPARK-44993: - Summary: Move compareChecksums from ShuffleChecksumTestHelpe to ShuffleChecksumUtils and move compareChecksums Key: SPARK-44993 URL:

[jira] [Commented] (SPARK-41279) Feature parity: DataFrame API in Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-41279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759732#comment-17759732 ] Johannes Alberti commented on SPARK-41279: -- [~gurwls223] thank you for your response. When

[jira] [Created] (SPARK-44992) Add support for rack information from an environment variable

Holden Karau created SPARK-44992: Summary: Add support for rack information from an environment variable Key: SPARK-44992 URL: https://issues.apache.org/jira/browse/SPARK-44992 Project: Spark

[jira] [Commented] (SPARK-44900) Cached DataFrame keeps growing

[ https://issues.apache.org/jira/browse/SPARK-44900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759703#comment-17759703 ] Varun Nalla commented on SPARK-44900: - [~yao] hope you got a chance to look into what [~yaud]

[jira] [Updated] (SPARK-44991) Spark json schema inference and fromJson api having inconsistent behavior

[ https://issues.apache.org/jira/browse/SPARK-44991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nirav patel updated SPARK-44991: Summary: Spark json schema inference and fromJson api having inconsistent behavior (was: Spark

[jira] [Updated] (SPARK-44991) Spark json datasource reader and fromJson api having inconsistent behavior

[ https://issues.apache.org/jira/browse/SPARK-44991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nirav patel updated SPARK-44991: Description: Spark json reader can infer datatype of a fields. I am ingesting millions of

[jira] [Created] (SPARK-44991) Spark json datasource reader and fromJson api having inconsistent behavior

nirav patel created SPARK-44991: --- Summary: Spark json datasource reader and fromJson api having inconsistent behavior Key: SPARK-44991 URL: https://issues.apache.org/jira/browse/SPARK-44991 Project:

[jira] [Assigned] (SPARK-44989) Add a directional message to promote JIRA_ACCESS_TOKEN

[ https://issues.apache.org/jira/browse/SPARK-44989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44989: - Assignee: Dongjoon Hyun > Add a directional message to promote JIRA_ACCESS_TOKEN >

[jira] [Resolved] (SPARK-44989) Add a directional message to promote JIRA_ACCESS_TOKEN

[ https://issues.apache.org/jira/browse/SPARK-44989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44989. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42704

[jira] [Resolved] (SPARK-44832) Fix connect client transitive classpath

[ https://issues.apache.org/jira/browse/SPARK-44832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-44832. --- Fix Version/s: 3.5.0 Assignee: Herman van Hövell Resolution: Fixed

[jira] [Created] (SPARK-44989) Add a directional message to promote JIRA_ACCESS_TOKEN

Dongjoon Hyun created SPARK-44989: - Summary: Add a directional message to promote JIRA_ACCESS_TOKEN Key: SPARK-44989 URL: https://issues.apache.org/jira/browse/SPARK-44989 Project: Spark

[jira] [Resolved] (SPARK-44985) Use toString instead of stacktrace for task reaper threadDump

[ https://issues.apache.org/jira/browse/SPARK-44985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44985. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42699

[jira] [Assigned] (SPARK-44985) Use toString instead of stacktrace for task reaper threadDump

[ https://issues.apache.org/jira/browse/SPARK-44985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44985: - Assignee: Kent Yao > Use toString instead of stacktrace for task reaper threadDump >

[jira] [Assigned] (SPARK-44972) Eagerly check if the token is valid to align with the behavior of username/password auth

[ https://issues.apache.org/jira/browse/SPARK-44972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44972: - Assignee: Kent Yao > Eagerly check if the token is valid to align with the behavior of

[jira] [Resolved] (SPARK-44972) Eagerly check if the token is valid to align with the behavior of username/password auth

[ https://issues.apache.org/jira/browse/SPARK-44972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44972. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42625

[jira] [Created] (SPARK-44988) Parquet INT64 (TIMESTAMP(NANOS,false)) throwing Illegal Parquet type

Flavio Odas created SPARK-44988: --- Summary: Parquet INT64 (TIMESTAMP(NANOS,false)) throwing Illegal Parquet type Key: SPARK-44988 URL: https://issues.apache.org/jira/browse/SPARK-44988 Project: Spark

[jira] [Commented] (SPARK-44805) Data lost after union using spark.sql.parquet.enableNestedColumnVectorizedReader=true

[ https://issues.apache.org/jira/browse/SPARK-44805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759615#comment-17759615 ] Jakub Wozniak commented on SPARK-44805: --- Hello, Is it possible to know any ETA on this one? Is

[jira] [Resolved] (SPARK-44983) Convert binary to string by to_char for the formats: hex, base64, utf-8

[ https://issues.apache.org/jira/browse/SPARK-44983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-44983. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42632

[jira] [Resolved] (SPARK-44974) Replace SparkSession/Dataset/KeyValueGroupedDataset with null during serialization

[ https://issues.apache.org/jira/browse/SPARK-44974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-44974. --- Fix Version/s: 3.5.0 Resolution: Fixed > Replace

[jira] [Updated] (SPARK-44976) Preserve full principal user name on executor side

[ https://issues.apache.org/jira/browse/SPARK-44976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YUBI LEE updated SPARK-44976: - Summary: Preserve full principal user name on executor side (was: Utils.getCurrentUserName should

[jira] [Assigned] (SPARK-44984) Remove _get_alias from DataFrame

[ https://issues.apache.org/jira/browse/SPARK-44984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-44984: - Assignee: Ruifeng Zheng > Remove _get_alias from DataFrame >

[jira] [Resolved] (SPARK-44984) Remove _get_alias from DataFrame

[ https://issues.apache.org/jira/browse/SPARK-44984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-44984. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42698

[jira] [Comment Edited] (SPARK-42905) pyspark.ml.stat.Correlation - Spearman Correlation method giving incorrect and inconsistent results for the same DataFrame if it has huge amount of Ties.

[ https://issues.apache.org/jira/browse/SPARK-42905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757858#comment-17757858 ] zhangzhenhao edited comment on SPARK-42905 at 8/28/23 11:04 AM: minimal

[jira] [Updated] (SPARK-44987) Assign name to the error class _LEGACY_ERROR_TEMP_1100

[ https://issues.apache.org/jira/browse/SPARK-44987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-44987: - Description: Assign a name and improve the error message format. > Assign name to the error class

[jira] [Updated] (SPARK-44987) Assign name to the error class _LEGACY_ERROR_TEMP_1100

[ https://issues.apache.org/jira/browse/SPARK-44987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-44987: - Reporter: Max Gekk (was: BingKun Pan) > Assign name to the error class _LEGACY_ERROR_TEMP_1100 >

[jira] [Assigned] (SPARK-44987) Assign name to the error class _LEGACY_ERROR_TEMP_1100

[ https://issues.apache.org/jira/browse/SPARK-44987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-44987: Assignee: Max Gekk > Assign name to the error class _LEGACY_ERROR_TEMP_1100 >

[jira] [Created] (SPARK-44987) Assign name to the error class _LEGACY_ERROR_TEMP_1100[1017,1073,1074,1076,1125,1126]

Max Gekk created SPARK-44987: Summary: Assign name to the error class _LEGACY_ERROR_TEMP_1100[1017,1073,1074,1076,1125,1126] Key: SPARK-44987 URL: https://issues.apache.org/jira/browse/SPARK-44987

[jira] [Updated] (SPARK-44987) Assign name to the error class _LEGACY_ERROR_TEMP_1100

[ https://issues.apache.org/jira/browse/SPARK-44987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-44987: - Summary: Assign name to the error class _LEGACY_ERROR_TEMP_1100 (was: Assign name to the error class

[jira] [Commented] (SPARK-44983) Convert binary to string by to_char for the formats: hex, base64, utf-8

[ https://issues.apache.org/jira/browse/SPARK-44983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759530#comment-17759530 ] Hudson commented on SPARK-44983: User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-44986) There should be a gap at the bottom of the HTML

[ https://issues.apache.org/jira/browse/SPARK-44986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759520#comment-17759520 ] ASF GitHub Bot commented on SPARK-44986: User 'panbingkun' has created a pull request for this

[jira] [Updated] (SPARK-44986) There should be a gap at the bottom of the HTML

[ https://issues.apache.org/jira/browse/SPARK-44986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-44986: Attachment: image-2023-08-28-16-47-11-582.png > There should be a gap at the bottom of the HTML >

[jira] [Updated] (SPARK-44986) There should be a gap at the bottom of the HTML

[ https://issues.apache.org/jira/browse/SPARK-44986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-44986: Description: Before: !image-2023-08-28-16-47-11-582.png|width=794,height=392!   After:

[jira] [Updated] (SPARK-44986) There should be a gap at the bottom of the HTML

[ https://issues.apache.org/jira/browse/SPARK-44986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-44986: Attachment: image-2023-08-28-16-46-04-705.png > There should be a gap at the bottom of the HTML >

[jira] [Created] (SPARK-44986) There should be a gap at the bottom of the HTML

BingKun Pan created SPARK-44986: --- Summary: There should be a gap at the bottom of the HTML Key: SPARK-44986 URL: https://issues.apache.org/jira/browse/SPARK-44986 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-44982) Mark Spark Connect configurations as static configuration

[ https://issues.apache.org/jira/browse/SPARK-44982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44982. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-44982) Mark Spark Connect configurations as static configuration

[ https://issues.apache.org/jira/browse/SPARK-44982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44982: Assignee: Hyukjin Kwon > Mark Spark Connect configurations as static configuration >

[jira] [Resolved] (SPARK-44981) Filter out static configurations used in local mode

[ https://issues.apache.org/jira/browse/SPARK-44981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44981. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-44981) Filter out static configurations used in local mode

[ https://issues.apache.org/jira/browse/SPARK-44981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44981: Assignee: Hyukjin Kwon > Filter out static configurations used in local mode >

[jira] [Created] (SPARK-44985) Use toString instead of stacktrace for task reaper threadDump

Kent Yao created SPARK-44985: Summary: Use toString instead of stacktrace for task reaper threadDump Key: SPARK-44985 URL: https://issues.apache.org/jira/browse/SPARK-44985 Project: Spark Issue

[jira] [Created] (SPARK-44984) Remove _get_alias from DataFrame

Ruifeng Zheng created SPARK-44984: - Summary: Remove _get_alias from DataFrame Key: SPARK-44984 URL: https://issues.apache.org/jira/browse/SPARK-44984 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-44819) Make Python the first language in all Spark code snippet

[ https://issues.apache.org/jira/browse/SPARK-44819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759468#comment-17759468 ] BingKun Pan edited comment on SPARK-44819 at 8/28/23 7:36 AM: -- This PR is

[jira] [Created] (SPARK-44983) Convert binary to string by to_char for the formats: hex, base64, utf-8

Max Gekk created SPARK-44983: Summary: Convert binary to string by to_char for the formats: hex, base64, utf-8 Key: SPARK-44983 URL: https://issues.apache.org/jira/browse/SPARK-44983 Project: Spark

[jira] [Assigned] (SPARK-44980) Fix inherited namedtuples to work in createDataFrame

[ https://issues.apache.org/jira/browse/SPARK-44980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44980: Assignee: Hyukjin Kwon > Fix inherited namedtuples to work in createDataFrame >

[jira] [Resolved] (SPARK-44980) Fix inherited namedtuples to work in createDataFrame

[ https://issues.apache.org/jira/browse/SPARK-44980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44980. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Commented] (SPARK-44819) Make Python the first language in all Spark code snippet

[ https://issues.apache.org/jira/browse/SPARK-44819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759468#comment-17759468 ] BingKun Pan commented on SPARK-44819: - I work on it. > Make Python the first language in all Spark

[jira] [Commented] (SPARK-44862) Adding metric tracking number of jobs running on a cluster

[ https://issues.apache.org/jira/browse/SPARK-44862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759459#comment-17759459 ] Kent Yao commented on SPARK-44862: -- There is a gauge named activeJobs in the DAGSchedulerSource. Is it