[jira] [Updated] (SPARK-49010) Add unit tests for XML case sensitivity

2024-07-25 Thread Shujing Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shujing Yang updated SPARK-49010: - Description: Currently, XML respects the case sensitivity SQLConf (default to false) in the

[jira] [Created] (SPARK-49010) Add unit tests for XML case sensitivity

2024-07-25 Thread Shujing Yang (Jira)
Shujing Yang created SPARK-49010: Summary: Add unit tests for XML case sensitivity Key: SPARK-49010 URL: https://issues.apache.org/jira/browse/SPARK-49010 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-48397) Add data write time metric to FileFormatDataWriter/BasicWriteJobStatsTracker

2024-05-23 Thread Eric Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848824#comment-17848824 ] Eric Yang edited comment on SPARK-48397 at 5/23/24 6:38 AM: The PR:

[jira] [Commented] (SPARK-48397) Add data write time metric to FileFormatDataWriter/BasicWriteJobStatsTracker

2024-05-23 Thread Eric Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848824#comment-17848824 ] Eric Yang commented on SPARK-48397: --- I'm working on a PR for it. > Add data write time metric to

[jira] [Created] (SPARK-48397) Add data write time metric to FileFormatDataWriter/BasicWriteJobStatsTracker

2024-05-23 Thread Eric Yang (Jira)
Eric Yang created SPARK-48397: - Summary: Add data write time metric to FileFormatDataWriter/BasicWriteJobStatsTracker Key: SPARK-48397 URL: https://issues.apache.org/jira/browse/SPARK-48397 Project:

[jira] [Comment Edited] (SPARK-48298) Add TCP mode to StatsdSink

2024-05-15 Thread Eric Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846789#comment-17846789 ] Eric Yang edited comment on SPARK-48298 at 5/16/24 4:48 AM: PR:

[jira] [Updated] (SPARK-48298) Add TCP mode to StatsdSink

2024-05-15 Thread Eric Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated SPARK-48298: -- Summary: Add TCP mode to StatsdSink (was: StatsdSink supports TCP mode) > Add TCP mode to StatsdSink

[jira] [Updated] (SPARK-48298) StatsdSink supports TCP mode

2024-05-15 Thread Eric Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated SPARK-48298: -- Description: Currently, the StatsdSink in Spark supports UDP mode only, which is the default mode of

[jira] [Created] (SPARK-48298) StatsdSink supports TCP mode

2024-05-15 Thread Eric Yang (Jira)
Eric Yang created SPARK-48298: - Summary: StatsdSink supports TCP mode Key: SPARK-48298 URL: https://issues.apache.org/jira/browse/SPARK-48298 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-48298) StatsdSink supports TCP mode

2024-05-15 Thread Eric Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846789#comment-17846789 ] Eric Yang commented on SPARK-48298: --- I'm preparing a PR for it. > StatsdSink supports TCP mode >

[jira] [Commented] (SPARK-47017) Show metrics of the physical plan of RDDScanExec's internal RDD in the history server

2024-05-06 Thread Eric Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844145#comment-17844145 ] Eric Yang commented on SPARK-47017: --- I'm preparing a PR for it.  > Show metrics of the physical plan

[jira] [Updated] (SPARK-48100) [SQL][XML] Fix issues in skipping nested structure fields not selected in schema

2024-05-02 Thread Shujing Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shujing Yang updated SPARK-48100: - Description: Previously, the XML parser can't skip nested structure data fields effectively

[jira] [Updated] (SPARK-48100) [SQL][XML] Fix issues in skipping nested structure fields not selected in schema

2024-05-02 Thread Shujing Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shujing Yang updated SPARK-48100: - Summary: [SQL][XML] Fix issues in skipping nested structure fields not selected in schema

[jira] [Created] (SPARK-48100) [SQL][XML] Fix projection issue when there's a nested struct

2024-05-02 Thread Shujing Yang (Jira)
Shujing Yang created SPARK-48100: Summary: [SQL][XML] Fix projection issue when there's a nested struct Key: SPARK-48100 URL: https://issues.apache.org/jira/browse/SPARK-48100 Project: Spark

[jira] [Created] (SPARK-47373) [SQL] Match FileSourceScanLike to get metadata instead of FileSourceScanExec

2024-03-12 Thread Binjie Yang (Jira)
Binjie Yang created SPARK-47373: --- Summary: [SQL] Match FileSourceScanLike to get metadata instead of FileSourceScanExec Key: SPARK-47373 URL: https://issues.apache.org/jira/browse/SPARK-47373 Project:

[jira] [Created] (SPARK-47314) [DOC] Correct the ExternalSorter#writePartitionedMapOutput method comment

2024-03-06 Thread Binjie Yang (Jira)
Binjie Yang created SPARK-47314: --- Summary: [DOC] Correct the ExternalSorter#writePartitionedMapOutput method comment Key: SPARK-47314 URL: https://issues.apache.org/jira/browse/SPARK-47314 Project:

[jira] [Created] (SPARK-47309) [XML] Add schema inference unit tests

2024-03-06 Thread Shujing Yang (Jira)
Shujing Yang created SPARK-47309: Summary: [XML] Add schema inference unit tests Key: SPARK-47309 URL: https://issues.apache.org/jira/browse/SPARK-47309 Project: Spark Issue Type:

[jira] [Created] (SPARK-47293) Build batchSchema with total sparkSchema instead of append one by one

2024-03-05 Thread Binjie Yang (Jira)
Binjie Yang created SPARK-47293: --- Summary: Build batchSchema with total sparkSchema instead of append one by one Key: SPARK-47293 URL: https://issues.apache.org/jira/browse/SPARK-47293 Project: Spark

[jira] [Resolved] (SPARK-47204) [CORE] Check whether enabled checksum before delete checksum file

2024-02-27 Thread Binjie Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Binjie Yang resolved SPARK-47204. - Resolution: Not A Problem We will check whether the checksum file is exists or not before try

[jira] [Created] (SPARK-47204) [CORE] Check whether enabled checksum before delete checksum file

2024-02-27 Thread Binjie Yang (Jira)
Binjie Yang created SPARK-47204: --- Summary: [CORE] Check whether enabled checksum before delete checksum file Key: SPARK-47204 URL: https://issues.apache.org/jira/browse/SPARK-47204 Project: Spark

[jira] [Comment Edited] (SPARK-47017) Show metrics of the physical plan of RDDScanExec's internal RDD in the history server

2024-02-15 Thread Eric Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17817786#comment-17817786 ] Eric Yang edited comment on SPARK-47017 at 2/15/24 9:30 PM: Here is a simple

[jira] [Comment Edited] (SPARK-47017) Show metrics of the physical plan of RDDScanExec's internal RDD in the history server

2024-02-15 Thread Eric Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17817786#comment-17817786 ] Eric Yang edited comment on SPARK-47017 at 2/15/24 9:27 PM: Here is a simple

[jira] [Commented] (SPARK-47017) Show metrics of the physical plan of RDDScanExec's internal RDD in the history server

2024-02-15 Thread Eric Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17817786#comment-17817786 ] Eric Yang commented on SPARK-47017: --- Here is a simple example of this issue (based on the example code

[jira] [Updated] (SPARK-47017) Show metrics of the physical plan of RDDScanExec's internal RDD in the history server

2024-02-15 Thread Eric Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated SPARK-47017: -- Attachment: eventLogs-local-1708032228180.zip > Show metrics of the physical plan of RDDScanExec's

[jira] [Updated] (SPARK-47017) Show metrics of the physical plan of RDDScanExec's internal RDD in the history server

2024-02-15 Thread Eric Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated SPARK-47017: -- Attachment: simple2.scala > Show metrics of the physical plan of RDDScanExec's internal RDD in the >

[jira] [Updated] (SPARK-47017) Show metrics of the physical plan of RDDScanExec's internal RDD in the history server

2024-02-09 Thread Eric Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated SPARK-47017: -- Attachment: ScanExistingRDD.jpg > Show metrics of the physical plan of RDDScanExec's internal RDD in

[jira] [Updated] (SPARK-47017) Show metrics of the physical plan of RDDScanExec's internal RDD in the history server

2024-02-09 Thread Eric Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated SPARK-47017: -- Description: The RDDScanExec wraps an internal RDD (as below). In our environment, we find that this

[jira] [Created] (SPARK-47017) Show metrics of the physical plan of RDDScanExec's internal RDD in the history server

2024-02-09 Thread Eric Yang (Jira)
Eric Yang created SPARK-47017: - Summary: Show metrics of the physical plan of RDDScanExec's internal RDD in the history server Key: SPARK-47017 URL: https://issues.apache.org/jira/browse/SPARK-47017

[jira] [Created] (SPARK-46848) XML: Add support to partial results

2024-01-24 Thread Shujing Yang (Jira)
Shujing Yang created SPARK-46848: Summary: XML: Add support to partial results Key: SPARK-46848 URL: https://issues.apache.org/jira/browse/SPARK-46848 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-46382) XML: Capture values interspersed between elements

2023-12-12 Thread Shujing Yang (Jira)
Shujing Yang created SPARK-46382: Summary: XML: Capture values interspersed between elements Key: SPARK-46382 URL: https://issues.apache.org/jira/browse/SPARK-46382 Project: Spark Issue

[jira] [Created] (SPARK-46248) Support ignoreCorruptFiles and ignoreMissingFiles options in XML

2023-12-04 Thread Shujing Yang (Jira)
Shujing Yang created SPARK-46248: Summary: Support ignoreCorruptFiles and ignoreMissingFiles options in XML Key: SPARK-46248 URL: https://issues.apache.org/jira/browse/SPARK-46248 Project: Spark

[jira] [Created] (SPARK-46133) Refine ShuffleWriteProcessor write methods comment

2023-11-27 Thread Binjie Yang (Jira)
Binjie Yang created SPARK-46133: --- Summary: Refine ShuffleWriteProcessor write methods comment Key: SPARK-46133 URL: https://issues.apache.org/jira/browse/SPARK-46133 Project: Spark Issue Type:

[jira] [Created] (SPARK-45928) Fix schema merging for nested structures

2023-11-14 Thread Shujing Yang (Jira)
Shujing Yang created SPARK-45928: Summary: Fix schema merging for nested structures Key: SPARK-45928 URL: https://issues.apache.org/jira/browse/SPARK-45928 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45912) Enhancement of XSDToSchema API: Change to HDFS API for cloud storage accessibility

2023-11-13 Thread Shujing Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shujing Yang updated SPARK-45912: - Summary: Enhancement of XSDToSchema API: Change to HDFS API for cloud storage accessibility

[jira] [Created] (SPARK-45912) Enhancement of XSDToSchema API: Transit to HDFS API for cloud storage accessibility

2023-11-13 Thread Shujing Yang (Jira)
Shujing Yang created SPARK-45912: Summary: Enhancement of XSDToSchema API: Transit to HDFS API for cloud storage accessibility Key: SPARK-45912 URL: https://issues.apache.org/jira/browse/SPARK-45912

[jira] [Created] (SPARK-45844) Implement case insensitivity for XML

2023-11-08 Thread Shujing Yang (Jira)
Shujing Yang created SPARK-45844: Summary: Implement case insensitivity for XML Key: SPARK-45844 URL: https://issues.apache.org/jira/browse/SPARK-45844 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-45653) Refractor XMLSuite to allow other test suites to easily extend and override.

2023-10-24 Thread Shujing Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shujing Yang resolved SPARK-45653. -- Resolution: Not A Problem > Refractor XMLSuite to allow other test suites to easily extend

[jira] [Updated] (SPARK-45653) Refractor XMLSuite to allow other test suites to easily extend and override.

2023-10-24 Thread Shujing Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shujing Yang updated SPARK-45653: - Summary: Refractor XMLSuite to allow other test suites to easily extend and override. (was:

[jira] [Created] (SPARK-45653) Refractor XMLSuite

2023-10-24 Thread Shujing Yang (Jira)
Shujing Yang created SPARK-45653: Summary: Refractor XMLSuite Key: SPARK-45653 URL: https://issues.apache.org/jira/browse/SPARK-45653 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-45244) Correct spelling in VolcanoTestsSuite

2023-09-20 Thread Binjie Yang (Jira)
Binjie Yang created SPARK-45244: --- Summary: Correct spelling in VolcanoTestsSuite Key: SPARK-45244 URL: https://issues.apache.org/jira/browse/SPARK-45244 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-45205) Since version 3.2.0, Spark SQL has taken longer to execute "show paritions",probably because of changes introduced by SPARK-35278

2023-09-18 Thread Qiang Yang (Jira)
Qiang Yang created SPARK-45205: -- Summary: Since version 3.2.0, Spark SQL has taken longer to execute "show paritions",probably because of changes introduced by SPARK-35278 Key: SPARK-45205 URL:

[jira] [Created] (SPARK-44906) Move substituteAppNExecIds logic into kubernetesConf.annotations method

2023-08-21 Thread Binjie Yang (Jira)
Binjie Yang created SPARK-44906: --- Summary: Move substituteAppNExecIds logic into kubernetesConf.annotations method Key: SPARK-44906 URL: https://issues.apache.org/jira/browse/SPARK-44906 Project:

[jira] [Commented] (SPARK-43801) Support unwrap date type to string type in UnwrapCastInBinaryComparison

2023-05-26 Thread Pucheng Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17726671#comment-17726671 ] Pucheng Yang commented on SPARK-43801: -- created PR https://github.com/apache/spark/pull/41332 >

[jira] [Commented] (SPARK-43801) Support unwrap date type to string type in UnwrapCastInBinaryComparison

2023-05-25 Thread Pucheng Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17726460#comment-17726460 ] Pucheng Yang commented on SPARK-43801: -- [~yumwang] Thanks, I did not know we have this one. I have

[jira] [Updated] (SPARK-43801) Support unwrap date type to string type in UnwrapCastInBinaryComparison

2023-05-25 Thread Pucheng Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pucheng Yang updated SPARK-43801: - Summary: Support unwrap date type to string type in UnwrapCastInBinaryComparison (was: Support

[jira] [Created] (SPARK-43801) Support unwrap date type to string type

2023-05-25 Thread Pucheng Yang (Jira)
Pucheng Yang created SPARK-43801: Summary: Support unwrap date type to string type Key: SPARK-43801 URL: https://issues.apache.org/jira/browse/SPARK-43801 Project: Spark Issue Type:

[jira] [Updated] (SPARK-43800) [duplicated] Support unwrap date type to string type

2023-05-25 Thread Pucheng Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pucheng Yang updated SPARK-43800: - Summary: [duplicated] Support unwrap date type to string type (was: Support unwrap date type

[jira] [Resolved] (SPARK-43800) Support unwrap date type to string type

2023-05-25 Thread Pucheng Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pucheng Yang resolved SPARK-43800. -- Resolution: Invalid Ticket cloned can modify the assignee, will create a new one. > Support

[jira] [Created] (SPARK-43800) Support unwrap date type to string type

2023-05-25 Thread Pucheng Yang (Jira)
Pucheng Yang created SPARK-43800: Summary: Support unwrap date type to string type Key: SPARK-43800 URL: https://issues.apache.org/jira/browse/SPARK-43800 Project: Spark Issue Type:

[jira] [Updated] (SPARK-43800) Support unwrap date type to string type

2023-05-25 Thread Pucheng Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pucheng Yang updated SPARK-43800: - Fix Version/s: (was: 3.5.0) > Support unwrap date type to string type >

[jira] [Updated] (SPARK-43298) predict_batch_udf with scalar input fails when batch size consists of a single value

2023-04-26 Thread Lee Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lee Yang updated SPARK-43298: - Description: This is related to SPARK-42250.  For scalar inputs, the predict_batch_udf will fail if

[jira] [Updated] (SPARK-43298) predict_batch_udf with scalar input fails when batch size consists of a single value

2023-04-26 Thread Lee Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lee Yang updated SPARK-43298: - Description: This is related to SPARK-42250.  For scalar inputs, the predict_batch_udf will fail if

[jira] [Created] (SPARK-43298) predict_batch_udf with scalar input fails when batch size consists of a single value

2023-04-26 Thread Lee Yang (Jira)
Lee Yang created SPARK-43298: Summary: predict_batch_udf with scalar input fails when batch size consists of a single value Key: SPARK-43298 URL: https://issues.apache.org/jira/browse/SPARK-43298

[jira] [Updated] (SPARK-43221) Executor obtained error information

2023-04-20 Thread Qiang Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Yang updated SPARK-43221: --- Description: Spark on Yarn Cluster When multiple executors exist on a node, and the same block

[jira] [Updated] (SPARK-43221) Executor obtained error information

2023-04-20 Thread Qiang Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Yang updated SPARK-43221: --- Description: Spark on Yarn Cluster When multiple executors exist on a node, and the same block

[jira] [Updated] (SPARK-43221) Executor obtained error information

2023-04-20 Thread Qiang Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Yang updated SPARK-43221: --- Attachment: image-2023-04-21-00-57-29-140.png > Executor obtained error information >

[jira] [Updated] (SPARK-43221) Executor obtained error information

2023-04-20 Thread Qiang Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Yang updated SPARK-43221: --- Attachment: image-2023-04-21-00-54-11-968.png > Executor obtained error information >

[jira] [Updated] (SPARK-43221) Executor obtained error information

2023-04-20 Thread Qiang Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Yang updated SPARK-43221: --- Attachment: image-2023-04-21-00-53-20-720.png > Executor obtained error information >

[jira] [Updated] (SPARK-43221) Executor obtained error information

2023-04-20 Thread Qiang Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Yang updated SPARK-43221: --- Attachment: image-2023-04-21-00-50-10-918.png > Executor obtained error information >

[jira] [Updated] (SPARK-43221) Executor obtained error information

2023-04-20 Thread Qiang Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Yang updated SPARK-43221: --- Description: Spark on Yarn Cluster When multiple executors exist on a node, and the same block

[jira] [Updated] (SPARK-43221) Executor obtained error information

2023-04-20 Thread Qiang Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Yang updated SPARK-43221: --- Attachment: image-2023-04-21-00-30-41-851.png > Executor obtained error information >

[jira] [Updated] (SPARK-43221) Executor obtained error information

2023-04-20 Thread Qiang Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Yang updated SPARK-43221: --- Attachment: image-2023-04-21-00-24-22-059.png > Executor obtained error information >

[jira] [Updated] (SPARK-43221) Executor obtained error information

2023-04-20 Thread Qiang Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Yang updated SPARK-43221: --- Attachment: image-2023-04-21-00-19-58-021.png > Executor obtained error information >

[jira] [Updated] (SPARK-43221) Executor obtained error information

2023-04-20 Thread Qiang Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Yang updated SPARK-43221: --- Description: Spark on Yarn Cluster When multiple executors exist on a node, and the same block

[jira] [Updated] (SPARK-43221) Executor obtained error information

2023-04-20 Thread Qiang Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Yang updated SPARK-43221: --- Description: Spark on Yarn Cluster When multiple executors exist on a node, and the same block

[jira] [Updated] (SPARK-43221) Executor obtained error information

2023-04-20 Thread Qiang Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Yang updated SPARK-43221: --- Description: Spark on Yarn Cluster When multiple executors exist on a node, and the same block

[jira] [Updated] (SPARK-43221) Executor obtained error information

2023-04-20 Thread Qiang Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Yang updated SPARK-43221: --- Description: Spark on Yarn Cluster When multiple executors exist on a node, and the same block

[jira] [Created] (SPARK-43221) Executor obtained error information

2023-04-20 Thread Qiang Yang (Jira)
Qiang Yang created SPARK-43221: -- Summary: Executor obtained error information Key: SPARK-43221 URL: https://issues.apache.org/jira/browse/SPARK-43221 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-42972) ExecutorAllocationManager cannot allocate new instances when all executors down.

2023-03-29 Thread Jiandan Yang (Jira)
Jiandan Yang created SPARK-42972: - Summary: ExecutorAllocationManager cannot allocate new instances when all executors down. Key: SPARK-42972 URL: https://issues.apache.org/jira/browse/SPARK-42972

[jira] [Created] (SPARK-42785) [K8S][Core] When spark submit without --deploy-mode, will face NPE in Kubernetes Case

2023-03-14 Thread binjie yang (Jira)
binjie yang created SPARK-42785: --- Summary: [K8S][Core] When spark submit without --deploy-mode, will face NPE in Kubernetes Case Key: SPARK-42785 URL: https://issues.apache.org/jira/browse/SPARK-42785

[jira] [Commented] (SPARK-41780) `regexp_replace('', '[a\\\\d]{0, 2}', 'x')` causes an internal error

2023-01-03 Thread Remzi Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654311#comment-17654311 ] Remzi Yang commented on SPARK-41780: It is a bug I guess, because an internal error is returned.

[jira] [Updated] (SPARK-41780) `regexp_replace('', '[a\\\\d]{0, 2}', 'x')` causes an internal error

2022-12-29 Thread Remzi Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remzi Yang updated SPARK-41780: --- Description: {code:scala} scala> spark.sql("select regexp_replace('', '[ad]{0,2}', 'x')").show

[jira] [Updated] (SPARK-41780) `regexp_replace('', '[a\\\\d]{0, 2}', 'x')` causes an internal error

2022-12-29 Thread Remzi Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remzi Yang updated SPARK-41780: --- Description: {code:scala} scala> spark.sql("select regexp_replace('', '[ad]\{0,2}',

[jira] [Updated] (SPARK-41780) `regexp_replace('', '[a\\\\d]{0, 2}', 'x')` causes an internal error

2022-12-29 Thread Remzi Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remzi Yang updated SPARK-41780: --- Description: scala> spark.sql("select regexp_replace('', '[ad]\{0,2}', 'x')").show

[jira] [Created] (SPARK-41780) `regexp_replace('', '[a\\\\d]{0, 2}', 'x')` causes an internal error

2022-12-29 Thread Remzi Yang (Jira)
Remzi Yang created SPARK-41780: -- Summary: `regexp_replace('', '[ad]{0, 2}', 'x')` causes an internal error Key: SPARK-41780 URL: https://issues.apache.org/jira/browse/SPARK-41780 Project: Spark

[jira] [Updated] (SPARK-41246) When RddId exceeds Integer.Max_VALUE, RddId becomes negative

2022-12-11 Thread Qiang Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Yang updated SPARK-41246: --- Description: The incrementAndGet method of AtomicInteger keeps increasing. When AtomicInteger

[jira] [Created] (SPARK-41246) When RddId exceeds Integer.Max_VALUE, RddId becomes negative

2022-11-23 Thread Qiang Yang (Jira)
Qiang Yang created SPARK-41246: -- Summary: When RddId exceeds Integer.Max_VALUE, RddId becomes negative Key: SPARK-41246 URL: https://issues.apache.org/jira/browse/SPARK-41246 Project: Spark

[jira] [Created] (SPARK-40763) Should expose driver service name to config for user features

2022-10-11 Thread binjie yang (Jira)
binjie yang created SPARK-40763: --- Summary: Should expose driver service name to config for user features Key: SPARK-40763 URL: https://issues.apache.org/jira/browse/SPARK-40763 Project: Spark

[jira] [Updated] (SPARK-40667) Refactor File Data Source Options

2022-10-05 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40667: - Description: Currently for each file data source, all options are placed sparsely in the

[jira] [Updated] (SPARK-40667) Refactor File Data Source Options

2022-10-05 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40667: - Summary: Refactor File Data Source Options (was: Refactor Data Source Options) > Refactor

[jira] [Updated] (SPARK-40667) Refactor Data Source Options

2022-10-05 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40667: - Description: Currently for each data source, all options are placed sparsely in the options

[jira] [Updated] (SPARK-40667) Refactor Data Source Options

2022-10-05 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40667: - Description: Currently for each data source, all options are placed sparsely in the options

[jira] [Updated] (SPARK-40667) Refactor Data Source Options

2022-10-05 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40667: - Description: Currently for each data source, all options are placed sparsely in the options

[jira] [Updated] (SPARK-40667) Refactor Data Source Options

2022-10-05 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40667: - Description: Refactor data source options like `CSVOptions`, `JsonOptions` for better code

[jira] [Updated] (SPARK-40667) Refactor Data Source Options

2022-10-05 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40667: - Description: Refactor data source options like `CSVOptions`, `JsonOptions`. (was: Refactor

[jira] [Created] (SPARK-40667) Refactor Data Source Options

2022-10-05 Thread Xiaonan Yang (Jira)
Xiaonan Yang created SPARK-40667: Summary: Refactor Data Source Options Key: SPARK-40667 URL: https://issues.apache.org/jira/browse/SPARK-40667 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-40650) Infer date type for Json schema inference

2022-10-04 Thread Xiaonan Yang (Jira)
Xiaonan Yang created SPARK-40650: Summary: Infer date type for Json schema inference Key: SPARK-40650 URL: https://issues.apache.org/jira/browse/SPARK-40650 Project: Spark Issue Type:

[jira] [Updated] (SPARK-40649) Infer date type for Json schema inference

2022-10-04 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40649: - Fix Version/s: (was: 3.4.0) > Infer date type for Json schema inference >

[jira] [Resolved] (SPARK-40649) Infer date type for Json schema inference

2022-10-04 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang resolved SPARK-40649. -- Resolution: Duplicate > Infer date type for Json schema inference >

[jira] [Created] (SPARK-40649) Infer date type for Json schema inference

2022-10-04 Thread Xiaonan Yang (Jira)
Xiaonan Yang created SPARK-40649: Summary: Infer date type for Json schema inference Key: SPARK-40649 URL: https://issues.apache.org/jira/browse/SPARK-40649 Project: Spark Issue Type:

[jira] [Updated] (SPARK-40474) Correct CSV schema inference and data parsing behavior on columns with mixed dates and timestamps

2022-09-21 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40474: - Description: In this ticket https://issues.apache.org/jira/browse/SPARK-39469, we introduced

[jira] [Updated] (SPARK-40474) Correct CSV schema inference and data parsing behavior on columns with mixed dates and timestamps

2022-09-21 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40474: - Description: In this ticket https://issues.apache.org/jira/browse/SPARK-39469, we introduced

[jira] [Updated] (SPARK-40474) Correct CSV schema inference and data parsing behavior on columns with mixed dates and timestamps

2022-09-21 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40474: - Summary: Correct CSV schema inference and data parsing behavior on columns with mixed dates and

[jira] [Updated] (SPARK-40474) Infer columns with mixed date and timestamp as String in CSV schema inference

2022-09-18 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40474: - Description: In this ticket https://issues.apache.org/jira/browse/SPARK-39469, we introduced

[jira] [Updated] (SPARK-40474) Infer columns with mixed date and timestamp as String in CSV schema inference

2022-09-16 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40474: - Description: In this ticket, we introduced the support of date type in CSV schema inference.

[jira] [Updated] (SPARK-40474) Infer columns with mixed date and timestamp as String in CSV schema inference

2022-09-16 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40474: - Description: In this ticket, we introduced the support of date type in CSV schema inference.

[jira] [Updated] (SPARK-40474) Infer columns with mixed date and timestamp as String in CSV schema inference

2022-09-16 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40474: - Description: In this [ticket|https://issues.apache.org/jira/browse/SPARK-39469], we introduced

[jira] [Updated] (SPARK-40474) Infer columns with mixed date and timestamp as String in CSV schema inference

2022-09-16 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40474: - Shepherd: (was: Xiaonan Yang) > Infer columns with mixed date and timestamp as String in CSV

[jira] [Updated] (SPARK-40474) Infer columns with mixed date and timestamp as String in CSV schema inference

2022-09-16 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40474: - Shepherd: Xiaonan Yang > Infer columns with mixed date and timestamp as String in CSV schema

[jira] [Created] (SPARK-40474) Infer columns with mixed date and timestamp as String in CSV schema inference

2022-09-16 Thread Xiaonan Yang (Jira)
Xiaonan Yang created SPARK-40474: Summary: Infer columns with mixed date and timestamp as String in CSV schema inference Key: SPARK-40474 URL: https://issues.apache.org/jira/browse/SPARK-40474

  1   2   3   4   5   6   7   8   9   10   >