[jira] [Resolved] (SPARK-38557) What may be a cause for HDFSMetadataCommitter: Error while fetching MetaData and how to fix or work around this?

2022-03-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38557. -- Resolution: Invalid > What may be a cause for HDFSMetadataCommitter: Error while fetching

[jira] [Commented] (SPARK-38557) What may be a cause for HDFSMetadataCommitter: Error while fetching MetaData and how to fix or work around this?

2022-03-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507976#comment-17507976 ] Hyukjin Kwon commented on SPARK-38557: -- For questions, we should better interact with Spark mailing

[jira] [Updated] (SPARK-38578) Avoid unnecessary sort in FileFormatWriter if user has specified sort in AQE

2022-03-16 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-38578: -- Summary: Avoid unnecessary sort in FileFormatWriter if user has specified sort in AQE (was: Avoid

[jira] [Updated] (SPARK-38581) List of supported pandas APIs for pandas API on Spark docs.

2022-03-16 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-38581: Issue Type: Documentation (was: Bug) > List of supported pandas APIs for pandas API on Spark

[jira] [Commented] (SPARK-38581) List of supported pandas APIs for pandas API on Spark docs.

2022-03-16 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507962#comment-17507962 ] Haejoon Lee commented on SPARK-38581: - I'm working on it > List of supported pandas APIs for pandas

[jira] [Created] (SPARK-38581) List of supported pandas APIs for pandas API on Spark docs.

2022-03-16 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-38581: --- Summary: List of supported pandas APIs for pandas API on Spark docs. Key: SPARK-38581 URL: https://issues.apache.org/jira/browse/SPARK-38581 Project: Spark

[jira] [Commented] (SPARK-38579) Requesting Restful API can cause NullPointerException

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507958#comment-17507958 ] Apache Spark commented on SPARK-38579: -- User 'yym1995' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38579) Requesting Restful API can cause NullPointerException

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38579: Assignee: Apache Spark > Requesting Restful API can cause NullPointerException >

[jira] [Assigned] (SPARK-38579) Requesting Restful API can cause NullPointerException

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38579: Assignee: (was: Apache Spark) > Requesting Restful API can cause

[jira] [Resolved] (SPARK-38580) Requesting Restful API can cause NullPointerException

2022-03-16 Thread Yimin Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yimin Yang resolved SPARK-38580. Resolution: Duplicate > Requesting Restful API can cause NullPointerException >

[jira] [Created] (SPARK-38580) Requesting Restful API can cause NullPointerException

2022-03-16 Thread Yimin Yang (Jira)
Yimin Yang created SPARK-38580: -- Summary: Requesting Restful API can cause NullPointerException Key: SPARK-38580 URL: https://issues.apache.org/jira/browse/SPARK-38580 Project: Spark Issue

[jira] [Created] (SPARK-38579) Requesting Restful API can cause NullPointerException

2022-03-16 Thread Yimin Yang (Jira)
Yimin Yang created SPARK-38579: -- Summary: Requesting Restful API can cause NullPointerException Key: SPARK-38579 URL: https://issues.apache.org/jira/browse/SPARK-38579 Project: Spark Issue

[jira] [Commented] (SPARK-38575) Duduplicate branch specification in GitHub Actions workflow

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507946#comment-17507946 ] Apache Spark commented on SPARK-38575: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-38578) Avoid unnecessary sort in FileFormatWriter if user has specified sort

2022-03-16 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-38578: -- Parent: SPARK-37063 Issue Type: Sub-task (was: Improvement) > Avoid unnecessary sort in

[jira] [Created] (SPARK-38578) Avoid unnecessary sort in FileFormatWriter if user has specified sort

2022-03-16 Thread XiDuo You (Jira)
XiDuo You created SPARK-38578: - Summary: Avoid unnecessary sort in FileFormatWriter if user has specified sort Key: SPARK-38578 URL: https://issues.apache.org/jira/browse/SPARK-38578 Project: Spark

[jira] [Assigned] (SPARK-38575) Duduplicate branch specification in GitHub Actions workflow

2022-03-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38575: Assignee: Hyukjin Kwon > Duduplicate branch specification in GitHub Actions workflow >

[jira] [Resolved] (SPARK-38575) Duduplicate branch specification in GitHub Actions workflow

2022-03-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38575. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 35882

[jira] [Updated] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38568: Reporter: Yuming Wang (was: Dongjoon Hyun) > Upgrade ZSTD-JNI to 1.5.2-2 >

[jira] [Commented] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507942#comment-17507942 ] Yuming Wang commented on SPARK-38568: - OK > Upgrade ZSTD-JNI to 1.5.2-2 >

[jira] [Updated] (SPARK-38577) Interval types are not truncated to the expected endField when creating a DataFrame via Duration

2022-03-16 Thread chong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chong updated SPARK-38577: -- Description: *Problem:* ANSI interval types are store as long internally. The long value are not truncated

[jira] [Updated] (SPARK-38520) Overflow occurs when reading ANSI day time interval from CSV file

2022-03-16 Thread chong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chong updated SPARK-38520: -- Description: *Problem:* Overflow occurs when reading the following positive intervals, the results become

[jira] [Updated] (SPARK-38520) Overflow occurs when reading ANSI day time interval from CSV file

2022-03-16 Thread chong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chong updated SPARK-38520: -- Description: *Problem:* Overflow occurs when reading the following positive intervals, the results become

[jira] [Assigned] (SPARK-38576) Implement `numeric_only` parameter for `DataFrame/Series.rank` to rank numeric columns only

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38576: Assignee: (was: Apache Spark) > Implement `numeric_only` parameter for

[jira] [Assigned] (SPARK-38576) Implement `numeric_only` parameter for `DataFrame/Series.rank` to rank numeric columns only

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38576: Assignee: Apache Spark > Implement `numeric_only` parameter for `DataFrame/Series.rank`

[jira] [Commented] (SPARK-38576) Implement `numeric_only` parameter for `DataFrame/Series.rank` to rank numeric columns only

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507931#comment-17507931 ] Apache Spark commented on SPARK-38576: -- User 'xinrong-databricks' has created a pull request for

[jira] [Commented] (SPARK-38576) Implement `numeric_only` parameter for `DataFrame/Series.rank` to rank numeric columns only

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507930#comment-17507930 ] Apache Spark commented on SPARK-38576: -- User 'xinrong-databricks' has created a pull request for

[jira] [Created] (SPARK-38577) Interval types are not truncated to the expected endField when creating a DataFrame via Duration

2022-03-16 Thread chong (Jira)
chong created SPARK-38577: - Summary: Interval types are not truncated to the expected endField when creating a DataFrame via Duration Key: SPARK-38577 URL: https://issues.apache.org/jira/browse/SPARK-38577

[jira] [Created] (SPARK-38576) Implement `numeric_only` parameter for `DataFrame/Series.rank` to rank numeric columns only

2022-03-16 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-38576: Summary: Implement `numeric_only` parameter for `DataFrame/Series.rank` to rank numeric columns only Key: SPARK-38576 URL: https://issues.apache.org/jira/browse/SPARK-38576

[jira] [Resolved] (SPARK-38556) Disable Pandas usage logging for method calls inside @contextmanager functions

2022-03-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38556. -- Target Version/s: 3.2.1, 3.3.0 Resolution: Fixed Fixed in

[jira] [Resolved] (SPARK-38441) Support string and bool `regex` in `Series.replace`

2022-03-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38441. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 35747

[jira] [Assigned] (SPARK-38441) Support string and bool `regex` in `Series.replace`

2022-03-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38441: Assignee: Xinrong Meng > Support string and bool `regex` in `Series.replace` >

[jira] [Resolved] (SPARK-38572) Setting version to 3.4.0-SNAPSHOT

2022-03-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38572. --- Resolution: Fixed Issue resolved by pull request 35879

[jira] [Commented] (SPARK-38575) Duduplicate branch specification in GitHub Actions workflow

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507921#comment-17507921 ] Apache Spark commented on SPARK-38575: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-38575) Duduplicate branch specification in GitHub Actions workflow

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38575: Assignee: (was: Apache Spark) > Duduplicate branch specification in GitHub Actions

[jira] [Assigned] (SPARK-38575) Duduplicate branch specification in GitHub Actions workflow

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38575: Assignee: Apache Spark > Duduplicate branch specification in GitHub Actions workflow >

[jira] [Created] (SPARK-38575) Duduplicate branch specification in GitHub Actions workflow

2022-03-16 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-38575: Summary: Duduplicate branch specification in GitHub Actions workflow Key: SPARK-38575 URL: https://issues.apache.org/jira/browse/SPARK-38575 Project: Spark

[jira] [Updated] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38568: -- Affects Version/s: 3.4.0 (was: 3.3.0) > Upgrade ZSTD-JNI to

[jira] [Commented] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507907#comment-17507907 ] Dongjoon Hyun commented on SPARK-38568: --- Hi, [~yumwang]. When you clone an issue, you should

[jira] [Commented] (SPARK-36664) Log time spent waiting for cluster resources

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507899#comment-17507899 ] Apache Spark commented on SPARK-36664: -- User 'holdenk' has created a pull request for this issue:

[jira] [Commented] (SPARK-36664) Log time spent waiting for cluster resources

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507900#comment-17507900 ] Apache Spark commented on SPARK-36664: -- User 'holdenk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38574) Enrich Avro data source documentation

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38574: Assignee: Apache Spark > Enrich Avro data source documentation >

[jira] [Assigned] (SPARK-38574) Enrich Avro data source documentation

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38574: Assignee: (was: Apache Spark) > Enrich Avro data source documentation >

[jira] [Commented] (SPARK-38574) Enrich Avro data source documentation

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507898#comment-17507898 ] Apache Spark commented on SPARK-38574: -- User 'tianhanhu' has created a pull request for this issue:

[jira] [Created] (SPARK-38574) Enrich Avro data source documentation

2022-03-16 Thread Tianhan Hu (Jira)
Tianhan Hu created SPARK-38574: -- Summary: Enrich Avro data source documentation Key: SPARK-38574 URL: https://issues.apache.org/jira/browse/SPARK-38574 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-38555) Avoid contention and get or create clientPools quickly in the TransportClientFactory

2022-03-16 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-38555. - Assignee: weixiuli Resolution: Fixed > Avoid contention and get or

[jira] [Updated] (SPARK-38555) Avoid contention and get or create clientPools quickly in the TransportClientFactory

2022-03-16 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-38555: Fix Version/s: 3.4.0 > Avoid contention and get or create clientPools quickly in

[jira] [Created] (SPARK-38573) Support Partition Level Statistics Collection

2022-03-16 Thread Kazuyuki Tanimura (Jira)
Kazuyuki Tanimura created SPARK-38573: - Summary: Support Partition Level Statistics Collection Key: SPARK-38573 URL: https://issues.apache.org/jira/browse/SPARK-38573 Project: Spark

[jira] [Resolved] (SPARK-38545) Upgarde scala-maven-plugin from 4.4.0 to 4.5.6

2022-03-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38545. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 35841

[jira] [Assigned] (SPARK-38545) Upgarde scala-maven-plugin from 4.4.0 to 4.5.6

2022-03-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-38545: Assignee: Yang Jie > Upgarde scala-maven-plugin from 4.4.0 to 4.5.6 >

[jira] [Commented] (SPARK-38572) Setting version to 3.4.0-SNAPSHOT

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507808#comment-17507808 ] Apache Spark commented on SPARK-38572: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-38572) Setting version to 3.4.0-SNAPSHOT

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507807#comment-17507807 ] Apache Spark commented on SPARK-38572: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38572) Setting version to 3.4.0-SNAPSHOT

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38572: Assignee: Max Gekk (was: Apache Spark) > Setting version to 3.4.0-SNAPSHOT >

[jira] [Assigned] (SPARK-38572) Setting version to 3.4.0-SNAPSHOT

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38572: Assignee: Apache Spark (was: Max Gekk) > Setting version to 3.4.0-SNAPSHOT >

[jira] [Resolved] (SPARK-38561) Add doc for "Customized Kubernetes Schedulers"

2022-03-16 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-38561. -- Fix Version/s: 3.3.0 Assignee: Yikun Jiang Resolution: Fixed > Add doc for

[jira] [Updated] (SPARK-38503) Add warn for getAdditionalPreKubernetesResources in executor side

2022-03-16 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-38503: - Target Version/s: 3.4.0 > Add warn for getAdditionalPreKubernetesResources in executor side >

[jira] [Commented] (SPARK-38572) Setting version to 3.4.0-SNAPSHOT

2022-03-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507780#comment-17507780 ] Max Gekk commented on SPARK-38572: -- FYI, I am working on this. > Setting version to 3.4.0-SNAPSHOT >

[jira] [Updated] (SPARK-38572) Setting version to 3.4.0-SNAPSHOT

2022-03-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-38572: - Reporter: Max Gekk (was: Dongjoon Hyun) > Setting version to 3.4.0-SNAPSHOT >

[jira] [Updated] (SPARK-38572) Setting version to 3.4.0-SNAPSHOT

2022-03-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-38572: - Fix Version/s: 3.4.0 (was: 3.3.0) > Setting version to 3.4.0-SNAPSHOT >

[jira] [Updated] (SPARK-38572) Setting version to 3.4.0-SNAPSHOT

2022-03-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-38572: - Affects Version/s: 3.4.0 (was: 3.3.0) > Setting version to 3.4.0-SNAPSHOT >

[jira] [Assigned] (SPARK-38572) Setting version to 3.4.0-SNAPSHOT

2022-03-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-38572: Assignee: Max Gekk (was: Dongjoon Hyun) > Setting version to 3.4.0-SNAPSHOT >

[jira] [Created] (SPARK-38572) Setting version to 3.4.0-SNAPSHOT

2022-03-16 Thread Max Gekk (Jira)
Max Gekk created SPARK-38572: Summary: Setting version to 3.4.0-SNAPSHOT Key: SPARK-38572 URL: https://issues.apache.org/jira/browse/SPARK-38572 Project: Spark Issue Type: Task

[jira] [Updated] (SPARK-38194) Make Yarn memory overhead factor configurable

2022-03-16 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-38194: -- Fix Version/s: 3.3.0 (was: 3.4.0) > Make Yarn memory overhead factor

[jira] [Updated] (SPARK-38194) Make Yarn memory overhead factor configurable

2022-03-16 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-38194: -- Fix Version/s: 3.4.0 (was: 3.3.0) > Make Yarn memory overhead factor

[jira] [Resolved] (SPARK-38194) Make Yarn memory overhead factor configurable

2022-03-16 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-38194. --- Fix Version/s: 3.3.0 Assignee: Adam Binford Resolution: Fixed > Make Yarn

[jira] [Updated] (SPARK-38571) Week of month from a date is missing in spark3 for return values of 1 to 6

2022-03-16 Thread Kevin Appel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Appel updated SPARK-38571: Description: In Spark2 we could use the date_format function with either the W or F flags to

[jira] [Created] (SPARK-38571) Week of month from a date is missing in spark3 for return values of 1 to 6

2022-03-16 Thread Kevin Appel (Jira)
Kevin Appel created SPARK-38571: --- Summary: Week of month from a date is missing in spark3 for return values of 1 to 6 Key: SPARK-38571 URL: https://issues.apache.org/jira/browse/SPARK-38571 Project:

[jira] [Commented] (SPARK-38570) Incorrect DynamicPartitionPruning caused by Literal

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507626#comment-17507626 ] Apache Spark commented on SPARK-38570: -- User 'mcdull-zhang' has created a pull request for this

[jira] [Assigned] (SPARK-38570) Incorrect DynamicPartitionPruning caused by Literal

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38570: Assignee: Apache Spark > Incorrect DynamicPartitionPruning caused by Literal >

[jira] [Assigned] (SPARK-38570) Incorrect DynamicPartitionPruning caused by Literal

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38570: Assignee: (was: Apache Spark) > Incorrect DynamicPartitionPruning caused by Literal

[jira] [Updated] (SPARK-38570) Incorrect DynamicPartitionPruning caused by Literal

2022-03-16 Thread mcdull_zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mcdull_zhang updated SPARK-38570: - Description: The return value of Literal.references is an empty AttributeSet, so Literal is

[jira] [Created] (SPARK-38570) Incorrect DynamicPartitionPruning caused by Literal

2022-03-16 Thread mcdull_zhang (Jira)
mcdull_zhang created SPARK-38570: Summary: Incorrect DynamicPartitionPruning caused by Literal Key: SPARK-38570 URL: https://issues.apache.org/jira/browse/SPARK-38570 Project: Spark Issue

[jira] [Commented] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507585#comment-17507585 ] Apache Spark commented on SPARK-38568: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38568: Assignee: Apache Spark > Upgrade ZSTD-JNI to 1.5.2-2 > --- > >

[jira] [Assigned] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38568: Assignee: (was: Apache Spark) > Upgrade ZSTD-JNI to 1.5.2-2 >

[jira] [Commented] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507584#comment-17507584 ] Apache Spark commented on SPARK-38568: -- User 'wangyum' has created a pull request for this issue:

[jira] [Commented] (SPARK-38569) external top-level directory is problematic for bazel

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507583#comment-17507583 ] Apache Spark commented on SPARK-38569: -- User 'alkis' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38569) external top-level directory is problematic for bazel

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38569: Assignee: Apache Spark > external top-level directory is problematic for bazel >

[jira] [Assigned] (SPARK-38569) external top-level directory is problematic for bazel

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38569: Assignee: (was: Apache Spark) > external top-level directory is problematic for

[jira] [Created] (SPARK-38569) external top-level directory is problematic for bazel

2022-03-16 Thread Alkis Evlogimenos (Jira)
Alkis Evlogimenos created SPARK-38569: - Summary: external top-level directory is problematic for bazel Key: SPARK-38569 URL: https://issues.apache.org/jira/browse/SPARK-38569 Project: Spark

[jira] [Created] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38568: --- Summary: Upgrade ZSTD-JNI to 1.5.2-2 Key: SPARK-38568 URL: https://issues.apache.org/jira/browse/SPARK-38568 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38568: Fix Version/s: (was: 3.3.0) > Upgrade ZSTD-JNI to 1.5.2-2 > --- > >

[jira] [Assigned] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38568: --- Assignee: (was: Dongjoon Hyun) > Upgrade ZSTD-JNI to 1.5.2-2 >

[jira] [Commented] (SPARK-38330) Certificate doesn't match any of the subject alternative names: [*.s3.amazonaws.com, s3.amazonaws.com]

2022-03-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507557#comment-17507557 ] Steve Loughran commented on SPARK-38330: sorry about that. try enabling path style access and

[jira] [Assigned] (SPARK-38567) Enable GitHub Action build_and_test on branch-3.3

2022-03-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38567: Assignee: Max Gekk > Enable GitHub Action build_and_test on branch-3.3 >

[jira] [Resolved] (SPARK-38567) Enable GitHub Action build_and_test on branch-3.3

2022-03-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38567. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35876

[jira] [Assigned] (SPARK-38567) Enable GitHub Action build_and_test on branch-3.3

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38567: Assignee: Apache Spark > Enable GitHub Action build_and_test on branch-3.3 >

[jira] [Assigned] (SPARK-38567) Enable GitHub Action build_and_test on branch-3.3

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38567: Assignee: (was: Apache Spark) > Enable GitHub Action build_and_test on branch-3.3 >

[jira] [Commented] (SPARK-38567) Enable GitHub Action build_and_test on branch-3.3

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507532#comment-17507532 ] Apache Spark commented on SPARK-38567: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-38108) Use error classes in the compilation errors of UDF/UDAF

2022-03-16 Thread huangtengfei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507528#comment-17507528 ] huangtengfei commented on SPARK-38108: -- I am working on this. Thanks [~maxgekk] > Use error

[jira] [Created] (SPARK-38567) Enable GitHub Action build_and_test on branch-3.3

2022-03-16 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-38567: Summary: Enable GitHub Action build_and_test on branch-3.3 Key: SPARK-38567 URL: https://issues.apache.org/jira/browse/SPARK-38567 Project: Spark Issue

[jira] [Commented] (SPARK-38566) Revert the parser changes for DEFAULT column support

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507507#comment-17507507 ] Apache Spark commented on SPARK-38566: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38566) Revert the parser changes for DEFAULT column support

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38566: Assignee: (was: Apache Spark) > Revert the parser changes for DEFAULT column support

[jira] [Commented] (SPARK-38566) Revert the parser changes for DEFAULT column support

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507505#comment-17507505 ] Apache Spark commented on SPARK-38566: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38566) Revert the parser changes for DEFAULT column support

2022-03-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38566: Assignee: Apache Spark > Revert the parser changes for DEFAULT column support >

[jira] [Updated] (SPARK-38566) Revert the parser changes for DEFAULT column support

2022-03-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-38566: - Issue Type: Task (was: Improvement) > Revert the parser changes for DEFAULT column support >

[jira] [Updated] (SPARK-38566) Revert the parser changes for DEFAULT column support

2022-03-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-38566: - Priority: Blocker (was: Major) > Revert the parser changes for DEFAULT column support >

[jira] [Created] (SPARK-38566) Revert the parser changes for DEFAULT column support

2022-03-16 Thread Max Gekk (Jira)
Max Gekk created SPARK-38566: Summary: Revert the parser changes for DEFAULT column support Key: SPARK-38566 URL: https://issues.apache.org/jira/browse/SPARK-38566 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-38330) Certificate doesn't match any of the subject alternative names: [*.s3.amazonaws.com, s3.amazonaws.com]

2022-03-16 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507490#comment-17507490 ] André F. edited comment on SPARK-38330 at 3/16/22, 10:52 AM: - Created ticket

[jira] [Commented] (SPARK-38330) Certificate doesn't match any of the subject alternative names: [*.s3.amazonaws.com, s3.amazonaws.com]

2022-03-16 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507490#comment-17507490 ] André F. commented on SPARK-38330: -- Created ticket on

[jira] [Updated] (SPARK-38565) Support Left Semi join in row level runtime filters

2022-03-16 Thread Abhishek Somani (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Somani updated SPARK-38565: Description: Support Left Semi join in the runtime filtering as well. This is a follow

  1   2   >