[jira] [Assigned] (SPARK-38994) Add an Python example of StreamingQueryListener

2022-04-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38994: Assignee: Hyukjin Kwon > Add an Python example of StreamingQueryListener >

[jira] [Resolved] (SPARK-38986) Prepend error class tag to error messages

2022-04-21 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-38986. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36287

[jira] [Commented] (SPARK-38732) Test the error class: INCOMPARABLE_PIVOT_COLUMN

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526192#comment-17526192 ] Apache Spark commented on SPARK-38732: -- User 'lvshaokang' has created a pull request for this

[jira] [Commented] (SPARK-38732) Test the error class: INCOMPARABLE_PIVOT_COLUMN

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526191#comment-17526191 ] Apache Spark commented on SPARK-38732: -- User 'lvshaokang' has created a pull request for this

[jira] [Commented] (SPARK-28330) ANSI SQL: Top-level in

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526190#comment-17526190 ] Apache Spark commented on SPARK-28330: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38994) Add an Python example of StreamingQueryListener

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38994: Assignee: (was: Apache Spark) > Add an Python example of StreamingQueryListener >

[jira] [Commented] (SPARK-38994) Add an Python example of StreamingQueryListener

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526189#comment-17526189 ] Apache Spark commented on SPARK-38994: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-38994) Add an Python example of StreamingQueryListener

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38994: Assignee: Apache Spark > Add an Python example of StreamingQueryListener >

[jira] [Created] (SPARK-38994) Add an Python example of StreamingQueryListener

2022-04-21 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-38994: Summary: Add an Python example of StreamingQueryListener Key: SPARK-38994 URL: https://issues.apache.org/jira/browse/SPARK-38994 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-38990) date_trunc and trunc both fail with format from column in inline table

2022-04-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38990. -- Fix Version/s: 3.3.0 3.0.4 3.2.2

[jira] [Assigned] (SPARK-38990) date_trunc and trunc both fail with format from column in inline table

2022-04-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38990: Assignee: Bruce Robbins > date_trunc and trunc both fail with format from column in

[jira] [Resolved] (SPARK-38974) List functions should only list registered functions in the specified database

2022-04-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38974. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 36291

[jira] [Assigned] (SPARK-38974) List functions should only list registered functions in the specified database

2022-04-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-38974: --- Assignee: Allison Wang > List functions should only list registered functions in the

[jira] [Commented] (SPARK-38993) Impl DataFrame.boxplot and DataFrame.plot.box

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526182#comment-17526182 ] Apache Spark commented on SPARK-38993: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-38993) Impl DataFrame.boxplot and DataFrame.plot.box

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526181#comment-17526181 ] Apache Spark commented on SPARK-38993: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-38993) Impl DataFrame.boxplot and DataFrame.plot.box

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38993: Assignee: (was: Apache Spark) > Impl DataFrame.boxplot and DataFrame.plot.box >

[jira] [Assigned] (SPARK-38993) Impl DataFrame.boxplot and DataFrame.plot.box

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38993: Assignee: Apache Spark > Impl DataFrame.boxplot and DataFrame.plot.box >

[jira] [Commented] (SPARK-38813) Remove TimestampNTZ type support in Spark 3.3

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526180#comment-17526180 ] Apache Spark commented on SPARK-38813: -- User 'gengliangwang' has created a pull request for this

[jira] [Created] (SPARK-38993) Impl DataFrame.boxplot and DataFrame.plot.box

2022-04-21 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-38993: Summary: Impl DataFrame.boxplot and DataFrame.plot.box Key: SPARK-38993 URL: https://issues.apache.org/jira/browse/SPARK-38993 Project: Spark Issue Type:

[jira] [Commented] (SPARK-38992) Avoid using bash -c in ShellBasedGroupsMappingProvider

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526178#comment-17526178 ] Apache Spark commented on SPARK-38992: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Resolved] (SPARK-38666) Missing aggregate filter checks

2022-04-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38666. - Fix Version/s: 3.3.0 Assignee: Bruce Robbins Resolution: Fixed > Missing

[jira] [Commented] (SPARK-38992) Avoid using bash -c in ShellBasedGroupsMappingProvider

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526177#comment-17526177 ] Apache Spark commented on SPARK-38992: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-38992) Avoid using bash -c in ShellBasedGroupsMappingProvider

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38992: Assignee: Apache Spark > Avoid using bash -c in ShellBasedGroupsMappingProvider >

[jira] [Assigned] (SPARK-38992) Avoid using bash -c in ShellBasedGroupsMappingProvider

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38992: Assignee: (was: Apache Spark) > Avoid using bash -c in

[jira] [Created] (SPARK-38992) Avoid using bash -c in ShellBasedGroupsMappingProvider

2022-04-21 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-38992: Summary: Avoid using bash -c in ShellBasedGroupsMappingProvider Key: SPARK-38992 URL: https://issues.apache.org/jira/browse/SPARK-38992 Project: Spark Issue

[jira] [Resolved] (SPARK-38938) Implement `inplace` and `columns` parameters of `Series.drop`

2022-04-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38938. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36215

[jira] [Assigned] (SPARK-38938) Implement `inplace` and `columns` parameters of `Series.drop`

2022-04-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38938: Assignee: Xinrong Meng > Implement `inplace` and `columns` parameters of `Series.drop` >

[jira] [Commented] (SPARK-38734) Test the error class: INDEX_OUT_OF_BOUNDS

2022-04-21 Thread panbingkun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526173#comment-17526173 ] panbingkun commented on SPARK-38734: ignore it! > Test the error class: INDEX_OUT_OF_BOUNDS >

[jira] [Assigned] (SPARK-38952) Implement `numeric_only` of `GroupBy.first` and `GroupBy.last`

2022-04-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38952: Assignee: Xinrong Meng > Implement `numeric_only` of `GroupBy.first` and `GroupBy.last`

[jira] [Resolved] (SPARK-38952) Implement `numeric_only` of `GroupBy.first` and `GroupBy.last`

2022-04-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38952. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36266

[jira] [Resolved] (SPARK-38955) from_csv can corrupt surrounding lines if a lineSep is in the data

2022-04-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38955. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 36294

[jira] [Assigned] (SPARK-38955) from_csv can corrupt surrounding lines if a lineSep is in the data

2022-04-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38955: Assignee: Hyukjin Kwon > from_csv can corrupt surrounding lines if a lineSep is in the

[jira] [Commented] (SPARK-38736) Test the error classes: INVALID_ARRAY_INDEX*

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526168#comment-17526168 ] Apache Spark commented on SPARK-38736: -- User 'panbingkun' has created a pull request for this

[jira] [Assigned] (SPARK-38736) Test the error classes: INVALID_ARRAY_INDEX*

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38736: Assignee: (was: Apache Spark) > Test the error classes: INVALID_ARRAY_INDEX* >

[jira] [Assigned] (SPARK-38736) Test the error classes: INVALID_ARRAY_INDEX*

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38736: Assignee: Apache Spark > Test the error classes: INVALID_ARRAY_INDEX* >

[jira] [Commented] (SPARK-38734) Test the error class: INDEX_OUT_OF_BOUNDS

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526165#comment-17526165 ] Apache Spark commented on SPARK-38734: -- User 'panbingkun' has created a pull request for this

[jira] [Commented] (SPARK-38734) Test the error class: INDEX_OUT_OF_BOUNDS

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526163#comment-17526163 ] Apache Spark commented on SPARK-38734: -- User 'panbingkun' has created a pull request for this

[jira] [Assigned] (SPARK-38734) Test the error class: INDEX_OUT_OF_BOUNDS

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38734: Assignee: Apache Spark > Test the error class: INDEX_OUT_OF_BOUNDS >

[jira] [Assigned] (SPARK-38734) Test the error class: INDEX_OUT_OF_BOUNDS

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38734: Assignee: (was: Apache Spark) > Test the error class: INDEX_OUT_OF_BOUNDS >

[jira] [Updated] (SPARK-37528) Schedule Tasks By Input Size

2022-04-21 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-37528: -- Description: In general, the larger input data size means longer running time. So ideally, we can

[jira] [Updated] (SPARK-37528) Schedule Tasks By Input Size

2022-04-21 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-37528: -- Description: In general, the larger input data size means longer running time. So ideally, we can

[jira] [Updated] (SPARK-37528) Schedule Tasks By Input Size

2022-04-21 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-37528: -- Description: In general, the larger input data size means longer running time. So ideally, we can

[jira] [Commented] (SPARK-38990) date_trunc and trunc both fail with format from column in inline table

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526148#comment-17526148 ] Apache Spark commented on SPARK-38990: -- User 'bersprockets' has created a pull request for this

[jira] [Assigned] (SPARK-38990) date_trunc and trunc both fail with format from column in inline table

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38990: Assignee: (was: Apache Spark) > date_trunc and trunc both fail with format from

[jira] [Assigned] (SPARK-38990) date_trunc and trunc both fail with format from column in inline table

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38990: Assignee: Apache Spark > date_trunc and trunc both fail with format from column in

[jira] [Commented] (SPARK-34960) Aggregate (Min/Max/Count) push down for ORC

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526145#comment-17526145 ] Apache Spark commented on SPARK-34960: -- User 'c21' has created a pull request for this issue:

[jira] [Commented] (SPARK-34960) Aggregate (Min/Max/Count) push down for ORC

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526144#comment-17526144 ] Apache Spark commented on SPARK-34960: -- User 'c21' has created a pull request for this issue:

[jira] [Commented] (SPARK-38581) List of supported pandas APIs for pandas API on Spark docs.

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526135#comment-17526135 ] Apache Spark commented on SPARK-38581: -- User 'itholic' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38985) Support sub-error-class for UNSUPPORTED_FEATURE et al

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38985: Assignee: Apache Spark > Support sub-error-class for UNSUPPORTED_FEATURE et al >

[jira] [Assigned] (SPARK-38985) Support sub-error-class for UNSUPPORTED_FEATURE et al

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38985: Assignee: (was: Apache Spark) > Support sub-error-class for UNSUPPORTED_FEATURE et

[jira] [Commented] (SPARK-38985) Support sub-error-class for UNSUPPORTED_FEATURE et al

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526132#comment-17526132 ] Apache Spark commented on SPARK-38985: -- User 'srielau' has created a pull request for this issue:

[jira] [Created] (SPARK-38991) Implement `numeric_only` of `GroupBy.mean` and `GroupBy.sum`

2022-04-21 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-38991: Summary: Implement `numeric_only` of `GroupBy.mean` and `GroupBy.sum` Key: SPARK-38991 URL: https://issues.apache.org/jira/browse/SPARK-38991 Project: Spark

[jira] [Commented] (SPARK-38991) Implement `numeric_only` of `GroupBy.mean` and `GroupBy.sum`

2022-04-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526121#comment-17526121 ] Xinrong Meng commented on SPARK-38991: -- I am working on that. > Implement `numeric_only` of

[jira] [Created] (SPARK-38990) date_trunc and trunc both fail with format from column in inline table

2022-04-21 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-38990: - Summary: date_trunc and trunc both fail with format from column in inline table Key: SPARK-38990 URL: https://issues.apache.org/jira/browse/SPARK-38990 Project:

[jira] [Updated] (SPARK-38988) Pandas API - "PerformanceWarning: DataFrame is highly fragmented." get printed many times.

2022-04-21 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-38988: Summary: Pandas API - "PerformanceWarning: DataFrame is highly fragmented." get printed

[jira] [Updated] (SPARK-38988) Pandas API - "PerformanceWarning: DataFrame is highly fragmented." get printed to many times.

2022-04-21 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-38988: Description: I add a file and a notebook with the info msg I get when I run df.info()

[jira] [Updated] (SPARK-38988) Pandas API - "PerformanceWarning: DataFrame is highly fragmented." get printed to many times.

2022-04-21 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-38988: Description: I add a file now with the info msg I get when I run df.info() Spark master

[jira] [Updated] (SPARK-38988) Pandas API - "PerformanceWarning: DataFrame is highly fragmented." get printed to many times.

2022-04-21 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-38988: Attachment: Untitled.html > Pandas API - "PerformanceWarning: DataFrame is highly

[jira] [Commented] (SPARK-34827) Support fetching shuffle blocks in batch with i/o encryption

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525957#comment-17525957 ] Apache Spark commented on SPARK-34827: -- User 'xinrong-databricks' has created a pull request for

[jira] [Assigned] (SPARK-34827) Support fetching shuffle blocks in batch with i/o encryption

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34827: Assignee: (was: Apache Spark) > Support fetching shuffle blocks in batch with i/o

[jira] [Commented] (SPARK-34827) Support fetching shuffle blocks in batch with i/o encryption

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525956#comment-17525956 ] Apache Spark commented on SPARK-34827: -- User 'xinrong-databricks' has created a pull request for

[jira] [Assigned] (SPARK-34827) Support fetching shuffle blocks in batch with i/o encryption

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34827: Assignee: Apache Spark > Support fetching shuffle blocks in batch with i/o encryption >

[jira] [Created] (SPARK-38989) Implement `ignore_index` of `DataFrame/Series.sample`

2022-04-21 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-38989: Summary: Implement `ignore_index` of `DataFrame/Series.sample` Key: SPARK-38989 URL: https://issues.apache.org/jira/browse/SPARK-38989 Project: Spark Issue

[jira] [Created] (SPARK-38988) Pandas API - "PerformanceWarning: DataFrame is highly fragmented." get printed to many times.

2022-04-21 Thread Jira
Bjørn Jørgensen created SPARK-38988: --- Summary: Pandas API - "PerformanceWarning: DataFrame is highly fragmented." get printed to many times. Key: SPARK-38988 URL:

[jira] [Updated] (SPARK-38988) Pandas API - "PerformanceWarning: DataFrame is highly fragmented." get printed to many times.

2022-04-21 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-38988: Attachment: info.txt > Pandas API - "PerformanceWarning: DataFrame is highly fragmented."

[jira] [Commented] (SPARK-38987) Handle fallback when merged shuffle blocks are corrupted and spark.shuffle.detectCorrupt is set to true

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525915#comment-17525915 ] Apache Spark commented on SPARK-38987: -- User 'zhouyejoe' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38987) Handle fallback when merged shuffle blocks are corrupted and spark.shuffle.detectCorrupt is set to true

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38987: Assignee: (was: Apache Spark) > Handle fallback when merged shuffle blocks are

[jira] [Assigned] (SPARK-38987) Handle fallback when merged shuffle blocks are corrupted and spark.shuffle.detectCorrupt is set to true

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38987: Assignee: Apache Spark > Handle fallback when merged shuffle blocks are corrupted and >

[jira] (SPARK-37174) WARN WindowExec: No Partition Defined is being printed 4 times.

2022-04-21 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-37174 ] Bjørn Jørgensen deleted comment on SPARK-37174: - was (Author: bjornjorgensen): I add a file now with the info msg I get when I run df.info() Spark master build from last week. I will

[jira] [Commented] (SPARK-38987) Handle fallback when merged shuffle blocks are corrupted and spark.shuffle.detectCorrupt is set to true

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525912#comment-17525912 ] Apache Spark commented on SPARK-38987: -- User 'zhouyejoe' has created a pull request for this issue:

[jira] [Commented] (SPARK-37174) WARN WindowExec: No Partition Defined is being printed 4 times.

2022-04-21 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-37174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525911#comment-17525911 ] Bjørn Jørgensen commented on SPARK-37174: - I add a file now with the info msg I get when I run

[jira] [Updated] (SPARK-37174) WARN WindowExec: No Partition Defined is being printed 4 times.

2022-04-21 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-37174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-37174: Attachment: info.txt > WARN WindowExec: No Partition Defined is being printed 4 times. >

[jira] [Resolved] (SPARK-38984) Allow comparison between TimestampNTZ and Timestamp/Date

2022-04-21 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-38984. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36300

[jira] [Resolved] (SPARK-38980) Move error class tests requiring ANSI SQL mode to QueryExecutionAnsiErrorsSuite

2022-04-21 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-38980. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36299

[jira] [Updated] (SPARK-38980) Move error class tests requiring ANSI SQL mode to QueryExecutionAnsiErrorsSuite

2022-04-21 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-38980: --- Affects Version/s: 3.4.0 (was: 3.3.0) > Move error class tests

[jira] [Created] (SPARK-38987) Handle fallback when merged shuffle blocks are corrupted and spark.shuffle.detectCorrupt is set to true

2022-04-21 Thread Ye Zhou (Jira)
Ye Zhou created SPARK-38987: --- Summary: Handle fallback when merged shuffle blocks are corrupted and spark.shuffle.detectCorrupt is set to true Key: SPARK-38987 URL: https://issues.apache.org/jira/browse/SPARK-38987

[jira] [Assigned] (SPARK-38959) DataSource V2: Support runtime group filtering in row-level commands

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38959: Assignee: (was: Apache Spark) > DataSource V2: Support runtime group filtering in

[jira] [Commented] (SPARK-38959) DataSource V2: Support runtime group filtering in row-level commands

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525872#comment-17525872 ] Apache Spark commented on SPARK-38959: -- User 'aokolnychyi' has created a pull request for this

[jira] [Assigned] (SPARK-38959) DataSource V2: Support runtime group filtering in row-level commands

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38959: Assignee: Apache Spark > DataSource V2: Support runtime group filtering in row-level

[jira] [Commented] (SPARK-38959) DataSource V2: Support runtime group filtering in row-level commands

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525876#comment-17525876 ] Apache Spark commented on SPARK-38959: -- User 'aokolnychyi' has created a pull request for this

[jira] [Assigned] (SPARK-38977) Fix schema pruning with correlated subqueries

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38977: Assignee: Apache Spark > Fix schema pruning with correlated subqueries >

[jira] [Assigned] (SPARK-38977) Fix schema pruning with correlated subqueries

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38977: Assignee: (was: Apache Spark) > Fix schema pruning with correlated subqueries >

[jira] [Commented] (SPARK-38977) Fix schema pruning with correlated subqueries

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525844#comment-17525844 ] Apache Spark commented on SPARK-38977: -- User 'aokolnychyi' has created a pull request for this

[jira] [Created] (SPARK-38986) Prepend error class tag to error messages

2022-04-21 Thread Max Gekk (Jira)
Max Gekk created SPARK-38986: Summary: Prepend error class tag to error messages Key: SPARK-38986 URL: https://issues.apache.org/jira/browse/SPARK-38986 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-38986) Prepend error class tag to error messages

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525837#comment-17525837 ] Apache Spark commented on SPARK-38986: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38986) Prepend error class tag to error messages

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38986: Assignee: Max Gekk (was: Apache Spark) > Prepend error class tag to error messages >

[jira] [Assigned] (SPARK-38986) Prepend error class tag to error messages

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38986: Assignee: Apache Spark (was: Max Gekk) > Prepend error class tag to error messages >

[jira] [Commented] (SPARK-38986) Prepend error class tag to error messages

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525835#comment-17525835 ] Apache Spark commented on SPARK-38986: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-38950) Return Array of Predicate for SupportsPushDownCatalystFilters.pushedFilters

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525816#comment-17525816 ] Apache Spark commented on SPARK-38950: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Commented] (SPARK-37259) JDBC read is always going to wrap the query in a select statement

2022-04-21 Thread Kevin Appel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525784#comment-17525784 ] Kevin Appel commented on SPARK-37259: - Both of these PR's were closed due to inactivity, I had

[jira] [Resolved] (SPARK-38950) Return Array of Predicate for SupportsPushDownCatalystFilters.pushedFilters

2022-04-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38950. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 36264

[jira] [Assigned] (SPARK-38950) Return Array of Predicate for SupportsPushDownCatalystFilters.pushedFilters

2022-04-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-38950: --- Assignee: Huaxin Gao > Return Array of Predicate for

[jira] [Created] (SPARK-38985) Support sub-error-class for UNSUPPORTED_FEATURE et al

2022-04-21 Thread Serge Rielau (Jira)
Serge Rielau created SPARK-38985: Summary: Support sub-error-class for UNSUPPORTED_FEATURE et al Key: SPARK-38985 URL: https://issues.apache.org/jira/browse/SPARK-38985 Project: Spark Issue

[jira] [Commented] (SPARK-21697) NPE & ExceptionInInitializerError trying to load UDF from HDFS

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525756#comment-17525756 ] Apache Spark commented on SPARK-21697: -- User 'cxzl25' has created a pull request for this issue:

[jira] [Commented] (SPARK-21697) NPE & ExceptionInInitializerError trying to load UDF from HDFS

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525755#comment-17525755 ] Apache Spark commented on SPARK-21697: -- User 'cxzl25' has created a pull request for this issue:

[jira] [Commented] (SPARK-38984) Allow comparison between TimestampNTZ and Timestamp/Date

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525750#comment-17525750 ] Apache Spark commented on SPARK-38984: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-38984) Allow comparison between TimestampNTZ and Timestamp/Date

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38984: Assignee: Apache Spark (was: Gengliang Wang) > Allow comparison between TimestampNTZ

[jira] [Assigned] (SPARK-38984) Allow comparison between TimestampNTZ and Timestamp/Date

2022-04-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38984: Assignee: Gengliang Wang (was: Apache Spark) > Allow comparison between TimestampNTZ

[jira] [Created] (SPARK-38984) Allow comparison between TimestampNTZ and Timestamp/Date

2022-04-21 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-38984: -- Summary: Allow comparison between TimestampNTZ and Timestamp/Date Key: SPARK-38984 URL: https://issues.apache.org/jira/browse/SPARK-38984 Project: Spark

[jira] [Created] (SPARK-38983) Pyspark throws AnalysisException with incorrect error message when using .grouping() or .groupingId() (AnalysisException: grouping() can only be used with GroupingSets/C

2022-04-21 Thread Chris Kimmel (Jira)
Chris Kimmel created SPARK-38983: Summary: Pyspark throws AnalysisException with incorrect error message when using .grouping() or .groupingId() (AnalysisException: grouping() can only be used with GroupingSets/Cube/Rollup;)

  1   2   >