[jira] [Assigned] (SPARK-38770) Simply steps to re write primary resource in k8s spark application

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38770: Assignee: (was: Apache Spark) > Simply steps to re write primary resource in k8s

[jira] [Assigned] (SPARK-38770) Simply steps to re write primary resource in k8s spark application

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38770: Assignee: Apache Spark > Simply steps to re write primary resource in k8s spark

[jira] [Commented] (SPARK-38770) Simply steps to re write primary resource in k8s spark application

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516228#comment-17516228 ] Apache Spark commented on SPARK-38770: -- User 'dcoliversun' has created a pull request for this

[jira] [Updated] (SPARK-38770) Simply steps to re write primary resource in k8s spark application

2022-04-01 Thread qian (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] qian updated SPARK-38770: - Fix Version/s: 3.4.0 (was: 3.3.0) > Simply steps to re write primary resource in k8s

[jira] [Created] (SPARK-38770) Simply steps to re write primary resource in k8s spark application

2022-04-01 Thread qian (Jira)
qian created SPARK-38770: Summary: Simply steps to re write primary resource in k8s spark application Key: SPARK-38770 URL: https://issues.apache.org/jira/browse/SPARK-38770 Project: Spark Issue

[jira] [Commented] (SPARK-38769) [SQL] behavior of schema_of_json not same with 2.4.0

2022-04-01 Thread gabrywu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516219#comment-17516219 ] gabrywu commented on SPARK-38769: - [~maxgekk]  > [SQL] behavior of schema_of_json not same with 2.4.0 >

[jira] [Commented] (SPARK-38768) If limit could pushed down and Data source only have one partition, DS V2 should not do limit again

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516209#comment-17516209 ] Apache Spark commented on SPARK-38768: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38768) If limit could pushed down and Data source only have one partition, DS V2 should not do limit again

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38768: Assignee: (was: Apache Spark) > If limit could pushed down and Data source only have

[jira] [Assigned] (SPARK-38768) If limit could pushed down and Data source only have one partition, DS V2 should not do limit again

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38768: Assignee: Apache Spark > If limit could pushed down and Data source only have one

[jira] [Commented] (SPARK-38768) If limit could pushed down and Data source only have one partition, DS V2 should not do limit again

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516210#comment-17516210 ] Apache Spark commented on SPARK-38768: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38768) If limit could pushed down and Data source only have one partition, DS V2 should not do limit again

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38768: Assignee: Apache Spark > If limit could pushed down and Data source only have one

[jira] [Created] (SPARK-38769) [SQL] behavior schema_of_json not same with 2.4.0

2022-04-01 Thread gabrywu (Jira)
gabrywu created SPARK-38769: --- Summary: [SQL] behavior schema_of_json not same with 2.4.0 Key: SPARK-38769 URL: https://issues.apache.org/jira/browse/SPARK-38769 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-38769) [SQL] behavior of schema_of_json not same with 2.4.0

2022-04-01 Thread gabrywu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gabrywu updated SPARK-38769: Summary: [SQL] behavior of schema_of_json not same with 2.4.0 (was: [SQL] behavior schema_of_json not

[jira] [Created] (SPARK-38768) If limit could pushed down and Data source only have one partition, DS V2 should not do limit again

2022-04-01 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-38768: -- Summary: If limit could pushed down and Data source only have one partition, DS V2 should not do limit again Key: SPARK-38768 URL: https://issues.apache.org/jira/browse/SPARK-38768

[jira] [Resolved] (SPARK-34863) Support nested column in Spark Parquet vectorized readers

2022-04-01 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-34863. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34659

[jira] [Assigned] (SPARK-38620) Replace `value.formatted(formatString)` with `formatString.format(value)` to clean up compilation warning

2022-04-01 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-38620: Assignee: Yang Jie > Replace `value.formatted(formatString)` with

[jira] [Resolved] (SPARK-38620) Replace `value.formatted(formatString)` with `formatString.format(value)` to clean up compilation warning

2022-04-01 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38620. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 35930

[jira] [Updated] (SPARK-38620) Replace `value.formatted(formatString)` with `formatString.format(value)` to clean up compilation warning

2022-04-01 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-38620: - Priority: Trivial (was: Minor) > Replace `value.formatted(formatString)` with

[jira] [Created] (SPARK-38767) Support ignoreCorruptFiles and ignoreMissingFiles in Data Source options

2022-04-01 Thread Yaohua Zhao (Jira)
Yaohua Zhao created SPARK-38767: --- Summary: Support ignoreCorruptFiles and ignoreMissingFiles in Data Source options Key: SPARK-38767 URL: https://issues.apache.org/jira/browse/SPARK-38767 Project:

[jira] [Commented] (SPARK-38763) Pandas API on spark Can`t apply lamda to columns.

2022-04-01 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516177#comment-17516177 ] Xinrong Meng commented on SPARK-38763: -- I will backport the fix after approved and merged. >

[jira] [Commented] (SPARK-38763) Pandas API on spark Can`t apply lamda to columns.

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516176#comment-17516176 ] Apache Spark commented on SPARK-38763: -- User 'xinrong-databricks' has created a pull request for

[jira] [Commented] (SPARK-38763) Pandas API on spark Can`t apply lamda to columns.

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516175#comment-17516175 ] Apache Spark commented on SPARK-38763: -- User 'xinrong-databricks' has created a pull request for

[jira] [Assigned] (SPARK-38763) Pandas API on spark Can`t apply lamda to columns.

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38763: Assignee: Apache Spark > Pandas API on spark Can`t apply lamda to columns. >

[jira] [Assigned] (SPARK-38763) Pandas API on spark Can`t apply lamda to columns.

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38763: Assignee: (was: Apache Spark) > Pandas API on spark Can`t apply lamda to columns.

[jira] [Comment Edited] (SPARK-38763) Pandas API on spark Can`t apply lamda to columns.

2022-04-01 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516172#comment-17516172 ] Xinrong Meng edited comment on SPARK-38763 at 4/1/22 11:56 PM: --- Hi

[jira] [Resolved] (SPARK-38766) Support lambda `column` parameter of `DataFrame.rename`

2022-04-01 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-38766. -- Resolution: Duplicate > Support lambda `column` parameter of `DataFrame.rename` >

[jira] [Commented] (SPARK-38763) Pandas API on spark Can`t apply lamda to columns.

2022-04-01 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516172#comment-17516172 ] Xinrong Meng commented on SPARK-38763: -- Hi [~bjornjorgensen], thanks for raising that! The

[jira] [Commented] (SPARK-38766) Support lambda `column` parameter of `DataFrame.rename`

2022-04-01 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516173#comment-17516173 ] Xinrong Meng commented on SPARK-38766: -- I am working on that. > Support lambda `column` parameter

[jira] [Created] (SPARK-38766) Support lambda `column` parameter of `DataFrame.rename`

2022-04-01 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-38766: Summary: Support lambda `column` parameter of `DataFrame.rename` Key: SPARK-38766 URL: https://issues.apache.org/jira/browse/SPARK-38766 Project: Spark

[jira] [Commented] (SPARK-38763) Pandas API on spark Can`t apply lamda to columns.

2022-04-01 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516087#comment-17516087 ] Bjørn Jørgensen commented on SPARK-38763: - This error is

[jira] [Commented] (SPARK-38765) Implement `inplace` parameter of `Series.clip`

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516075#comment-17516075 ] Apache Spark commented on SPARK-38765: -- User 'xinrong-databricks' has created a pull request for

[jira] [Assigned] (SPARK-38765) Implement `inplace` parameter of `Series.clip`

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38765: Assignee: Apache Spark > Implement `inplace` parameter of `Series.clip` >

[jira] [Assigned] (SPARK-38765) Implement `inplace` parameter of `Series.clip`

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38765: Assignee: (was: Apache Spark) > Implement `inplace` parameter of `Series.clip` >

[jira] [Commented] (SPARK-38765) Implement `inplace` parameter of `Series.clip`

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516074#comment-17516074 ] Apache Spark commented on SPARK-38765: -- User 'xinrong-databricks' has created a pull request for

[jira] [Created] (SPARK-38765) Implement `inplace` parameter of `Series.clip`

2022-04-01 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-38765: Summary: Implement `inplace` parameter of `Series.clip` Key: SPARK-38765 URL: https://issues.apache.org/jira/browse/SPARK-38765 Project: Spark Issue Type:

[jira] [Created] (SPARK-38764) spark thrift server issue: Length field is empty for varchar fields

2022-04-01 Thread Ayan Ray (Jira)
Ayan Ray created SPARK-38764: Summary: spark thrift server issue: Length field is empty for varchar fields Key: SPARK-38764 URL: https://issues.apache.org/jira/browse/SPARK-38764 Project: Spark

[jira] [Commented] (SPARK-37903) Replace string_typehints with get_type_hints.

2022-04-01 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-37903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516012#comment-17516012 ] Bjørn Jørgensen commented on SPARK-37903: - I do have some problems now when I use lamba on

[jira] (SPARK-38763) Pandas API on spark Can`t apply lamda to columns.

2022-04-01 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38763 ] Bjørn Jørgensen deleted comment on SPARK-38763: - was (Author: bjornjorgensen): [~XinrongM] > Pandas API on spark Can`t apply lamda to columns. >

[jira] [Commented] (SPARK-38763) Pandas API on spark Can`t apply lamda to columns.

2022-04-01 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515991#comment-17515991 ] Bjørn Jørgensen commented on SPARK-38763: - [~XinrongM] > Pandas API on spark Can`t apply lamda

[jira] [Created] (SPARK-38763) Pandas API on spark Can`t apply lamda to columns.

2022-04-01 Thread Jira
Bjørn Jørgensen created SPARK-38763: --- Summary: Pandas API on spark Can`t apply lamda to columns. Key: SPARK-38763 URL: https://issues.apache.org/jira/browse/SPARK-38763 Project: Spark

[jira] [Updated] (SPARK-38614) After Spark update, df.show() shows incorrect F.percent_rank results

2022-04-01 Thread ZygD (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZygD updated SPARK-38614: - Component/s: SQL > After Spark update, df.show() shows incorrect F.percent_rank results >

[jira] [Comment Edited] (SPARK-38703) High GC and memory footprint after switch to ZSTD

2022-04-01 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515907#comment-17515907 ] Cheng Pan edited comment on SPARK-38703 at 4/1/22 12:56 PM: SPARK-34390 may

[jira] [Comment Edited] (SPARK-38703) High GC and memory footprint after switch to ZSTD

2022-04-01 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515907#comment-17515907 ] Cheng Pan edited comment on SPARK-38703 at 4/1/22 12:55 PM: SPARK-34390 may

[jira] [Commented] (SPARK-38703) High GC and memory footprint after switch to ZSTD

2022-04-01 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515907#comment-17515907 ] Cheng Pan commented on SPARK-38703: --- SPARK-34390 may helps, our benchmark of 1T TPC-DS shows the

[jira] [Assigned] (SPARK-38762) Provide query context in Decimal overflow errors

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38762: Assignee: Gengliang Wang (was: Apache Spark) > Provide query context in Decimal

[jira] [Commented] (SPARK-38762) Provide query context in Decimal overflow errors

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515890#comment-17515890 ] Apache Spark commented on SPARK-38762: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-38762) Provide query context in Decimal overflow errors

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515889#comment-17515889 ] Apache Spark commented on SPARK-38762: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-38762) Provide query context in Decimal overflow errors

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38762: Assignee: Apache Spark (was: Gengliang Wang) > Provide query context in Decimal

[jira] [Created] (SPARK-38762) Provide query context in Decimal overflow errors

2022-04-01 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-38762: -- Summary: Provide query context in Decimal overflow errors Key: SPARK-38762 URL: https://issues.apache.org/jira/browse/SPARK-38762 Project: Spark Issue

[jira] [Assigned] (SPARK-38758) Web UI add heap dump

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38758: Assignee: Apache Spark > Web UI add heap dump > - > >

[jira] [Commented] (SPARK-38758) Web UI add heap dump

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515865#comment-17515865 ] Apache Spark commented on SPARK-38758: -- User 'cutiechi' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38758) Web UI add heap dump

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38758: Assignee: (was: Apache Spark) > Web UI add heap dump > - > >

[jira] [Assigned] (SPARK-38759) Add StreamingQueryListener support in PySpark

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38759: Assignee: (was: Apache Spark) > Add StreamingQueryListener support in PySpark >

[jira] [Assigned] (SPARK-38761) DS V2 supports push down misc non-aggregate functions

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38761: Assignee: Apache Spark > DS V2 supports push down misc non-aggregate functions >

[jira] [Commented] (SPARK-38761) DS V2 supports push down misc non-aggregate functions

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515861#comment-17515861 ] Apache Spark commented on SPARK-38761: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38759) Add StreamingQueryListener support in PySpark

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38759: Assignee: Apache Spark > Add StreamingQueryListener support in PySpark >

[jira] [Assigned] (SPARK-38761) DS V2 supports push down misc non-aggregate functions

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38761: Assignee: (was: Apache Spark) > DS V2 supports push down misc non-aggregate

[jira] [Commented] (SPARK-38759) Add StreamingQueryListener support in PySpark

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515862#comment-17515862 ] Apache Spark commented on SPARK-38759: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Created] (SPARK-38761) DS V2 supports push down misc non-aggregate functions

2022-04-01 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-38761: -- Summary: DS V2 supports push down misc non-aggregate functions Key: SPARK-38761 URL: https://issues.apache.org/jira/browse/SPARK-38761 Project: Spark Issue

[jira] [Assigned] (SPARK-38684) Stream-stream outer join has a possible correctness issue due to weakly read consistent on outer iterators

2022-04-01 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-38684: Assignee: Jungtaek Lim > Stream-stream outer join has a possible correctness issue due

[jira] [Resolved] (SPARK-38684) Stream-stream outer join has a possible correctness issue due to weakly read consistent on outer iterators

2022-04-01 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-38684. -- Fix Version/s: 3.3.0 3.2.2 Resolution: Fixed Issue resolved by pull

[jira] [Created] (SPARK-38760) Implement DataFrame.observe in PySpark

2022-04-01 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-38760: Summary: Implement DataFrame.observe in PySpark Key: SPARK-38760 URL: https://issues.apache.org/jira/browse/SPARK-38760 Project: Spark Issue Type:

[jira] [Created] (SPARK-38759) Add StreamingQueryListener support in PySpark

2022-04-01 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-38759: Summary: Add StreamingQueryListener support in PySpark Key: SPARK-38759 URL: https://issues.apache.org/jira/browse/SPARK-38759 Project: Spark Issue Type:

[jira] [Commented] (SPARK-38358) Add migration guide for spark.sql.hive.convertMetastoreInsertDir and spark.sql.hive.convertMetastoreCtas

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515836#comment-17515836 ] Apache Spark commented on SPARK-38358: -- User 'cutiechi' has created a pull request for this issue:

[jira] [Created] (SPARK-38758) Web UI add heap dump

2022-04-01 Thread Jinpeng Chi (Jira)
Jinpeng Chi created SPARK-38758: --- Summary: Web UI add heap dump Key: SPARK-38758 URL: https://issues.apache.org/jira/browse/SPARK-38758 Project: Spark Issue Type: New Feature

[jira] [Assigned] (SPARK-38757) Update the Oracle docker image version used for test and integration

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38757: Assignee: Apache Spark > Update the Oracle docker image version used for test and

[jira] [Commented] (SPARK-38757) Update the Oracle docker image version used for test and integration

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515807#comment-17515807 ] Apache Spark commented on SPARK-38757: -- User 'LucaCanali' has created a pull request for this

[jira] [Assigned] (SPARK-38757) Update the Oracle docker image version used for test and integration

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38757: Assignee: (was: Apache Spark) > Update the Oracle docker image version used for test

[jira] [Updated] (SPARK-38757) Update the Oracle docker image version used for test and integration

2022-04-01 Thread Luca Canali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Canali updated SPARK-38757: Summary: Update the Oracle docker image version used for test and integration (was: Update the

[jira] [Created] (SPARK-38757) Update the Oracle docker image

2022-04-01 Thread Luca Canali (Jira)
Luca Canali created SPARK-38757: --- Summary: Update the Oracle docker image Key: SPARK-38757 URL: https://issues.apache.org/jira/browse/SPARK-38757 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-37528) Schedule Tasks By Input Size

2022-04-01 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-37528: -- Affects Version/s: 3.4.0 (was: 3.3.0) > Schedule Tasks By Input Size >

[jira] [Assigned] (SPARK-38756) Clean up useless security configs in `TransportConf`

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38756: Assignee: (was: Apache Spark) > Clean up useless security configs in `TransportConf`

[jira] [Assigned] (SPARK-38756) Clean up useless security configs in `TransportConf`

2022-04-01 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38756: Assignee: Apache Spark > Clean up useless security configs in `TransportConf` >

[jira] [Created] (SPARK-38756) Clean up useless security configs in `TransportConf`

2022-04-01 Thread Yang Jie (Jira)
Yang Jie created SPARK-38756: Summary: Clean up useless security configs in `TransportConf` Key: SPARK-38756 URL: https://issues.apache.org/jira/browse/SPARK-38756 Project: Spark Issue Type:

[jira] [Updated] (SPARK-37528) Schedule Tasks By Input Size

2022-04-01 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-37528: -- Summary: Schedule Tasks By Input Size (was: Support reorder tasks during scheduling by shuffle

[jira] [Updated] (SPARK-37528) Support reorder tasks during scheduling by shuffle partition size in AQE

2022-04-01 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-37528: -- Description: In general, the larger input data size means longer running time. So ideally, we can