[jira] [Assigned] (SPARK-34637) Support DPP in AQE when the boradcast exchange can be reused

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34637: Assignee: (was: Apache Spark) > Support DPP in AQE when the boradcast exchange can

[jira] [Assigned] (SPARK-34637) Support DPP in AQE when the boradcast exchange can be reused

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34637: Assignee: Apache Spark > Support DPP in AQE when the boradcast exchange can be reused >

[jira] [Commented] (SPARK-34637) Support DPP in AQE when the boradcast exchange can be reused

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295840#comment-17295840 ] Apache Spark commented on SPARK-34637: -- User 'JkSelf' has created a pull request for this issue:

[jira] [Commented] (SPARK-34637) Support DPP in AQE when the boradcast exchange can be reused

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295839#comment-17295839 ] Apache Spark commented on SPARK-34637: -- User 'JkSelf' has created a pull request for this issue:

[jira] [Updated] (SPARK-34637) Support DPP in AQE when the boradcast exchange can reused

2021-03-04 Thread Ke Jia (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ke Jia updated SPARK-34637: --- Description: We have supported DPP in AQE when the join is Broadcast hash join before applying the AQE

[jira] [Updated] (SPARK-34637) Support DPP in AQE when the boradcast exchange can be reused

2021-03-04 Thread Ke Jia (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ke Jia updated SPARK-34637: --- Summary: Support DPP in AQE when the boradcast exchange can be reused (was: Support DPP in AQE when the

[jira] [Resolved] (SPARK-34621) Unify output of ShowCreateTableAsSerdeCommand ShowCreateTableCommand

2021-03-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34621. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31737

[jira] [Assigned] (SPARK-34621) Unify output of ShowCreateTableAsSerdeCommand ShowCreateTableCommand

2021-03-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34621: --- Assignee: angerszhu > Unify output of ShowCreateTableAsSerdeCommand

[jira] [Resolved] (SPARK-22748) Error in query: grouping_id() can only be used with GroupingSets/Cube/Rollup;

2021-03-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22748. - Fix Version/s: 3.2 Assignee: Takeshi Yamamuro Resolution: Fixed > Error in

[jira] [Created] (SPARK-34637) Support DPP in AQE when the boradcast exchange can reused

2021-03-04 Thread Ke Jia (Jira)
Ke Jia created SPARK-34637: -- Summary: Support DPP in AQE when the boradcast exchange can reused Key: SPARK-34637 URL: https://issues.apache.org/jira/browse/SPARK-34637 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-34626) UnresolvedAttribute.sql may return an incorrect sql

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34626: Assignee: Apache Spark > UnresolvedAttribute.sql may return an incorrect sql >

[jira] [Assigned] (SPARK-34626) UnresolvedAttribute.sql may return an incorrect sql

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34626: Assignee: (was: Apache Spark) > UnresolvedAttribute.sql may return an incorrect sql

[jira] [Commented] (SPARK-34626) UnresolvedAttribute.sql may return an incorrect sql

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295828#comment-17295828 ] Apache Spark commented on SPARK-34626: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34636) sql method in UnresolvedAttribute, AttributeReference and Alias don't quote qualified names properly.

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34636: Assignee: Kousuke Saruta (was: Apache Spark) > sql method in UnresolvedAttribute,

[jira] [Commented] (SPARK-34392) Invalid ID for offset-based ZoneId since Spark 3.0

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295827#comment-17295827 ] Apache Spark commented on SPARK-34392: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34636) sql method in UnresolvedAttribute, AttributeReference and Alias don't quote qualified names properly.

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34636: Assignee: Apache Spark (was: Kousuke Saruta) > sql method in UnresolvedAttribute,

[jira] [Commented] (SPARK-34636) sql method in UnresolvedAttribute, AttributeReference and Alias don't quote qualified names properly.

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295826#comment-17295826 ] Apache Spark commented on SPARK-34636: -- User 'sarutak' has created a pull request for this issue:

[jira] [Updated] (SPARK-34636) sql method in UnresolvedAttribute, AttributeReference and Alias don't quote qualified names properly.

2021-03-04 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-34636: --- Description: UnresolvedReference, AttributeReference and Alias which take qualifier don't

[jira] [Commented] (SPARK-27997) kubernetes client token expired

2021-03-04 Thread rameshkrishnan muthusamy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295825#comment-17295825 ] rameshkrishnan muthusamy commented on SPARK-27997: -- The PR to upgrade the underlying

[jira] [Resolved] (SPARK-34635) Add trailing slash in URLs to reduce unnecessary redirect

2021-03-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34635. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31753

[jira] [Updated] (SPARK-34636) sql method in UnresolvedAttribute, AttributeReference and Alias don't quote qualified names properly.

2021-03-04 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-34636: --- Description: UnresolvedReference, AttributeReference and Alias which take qualifier don't

[jira] [Created] (SPARK-34636) sql method in UnresolvedAttribute, AttributeReference and Alias don't quote qualified names properly.

2021-03-04 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-34636: -- Summary: sql method in UnresolvedAttribute, AttributeReference and Alias don't quote qualified names properly. Key: SPARK-34636 URL:

[jira] [Updated] (SPARK-34631) Caught Hive MetaException when query by partition (partition col start with ‘$’)

2021-03-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-34631: Fix Version/s: (was: 2.4.4) > Caught Hive MetaException when query by partition (partition

[jira] [Updated] (SPARK-34631) Caught Hive MetaException when query by partition (partition col start with ‘$’)

2021-03-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-34631: Target Version/s: (was: 2.4.4) > Caught Hive MetaException when query by partition (partition

[jira] [Resolved] (SPARK-34609) unify resolveExpressionBottomUp and resolveExpressionTopDown

2021-03-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34609. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31728

[jira] [Commented] (SPARK-34626) UnresolvedAttribute.sql may return an incorrect sql

2021-03-04 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295782#comment-17295782 ] Kousuke Saruta commented on SPARK-34626: I've found some similar issues and trying to fix them

[jira] [Commented] (SPARK-34635) Add trailing slash in URLs to reduce unnecessary redirect

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295781#comment-17295781 ] Apache Spark commented on SPARK-34635: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-34635) Add trailing slash in URLs to reduce unnecessary redirect

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34635: Assignee: Apache Spark (was: Gengliang Wang) > Add trailing slash in URLs to reduce

[jira] [Assigned] (SPARK-34635) Add trailing slash in URLs to reduce unnecessary redirect

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34635: Assignee: Gengliang Wang (was: Apache Spark) > Add trailing slash in URLs to reduce

[jira] [Commented] (SPARK-34635) Add trailing slash in URLs to reduce unnecessary redirect

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295780#comment-17295780 ] Apache Spark commented on SPARK-34635: -- User 'gengliangwang' has created a pull request for this

[jira] [Created] (SPARK-34635) Add trailing slash in URLs to reduce unnecessary redirect

2021-03-04 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-34635: -- Summary: Add trailing slash in URLs to reduce unnecessary redirect Key: SPARK-34635 URL: https://issues.apache.org/jira/browse/SPARK-34635 Project: Spark

[jira] [Resolved] (SPARK-34608) Remove unused output in AddJarCommand

2021-03-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34608. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31725

[jira] [Assigned] (SPARK-34608) Remove unused output in AddJarCommand

2021-03-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34608: - Assignee: angerszhu > Remove unused output in AddJarCommand >

[jira] [Updated] (SPARK-34634) Self-join with script transformation failed to resolve attribute correctly

2021-03-04 Thread EdisonWang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] EdisonWang updated SPARK-34634: --- Description: To reproduce, ``` create temporary view t as select * from values 0, 1, 2 as t(a);

[jira] [Updated] (SPARK-34634) Self-join with script transformation failed to resolve attribute correctly

2021-03-04 Thread EdisonWang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] EdisonWang updated SPARK-34634: --- Description: To reproduce, {code:java} // code placeholder create temporary view t as select * from

[jira] [Updated] (SPARK-34634) Self-join with script transformation failed to resolve attribute correctly

2021-03-04 Thread EdisonWang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] EdisonWang updated SPARK-34634: --- Description: To reproduce, {code:java} // code placeholder create temporary view t as select * from

[jira] [Assigned] (SPARK-34634) Self-join with script transformation failed to resolve attribute correctly

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34634: Assignee: Apache Spark > Self-join with script transformation failed to resolve

[jira] [Assigned] (SPARK-34634) Self-join with script transformation failed to resolve attribute correctly

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34634: Assignee: Apache Spark > Self-join with script transformation failed to resolve

[jira] [Commented] (SPARK-34634) Self-join with script transformation failed to resolve attribute correctly

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295764#comment-17295764 ] Apache Spark commented on SPARK-34634: -- User 'WangGuangxin' has created a pull request for this

[jira] [Assigned] (SPARK-34634) Self-join with script transformation failed to resolve attribute correctly

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34634: Assignee: (was: Apache Spark) > Self-join with script transformation failed to

[jira] [Commented] (SPARK-34588) Support int64 buffer lengths in Java for pyspark Pandas UDF as buffer expanding

2021-03-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295763#comment-17295763 ] Hyukjin Kwon commented on SPARK-34588: -- Thank you [~dishka_krauch]. > Support int64 buffer lengths

[jira] [Commented] (SPARK-22748) Error in query: grouping_id() can only be used with GroupingSets/Cube/Rollup;

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295760#comment-17295760 ] Apache Spark commented on SPARK-22748: -- User 'maropu' has created a pull request for this issue:

[jira] [Created] (SPARK-34634) Self-join with script transformation failed to resolve attribute correctly

2021-03-04 Thread EdisonWang (Jira)
EdisonWang created SPARK-34634: -- Summary: Self-join with script transformation failed to resolve attribute correctly Key: SPARK-34634 URL: https://issues.apache.org/jira/browse/SPARK-34634 Project:

[jira] [Commented] (SPARK-22748) Error in query: grouping_id() can only be used with GroupingSets/Cube/Rollup;

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295759#comment-17295759 ] Apache Spark commented on SPARK-22748: -- User 'maropu' has created a pull request for this issue:

[jira] [Created] (SPARK-34633) Self-join with script transformation failed to resolve attribute correctly

2021-03-04 Thread EdisonWang (Jira)
EdisonWang created SPARK-34633: -- Summary: Self-join with script transformation failed to resolve attribute correctly Key: SPARK-34633 URL: https://issues.apache.org/jira/browse/SPARK-34633 Project:

[jira] [Resolved] (SPARK-34613) Fix view does not capture disable hint config

2021-03-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34613. - Fix Version/s: 3.1.2 3.2.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-34613) Fix view does not capture disable hint config

2021-03-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34613: --- Assignee: ulysses you > Fix view does not capture disable hint config >

[jira] [Commented] (SPARK-34629) Python type hints improvement

2021-03-04 Thread Rafal Wojdyla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295753#comment-17295753 ] Rafal Wojdyla commented on SPARK-34629: --- Thanks [~hyukjin.kwon]! > Python type hints improvement

[jira] [Created] (SPARK-34632) Can we create 'SessionState' with a username in 'HiveClientImpl'

2021-03-04 Thread HonglunChen (Jira)
HonglunChen created SPARK-34632: --- Summary: Can we create 'SessionState' with a username in 'HiveClientImpl' Key: SPARK-34632 URL: https://issues.apache.org/jira/browse/SPARK-34632 Project: Spark

[jira] [Assigned] (SPARK-34376) Support regexp as a function

2021-03-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-34376: Assignee: Kent Yao > Support regexp as a function > > >

[jira] [Resolved] (SPARK-34376) Support regexp as a function

2021-03-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34376. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31488

[jira] [Assigned] (SPARK-34628) Remove GlobalLimit operator if it's child max row <= limit

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34628: Assignee: Apache Spark > Remove GlobalLimit operator if it's child max row <= limit >

[jira] [Commented] (SPARK-34628) Remove GlobalLimit operator if it's child max row <= limit

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295704#comment-17295704 ] Apache Spark commented on SPARK-34628: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34628) Remove GlobalLimit operator if it's child max row <= limit

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34628: Assignee: Apache Spark > Remove GlobalLimit operator if it's child max row <= limit >

[jira] [Assigned] (SPARK-34628) Remove GlobalLimit operator if it's child max row <= limit

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34628: Assignee: (was: Apache Spark) > Remove GlobalLimit operator if it's child max row <=

[jira] [Created] (SPARK-34631) Caught Hive MetaException when query by partition (partition col start with ‘$’)

2021-03-04 Thread zhouyuan (Jira)
zhouyuan created SPARK-34631: Summary: Caught Hive MetaException when query by partition (partition col start with ‘$’) Key: SPARK-34631 URL: https://issues.apache.org/jira/browse/SPARK-34631 Project:

[jira] [Assigned] (SPARK-32924) Web UI sort on duration is wrong

2021-03-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32924: - Assignee: Baohe Zhang > Web UI sort on duration is wrong >

[jira] [Updated] (SPARK-34629) Python type hints improvement

2021-03-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34629: - Shepherd: Maciej Szymkiewicz > Python type hints improvement > - >

[jira] [Commented] (SPARK-34629) Python type hints improvement

2021-03-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295700#comment-17295700 ] Hyukjin Kwon commented on SPARK-34629: -- cc [~ravwojdyla] too FYI > Python type hints improvement >

[jira] [Updated] (SPARK-34629) Python type hints improvement

2021-03-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34629: - Description: We added PySpark type hints at SPARK-32681 However, looks like there are still

[jira] [Commented] (SPARK-34629) Python type hints improvement

2021-03-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295698#comment-17295698 ] Hyukjin Kwon commented on SPARK-34629: -- cc [~zero323], [~itholic], [~ueshin], [~XinrongM] FYI >

[jira] [Updated] (SPARK-34544) pyspark toPandas() should return pd.DataFrame

2021-03-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34544: - Parent: SPARK-34629 Issue Type: Sub-task (was: Bug) > pyspark toPandas() should return

[jira] [Created] (SPARK-34630) Add type hints of pyspark.__version__ and pyspark.sql.Column.contains

2021-03-04 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-34630: Summary: Add type hints of pyspark.__version__ and pyspark.sql.Column.contains Key: SPARK-34630 URL: https://issues.apache.org/jira/browse/SPARK-34630 Project: Spark

[jira] [Created] (SPARK-34629) Python type hints improvement

2021-03-04 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-34629: Summary: Python type hints improvement Key: SPARK-34629 URL: https://issues.apache.org/jira/browse/SPARK-34629 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-34628) Remove GlobalLimit operator if it's child max row <= limit

2021-03-04 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-34628: --- Summary: Remove GlobalLimit operator if it's child max row <= limit Key: SPARK-34628 URL: https://issues.apache.org/jira/browse/SPARK-34628 Project: Spark

[jira] [Commented] (SPARK-34627) Use FunctionIdentifier in UnresolvedTableValuedFunction

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295690#comment-17295690 ] Apache Spark commented on SPARK-34627: -- User 'allisonwang-db' has created a pull request for this

[jira] [Commented] (SPARK-34627) Use FunctionIdentifier in UnresolvedTableValuedFunction

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295689#comment-17295689 ] Apache Spark commented on SPARK-34627: -- User 'allisonwang-db' has created a pull request for this

[jira] [Assigned] (SPARK-34627) Use FunctionIdentifier in UnresolvedTableValuedFunction

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34627: Assignee: (was: Apache Spark) > Use FunctionIdentifier in

[jira] [Assigned] (SPARK-34627) Use FunctionIdentifier in UnresolvedTableValuedFunction

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34627: Assignee: Apache Spark > Use FunctionIdentifier in UnresolvedTableValuedFunction >

[jira] [Updated] (SPARK-34627) Use FunctionIdentifier in UnresolvedTableValuedFunction

2021-03-04 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-34627: - Description: Currently, UnresolvedTableValuedFunction uses a string as its function name, which

[jira] [Updated] (SPARK-34622) Push down limit through Project

2021-03-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-34622: Issue Type: Improvement (was: Bug) > Push down limit through Project >

[jira] [Updated] (SPARK-34622) Push down limit through Project

2021-03-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-34622: Summary: Push down limit through Project (was: Fix push down limit through join) > Push down

[jira] [Created] (SPARK-34627) Use FunctionIdentifier in UnresolvedTableValuedFunction

2021-03-04 Thread Allison Wang (Jira)
Allison Wang created SPARK-34627: Summary: Use FunctionIdentifier in UnresolvedTableValuedFunction Key: SPARK-34627 URL: https://issues.apache.org/jira/browse/SPARK-34627 Project: Spark

[jira] [Commented] (SPARK-34607) NewInstance.resolved should not throw malformed class name error

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295661#comment-17295661 ] Apache Spark commented on SPARK-34607: -- User 'maropu' has created a pull request for this issue:

[jira] [Commented] (SPARK-34607) NewInstance.resolved should not throw malformed class name error

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295658#comment-17295658 ] Apache Spark commented on SPARK-34607: -- User 'maropu' has created a pull request for this issue:

[jira] [Resolved] (SPARK-34607) NewInstance.resolved should not throw malformed class name error

2021-03-04 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-34607. -- Fix Version/s: 3.2.0 Assignee: Takeshi Yamamuro Resolution: Fixed

[jira] [Commented] (SPARK-34607) NewInstance.resolved should not throw malformed class name error

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295656#comment-17295656 ] Apache Spark commented on SPARK-34607: -- User 'maropu' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-34624) Filter non-jar dependencies from ivy/maven coordinates

2021-03-04 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295650#comment-17295650 ] Erik Krogen edited comment on SPARK-34624 at 3/5/21, 12:01 AM: --- Thanks for

[jira] [Commented] (SPARK-34624) Filter non-jar dependencies from ivy/maven coordinates

2021-03-04 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295650#comment-17295650 ] Erik Krogen commented on SPARK-34624: - Thanks for reporting this [~shardulm]! One question for you:

[jira] [Commented] (SPARK-32924) Web UI sort on duration is wrong

2021-03-04 Thread Baohe Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295639#comment-17295639 ] Baohe Zhang commented on SPARK-32924: - [~dongjoon] This is my Jira id. > Web UI sort on duration is

[jira] [Resolved] (SPARK-32924) Web UI sort on duration is wrong

2021-03-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32924. --- Fix Version/s: 3.0.3 3.1.2 2.4.8

[jira] [Updated] (SPARK-32924) Web UI sort on duration is wrong

2021-03-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32924: -- Affects Version/s: 3.2.0 2.4.7 3.0.2

[jira] [Updated] (SPARK-34626) UnresolvedAttribute.sql may return an incorrect sql

2021-03-04 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-34626: - Description: UnresolvedAttribute("a" :: "b" :: Nil).sql returns {code:java} `a.b`{code} The

[jira] [Created] (SPARK-34626) UnresolvedAttribute.sql may return an incorrect sql

2021-03-04 Thread Shixiong Zhu (Jira)
Shixiong Zhu created SPARK-34626: Summary: UnresolvedAttribute.sql may return an incorrect sql Key: SPARK-34626 URL: https://issues.apache.org/jira/browse/SPARK-34626 Project: Spark Issue

[jira] [Created] (SPARK-34625) Enable Arrow optimization for additional supported types when using SparkR

2021-03-04 Thread Matthew Summersgill (Jira)
Matthew Summersgill created SPARK-34625: --- Summary: Enable Arrow optimization for additional supported types when using SparkR Key: SPARK-34625 URL: https://issues.apache.org/jira/browse/SPARK-34625

[jira] [Updated] (SPARK-34625) Enable Arrow optimization for additional supported types when using SparkR

2021-03-04 Thread Matthew Summersgill (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Summersgill updated SPARK-34625: Description: I am opening this issue to track pull request here:

[jira] [Commented] (SPARK-33000) cleanCheckpoints config does not clean all checkpointed RDDs on shutdown

2021-03-04 Thread Haoyuan Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295612#comment-17295612 ] Haoyuan Wang commented on SPARK-33000: -- [~nchammas] absolutely...I left a comment, I think you've

[jira] [Assigned] (SPARK-32924) Web UI sort on duration is wrong

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32924: Assignee: Apache Spark > Web UI sort on duration is wrong >

[jira] [Commented] (SPARK-32924) Web UI sort on duration is wrong

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295576#comment-17295576 ] Apache Spark commented on SPARK-32924: -- User 'baohe-zhang' has created a pull request for this

[jira] [Assigned] (SPARK-32924) Web UI sort on duration is wrong

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32924: Assignee: (was: Apache Spark) > Web UI sort on duration is wrong >

[jira] [Commented] (SPARK-33000) cleanCheckpoints config does not clean all checkpointed RDDs on shutdown

2021-03-04 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295531#comment-17295531 ] Nicholas Chammas commented on SPARK-33000: -- [~caowang888] - If you're still interested in this

[jira] [Assigned] (SPARK-33000) cleanCheckpoints config does not clean all checkpointed RDDs on shutdown

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33000: Assignee: (was: Apache Spark) > cleanCheckpoints config does not clean all

[jira] [Commented] (SPARK-33000) cleanCheckpoints config does not clean all checkpointed RDDs on shutdown

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295529#comment-17295529 ] Apache Spark commented on SPARK-33000: -- User 'nchammas' has created a pull request for this issue:

[jira] [Commented] (SPARK-33000) cleanCheckpoints config does not clean all checkpointed RDDs on shutdown

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295528#comment-17295528 ] Apache Spark commented on SPARK-33000: -- User 'nchammas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33000) cleanCheckpoints config does not clean all checkpointed RDDs on shutdown

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33000: Assignee: Apache Spark > cleanCheckpoints config does not clean all checkpointed RDDs on

[jira] [Assigned] (SPARK-34624) Filter non-jar dependencies from ivy/maven coordinates

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34624: Assignee: (was: Apache Spark) > Filter non-jar dependencies from ivy/maven

[jira] [Assigned] (SPARK-34624) Filter non-jar dependencies from ivy/maven coordinates

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34624: Assignee: Apache Spark > Filter non-jar dependencies from ivy/maven coordinates >

[jira] [Commented] (SPARK-34624) Filter non-jar dependencies from ivy/maven coordinates

2021-03-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295503#comment-17295503 ] Apache Spark commented on SPARK-34624: -- User 'shardulm94' has created a pull request for this

[jira] [Created] (SPARK-34624) Filter non-jar dependencies from ivy/maven coordinates

2021-03-04 Thread Shardul Mahadik (Jira)
Shardul Mahadik created SPARK-34624: --- Summary: Filter non-jar dependencies from ivy/maven coordinates Key: SPARK-34624 URL: https://issues.apache.org/jira/browse/SPARK-34624 Project: Spark

[jira] [Assigned] (SPARK-34605) Support java.time.Duration as an external type of the day-time interval type

2021-03-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34605: --- Assignee: Maxim Gekk > Support java.time.Duration as an external type of the day-time

  1   2   >