[jira] [Commented] (SPARK-34105) In addition to killing exlcuded/flakey executors which should support decommissioning

2021-08-08 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395833#comment-17395833 ] Gengliang Wang commented on SPARK-34105: [~holden][~hyukjin.kwon]Shall we mark this as done? >

[jira] [Commented] (SPARK-34104) Allow users to specify a maximum decommissioning time

2021-08-08 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395832#comment-17395832 ] Gengliang Wang commented on SPARK-34104: [~holden][~hyukjin.kwon]Shall we mark this as done? >

[jira] [Created] (SPARK-36457) Review and fix issues in API docs

2021-08-08 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-36457: -- Summary: Review and fix issues in API docs Key: SPARK-36457 URL: https://issues.apache.org/jira/browse/SPARK-36457 Project: Spark Issue Type:

[jira] [Commented] (SPARK-34198) Add RocksDB StateStore implementation

2021-08-08 Thread Yuanjian Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395822#comment-17395822 ] Yuanjian Li commented on SPARK-34198: - [~Gengliang.Wang] Thanks for reminding. I'll submit the

[jira] [Commented] (SPARK-36456) Clean up the depredation usage of o.a.c.io.IOUtils.closeQuietly

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395737#comment-17395737 ] Apache Spark commented on SPARK-36456: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-36456) Clean up the depredation usage of o.a.c.io.IOUtils.closeQuietly

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36456: Assignee: (was: Apache Spark) > Clean up the depredation usage of

[jira] [Assigned] (SPARK-36456) Clean up the depredation usage of o.a.c.io.IOUtils.closeQuietly

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36456: Assignee: Apache Spark > Clean up the depredation usage of

[jira] [Assigned] (SPARK-36455) Provide an example of complex session window via flatMapGroupsWithState

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36455: Assignee: (was: Apache Spark) > Provide an example of complex session window via

[jira] [Commented] (SPARK-36455) Provide an example of complex session window via flatMapGroupsWithState

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395736#comment-17395736 ] Apache Spark commented on SPARK-36455: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Assigned] (SPARK-36455) Provide an example of complex session window via flatMapGroupsWithState

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36455: Assignee: Apache Spark > Provide an example of complex session window via

[jira] [Updated] (SPARK-36456) Clean up the depredation usage of o.a.c.io.IOUtils.closeQuietly

2021-08-08 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-36456: - Summary: Clean up the depredation usage of o.a.c.io.IOUtils.closeQuietly (was: Clean up the

[jira] [Created] (SPARK-36456) Clean up the depredation use of o.a.c.io.IOUtils.closeQuietly

2021-08-08 Thread Yang Jie (Jira)
Yang Jie created SPARK-36456: Summary: Clean up the depredation use of o.a.c.io.IOUtils.closeQuietly Key: SPARK-36456 URL: https://issues.apache.org/jira/browse/SPARK-36456 Project: Spark

[jira] [Updated] (SPARK-36455) Provide an example of complex session window via flatMapGroupsWithState

2021-08-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-36455: - Summary: Provide an example of complex session window via flatMapGroupsWithState (was: Provide

[jira] [Created] (SPARK-36455) Provide an example of complex session window

2021-08-08 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-36455: Summary: Provide an example of complex session window Key: SPARK-36455 URL: https://issues.apache.org/jira/browse/SPARK-36455 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-36369) Fix Index.union to follow pandas 1.3

2021-08-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36369. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33634

[jira] [Assigned] (SPARK-36369) Fix Index.union to follow pandas 1.3

2021-08-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36369: Assignee: Haejoon Lee > Fix Index.union to follow pandas 1.3 >

[jira] [Updated] (SPARK-32953) Lower memory usage in toPandas with Arrow self_destruct

2021-08-08 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-32953: Description: As described on the mailing list:

[jira] [Assigned] (SPARK-36454) Not push down partition filter to ORCScan for DSv2

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36454: Assignee: Apache Spark > Not push down partition filter to ORCScan for DSv2 >

[jira] [Commented] (SPARK-36454) Not push down partition filter to ORCScan for DSv2

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395683#comment-17395683 ] Apache Spark commented on SPARK-36454: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Commented] (SPARK-36454) Not push down partition filter to ORCScan for DSv2

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395684#comment-17395684 ] Apache Spark commented on SPARK-36454: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36454) Not push down partition filter to ORCScan for DSv2

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36454: Assignee: Apache Spark > Not push down partition filter to ORCScan for DSv2 >

[jira] [Assigned] (SPARK-36454) Not push down partition filter to ORCScan for DSv2

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36454: Assignee: (was: Apache Spark) > Not push down partition filter to ORCScan for DSv2 >

[jira] [Assigned] (SPARK-36432) Upgrade Jetty version to 9.4.43

2021-08-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36432: Assignee: Sajith A > Upgrade Jetty version to 9.4.43 > --- >

[jira] [Resolved] (SPARK-36432) Upgrade Jetty version to 9.4.43

2021-08-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36432. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33656

[jira] [Created] (SPARK-36454) Not push down partition filter to ORCScan for DSv2

2021-08-08 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-36454: -- Summary: Not push down partition filter to ORCScan for DSv2 Key: SPARK-36454 URL: https://issues.apache.org/jira/browse/SPARK-36454 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-36425) PySpark: support CrossValidatorModel get standard deviation of metrics for each paramMap

2021-08-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36425. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33652

[jira] [Commented] (SPARK-36453) Improve consistency processing floating point special literals

2021-08-08 Thread Pablo Langa Blanco (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395658#comment-17395658 ] Pablo Langa Blanco commented on SPARK-36453: I'm working on it > Improve consistency

[jira] [Created] (SPARK-36453) Improve consistency processing floating point special literals

2021-08-08 Thread Pablo Langa Blanco (Jira)
Pablo Langa Blanco created SPARK-36453: -- Summary: Improve consistency processing floating point special literals Key: SPARK-36453 URL: https://issues.apache.org/jira/browse/SPARK-36453 Project:

[jira] [Updated] (SPARK-36452) Add the support in Spark for having group by map datatype column for the scenario that works in Hive

2021-08-08 Thread Saurabh Chawla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saurabh Chawla updated SPARK-36452: --- Description: Add the support in Spark for having group by map datatype column for the

[jira] [Commented] (SPARK-34198) Add RocksDB StateStore implementation

2021-08-08 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395526#comment-17395526 ] Gengliang Wang commented on SPARK-34198: [~XuanYuan][~kabhwan][~viirya][~vkorukanti] Thanks for

[jira] [Commented] (SPARK-36041) Introduce the RocksDBStateStoreProvider in the programming guide

2021-08-08 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395525#comment-17395525 ] Gengliang Wang commented on SPARK-36041: [~XuanYuan][~kabhwan]What is the status of this one? >

[jira] [Commented] (SPARK-36452) Add the support in Spark for having group by map datatype column for the scenario that works in Hive

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395524#comment-17395524 ] Apache Spark commented on SPARK-36452: -- User 'SaurabhChawla100' has created a pull request for this

[jira] [Assigned] (SPARK-36452) Add the support in Spark for having group by map datatype column for the scenario that works in Hive

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36452: Assignee: Apache Spark > Add the support in Spark for having group by map datatype

[jira] [Commented] (SPARK-36452) Add the support in Spark for having group by map datatype column for the scenario that works in Hive

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395523#comment-17395523 ] Apache Spark commented on SPARK-36452: -- User 'SaurabhChawla100' has created a pull request for this

[jira] [Assigned] (SPARK-36452) Add the support in Spark for having group by map datatype column for the scenario that works in Hive

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36452: Assignee: (was: Apache Spark) > Add the support in Spark for having group by map

[jira] [Updated] (SPARK-36452) Add the support in Spark for having group by map datatype column for the scenario that works in Hive

2021-08-08 Thread Saurabh Chawla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saurabh Chawla updated SPARK-36452: --- Description: Add the support in Spark for having group by map datatype column for the

[jira] [Updated] (SPARK-36452) Add the support in Spark for having group by map datatype column for the scenario that works in Hive

2021-08-08 Thread Saurabh Chawla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saurabh Chawla updated SPARK-36452: --- Description: Add the support in Spark for having group by map datatype column for the

[jira] [Created] (SPARK-36452) Add the support in Spark for having group by map datatype column for the scenario that works in Hive

2021-08-08 Thread Saurabh Chawla (Jira)
Saurabh Chawla created SPARK-36452: -- Summary: Add the support in Spark for having group by map datatype column for the scenario that works in Hive Key: SPARK-36452 URL:

[jira] [Assigned] (SPARK-36451) Ivy skips looking for source and doc pom

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36451: Assignee: (was: Apache Spark) > Ivy skips looking for source and doc pom >

[jira] [Assigned] (SPARK-36451) Ivy skips looking for source and doc pom

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36451: Assignee: Apache Spark > Ivy skips looking for source and doc pom >

[jira] [Commented] (SPARK-36451) Ivy skips looking for source and doc pom

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395516#comment-17395516 ] Apache Spark commented on SPARK-36451: -- User 'cxzl25' has created a pull request for this issue:

[jira] [Updated] (SPARK-36451) Ivy skips looking for source and doc pom

2021-08-08 Thread dzcxzl (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dzcxzl updated SPARK-36451: --- Description: Because SPARK-35863 Upgrade Ivy to 2.5.0, it supports skip searching the source and doc pom,

[jira] [Created] (SPARK-36451) Ivy skips looking for source and doc pom

2021-08-08 Thread dzcxzl (Jira)
dzcxzl created SPARK-36451: -- Summary: Ivy skips looking for source and doc pom Key: SPARK-36451 URL: https://issues.apache.org/jira/browse/SPARK-36451 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-36450) Remove unused UnresolvedV2Relation

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395469#comment-17395469 ] Apache Spark commented on SPARK-36450: -- User 'imback82' has created a pull request for this issue:

[jira] [Commented] (SPARK-36450) Remove unused UnresolvedV2Relation

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395468#comment-17395468 ] Apache Spark commented on SPARK-36450: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36450) Remove unused UnresolvedV2Relation

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36450: Assignee: Apache Spark > Remove unused UnresolvedV2Relation >

[jira] [Assigned] (SPARK-36450) Remove unused UnresolvedV2Relation

2021-08-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36450: Assignee: (was: Apache Spark) > Remove unused UnresolvedV2Relation >

[jira] [Created] (SPARK-36450) Remove unused UnresolvedV2Relation

2021-08-08 Thread Terry Kim (Jira)
Terry Kim created SPARK-36450: - Summary: Remove unused UnresolvedV2Relation Key: SPARK-36450 URL: https://issues.apache.org/jira/browse/SPARK-36450 Project: Spark Issue Type: Improvement