[jira] [Assigned] (SPARK-37028) Add a 'kill' executor link in the Web UI.

2021-10-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37028: Assignee: Apache Spark > Add a 'kill' executor link in the Web UI. > ---

[jira] [Assigned] (SPARK-37028) Add a 'kill' executor link in the Web UI.

2021-10-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37028: Assignee: (was: Apache Spark) > Add a 'kill' executor link in the Web UI. >

[jira] [Commented] (SPARK-37028) Add a 'kill' executor link in the Web UI.

2021-10-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429652#comment-17429652 ] Apache Spark commented on SPARK-37028: -- User 'weixiuli' has created a pull request

[jira] [Updated] (SPARK-37028) Add a 'kill' executor link in the Web UI.

2021-10-16 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-37028: - Description: The executor which is running in a bad node(eg. The system is overloaded or disks are busy

[jira] [Updated] (SPARK-37028) Add a 'kill' executor link in the Web UI.

2021-10-16 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-37028: - Description: The executor which is running in a bad node(eg. The system is overloaded or disks are busy

[jira] [Updated] (SPARK-37028) Add a 'kill' executor link in the Web UI.

2021-10-16 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-37028: - Summary: Add a 'kill' executor link in the Web UI. (was: Add a 'kill' executor link in Web UI.) > A

[jira] [Updated] (SPARK-37028) Add a 'kill' executor link in Web UI.

2021-10-16 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-37028: - Description: The executor which is running in a bad node(eg. The system is overloaded or disks are busy

[jira] [Updated] (SPARK-37028) Add a 'kill' executor link in Web UI.

2021-10-16 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-37028: - Description: The executor which is running in a bad node(eg. The system is overloaded or disks are busy

[jira] [Updated] (SPARK-37028) Add a 'kill' executor link in Web UI.

2021-10-16 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-37028: - Summary: Add a 'kill' executor link in Web UI. (was: Add a 'kill' executors link in Web UI.) > Add

[jira] [Updated] (SPARK-37028) Add a 'kill' executors link in Web UI.

2021-10-16 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-37028: - Summary: Add a 'kill' executors link in Web UI. (was: Add 'kill' executors link in Web UI.) > Add

[jira] [Created] (SPARK-37028) Add 'kill' executors link in Web UI.

2021-10-16 Thread weixiuli (Jira)
weixiuli created SPARK-37028: Summary: Add 'kill' executors link in Web UI. Key: SPARK-37028 URL: https://issues.apache.org/jira/browse/SPARK-37028 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-37028) Add 'kill' executors link in Web UI.

2021-10-16 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-37028: - Description: Add a 'kill' executors link in Web UI. (was: Add 'kill' executors link in Web UI.) > Add

[jira] [Updated] (SPARK-37026) Ensure the element type of ResolvedRFormula.terms is scala.Seq for Scala 2.13

2021-10-16 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-37026: --- Component/s: Build > Ensure the element type of ResolvedRFormula.terms is scala.Seq for Scal

[jira] [Updated] (SPARK-37027) Fix behavior inconsistent in Hive table when ‘path’ is provided in SERDEPROPERTIES

2021-10-16 Thread Yuzhou Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuzhou Sun updated SPARK-37027: --- Attachment: SPARK-37027-test-example.patch > Fix behavior inconsistent in Hive table when ‘path’ is

[jira] [Created] (SPARK-37027) Fix behavior inconsistent in Hive table when ‘path’ is provided in SERDEPROPERTIES

2021-10-16 Thread Yuzhou Sun (Jira)
Yuzhou Sun created SPARK-37027: -- Summary: Fix behavior inconsistent in Hive table when ‘path’ is provided in SERDEPROPERTIES Key: SPARK-37027 URL: https://issues.apache.org/jira/browse/SPARK-37027 Projec

[jira] [Commented] (SPARK-37026) Ensure the element type of ResolvedRFormula.terms is scala.Seq for Scala 2.13

2021-10-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429644#comment-17429644 ] Apache Spark commented on SPARK-37026: -- User 'sarutak' has created a pull request f

[jira] [Assigned] (SPARK-37026) Ensure the element type of ResolvedRFormula.terms is scala.Seq for Scala 2.13

2021-10-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37026: Assignee: Apache Spark (was: Kousuke Saruta) > Ensure the element type of ResolvedRFormu

[jira] [Assigned] (SPARK-37026) Ensure the element type of ResolvedRFormula.terms is scala.Seq for Scala 2.13

2021-10-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37026: Assignee: Kousuke Saruta (was: Apache Spark) > Ensure the element type of ResolvedRFormu

[jira] [Updated] (SPARK-37026) Ensure the element type of ResolvedRFormula.terms is scala.Seq for Scala 2.13

2021-10-16 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-37026: --- Summary: Ensure the element type of ResolvedRFormula.terms is scala.Seq for Scala 2.13 (was

[jira] [Updated] (SPARK-37026) Ensure the element type of RFormula.terms is scala.Seq for Scala 2.13

2021-10-16 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-37026: --- Summary: Ensure the element type of RFormula.terms is scala.Seq for Scala 2.13 (was: Resolv

[jira] [Updated] (SPARK-37026) ResolvedRFormula.toString throws ClassCastException with Scala 2.13

2021-10-16 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-37026: --- Description: ResolvedRFormula.toString throws ClassCastException with Scala 2.13 because the

[jira] [Created] (SPARK-37026) ResolvedRFormula.toString throws ClassCastException with Scala 2.13

2021-10-16 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-37026: -- Summary: ResolvedRFormula.toString throws ClassCastException with Scala 2.13 Key: SPARK-37026 URL: https://issues.apache.org/jira/browse/SPARK-37026 Project: Spar

[jira] [Updated] (SPARK-37026) ResolvedRFormula.toString throws ClassCastException with Scala 2.13

2021-10-16 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-37026: --- Issue Type: Bug (was: Improvement) > ResolvedRFormula.toString throws ClassCastException wi

[jira] [Updated] (SPARK-36966) Spark evicts RDD partitions instead of allowing OOM

2021-10-16 Thread sam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam updated SPARK-36966: Affects Version/s: 1.6.0 > Spark evicts RDD partitions instead of allowing OOM > -

[jira] [Updated] (SPARK-36966) Spark evicts RDD partitions instead of allowing OOM

2021-10-16 Thread sam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam updated SPARK-36966: Description: In the past Spark (pre 1.6) jobs would give OOM if an RDD could not fit into memory (when trying to

[jira] [Updated] (SPARK-36966) Spark evicts RDD partitions instead of allowing OOM

2021-10-16 Thread sam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam updated SPARK-36966: Description: In the past Spark jobs would give OOM if an RDD could not fit into memory (when trying to cache with

[jira] [Assigned] (SPARK-37025) Upgrade RoaringBitmap to 0.9.22

2021-10-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37025: Assignee: Apache Spark (was: Lantao Jin) > Upgrade RoaringBitmap to 0.9.22 > ---

[jira] [Assigned] (SPARK-37025) Upgrade RoaringBitmap to 0.9.22

2021-10-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37025: Assignee: Lantao Jin (was: Apache Spark) > Upgrade RoaringBitmap to 0.9.22 > ---

[jira] [Commented] (SPARK-37025) Upgrade RoaringBitmap to 0.9.22

2021-10-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429583#comment-17429583 ] Apache Spark commented on SPARK-37025: -- User 'medb' has created a pull request for

[jira] [Updated] (SPARK-37025) Upgrade RoaringBitmap to 0.9.22

2021-10-16 Thread Igor Dvorzhak (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak updated SPARK-37025: -- Labels: (was: correctness) > Upgrade RoaringBitmap to 0.9.22 > -

[jira] [Updated] (SPARK-37025) Upgrade RoaringBitmap to 0.9.22

2021-10-16 Thread Igor Dvorzhak (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak updated SPARK-37025: -- Fix Version/s: (was: 2.4.2) (was: 2.3.4)

[jira] [Created] (SPARK-37025) Upgrade RoaringBitmap to 0.9.22

2021-10-16 Thread Igor Dvorzhak (Jira)
Igor Dvorzhak created SPARK-37025: - Summary: Upgrade RoaringBitmap to 0.9.22 Key: SPARK-37025 URL: https://issues.apache.org/jira/browse/SPARK-37025 Project: Spark Issue Type: Bug C

[jira] [Commented] (SPARK-27312) PropertyGraph <-> GraphX conversions

2021-10-16 Thread Munish Sharma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429577#comment-17429577 ] Munish Sharma commented on SPARK-27312: --- [~mengxr] can I start to work on this ? I

[jira] [Resolved] (SPARK-36992) Improve byte array sort perf by unify getPrefix function of UTF8String and ByteArray

2021-10-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-36992. -- Fix Version/s: 3.3.0 Assignee: XiDuo You Resolution: Fixed Resolved by https:/

[jira] [Updated] (SPARK-36992) Improve byte array sort perf by unify getPrefix function of UTF8String and ByteArray

2021-10-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-36992: - Priority: Minor (was: Major) > Improve byte array sort perf by unify getPrefix function of UTF8

[jira] [Assigned] (SPARK-37008) WholeStageCodegenSparkSubmitSuite Failed with Java 17

2021-10-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-37008: Assignee: Yang Jie > WholeStageCodegenSparkSubmitSuite Failed with Java 17 > ---

[jira] [Updated] (SPARK-37008) WholeStageCodegenSparkSubmitSuite Failed with Java 17

2021-10-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-37008: - Priority: Minor (was: Major) > WholeStageCodegenSparkSubmitSuite Failed with Java 17 > ---

[jira] [Resolved] (SPARK-37008) WholeStageCodegenSparkSubmitSuite Failed with Java 17

2021-10-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-37008. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34286 [https://gi

[jira] [Assigned] (SPARK-36900) "SPARK-36464: size returns correct positive number even with over 2GB data" will oom with JDK17

2021-10-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-36900: Assignee: (was: Yang Jie) > "SPARK-36464: size returns correct positive number even w

[jira] [Resolved] (SPARK-36900) "SPARK-36464: size returns correct positive number even with over 2GB data" will oom with JDK17

2021-10-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-36900. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34284 [https://gi

[jira] [Assigned] (SPARK-36900) "SPARK-36464: size returns correct positive number even with over 2GB data" will oom with JDK17

2021-10-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-36900: Assignee: Yang Jie > "SPARK-36464: size returns correct positive number even with over 2G

[jira] [Updated] (SPARK-36915) Pin actions to a full length commit SHA

2021-10-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-36915: - Issue Type: Improvement (was: Bug) Priority: Minor (was: Major) > Pin actions to a full

[jira] [Resolved] (SPARK-36915) Pin actions to a full length commit SHA

2021-10-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-36915. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34163 [https://gi

[jira] [Assigned] (SPARK-36915) Pin actions to a full length commit SHA

2021-10-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-36915: Assignee: Naveen S > Pin actions to a full length commit SHA > --

[jira] [Created] (SPARK-37024) Even more decimal overflow issues in average

2021-10-16 Thread Robert Joseph Evans (Jira)
Robert Joseph Evans created SPARK-37024: --- Summary: Even more decimal overflow issues in average Key: SPARK-37024 URL: https://issues.apache.org/jira/browse/SPARK-37024 Project: Spark Is

[jira] [Updated] (SPARK-37021) JDBC option "sessionInitStatement" is ignored in resolveTable

2021-10-16 Thread Valery Meleshkin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valery Meleshkin updated SPARK-37021: - Summary: JDBC option "sessionInitStatement" is ignored in resolveTable (was: JDBC optio

[jira] [Updated] (SPARK-37021) JDBC option "sessionInitStatement" does not execute set sql statement when resolving a table

2021-10-16 Thread Valery Meleshkin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valery Meleshkin updated SPARK-37021: - Description: If {{sessionInitStatement}} is required to grant permissions or resolve an

[jira] [Updated] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

2021-10-16 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37022: --- Description: [{{black}}|https://github.com/psf/black] is a popular Python code forma

[jira] [Commented] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

2021-10-16 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429532#comment-17429532 ] Maciej Szymkiewicz commented on SPARK-37022: Looking at the diffs, the major

[jira] [Resolved] (SPARK-36232) Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled

2021-10-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36232. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34299 [https://gi

[jira] [Assigned] (SPARK-36232) Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled

2021-10-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36232: Assignee: Yikun Jiang > Support creating a ps.Series/Index with `Decimal('NaN')` with Arr

[jira] [Resolved] (SPARK-36230) hasnans for Series of Decimal(`NaN`)

2021-10-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36230. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34299 [https://gi

[jira] [Assigned] (SPARK-36230) hasnans for Series of Decimal(`NaN`)

2021-10-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36230: Assignee: Yikun Jiang > hasnans for Series of Decimal(`NaN`) > --