[jira] [Assigned] (SPARK-38741) Test the error class: MAP_KEY_DOES_NOT_EXIST*

[ https://issues.apache.org/jira/browse/SPARK-38741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38741: Assignee: (was: Apache Spark) > Test the error class: MAP_KEY_DOES_NOT_EXIST* > -

[jira] [Commented] (SPARK-38741) Test the error class: MAP_KEY_DOES_NOT_EXIST*

[ https://issues.apache.org/jira/browse/SPARK-38741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523272#comment-17523272 ] Apache Spark commented on SPARK-38741: -- User 'panbingkun' has created a pull reques

[jira] [Assigned] (SPARK-38741) Test the error class: MAP_KEY_DOES_NOT_EXIST*

[ https://issues.apache.org/jira/browse/SPARK-38741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38741: Assignee: Apache Spark > Test the error class: MAP_KEY_DOES_NOT_EXIST* >

[jira] [Commented] (SPARK-38826) dropFieldIfAllNull option does not work for empty JSON struct

[ https://issues.apache.org/jira/browse/SPARK-38826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523270#comment-17523270 ] morvenhuang commented on SPARK-38826: - [~srowen] Sean, thanks for the comment, :). I

[jira] [Assigned] (SPARK-38784) Upgrade Jetty to 9.4.46

[ https://issues.apache.org/jira/browse/SPARK-38784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38784: - Assignee: Sean R. Owen > Upgrade Jetty to 9.4.46 > --- > >

[jira] [Resolved] (SPARK-38784) Upgrade Jetty to 9.4.46

[ https://issues.apache.org/jira/browse/SPARK-38784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38784. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 36229 [https://

[jira] [Resolved] (SPARK-38881) PySpark Kinesis Streaming should expose metricsLevel CloudWatch config that is already supported in the Scala/Java APIs

[ https://issues.apache.org/jira/browse/SPARK-38881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38881. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36201 [https://gi

[jira] [Assigned] (SPARK-38881) PySpark Kinesis Streaming should expose metricsLevel CloudWatch config that is already supported in the Scala/Java APIs

[ https://issues.apache.org/jira/browse/SPARK-38881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-38881: Assignee: Mark Khaitman > PySpark Kinesis Streaming should expose metricsLevel CloudWatch

[jira] [Commented] (SPARK-37787) Long running Spark Job(Spark ThriftServer) throw HDFS_DELEGATE_TOKEN not found in cache Exception

[ https://issues.apache.org/jira/browse/SPARK-37787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523265#comment-17523265 ] huangzhir commented on SPARK-37787: --- Another jira is open for similar issue. https://

[jira] [Resolved] (SPARK-38920) Add ORC blockSize tests to BloomFilterBenchmark

[ https://issues.apache.org/jira/browse/SPARK-38920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38920. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36218 [https://

[jira] [Assigned] (SPARK-38920) Add ORC blockSize tests to BloomFilterBenchmark

[ https://issues.apache.org/jira/browse/SPARK-38920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38920: - Assignee: Dongjoon Hyun > Add ORC blockSize tests to BloomFilterBenchmark > ---

[jira] [Commented] (SPARK-38884) java.util.NoSuchElementException: key not found: numPartitions

[ https://issues.apache.org/jira/browse/SPARK-38884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523262#comment-17523262 ] chopperChen commented on SPARK-38884: - [~srowen] ，ok，thanks。 > java.util.NoSuchElem

[jira] [Commented] (SPARK-38826) dropFieldIfAllNull option does not work for empty JSON struct

[ https://issues.apache.org/jira/browse/SPARK-38826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523261#comment-17523261 ] Sean R. Owen commented on SPARK-38826: -- Ah right it was there inline in the code. I

[jira] [Assigned] (SPARK-38925) update guava to 30.1.1-jre

[ https://issues.apache.org/jira/browse/SPARK-38925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38925: Assignee: (was: Apache Spark) > update guava to 30.1.1-jre >

[jira] [Assigned] (SPARK-38925) update guava to 30.1.1-jre

[ https://issues.apache.org/jira/browse/SPARK-38925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38925: Assignee: Apache Spark > update guava to 30.1.1-jre > -- > >

[jira] [Commented] (SPARK-38925) update guava to 30.1.1-jre

[ https://issues.apache.org/jira/browse/SPARK-38925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523260#comment-17523260 ] Apache Spark commented on SPARK-38925: -- User 'dcoliversun' has created a pull reque

[jira] [Assigned] (SPARK-38925) update guava to 30.1.1-jre

[ https://issues.apache.org/jira/browse/SPARK-38925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38925: Assignee: Apache Spark > update guava to 30.1.1-jre > -- > >

[jira] [Created] (SPARK-38925) update guava to 30.1.1-jre

qian created SPARK-38925: Summary: update guava to 30.1.1-jre Key: SPARK-38925 URL: https://issues.apache.org/jira/browse/SPARK-38925 Project: Spark Issue Type: Improvement Components: Buil

[jira] [Updated] (SPARK-38750) Test the error class: SECOND_FUNCTION_ARGUMENT_NOT_INTEGER

[ https://issues.apache.org/jira/browse/SPARK-38750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38750: -- Fix Version/s: (was: 3.4.0) > Test the error class: SECOND_FUNCTION_ARGUMENT_NOT_INTEGER >

[jira] [Reopened] (SPARK-38750) Test the error class: SECOND_FUNCTION_ARGUMENT_NOT_INTEGER

[ https://issues.apache.org/jira/browse/SPARK-38750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-38750: --- Assignee: (was: panbingkun) This is reverted. > Test the error class: SECOND_FUNCTION

[jira] [Commented] (SPARK-38868) `assert_true` fails unconditionnaly after `left_outer` joins

[ https://issues.apache.org/jira/browse/SPARK-38868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523257#comment-17523257 ] Apache Spark commented on SPARK-38868: -- User 'bersprockets' has created a pull requ

[jira] [Assigned] (SPARK-38868) `assert_true` fails unconditionnaly after `left_outer` joins

[ https://issues.apache.org/jira/browse/SPARK-38868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38868: Assignee: (was: Apache Spark) > `assert_true` fails unconditionnaly after `left_outer

[jira] [Assigned] (SPARK-38868) `assert_true` fails unconditionnaly after `left_outer` joins

[ https://issues.apache.org/jira/browse/SPARK-38868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38868: Assignee: Apache Spark > `assert_true` fails unconditionnaly after `left_outer` joins > -

[jira] [Commented] (SPARK-38826) dropFieldIfAllNull option does not work for empty JSON struct

[ https://issues.apache.org/jira/browse/SPARK-38826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523251#comment-17523251 ] morvenhuang commented on SPARK-38826: - [~srowen] , Hi, Sean, here is the JSON I used

[jira] [Resolved] (SPARK-38077) Spark 3.2.1 breaks binary compatibility with Spark 3.2.0

[ https://issues.apache.org/jira/browse/SPARK-38077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38077. -- Resolution: Won't Fix > Spark 3.2.1 breaks binary compatibility with Spark 3.2.0 > ---

[jira] [Commented] (SPARK-38077) Spark 3.2.1 breaks binary compatibility with Spark 3.2.0

[ https://issues.apache.org/jira/browse/SPARK-38077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523227#comment-17523227 ] Sean R. Owen commented on SPARK-38077: -- Binary and source compatibility for public

[jira] [Resolved] (SPARK-38152) Upgrade Jetty version from 9.4.40 to 9.4.44

[ https://issues.apache.org/jira/browse/SPARK-38152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38152. -- Resolution: Duplicate We already have a ticket for 9.4.46 > Upgrade Jetty version from 9.4.40

[jira] [Resolved] (SPARK-38262) Upgrade Google guava to version 30.0-jre

[ https://issues.apache.org/jira/browse/SPARK-38262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38262. -- Resolution: Duplicate yes, this is a duplicate. Been discussed many times. > Upgrade Google g

[jira] [Resolved] (SPARK-38248) Add @nowarn to suppress warnings related to Serializable interface

[ https://issues.apache.org/jira/browse/SPARK-38248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38248. -- Resolution: Not A Problem > Add @nowarn to suppress warnings related to Serializable interface

[jira] [Resolved] (SPARK-38263) StructType explode

[ https://issues.apache.org/jira/browse/SPARK-38263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38263. -- Resolution: Not A Problem You can already use 'select mystruct.*' to turn the fields into cols

[jira] [Resolved] (SPARK-38313) Remove deprecated logic in SortMergeJoinExec

[ https://issues.apache.org/jira/browse/SPARK-38313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38313. -- Resolution: Not A Problem > Remove deprecated logic in SortMergeJoinExec > ---

[jira] [Resolved] (SPARK-38329) High I/O wait when Spark Structured Streaming checkpoint changed to EFS

[ https://issues.apache.org/jira/browse/SPARK-38329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38329. -- Resolution: Not A Problem > High I/O wait when Spark Structured Streaming checkpoint changed t

[jira] [Resolved] (SPARK-38330) Certificate doesn't match any of the subject alternative names: [*.s3.amazonaws.com, s3.amazonaws.com]

[ https://issues.apache.org/jira/browse/SPARK-38330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38330. -- Resolution: Not A Problem > Certificate doesn't match any of the subject alternative names: >

[jira] [Commented] (SPARK-38409) PrometheusServlet exports gauges with null values

[ https://issues.apache.org/jira/browse/SPARK-38409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523222#comment-17523222 ] Sean R. Owen commented on SPARK-38409: -- Yep, can you open a PR? > PrometheusServle

[jira] [Resolved] (SPARK-38340) Upgrade protobuf-java from 2.5.0 to 3.16.1

[ https://issues.apache.org/jira/browse/SPARK-38340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38340. -- Resolution: Won't Fix > Upgrade protobuf-java from 2.5.0 to 3.16.1 > -

[jira] [Resolved] (SPARK-38421) Cipher Block Chaining IVs should be unpredictable

[ https://issues.apache.org/jira/browse/SPARK-38421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38421. -- Resolution: Not A Problem Just not clear you've argued there is a problem here w.r.t Spark's u

[jira] [Commented] (SPARK-38422) Encryption algorithms should be used with secure mode and padding scheme

[ https://issues.apache.org/jira/browse/SPARK-38422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523220#comment-17523220 ] Sean R. Owen commented on SPARK-38422: -- This may be a good argument in general, but

[jira] [Assigned] (SPARK-38784) Upgrade Jetty to 9.4.46

[ https://issues.apache.org/jira/browse/SPARK-38784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38784: Assignee: (was: Apache Spark) > Upgrade Jetty to 9.4.46 > --- > >

[jira] [Commented] (SPARK-38784) Upgrade Jetty to 9.4.46

[ https://issues.apache.org/jira/browse/SPARK-38784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523213#comment-17523213 ] Apache Spark commented on SPARK-38784: -- User 'srowen' has created a pull request fo

[jira] [Assigned] (SPARK-38784) Upgrade Jetty to 9.4.46

[ https://issues.apache.org/jira/browse/SPARK-38784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38784: Assignee: Apache Spark > Upgrade Jetty to 9.4.46 > --- > >

[jira] [Resolved] (SPARK-38439) Add Braces with if,else,for,do and while statements

[ https://issues.apache.org/jira/browse/SPARK-38439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38439. -- Resolution: Won't Fix > Add Braces with if,else,for,do and while statements >

[jira] [Commented] (SPARK-38485) Non-deterministic UDF executed multiple times when combined with withField

[ https://issues.apache.org/jira/browse/SPARK-38485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523209#comment-17523209 ] Sean R. Owen commented on SPARK-38485: -- I don't think there are guarantees about th

[jira] [Commented] (SPARK-38536) Spark 3 can not read mixed format partitions

[ https://issues.apache.org/jira/browse/SPARK-38536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523208#comment-17523208 ] Sean R. Owen commented on SPARK-38536: -- 3.0.x is EOL; 3.1.x is not, but I'm not cle

[jira] [Resolved] (SPARK-38536) Spark 3 can not read mixed format partitions

[ https://issues.apache.org/jira/browse/SPARK-38536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38536. -- Resolution: Duplicate > Spark 3 can not read mixed format partitions > ---

[jira] [Commented] (SPARK-38649) Fix SECURITY.md

[ https://issues.apache.org/jira/browse/SPARK-38649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523207#comment-17523207 ] Sean R. Owen commented on SPARK-38649: -- That is documentation for Spark, about secu

[jira] [Resolved] (SPARK-38649) Fix SECURITY.md

[ https://issues.apache.org/jira/browse/SPARK-38649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38649. -- Resolution: Not A Problem > Fix SECURITY.md > --- > > Key: SPARK-3

[jira] [Commented] (SPARK-38693) Spark does not use SessionManager

[ https://issues.apache.org/jira/browse/SPARK-38693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523206#comment-17523206 ] Sean R. Owen commented on SPARK-38693: -- Just to back up - what is keycloak and why

[jira] [Resolved] (SPARK-38703) High GC and memory footprint after switch to ZSTD

[ https://issues.apache.org/jira/browse/SPARK-38703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38703. -- Resolution: Invalid I'd put questions on the mailing list. Desipte the "Question" type here, t

[jira] [Resolved] (SPARK-38712) Fix perf regression in ScanOperation

[ https://issues.apache.org/jira/browse/SPARK-38712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38712. -- Resolution: Duplicate > Fix perf regression in ScanOperation > ---

[jira] [Updated] (SPARK-38784) Upgrade Jetty to 9.4.46

[ https://issues.apache.org/jira/browse/SPARK-38784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-38784: - Summary: Upgrade Jetty to 9.4.46 (was: Upgrade Jetty to 9.4.45) > Upgrade Jetty to 9.4.46 > ---

[jira] [Updated] (SPARK-38784) Upgrade Jetty to 9.4.45

[ https://issues.apache.org/jira/browse/SPARK-38784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-38784: - Issue Type: Task (was: Bug) Priority: Minor (was: Major) > Upgrade Jetty to 9.4.45 > ---

[jira] [Commented] (SPARK-38784) Upgrade Jetty to 9.4.45

[ https://issues.apache.org/jira/browse/SPARK-38784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523204#comment-17523204 ] Sean R. Owen commented on SPARK-38784: -- Do any of these appear to affect Spark? Cou

[jira] [Commented] (SPARK-38801) ISO-8859-1 encoding doesn't work for text format

[ https://issues.apache.org/jira/browse/SPARK-38801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523203#comment-17523203 ] Sean R. Owen commented on SPARK-38801: -- Yeah, Spark and Hadoop strongly assume UTF-

[jira] [Updated] (SPARK-38805) Remove an expired indexFilePath from the ESS shuffleIndexCache or the PBS indexCache to save memory.

[ https://issues.apache.org/jira/browse/SPARK-38805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-38805: - Issue Type: Improvement (was: Bug) Priority: Minor (was: Major) > Remove an expired inde

[jira] [Resolved] (SPARK-38808) Windows, Spark 3.2.1: spark-shell command throwing this error: SparkContext: Error initializing SparkContext

[ https://issues.apache.org/jira/browse/SPARK-38808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38808. -- Resolution: Duplicate > Windows, Spark 3.2.1: spark-shell command throwing this error: SparkCo

[jira] [Commented] (SPARK-38816) Wrong comment in random matrix generator in spark-als algorithm

[ https://issues.apache.org/jira/browse/SPARK-38816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523202#comment-17523202 ] Apache Spark commented on SPARK-38816: -- User 'srowen' has created a pull request fo

[jira] [Assigned] (SPARK-38816) Wrong comment in random matrix generator in spark-als algorithm

[ https://issues.apache.org/jira/browse/SPARK-38816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38816: Assignee: (was: Apache Spark) > Wrong comment in random matrix generator in spark-als

[jira] [Assigned] (SPARK-38816) Wrong comment in random matrix generator in spark-als algorithm

[ https://issues.apache.org/jira/browse/SPARK-38816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38816: Assignee: Apache Spark > Wrong comment in random matrix generator in spark-als algorithm

[jira] [Commented] (SPARK-38816) Wrong comment in random matrix generator in spark-als algorithm

[ https://issues.apache.org/jira/browse/SPARK-38816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523200#comment-17523200 ] Apache Spark commented on SPARK-38816: -- User 'srowen' has created a pull request fo

[jira] [Commented] (SPARK-38810) Upgrade Python black in /dev/reformat-python

[ https://issues.apache.org/jira/browse/SPARK-38810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523199#comment-17523199 ] Sean R. Owen commented on SPARK-38810: -- Is this something Spark needs to change? >

[jira] [Resolved] (SPARK-38812) when i clean data ,I hope one rdd spill two rdd according clean data rule

[ https://issues.apache.org/jira/browse/SPARK-38812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38812. -- Resolution: Invalid I can't make out what this is reporting or asking. > when i clean data ,I

[jira] [Commented] (SPARK-38868) `assert_true` fails unconditionnaly after `left_outer` joins

[ https://issues.apache.org/jira/browse/SPARK-38868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523197#comment-17523197 ] Bruce Robbins commented on SPARK-38868: --- The issue is in {{EliminateOuterJoin}}. T

[jira] [Commented] (SPARK-38750) Test the error class: SECOND_FUNCTION_ARGUMENT_NOT_INTEGER

[ https://issues.apache.org/jira/browse/SPARK-38750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523195#comment-17523195 ] Apache Spark commented on SPARK-38750: -- User 'MaxGekk' has created a pull request f

[jira] [Updated] (SPARK-38816) Wrong comment in random matrix generator in spark-als algorithm

[ https://issues.apache.org/jira/browse/SPARK-38816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-38816: - Summary: Wrong comment in random matrix generator in spark-als algorithm (was: Wrong code resu

[jira] [Updated] (SPARK-38816) Wrong code results in random matrix generator in spark-als algorithm

[ https://issues.apache.org/jira/browse/SPARK-38816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-38816: - Issue Type: Improvement (was: Bug) Priority: Minor (was: Major) > Wrong code results in

[jira] [Commented] (SPARK-38816) Wrong code results in random matrix generator in spark-als algorithm

[ https://issues.apache.org/jira/browse/SPARK-38816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523188#comment-17523188 ] Sean R. Owen commented on SPARK-38816: -- Yeah the code and comment are inconsistent

[jira] [Commented] (SPARK-38826) dropFieldIfAllNull option does not work for empty JSON struct

[ https://issues.apache.org/jira/browse/SPARK-38826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523187#comment-17523187 ] Sean R. Owen commented on SPARK-38826: -- Wait, can you show the JSON? it's not clear

[jira] [Commented] (SPARK-38846) Teradata's Number is either converted to its floor value or ceiling value despite its fractional part.

[ https://issues.apache.org/jira/browse/SPARK-38846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523186#comment-17523186 ] Sean R. Owen commented on SPARK-38846: -- Do you know what JDBC type the teradata dri

[jira] [Commented] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523185#comment-17523185 ] Sean R. Owen commented on SPARK-38853: -- Can this be simplified? it's hard to make o

[jira] [Commented] (SPARK-38916) Tasks not killed caused by race conditions between killTask() and launchTask()

[ https://issues.apache.org/jira/browse/SPARK-38916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523183#comment-17523183 ] Wei Xue commented on SPARK-38916: - With a PR, I suppose? Will push it out soon, but its

[jira] [Commented] (SPARK-38856) Fix a rejectedExecutionException error when push-based shuffle is enabled

[ https://issues.apache.org/jira/browse/SPARK-38856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523179#comment-17523179 ] Sean R. Owen commented on SPARK-38856: -- [~weixiuli] There's no detail here, come on

[jira] [Commented] (SPARK-38858) PythonException - socke.timeout: timed out - socket.py line 707

[ https://issues.apache.org/jira/browse/SPARK-38858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523178#comment-17523178 ] Sean R. Owen commented on SPARK-38858: -- This does not look like it's related to Spa

[jira] [Commented] (SPARK-38861) GroupState.getCurrentWatermarkMs() on static Dataframe throws exception but should return -1

[ https://issues.apache.org/jira/browse/SPARK-38861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523177#comment-17523177 ] Sean R. Owen commented on SPARK-38861: -- Can you show the trace, where the UOE comes

[jira] [Commented] (SPARK-38868) `assert_true` fails unconditionnaly after `left_outer` joins

[ https://issues.apache.org/jira/browse/SPARK-38868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523176#comment-17523176 ] Sean R. Owen commented on SPARK-38868: -- That does seem like a legit bug; I don't kn

[jira] [Updated] (SPARK-38881) PySpark Kinesis Streaming should expose metricsLevel CloudWatch config that is already supported in the Scala/Java APIs

[ https://issues.apache.org/jira/browse/SPARK-38881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-38881: - Priority: Minor (was: Major) > PySpark Kinesis Streaming should expose metricsLevel CloudWatch

[jira] [Commented] (SPARK-38884) java.util.NoSuchElementException: key not found: numPartitions

[ https://issues.apache.org/jira/browse/SPARK-38884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523175#comment-17523175 ] Sean R. Owen commented on SPARK-38884: -- Please test this vs master. 3.0.x is EOL no

[jira] [Updated] (SPARK-38915) Log executed SQL queries

[ https://issues.apache.org/jira/browse/SPARK-38915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-38915: - Priority: Minor (was: Major) > Log executed SQL queries > > >

[jira] [Commented] (SPARK-38916) Tasks not killed caused by race conditions between killTask() and launchTask()

[ https://issues.apache.org/jira/browse/SPARK-38916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523174#comment-17523174 ] Sean R. Owen commented on SPARK-38916: -- OK, how would we resolve this though? > Ta

[jira] [Updated] (SPARK-38919) Filter stale event log before parse logs

[ https://issues.apache.org/jira/browse/SPARK-38919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-38919: - Fix Version/s: (was: 3.2.0) Target Version/s: (was: 3.2.0, 3.2.1) Don't set fix or

[jira] [Updated] (SPARK-38921) Use k8s-client to create queue resource in Volcano IT

[ https://issues.apache.org/jira/browse/SPARK-38921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-38921: - Issue Type: Improvement (was: Bug) Priority: Minor (was: Major) > Use k8s-client to crea

[jira] [Updated] (SPARK-38920) Add ORC blockSize tests to BloomFilterBenchmark

[ https://issues.apache.org/jira/browse/SPARK-38920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-38920: - Priority: Minor (was: Major) > Add ORC blockSize tests to BloomFilterBenchmark > --

[jira] [Resolved] (SPARK-38660) PySpark DeprecationWarning: distutils Version classes are deprecated

[ https://issues.apache.org/jira/browse/SPARK-38660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38660. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 35977 [https://gi

[jira] [Updated] (SPARK-38660) PySpark DeprecationWarning: distutils Version classes are deprecated

[ https://issues.apache.org/jira/browse/SPARK-38660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-38660: - Issue Type: Improvement (was: Bug) Priority: Minor (was: Major) > PySpark DeprecationWar

[jira] [Assigned] (SPARK-38660) PySpark DeprecationWarning: distutils Version classes are deprecated

[ https://issues.apache.org/jira/browse/SPARK-38660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-38660: Assignee: Gergely Kalmar > PySpark DeprecationWarning: distutils Version classes are depr

[jira] [Assigned] (SPARK-38924) Update dataTables to 1.10.25 for security issue

[ https://issues.apache.org/jira/browse/SPARK-38924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38924: Assignee: Apache Spark > Update dataTables to 1.10.25 for security issue > --

[jira] [Assigned] (SPARK-38924) Update dataTables to 1.10.25 for security issue

[ https://issues.apache.org/jira/browse/SPARK-38924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38924: Assignee: (was: Apache Spark) > Update dataTables to 1.10.25 for security issue > ---

[jira] [Commented] (SPARK-38924) Update dataTables to 1.10.25 for security issue

[ https://issues.apache.org/jira/browse/SPARK-38924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523171#comment-17523171 ] Apache Spark commented on SPARK-38924: -- User 'srowen' has created a pull request fo

[jira] [Updated] (SPARK-38924) Update dataTables to 1.10.25 for security issue

[ https://issues.apache.org/jira/browse/SPARK-38924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-38924: - Target Version/s: 3.4.0 Description: https://nvd.nist.gov/vuln/detail/CVE-2020-28458 aff

[jira] [Created] (SPARK-38924) Update dataTables to 1.10.25 for security issue

Sean R. Owen created SPARK-38924: Summary: Update dataTables to 1.10.25 for security issue Key: SPARK-38924 URL: https://issues.apache.org/jira/browse/SPARK-38924 Project: Spark Issue Type: I

[jira] [Commented] (SPARK-38659) PySpark ResourceWarning: unclosed socket

[ https://issues.apache.org/jira/browse/SPARK-38659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523168#comment-17523168 ] Sean R. Owen commented on SPARK-38659: -- Not quite sure what to make of that, but is

[jira] [Updated] (SPARK-38659) PySpark ResourceWarning: unclosed socket

[ https://issues.apache.org/jira/browse/SPARK-38659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-38659: - Priority: Minor (was: Major) > PySpark ResourceWarning: unclosed socket > -

[jira] [Resolved] (SPARK-38738) Test the error class: INVALID_FRACTION_OF_SECOND

[ https://issues.apache.org/jira/browse/SPARK-38738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-38738. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36211 [https://github.com

[jira] [Assigned] (SPARK-38738) Test the error class: INVALID_FRACTION_OF_SECOND

[ https://issues.apache.org/jira/browse/SPARK-38738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-38738: Assignee: panbingkun > Test the error class: INVALID_FRACTION_OF_SECOND > ---

[jira] [Resolved] (SPARK-38923) Regenerate benchmark results

[ https://issues.apache.org/jira/browse/SPARK-38923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38923. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36225 [https://

[jira] [Assigned] (SPARK-38923) Regenerate benchmark results

[ https://issues.apache.org/jira/browse/SPARK-38923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38923: - Assignee: Dongjoon Hyun > Regenerate benchmark results > >

[jira] [Commented] (SPARK-38923) Regenerate benchmark results

[ https://issues.apache.org/jira/browse/SPARK-38923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523149#comment-17523149 ] Apache Spark commented on SPARK-38923: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-38923) Regenerate benchmark results

[ https://issues.apache.org/jira/browse/SPARK-38923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38923: Assignee: (was: Apache Spark) > Regenerate benchmark results > --

[jira] [Commented] (SPARK-38923) Regenerate benchmark results

[ https://issues.apache.org/jira/browse/SPARK-38923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523148#comment-17523148 ] Apache Spark commented on SPARK-38923: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-38923) Regenerate benchmark results

[ https://issues.apache.org/jira/browse/SPARK-38923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38923: Assignee: Apache Spark > Regenerate benchmark results > > >

[jira] [Created] (SPARK-38923) Regenerate benchmark results

Dongjoon Hyun created SPARK-38923: - Summary: Regenerate benchmark results Key: SPARK-38923 URL: https://issues.apache.org/jira/browse/SPARK-38923 Project: Spark Issue Type: Test Com

  1   2   >