[jira] [Created] (SPARK-35748) Fix StreamingJoinHelper to be able to handle day-time interval

2021-06-13 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35748: -- Summary: Fix StreamingJoinHelper to be able to handle day-time interval Key: SPARK-35748 URL: https://issues.apache.org/jira/browse/SPARK-35748 Project: Spark

[jira] [Resolved] (SPARK-35746) Task id in the Stage page timeline is incorrect

2021-06-12 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35746. Target Version/s: 3.0.3, 3.2.0, 3.1.3 Assignee: shahid Resolution:

[jira] [Commented] (SPARK-35740) ChromeUIHistoryServerSuite and ChromeUISeleniumSuite failed on Mac 10.15

2021-06-12 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362243#comment-17362243 ] Kousuke Saruta commented on SPARK-35740: [~abhinavofficial] Could you try with the following

[jira] [Resolved] (SPARK-35741) Variance of 1 record gives NULL in Spark 3.x and NaN in Spark 2.x

2021-06-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35741. Resolution: Not A Problem > Variance of 1 record gives NULL in Spark 3.x and NaN in Spark

[jira] [Commented] (SPARK-35741) Variance of 1 record gives NULL in Spark 3.x and NaN in Spark 2.x

2021-06-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362236#comment-17362236 ] Kousuke Saruta commented on SPARK-35741: [~AbdealiJK] The behavior was changed in Spark 3.1. You

[jira] [Commented] (SPARK-35637) 2 active tasks shown in Spark UI but executor core is 1 and task cpu is 1

2021-06-07 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359041#comment-17359041 ] Kousuke Saruta commented on SPARK-35637: Hmm, the cause seems task-result-getter thread runs

[jira] [Commented] (SPARK-35632) SELECT rand(cast(3 / 7 AS int)) value gives different resultset in Spark 3.1 as compared to Spark 2.4.5

2021-06-05 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17358020#comment-17358020 ] Kousuke Saruta commented on SPARK-35632: Hi [~chetdb], this is caused by the change of

[jira] [Updated] (SPARK-35656) Upgrade SBT to 1.5.3

2021-06-05 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35656: --- Description: Recently, SBT 1.5.3 was released and includes a bug fix for Scala 2.13.16+ and

[jira] [Created] (SPARK-35656) Upgrade SBT to 1.5.3

2021-06-05 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35656: -- Summary: Upgrade SBT to 1.5.3 Key: SPARK-35656 URL: https://issues.apache.org/jira/browse/SPARK-35656 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-35655) Upgrade HtmlUnit and its related artifacts to 2.50.

2021-06-05 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35655: -- Summary: Upgrade HtmlUnit and its related artifacts to 2.50. Key: SPARK-35655 URL: https://issues.apache.org/jira/browse/SPARK-35655 Project: Spark

[jira] [Commented] (SPARK-35649) Failed to read data from Db2 on the column of DECFLOAT data type

2021-06-04 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17357124#comment-17357124 ] Kousuke Saruta commented on SPARK-35649: Hi [~sunguor], Spark enforces the maximum precision of

[jira] [Commented] (SPARK-35602) Job crashes with java.io.UTFDataFormatException: encoded string too long

2021-06-03 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356614#comment-17356614 ] Kousuke Saruta commented on SPARK-35602: [~dej...@gmail.com] Thank you for reporting. I

[jira] [Resolved] (SPARK-35609) Add style rules to prohibit to use a Guava's API which is incompatible with newer versions

2021-06-03 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35609. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved in

[jira] [Commented] (SPARK-35602) Job crashes with java.io.UTFDataFormatException: encoded string too long

2021-06-02 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356088#comment-17356088 ] Kousuke Saruta commented on SPARK-35602: Hi [~dej...@gmail.com] , could you provide reproducible

[jira] [Updated] (SPARK-35609) Add style rules to prohibit to use a Guava's API which is incompatible with newer versions

2021-06-02 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35609: --- Description: SPARK-30272 replaced Objects.toStringHelper which is an APIs Guava 14 provides

[jira] [Created] (SPARK-35609) Add style rules to prohibit to use a Guava's API which is incompatible with newer versions

2021-06-02 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35609: -- Summary: Add style rules to prohibit to use a Guava's API which is incompatible with newer versions Key: SPARK-35609 URL: https://issues.apache.org/jira/browse/SPARK-35609

[jira] [Assigned] (SPARK-35516) Storage UI tab Storage Level tool tip correction

2021-06-01 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta reassigned SPARK-35516: -- Assignee: Lidiya Nixon > Storage UI tab Storage Level tool tip correction >

[jira] [Resolved] (SPARK-35516) Storage UI tab Storage Level tool tip correction

2021-06-01 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35516. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved in

[jira] [Updated] (SPARK-35586) Set a default value for spark.kubernetes.test.sparkTgz in pom.xml for Kubernetes integration tests

2021-05-31 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35586: --- Description: In kubernetes/integration-tests/pom.xml, there is no default value for

[jira] [Created] (SPARK-35586) Set a default value for spark.kubernetes.test.sparkTgz in pom.xml for Kubernetes integration tests

2021-05-31 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35586: -- Summary: Set a default value for spark.kubernetes.test.sparkTgz in pom.xml for Kubernetes integration tests Key: SPARK-35586 URL:

[jira] [Created] (SPARK-35577) Allow to log container output for docker integration tests

2021-05-31 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35577: -- Summary: Allow to log container output for docker integration tests Key: SPARK-35577 URL: https://issues.apache.org/jira/browse/SPARK-35577 Project: Spark

[jira] [Created] (SPARK-35530) Fix rounding error in DifferentiableLossAggregatorSuite with Java 11

2021-05-26 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35530: -- Summary: Fix rounding error in DifferentiableLossAggregatorSuite with Java 11 Key: SPARK-35530 URL: https://issues.apache.org/jira/browse/SPARK-35530 Project:

[jira] [Created] (SPARK-35527) Fix HiveExternalCatalogVersionsSuite to pass with Java 11

2021-05-25 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35527: -- Summary: Fix HiveExternalCatalogVersionsSuite to pass with Java 11 Key: SPARK-35527 URL: https://issues.apache.org/jira/browse/SPARK-35527 Project: Spark

[jira] [Created] (SPARK-35501) Add a feature for removing pulled container image for docker integration tests

2021-05-24 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35501: -- Summary: Add a feature for removing pulled container image for docker integration tests Key: SPARK-35501 URL: https://issues.apache.org/jira/browse/SPARK-35501

[jira] [Resolved] (SPARK-35488) Upgrade ASM to 7.3.1

2021-05-22 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35488. Fix Version/s: 3.20 Assignee: Vinod KC Resolution: Fixed Resolved in

[jira] [Commented] (SPARK-35483) Add a new GA test job for the docker integration tests

2021-05-22 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17349614#comment-17349614 ] Kousuke Saruta commented on SPARK-35483: I'll open a PR now if you don't mind. > Add a new

[jira] [Commented] (SPARK-35483) Add a new GA test job for the integration tests

2021-05-21 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17349592#comment-17349592 ] Kousuke Saruta commented on SPARK-35483: Actually, I've been working on this issue for

[jira] [Created] (SPARK-35487) Upgrade dropwizard metrics to 4.2.0

2021-05-21 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35487: -- Summary: Upgrade dropwizard metrics to 4.2.0 Key: SPARK-35487 URL: https://issues.apache.org/jira/browse/SPARK-35487 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-35418) Add sentences function to functions.{scala,py}

2021-05-19 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35418. Fix Version/s: 3.2.0 Resolution: Fixed Resolved in

[jira] [Updated] (SPARK-35434) Upgrade scalatestplus artifacts to 3.2.9.0

2021-05-18 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35434: --- Summary: Upgrade scalatestplus artifacts to 3.2.9.0 (was: Upgrade scalatestplus artifacts

[jira] [Created] (SPARK-35434) Upgrade scalatestplus artifacts 3.2.9.0

2021-05-18 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35434: -- Summary: Upgrade scalatestplus artifacts 3.2.9.0 Key: SPARK-35434 URL: https://issues.apache.org/jira/browse/SPARK-35434 Project: Spark Issue Type:

[jira] [Updated] (SPARK-35420) Replace the usage of toStringHelper with ToStringBuilder

2021-05-18 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35420: --- Fix Version/s: 3.2.0 > Replace the usage of toStringHelper with ToStringBuilder >

[jira] [Resolved] (SPARK-35425) Add note about Jinja2 as a required dependency for document build.

2021-05-18 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35425. Target Version/s: 3.20 Resolution: Fixed This issue was resolved in

[jira] [Resolved] (SPARK-35420) Replace the usage of toStringHelper with ToStringBuilder

2021-05-17 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35420. Resolution: Fixed This issue was resolved in https://github.com/apache/spark/pull/32567.

[jira] [Created] (SPARK-35425) Add note about Jinja2 as a required dependency for document build.

2021-05-17 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35425: -- Summary: Add note about Jinja2 as a required dependency for document build. Key: SPARK-35425 URL: https://issues.apache.org/jira/browse/SPARK-35425 Project:

[jira] [Created] (SPARK-35421) Remove redundant ProjectExec from streaming queries with V2Relation

2021-05-17 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35421: -- Summary: Remove redundant ProjectExec from streaming queries with V2Relation Key: SPARK-35421 URL: https://issues.apache.org/jira/browse/SPARK-35421 Project:

[jira] [Created] (SPARK-35420) Replace the usage of toStringHelper with ToStringBuilder

2021-05-16 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35420: -- Summary: Replace the usage of toStringHelper with ToStringBuilder Key: SPARK-35420 URL: https://issues.apache.org/jira/browse/SPARK-35420 Project: Spark

[jira] [Created] (SPARK-35418) Add sentences function to functions.scala

2021-05-16 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35418: -- Summary: Add sentences function to functions.scala Key: SPARK-35418 URL: https://issues.apache.org/jira/browse/SPARK-35418 Project: Spark Issue Type:

[jira] [Updated] (SPARK-35418) Add sentences function to functions.{scala,py}

2021-05-16 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35418: --- Summary: Add sentences function to functions.{scala,py} (was: Add sentences function to

[jira] [Updated] (SPARK-35377) Add JS linter to GA

2021-05-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35377: --- Issue Type: Improvement (was: Bug) > Add JS linter to GA > --- > >

[jira] [Created] (SPARK-35377) Add JS linter to GA

2021-05-11 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35377: -- Summary: Add JS linter to GA Key: SPARK-35377 URL: https://issues.apache.org/jira/browse/SPARK-35377 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-35375) Use Jinja2 < 3.0.0 for Python linter dependency in GA

2021-05-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35375: --- Summary: Use Jinja2 < 3.0.0 for Python linter dependency in GA (was: Use Jinja2 < 3.0.0

[jira] [Updated] (SPARK-35375) Use Jinja2 < 3.0.0 for linter dependency in GA

2021-05-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35375: --- Description: >From a few hours ago, Python linter fails in GA. The latest Jinja 3.0.0 seems

[jira] [Updated] (SPARK-35375) Use Jinja2 < 3.0.0 for linter dependency in GA

2021-05-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35375: --- Description: >From a few hours ago, Python linter fails in GA. The latest Jinja 3.0.0 seems

[jira] [Created] (SPARK-35375) Use Jinja2 < 3.0.0 for linter dependency in GA

2021-05-11 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35375: -- Summary: Use Jinja2 < 3.0.0 for linter dependency in GA Key: SPARK-35375 URL: https://issues.apache.org/jira/browse/SPARK-35375 Project: Spark Issue

[jira] [Resolved] (SPARK-35372) JDK 11 compilation failure due to StackOverflowError

2021-05-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35372. Fix Version/s: 3.2.0 Assignee: Hyukjin Kwon Resolution: Fixed This issue

[jira] [Created] (SPARK-35374) Add string-to-number conversion support to JacksonParser

2021-05-11 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35374: -- Summary: Add string-to-number conversion support to JacksonParser Key: SPARK-35374 URL: https://issues.apache.org/jira/browse/SPARK-35374 Project: Spark

[jira] [Resolved] (SPARK-35175) Add linter for JavaScript source files

2021-05-07 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35175. Fix Version/s: 3.2.0 Resolution: Fixed This issue was resolved in

[jira] [Created] (SPARK-35326) Upgrade Jersey to 2.34

2021-05-06 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35326: -- Summary: Upgrade Jersey to 2.34 Key: SPARK-35326 URL: https://issues.apache.org/jira/browse/SPARK-35326 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-35268) Upgrade GenJavadoc to 0.17

2021-04-29 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35268: --- Description: GenJavadoc 0.17 was released and it seems to include a fix for an issue which

[jira] [Created] (SPARK-35268) Upgrade GenJavadoc to 0.17

2021-04-29 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35268: -- Summary: Upgrade GenJavadoc to 0.17 Key: SPARK-35268 URL: https://issues.apache.org/jira/browse/SPARK-35268 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-35105) Support multiple paths for ADD FILE/JAR/ARCHIVE commands

2021-04-28 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35105. Fix Version/s: 3.2.0 Resolution: Fixed This issue was resolved in

[jira] [Resolved] (SPARK-35226) JDBC datasources should accept refreshKrb5Config parameter

2021-04-28 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35226. Fix Version/s: 3.2.0 3.1.2 Resolution: Fixed This issue was

[jira] [Commented] (SPARK-35229) Spark Job web page is extremely slow while there are more than 1500 events in timeline

2021-04-28 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17334511#comment-17334511 ] Kousuke Saruta commented on SPARK-35229: [~tiehexue]Thanks for the report. I'll try to mitigate

[jira] [Created] (SPARK-35247) Add `mask` built-in function as a data masking function

2021-04-27 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35247: -- Summary: Add `mask` built-in function as a data masking function Key: SPARK-35247 URL: https://issues.apache.org/jira/browse/SPARK-35247 Project: Spark

[jira] [Updated] (SPARK-35247) Add `mask` built-in function as a data masking function

2021-04-27 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35247: --- Issue Type: New Feature (was: Bug) > Add `mask` built-in function as a data masking

[jira] [Updated] (SPARK-35236) Support archive files as resources for CREATE FUNCTION USING syntax

2021-04-26 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35236: --- Description: CREATE FUNCTION USING syntax doesn't support archives as resources. The reason

[jira] [Created] (SPARK-35236) Support archive files as resources for CREATE FUNCTION USING syntax

2021-04-26 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35236: -- Summary: Support archive files as resources for CREATE FUNCTION USING syntax Key: SPARK-35236 URL: https://issues.apache.org/jira/browse/SPARK-35236 Project:

[jira] [Updated] (SPARK-35236) Support archive files as resources for CREATE FUNCTION USING syntax

2021-04-26 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35236: --- Issue Type: Improvement (was: Bug) > Support archive files as resources for CREATE

[jira] [Updated] (SPARK-35226) JDBC datasources should accept refreshKrb5Config parameter

2021-04-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35226: --- Summary: JDBC datasources should accept refreshKrb5Config parameter (was: JDBCOptions

[jira] [Updated] (SPARK-35226) JDBC datasources should accept refreshKrb5Config parameter

2021-04-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35226: --- Description: In the current master, JDBC datasources can't accept refreshKrb5Config which

[jira] [Updated] (SPARK-35226) JDBCOption should accept refreshKrb5Config parameter

2021-04-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35226: --- Description: In the current master, JDBCOptions can't accept refreshKrb5Config which is

[jira] [Updated] (SPARK-35226) JDBCOptions should accept refreshKrb5Config parameter

2021-04-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35226: --- Summary: JDBCOptions should accept refreshKrb5Config parameter (was: JDBCOption should

[jira] [Created] (SPARK-35226) JDBCOption should accept refreshKrb5Config parameter

2021-04-25 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35226: -- Summary: JDBCOption should accept refreshKrb5Config parameter Key: SPARK-35226 URL: https://issues.apache.org/jira/browse/SPARK-35226 Project: Spark

[jira] [Resolved] (SPARK-35087) Some columns in table ` Aggregated Metrics by Executor` of stage-detail page shows incorrectly.

2021-04-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35087. Resolution: Fixed This issue was resolved in https://github.com/apache/spark/pull/32190.

[jira] [Assigned] (SPARK-35087) Some columns in table ` Aggregated Metrics by Executor` of stage-detail page shows incorrectly.

2021-04-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta reassigned SPARK-35087: -- Assignee: akiyamaneko > Some columns in table ` Aggregated Metrics by Executor` of

[jira] [Updated] (SPARK-35087) Some columns in table ` Aggregated Metrics by Executor` of stage-detail page shows incorrectly.

2021-04-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35087: --- Fix Version/s: 3.2.0 3.1.2 > Some columns in table ` Aggregated Metrics

[jira] [Updated] (SPARK-35210) Upgrade Jetty to 9.4.40 to fix ERR_CONNECTION_RESET issue

2021-04-24 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35210: --- Fix Version/s: 3.1.2 3.0.3 > Upgrade Jetty to 9.4.40 to fix

[jira] [Resolved] (SPARK-35210) Upgrade Jetty to 9.4.40 to fix ERR_CONNECTION_RESET issue

2021-04-24 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35210. Fix Version/s: 3.2.0 Resolution: Fixed This issue is resolved in

[jira] [Updated] (SPARK-35210) Upgrade Jetty to 9.4.40 to fix ERR_CONNECTION_RESET issue

2021-04-24 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35210: --- Component/s: (was: Spark Core) Build > Upgrade Jetty to 9.4.40 to fix

[jira] [Created] (SPARK-35210) Upgrade Jetty to 9.4.40 to fix ERR_CONNECTION_RESET issue

2021-04-24 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35210: -- Summary: Upgrade Jetty to 9.4.40 to fix ERR_CONNECTION_RESET issue Key: SPARK-35210 URL: https://issues.apache.org/jira/browse/SPARK-35210 Project: Spark

[jira] [Resolved] (SPARK-35127) When we switch between different stage-detail pages, the entry item in the newly-opened page may be blank.

2021-04-22 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35127. Fix Version/s: 3.2.0 3.1.2 3.0.3 Assignee:

[jira] [Created] (SPARK-35180) Allow to build SparkR with SBT

2021-04-21 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35180: -- Summary: Allow to build SparkR with SBT Key: SPARK-35180 URL: https://issues.apache.org/jira/browse/SPARK-35180 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-35175) Add linter for JavaScript source files

2021-04-21 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35175: --- Summary: Add linter for JavaScript source files (was: Add linter for JavaScript sources)

[jira] [Created] (SPARK-35175) Add linter for JavaScript sources

2021-04-21 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35175: -- Summary: Add linter for JavaScript sources Key: SPARK-35175 URL: https://issues.apache.org/jira/browse/SPARK-35175 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-35117) UI progress bar no longer highlights in progress tasks

2021-04-19 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35117. Fix Version/s: 3.2.0 3.1.2 Assignee: Adam Binford

[jira] [Resolved] (SPARK-35092) the auto-generated rdd's name in the storage tab should be truncated if it is too long.

2021-04-18 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35092. Fix Version/s: 3.2.0 Assignee: akiyamaneko (was: Apache Spark)

[jira] [Updated] (SPARK-35092) the auto-generated rdd's name in the storage tab should be truncated if it is too long.

2021-04-18 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35092: --- Affects Version/s: 3.1.1 > the auto-generated rdd's name in the storage tab should be

[jira] [Created] (SPARK-35105) Support multiple paths for ADD FILE/JAR/ARCHIVE commands

2021-04-16 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35105: -- Summary: Support multiple paths for ADD FILE/JAR/ARCHIVE commands Key: SPARK-35105 URL: https://issues.apache.org/jira/browse/SPARK-35105 Project: Spark

[jira] [Updated] (SPARK-35104) Fix ugly indentation of multiple JSON records in a single split file generated by JacksonGenerator when pretty option is true

2021-04-16 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35104: --- Description: When writing multiple JSON records into a single split file with pretty option

[jira] [Created] (SPARK-35104) Fix ugly indentation of multiple JSON records in a single split file generated by JacksonGenerator when pretty option is true

2021-04-16 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35104: -- Summary: Fix ugly indentation of multiple JSON records in a single split file generated by JacksonGenerator when pretty option is true Key: SPARK-35104 URL:

[jira] [Resolved] (SPARK-34977) LIST FILES/JARS/ARCHIVES cannot handle multiple arguments properly when at least one path is quoted.

2021-04-13 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-34977. Fix Version/s: 3.2.0 Resolution: Fixed This issue is resolved in

[jira] [Commented] (SPARK-34993) from_json() acts differently on created and literal strings with backslashes

2021-04-13 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320196#comment-17320196 ] Kousuke Saruta commented on SPARK-34993: The reason {code:java} "\\asd", "\\kkk" and "asd\\"

[jira] [Updated] (SPARK-35047) Allow Json datasources to write non-ascii characters as codepoints

2021-04-13 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35047: --- Summary: Allow Json datasources to write non-ascii characters as codepoints (was: Allow

[jira] [Created] (SPARK-35047) Allow Json datasources to write non-ascii characters as code points

2021-04-13 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35047: -- Summary: Allow Json datasources to write non-ascii characters as code points Key: SPARK-35047 URL: https://issues.apache.org/jira/browse/SPARK-35047 Project:

[jira] [Comment Edited] (SPARK-34993) from_json() acts differently on created and literal strings with backslashes

2021-04-13 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17319958#comment-17319958 ] Kousuke Saruta edited comment on SPARK-34993 at 4/13/21, 7:14 AM: -- Hi

[jira] [Comment Edited] (SPARK-34993) from_json() acts differently on created and literal strings with backslashes

2021-04-13 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17319958#comment-17319958 ] Kousuke Saruta edited comment on SPARK-34993 at 4/13/21, 7:01 AM: -- Hi

[jira] [Commented] (SPARK-34993) from_json() acts differently on created and literal strings with backslashes

2021-04-13 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17319958#comment-17319958 ] Kousuke Saruta commented on SPARK-34993: Hi [~laurikoobas]. I think all of the results is

[jira] [Updated] (SPARK-34988) Upgrade Jetty for CVE-2021-28165

2021-04-08 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-34988: --- Description: CVE-2021-28165 affects the version of Jetty that Spark uses and it seems to be

[jira] [Created] (SPARK-34988) Upgrade Jetty for CVE-2021-28165

2021-04-08 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-34988: -- Summary: Upgrade Jetty for CVE-2021-28165 Key: SPARK-34988 URL: https://issues.apache.org/jira/browse/SPARK-34988 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-34977) LIST FILES/JARS/ARCHIVES cannot handle multiple arguments properly when at least one path is quoted.

2021-04-07 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-34977: --- Description: `LIST FILES/JARS/ARCHIVES path1 path2 ...` cannot list all paths if at least

[jira] [Created] (SPARK-34977) LIST FILES/JARS/ARCHIVES cannot handle multiple arguments properly when at least one path is quoted.

2021-04-06 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-34977: -- Summary: LIST FILES/JARS/ARCHIVES cannot handle multiple arguments properly when at least one path is quoted. Key: SPARK-34977 URL:

[jira] [Created] (SPARK-34955) ADD JAR command cannot add jar files which contains whitespaces in the path

2021-04-05 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-34955: -- Summary: ADD JAR command cannot add jar files which contains whitespaces in the path Key: SPARK-34955 URL: https://issues.apache.org/jira/browse/SPARK-34955

[jira] [Created] (SPARK-34933) Remove the description that || and && can be used as logical operators from the document.

2021-04-01 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-34933: -- Summary: Remove the description that || and && can be used as logical operators from the document. Key: SPARK-34933 URL: https://issues.apache.org/jira/browse/SPARK-34933

[jira] [Updated] (SPARK-34872) quoteIfNeeded should quote a name which contains non-word characters

2021-03-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-34872: --- Description: quoteIfNeeded quotes a name only when it contains . (dots) or ` (backticks)

[jira] [Created] (SPARK-34872) quoteIfNeeded should quote a name which contains non-word characters

2021-03-25 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-34872: -- Summary: quoteIfNeeded should quote a name which contains non-word characters Key: SPARK-34872 URL: https://issues.apache.org/jira/browse/SPARK-34872 Project:

[jira] [Resolved] (SPARK-34225) Jars or file paths which contain spaces are generating FileNotFoundException exception

2021-03-21 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-34225. Fix Version/s: 3.1.2 3.2.0 Assignee: Kousuke Saruta

[jira] [Updated] (SPARK-34773) change-scala-version.sh fails to modify pom.xml if it changes the version from 2.13 to 2.12

2021-03-17 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-34773: --- Summary: change-scala-version.sh fails to modify pom.xml if it changes the version from

[jira] [Created] (SPARK-34773) change-scala-version fails to modify pom.xml if it changes the version from 2.13 to 2.12

2021-03-17 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-34773: -- Summary: change-scala-version fails to modify pom.xml if it changes the version from 2.13 to 2.12 Key: SPARK-34773 URL: https://issues.apache.org/jira/browse/SPARK-34773

[jira] [Updated] (SPARK-34763) col(), $"" and df("name") should handle quoted column names properly.

2021-03-16 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-34763: --- Summary: col(), $"" and df("name") should handle quoted column names properly. (was:

<    1   2   3   4   5   6   7   8   9   10   >