[jira] [Assigned] (SPARK-28012) Hive UDF supports struct type foldable expression

[ https://issues.apache.org/jira/browse/SPARK-28012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-28012: Assignee: dzcxzl > Hive UDF supports struct type foldable expression >

[jira] [Resolved] (SPARK-28012) Hive UDF supports struct type foldable expression

[ https://issues.apache.org/jira/browse/SPARK-28012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28012. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24846

[jira] [Resolved] (SPARK-23263) create table stored as parquet should update table size if automatic update table size is enabled

[ https://issues.apache.org/jira/browse/SPARK-23263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23263. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 20430

[jira] [Assigned] (SPARK-23263) create table stored as parquet should update table size if automatic update table size is enabled

[ https://issues.apache.org/jira/browse/SPARK-23263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23263: Assignee: yuming.wang > create table stored as parquet should update table size if

[jira] [Assigned] (SPARK-28118) Add `spark.eventLog.compression.codec` configuration

[ https://issues.apache.org/jira/browse/SPARK-28118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28118: Assignee: (was: Apache Spark) > Add `spark.eventLog.compression.codec` configuration

[jira] [Assigned] (SPARK-28118) Add `spark.eventLog.compression.codec` configuration

[ https://issues.apache.org/jira/browse/SPARK-28118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28118: Assignee: Apache Spark > Add `spark.eventLog.compression.codec` configuration >

[jira] [Created] (SPARK-28118) Add `spark.eventLog.compression.codec` configuration

Dongjoon Hyun created SPARK-28118: - Summary: Add `spark.eventLog.compression.codec` configuration Key: SPARK-28118 URL: https://issues.apache.org/jira/browse/SPARK-28118 Project: Spark Issue

[jira] [Assigned] (SPARK-28117) LDA and BisectingKMeans cache the input dataset if necessary

[ https://issues.apache.org/jira/browse/SPARK-28117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28117: Assignee: Apache Spark > LDA and BisectingKMeans cache the input dataset if necessary >

[jira] [Assigned] (SPARK-28117) LDA and BisectingKMeans cache the input dataset if necessary

[ https://issues.apache.org/jira/browse/SPARK-28117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28117: Assignee: (was: Apache Spark) > LDA and BisectingKMeans cache the input dataset if

[jira] [Updated] (SPARK-27761) Make UDF nondeterministic by default

[ https://issues.apache.org/jira/browse/SPARK-27761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27761: - Summary: Make UDF nondeterministic by default (was: Make UDF nondeterministic by default(?))

[jira] [Resolved] (SPARK-28089) File source v2: support reading output of file streaming Sink

[ https://issues.apache.org/jira/browse/SPARK-28089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28089. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24900

[jira] [Created] (SPARK-28117) LDA and BisectingKMeans cache the input dataset if necessary

zhengruifeng created SPARK-28117: Summary: LDA and BisectingKMeans cache the input dataset if necessary Key: SPARK-28117 URL: https://issues.apache.org/jira/browse/SPARK-28117 Project: Spark

[jira] [Assigned] (SPARK-28089) File source v2: support reading output of file streaming Sink

[ https://issues.apache.org/jira/browse/SPARK-28089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-28089: --- Assignee: Gengliang Wang > File source v2: support reading output of file streaming Sink >

[jira] [Resolved] (SPARK-27990) Provide a way to recursively load data from datasource

[ https://issues.apache.org/jira/browse/SPARK-27990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-27990. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24830

[jira] [Assigned] (SPARK-27990) Provide a way to recursively load data from datasource

[ https://issues.apache.org/jira/browse/SPARK-27990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-27990: --- Assignee: Weichen Xu > Provide a way to recursively load data from datasource >

[jira] [Closed] (SPARK-28116) Fix Flaky `SparkContextSuite.test resource scheduling under local-cluster mode`

[ https://issues.apache.org/jira/browse/SPARK-28116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-28116. - > Fix Flaky `SparkContextSuite.test resource scheduling under local-cluster > mode` >

[jira] [Commented] (SPARK-28098) Native ORC reader doesn't support subdirectories with Hive tables

[ https://issues.apache.org/jira/browse/SPARK-28098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868250#comment-16868250 ] Hyukjin Kwon commented on SPARK-28098: -- [~ddrinka], do you mind if I ask to check similar stuff as

[jira] [Commented] (SPARK-28099) Assertion when querying unpartitioned Hive table with partition-like naming

[ https://issues.apache.org/jira/browse/SPARK-28099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868249#comment-16868249 ] Hyukjin Kwon commented on SPARK-28099: -- Hi [~ddrinka], can you confirm if: 1. This issue is

[jira] [Resolved] (SPARK-28085) Spark Scala API documentation URLs not working properly in Chrome

[ https://issues.apache.org/jira/browse/SPARK-28085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28085. -- Resolution: Not A Problem > Spark Scala API documentation URLs not working properly in Chrome

[jira] [Commented] (SPARK-28085) Spark Scala API documentation URLs not working properly in Chrome

[ https://issues.apache.org/jira/browse/SPARK-28085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868246#comment-16868246 ] Hyukjin Kwon commented on SPARK-28085: -- I think it's not clear which side's bug for the current

[jira] [Commented] (SPARK-28080) There is a problem to download and watch offline the history of an application with multiple attempts due to UI inconsistency

[ https://issues.apache.org/jira/browse/SPARK-28080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868247#comment-16868247 ] Hyukjin Kwon commented on SPARK-28080: -- Can you attach screenshots of UI? > There is a problem to

[jira] [Commented] (SPARK-28079) CSV fails to detect corrupt record unless "columnNameOfCorruptRecord" is manually added to the schema

[ https://issues.apache.org/jira/browse/SPARK-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868235#comment-16868235 ] Hyukjin Kwon commented on SPARK-28079: -- I think he's saying the last row is not in the corrupt row

[jira] [Resolved] (SPARK-28095) Pyspark with kubernetes doesn't parse arguments with spaces as expected.

[ https://issues.apache.org/jira/browse/SPARK-28095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28095. -- Resolution: Invalid > Pyspark with kubernetes doesn't parse arguments with spaces as

[jira] [Commented] (SPARK-28095) Pyspark with kubernetes doesn't parse arguments with spaces as expected.

[ https://issues.apache.org/jira/browse/SPARK-28095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868239#comment-16868239 ] Hyukjin Kwon commented on SPARK-28095: -- ? shouldn't the white space encoded with %20? {code}

[jira] [Resolved] (SPARK-28079) CSV fails to detect corrupt record unless "columnNameOfCorruptRecord" is manually added to the schema

[ https://issues.apache.org/jira/browse/SPARK-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28079. -- Resolution: Duplicate > CSV fails to detect corrupt record unless "columnNameOfCorruptRecord"

[jira] [Updated] (SPARK-28106) Spark SQL add jar with wrong hdfs path, SparkContext still add it to jar path ,and cause Task Failed

[ https://issues.apache.org/jira/browse/SPARK-28106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-28106: -- Attachment: image-2019-06-20-11-51-06-889.png > Spark SQL add jar with wrong hdfs path, SparkContext

[jira] [Updated] (SPARK-28106) Spark SQL add jar with wrong hdfs path, SparkContext still add it to jar path ,and cause Task Failed

[ https://issues.apache.org/jira/browse/SPARK-28106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-28106: -- Description: When we use SparkSQL, about add jar command, if we add a wrong path of HDFS such as

[jira] [Updated] (SPARK-28106) Spark SQL add jar with wrong hdfs path, SparkContext still add it to jar path ,and cause Task Failed

[ https://issues.apache.org/jira/browse/SPARK-28106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-28106: -- Attachment: image-2019-06-20-11-50-36-418.png > Spark SQL add jar with wrong hdfs path, SparkContext

[jira] [Updated] (SPARK-28106) Spark SQL add jar with wrong hdfs path, SparkContext still add it to jar path ,and cause Task Failed

[ https://issues.apache.org/jira/browse/SPARK-28106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-28106: -- Attachment: image-2019-06-20-11-49-13-691.png > Spark SQL add jar with wrong hdfs path, SparkContext

[jira] [Updated] (SPARK-28106) Spark SQL add jar with wrong hdfs path, SparkContext still add it to jar path ,and cause Task Failed

[ https://issues.apache.org/jira/browse/SPARK-28106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-28106: -- Description: When we use SparkSQL, about add jar command, if we add a wrong path of HDFS such as

[jira] [Updated] (SPARK-28106) Spark SQL add jar with wrong hdfs path, SparkContext still add it to jar path ,and cause Task Failed

[ https://issues.apache.org/jira/browse/SPARK-28106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-28106: -- Description: When we use SparkSQL, about add jar command, if we add a wrong path of HDFS such as

[jira] [Commented] (SPARK-23128) A new approach to do adaptive execution in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-23128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868214#comment-16868214 ] Wenchen Fan commented on SPARK-23128: - > a. dynamic parallelism I believe [~carsonwang] is working

[jira] [Resolved] (SPARK-28116) Fix Flaky `SparkContextSuite.test resource scheduling under local-cluster mode`

[ https://issues.apache.org/jira/browse/SPARK-28116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28116. --- Resolution: Duplicate > Fix Flaky `SparkContextSuite.test resource scheduling under

[jira] [Updated] (SPARK-28115) Fix flaky test: SparkContextSuite.test resource scheduling under local-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-28115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28115: -- Description: This test suite has two kind of failures.

[jira] [Assigned] (SPARK-28077) ANSI SQL: OVERLAY function(T312)

[ https://issues.apache.org/jira/browse/SPARK-28077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28077: Assignee: (was: Apache Spark) > ANSI SQL: OVERLAY function(T312) >

[jira] [Assigned] (SPARK-28077) ANSI SQL: OVERLAY function(T312)

[ https://issues.apache.org/jira/browse/SPARK-28077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28077: Assignee: Apache Spark > ANSI SQL: OVERLAY function(T312) >

[jira] [Assigned] (SPARK-28114) Add Jenkins job for `Hadoop-3.2` profile

[ https://issues.apache.org/jira/browse/SPARK-28114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-28114: --- Assignee: shane knapp > Add Jenkins job for `Hadoop-3.2` profile >

[jira] [Commented] (SPARK-28114) Add Jenkins job for `Hadoop-3.2` profile

[ https://issues.apache.org/jira/browse/SPARK-28114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868194#comment-16868194 ] Xiao Li commented on SPARK-28114: - Thank you, [~shaneknapp]   > Add Jenkins job for `Hadoop-3.2`

[jira] [Resolved] (SPARK-28112) Fix Kryo exception perf. bottleneck in tests due to absence of ML/MLlib classes

[ https://issues.apache.org/jira/browse/SPARK-28112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-28112. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24916

[jira] [Commented] (SPARK-26555) Thread safety issue causes createDataset to fail with misleading errors

[ https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868178#comment-16868178 ] Josh Rosen commented on SPARK-26555: Backported for 2.4.4 in

[jira] [Updated] (SPARK-26555) Thread safety issue causes createDataset to fail with misleading errors

[ https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-26555: --- Fix Version/s: 2.4.4 > Thread safety issue causes createDataset to fail with misleading errors >

[jira] [Commented] (SPARK-28094) Multiple left joins or aggregations in one query produce incorrect results

[ https://issues.apache.org/jira/browse/SPARK-28094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868174#comment-16868174 ] Jungtaek Lim commented on SPARK-28094: -- Unfortunately, it is closer to 2). Even you understand the

[jira] [Commented] (SPARK-25128) multiple simultaneous job submissions against k8s backend cause driver pods to hang

[ https://issues.apache.org/jira/browse/SPARK-25128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868138#comment-16868138 ] Suman Somasundar commented on SPARK-25128: -- I have the same issue. When multiple jobs are

[jira] [Commented] (SPARK-28116) Fix Flaky `SparkContextSuite.test resource scheduling under local-cluster mode`

[ https://issues.apache.org/jira/browse/SPARK-28116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868114#comment-16868114 ] Dongjoon Hyun commented on SPARK-28116: --- Hi, [~tgraves]. Could you take a look at this, please? >

[jira] [Created] (SPARK-28116) Fix Flaky `SparkContextSuite.test resource scheduling under local-cluster mode`

Dongjoon Hyun created SPARK-28116: - Summary: Fix Flaky `SparkContextSuite.test resource scheduling under local-cluster mode` Key: SPARK-28116 URL: https://issues.apache.org/jira/browse/SPARK-28116

[jira] [Assigned] (SPARK-28115) Fix flaky test: SparkContextSuite.test resource scheduling under local-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-28115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28115: Assignee: Xingbo Jiang (was: Apache Spark) > Fix flaky test: SparkContextSuite.test

[jira] [Assigned] (SPARK-28115) Fix flaky test: SparkContextSuite.test resource scheduling under local-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-28115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28115: Assignee: Apache Spark (was: Xingbo Jiang) > Fix flaky test: SparkContextSuite.test

[jira] [Created] (SPARK-28115) Fix flaky test: SparkContextSuite.test resource scheduling under local-cluster mode

Xingbo Jiang created SPARK-28115: Summary: Fix flaky test: SparkContextSuite.test resource scheduling under local-cluster mode Key: SPARK-28115 URL: https://issues.apache.org/jira/browse/SPARK-28115

[jira] [Updated] (SPARK-28114) Add Jenkins job for `Hadoop-3.2` profile

[ https://issues.apache.org/jira/browse/SPARK-28114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28114: -- Description: Spark 3.0 is a major version change. We want to have the following new Jobs. 1.

[jira] [Updated] (SPARK-28114) Add Jenkins job for `Hadoop-3.2` profile

[ https://issues.apache.org/jira/browse/SPARK-28114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28114: -- Description: Spark 3.0 is a major version change. We want to have the following new Jobs. 1.

[jira] [Updated] (SPARK-28114) Add Jenkins job for `Hadoop-3.2` profile

[ https://issues.apache.org/jira/browse/SPARK-28114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28114: -- Description: Spark 3.0 is a major version change. We want to have the following new Jobs. 1.

[jira] [Created] (SPARK-28114) Add Jenkins job for `Hadoop-3.2` profile

Dongjoon Hyun created SPARK-28114: - Summary: Add Jenkins job for `Hadoop-3.2` profile Key: SPARK-28114 URL: https://issues.apache.org/jira/browse/SPARK-28114 Project: Spark Issue Type:

[jira] [Commented] (SPARK-28114) Add Jenkins job for `Hadoop-3.2` profile

[ https://issues.apache.org/jira/browse/SPARK-28114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868063#comment-16868063 ] Dongjoon Hyun commented on SPARK-28114: --- Hi, Shane. Could you create those jobs? > Add Jenkins

[jira] [Resolved] (SPARK-28102) Failed LZ4 JNI initialization is repeatedly re-attempted, causing lock contention issues

[ https://issues.apache.org/jira/browse/SPARK-28102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-28102. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24905

[jira] [Assigned] (SPARK-28112) Fix Kryo exception perf. bottleneck in tests due to absence of ML/MLlib classes

[ https://issues.apache.org/jira/browse/SPARK-28112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28112: Assignee: Josh Rosen (was: Apache Spark) > Fix Kryo exception perf. bottleneck in tests

[jira] [Assigned] (SPARK-28112) Fix Kryo exception perf. bottleneck in tests due to absence of ML/MLlib classes

[ https://issues.apache.org/jira/browse/SPARK-28112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28112: Assignee: Apache Spark (was: Josh Rosen) > Fix Kryo exception perf. bottleneck in tests

[jira] [Resolved] (SPARK-27839) Improve UTF8String.replace() / StringReplace performance

[ https://issues.apache.org/jira/browse/SPARK-27839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-27839. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24707

[jira] [Created] (SPARK-28113) Lazy val performance pitfall on Spark LogicalPlan's output method

Yesheng Ma created SPARK-28113: -- Summary: Lazy val performance pitfall on Spark LogicalPlan's output method Key: SPARK-28113 URL: https://issues.apache.org/jira/browse/SPARK-28113 Project: Spark

[jira] [Created] (SPARK-28112) Fix Kryo exception perf. bottleneck in tests due to absence of ML/MLlib classes

Xiao Li created SPARK-28112: --- Summary: Fix Kryo exception perf. bottleneck in tests due to absence of ML/MLlib classes Key: SPARK-28112 URL: https://issues.apache.org/jira/browse/SPARK-28112 Project: Spark

[jira] [Assigned] (SPARK-28111) Upgrade `xbean-asm7-shaded` to 4.14

[ https://issues.apache.org/jira/browse/SPARK-28111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28111: Assignee: (was: Apache Spark) > Upgrade `xbean-asm7-shaded` to 4.14 >

[jira] [Assigned] (SPARK-28111) Upgrade `xbean-asm7-shaded` to 4.14

[ https://issues.apache.org/jira/browse/SPARK-28111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28111: Assignee: Apache Spark > Upgrade `xbean-asm7-shaded` to 4.14 >

[jira] [Created] (SPARK-28111) Upgrade `xbean-asm7-shaded` to 4.14

Dongjoon Hyun created SPARK-28111: - Summary: Upgrade `xbean-asm7-shaded` to 4.14 Key: SPARK-28111 URL: https://issues.apache.org/jira/browse/SPARK-28111 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-26412) Allow Pandas UDF to take an iterator of pd.DataFrames

[ https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867954#comment-16867954 ] Terry Kim commented on SPARK-26412: --- [~WeichenXu123] and [~mengxr] do you plan to do something similar

[jira] [Resolved] (SPARK-28109) TRIM(type trimStr FROM str) returns incorrect result

[ https://issues.apache.org/jira/browse/SPARK-28109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28109. --- Resolution: Fixed Assignee: Yuming Wang Fix Version/s: 3.0.0 This is

[jira] [Commented] (SPARK-26839) Work around classloader changes in Java 9 for Hive isolation

[ https://issues.apache.org/jira/browse/SPARK-26839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867921#comment-16867921 ] Dongjoon Hyun commented on SPARK-26839: --- Thanks. Yes, I'll take a look at them in the new JIRA~ >

[jira] [Updated] (SPARK-28110) on JDK11, IsolatedClientLoader must be able to load java.sql classes

[ https://issues.apache.org/jira/browse/SPARK-28110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28110: -- Summary: on JDK11, IsolatedClientLoader must be able to load java.sql classes (was: CLONE -

[jira] [Commented] (SPARK-26839) Work around classloader changes in Java 9 for Hive isolation

[ https://issues.apache.org/jira/browse/SPARK-26839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867912#comment-16867912 ] Sean Owen commented on SPARK-26839: --- There is still a classloader and datanucleus and Hive issue here

[jira] [Updated] (SPARK-26839) Work around classloader changes in Java 9 for Hive isolation

[ https://issues.apache.org/jira/browse/SPARK-26839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26839: -- Description: Java 9+ changed how ClassLoaders work. The two most salient points: The boot

[jira] [Resolved] (SPARK-26839) Work around classloader changes in Java 9 for Hive isolation

[ https://issues.apache.org/jira/browse/SPARK-26839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-26839. --- Resolution: Fixed Assignee: Sean Owen Fix Version/s: 3.0.0 This is resolved

[jira] [Updated] (SPARK-26839) Work around classloader changes in Java 9 for Hive isolation

[ https://issues.apache.org/jira/browse/SPARK-26839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26839: -- Summary: Work around classloader changes in Java 9 for Hive isolation (was: on JDK11,

[jira] [Created] (SPARK-28110) CLONE - on JDK11, IsolatedClientLoader must be able to load java.sql classes

Dongjoon Hyun created SPARK-28110: - Summary: CLONE - on JDK11, IsolatedClientLoader must be able to load java.sql classes Key: SPARK-28110 URL: https://issues.apache.org/jira/browse/SPARK-28110

[jira] [Comment Edited] (SPARK-26839) on JDK11, IsolatedClientLoader must be able to load java.sql classes

[ https://issues.apache.org/jira/browse/SPARK-26839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867906#comment-16867906 ] Dongjoon Hyun edited comment on SPARK-26839 at 6/19/19 7:02 PM: Hi,

[jira] [Comment Edited] (SPARK-26839) on JDK11, IsolatedClientLoader must be able to load java.sql classes

[ https://issues.apache.org/jira/browse/SPARK-26839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867906#comment-16867906 ] Dongjoon Hyun edited comment on SPARK-26839 at 6/19/19 7:01 PM: Hi,

[jira] [Comment Edited] (SPARK-26839) on JDK11, IsolatedClientLoader must be able to load java.sql classes

[ https://issues.apache.org/jira/browse/SPARK-26839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867906#comment-16867906 ] Dongjoon Hyun edited comment on SPARK-26839 at 6/19/19 7:01 PM: Hi,

[jira] [Commented] (SPARK-26839) on JDK11, IsolatedClientLoader must be able to load java.sql classes

[ https://issues.apache.org/jira/browse/SPARK-26839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867906#comment-16867906 ] Dongjoon Hyun commented on SPARK-26839: --- Hi, [~srowen]. Did you open your PR? > on JDK11,

[jira] [Commented] (SPARK-27529) Spark Streaming consumer dies with kafka.common.OffsetOutOfRangeException

[ https://issues.apache.org/jira/browse/SPARK-27529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867809#comment-16867809 ] Yurii commented on SPARK-27529: --- Hi, [~hyukjin.kwon], I have same problem * Spark 2.4.3 *

[jira] [Assigned] (SPARK-28103) Cannot infer filters from union table with empty local relation table properly

[ https://issues.apache.org/jira/browse/SPARK-28103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28103: Assignee: (was: Apache Spark) > Cannot infer filters from union table with empty

[jira] [Assigned] (SPARK-28103) Cannot infer filters from union table with empty local relation table properly

[ https://issues.apache.org/jira/browse/SPARK-28103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28103: Assignee: Apache Spark > Cannot infer filters from union table with empty local relation

[jira] [Assigned] (SPARK-28109) TRIM(type trimStr FROM str) returns incorrect result

[ https://issues.apache.org/jira/browse/SPARK-28109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28109: Assignee: (was: Apache Spark) > TRIM(type trimStr FROM str) returns incorrect result

[jira] [Assigned] (SPARK-28109) TRIM(type trimStr FROM str) returns incorrect result

[ https://issues.apache.org/jira/browse/SPARK-28109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28109: Assignee: Apache Spark > TRIM(type trimStr FROM str) returns incorrect result >

[jira] [Assigned] (SPARK-28108) Simplify OrcFilters

[ https://issues.apache.org/jira/browse/SPARK-28108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28108: Assignee: (was: Apache Spark) > Simplify OrcFilters > --- > >

[jira] [Commented] (SPARK-27946) Hive DDL to Spark DDL conversion USING "show create table"

[ https://issues.apache.org/jira/browse/SPARK-27946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867771#comment-16867771 ] Liang-Chi Hsieh commented on SPARK-27946: - [~smilegator] Thanks for pinging me. I'd like to do,

[jira] [Created] (SPARK-28109) TRIM(type trimStr FROM str) returns incorrect result

Yuming Wang created SPARK-28109: --- Summary: TRIM(type trimStr FROM str) returns incorrect result Key: SPARK-28109 URL: https://issues.apache.org/jira/browse/SPARK-28109 Project: Spark Issue

[jira] [Assigned] (SPARK-28108) Simplify OrcFilters

[ https://issues.apache.org/jira/browse/SPARK-28108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28108: Assignee: Apache Spark > Simplify OrcFilters > --- > >

[jira] [Created] (SPARK-28108) Simplify OrcFilters

Gengliang Wang created SPARK-28108: -- Summary: Simplify OrcFilters Key: SPARK-28108 URL: https://issues.apache.org/jira/browse/SPARK-28108 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-28102) Failed LZ4 JNI initialization is repeatedly re-attempted, causing lock contention issues

[ https://issues.apache.org/jira/browse/SPARK-28102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-28102: --- Description: Spark's use of {{lz4-java}} ends up calling {{LZ4Factory.fastestInstance}}, which

[jira] [Updated] (SPARK-28102) Failed LZ4 JNI initialization is repeatedly re-attempted, causing lock contention issues

[ https://issues.apache.org/jira/browse/SPARK-28102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-28102: --- Summary: Failed LZ4 JNI initialization is repeatedly re-attempted, causing lock contention issues

[jira] [Updated] (SPARK-28107) Interval type conversion syntax support

[ https://issues.apache.org/jira/browse/SPARK-28107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28107: Issue Type: Sub-task (was: Improvement) Parent: SPARK-27764 > Interval type conversion

[jira] [Resolved] (SPARK-28101) Fix Flaky Test: `InputStreamsSuite.Modified files are correctly detected` in JDK9+

[ https://issues.apache.org/jira/browse/SPARK-28101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28101. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 3.0.0 This is

[jira] [Commented] (SPARK-28094) Multiple left joins or aggregations in one query produce incorrect results

[ https://issues.apache.org/jira/browse/SPARK-28094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867708#comment-16867708 ] Joe Ammann commented on SPARK-28094: Hi [~kabhwan] I see that SPARK-28074 is mainly a documentation

[jira] [Assigned] (SPARK-28062) HuberAggregator copies coefficients vector every time an instance is added

[ https://issues.apache.org/jira/browse/SPARK-28062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-28062: - Assignee: Andrew Crosby > HuberAggregator copies coefficients vector every time an instance is

[jira] [Resolved] (SPARK-28062) HuberAggregator copies coefficients vector every time an instance is added

[ https://issues.apache.org/jira/browse/SPARK-28062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28062. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24880

[jira] [Assigned] (SPARK-28044) MulticlassClassificationEvaluator support more metrics

[ https://issues.apache.org/jira/browse/SPARK-28044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-28044: - Assignee: zhengruifeng > MulticlassClassificationEvaluator support more metrics >

[jira] [Resolved] (SPARK-28044) MulticlassClassificationEvaluator support more metrics

[ https://issues.apache.org/jira/browse/SPARK-28044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28044. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24868

[jira] [Updated] (SPARK-28106) Spark SQL add jar with wrong hdfs path, SparkContext still add it to jar path ,and cause Task Failed

[ https://issues.apache.org/jira/browse/SPARK-28106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28106: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) > Spark SQL add jar with

[jira] [Commented] (SPARK-28093) Built-in function trim/ltrim/rtrim has bug when using trimStr

[ https://issues.apache.org/jira/browse/SPARK-28093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867621#comment-16867621 ] Sean Owen commented on SPARK-28093: --- Should we even call it a correctness problem? Like is this

[jira] [Updated] (SPARK-28093) Built-in function trim/ltrim/rtrim has bug when using trimStr

[ https://issues.apache.org/jira/browse/SPARK-28093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28093: -- Labels: release-notes (was: ) > Built-in function trim/ltrim/rtrim has bug when using trimStr >

[jira] [Updated] (SPARK-28106) Spark SQL add jar with wrong hdfs path, SparkContext still add it to jar path ,and cause Task Failed

[ https://issues.apache.org/jira/browse/SPARK-28106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-28106: -- Description: When we use SparkSQL, about add jar command, if we add a wrong path of HDFS such as

[jira] [Updated] (SPARK-28106) Spark SQL add jar with wrong hdfs path, SparkContext still add it to jar path ,and cause Task Failed

[ https://issues.apache.org/jira/browse/SPARK-28106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-28106: -- Attachment: image-2019-06-19-21-23-22-061.png > Spark SQL add jar with wrong hdfs path, SparkContext

[jira] [Updated] (SPARK-28106) Spark SQL add jar with wrong hdfs path, SparkContext still add it to jar path ,and cause Task Failed

[ https://issues.apache.org/jira/browse/SPARK-28106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-28106: -- Description: When we use SparkSQL, about add jar command, if we add a wrong path of HDFS such as

  1   2   >