[jira] [Updated] (SPARK-29168) Use a unique color on selected item on timeline view

[ https://issues.apache.org/jira/browse/SPARK-29168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29168: -- Issue Type: Improvement (was: Bug) > Use a unique color on selected item on timeline view > -

[jira] [Updated] (SPARK-29168) Use a unique color on selected item on timeline view

[ https://issues.apache.org/jira/browse/SPARK-29168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29168: -- Summary: Use a unique color on selected item on timeline view (was: Fix the appearance issue

[jira] [Assigned] (SPARK-29168) Use a unique color on selected item on timeline view

[ https://issues.apache.org/jira/browse/SPARK-29168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29168: - Assignee: Tomoko Komiyama > Use a unique color on selected item on timeline view >

[jira] [Commented] (SPARK-29168) Use a unique color on selected item on timeline view

[ https://issues.apache.org/jira/browse/SPARK-29168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936480#comment-16936480 ] Dongjoon Hyun commented on SPARK-29168: --- You are added to the Apache Spark contrib

[jira] [Commented] (SPARK-28558) DatasetWriter partitionBy is changing the group file permissions in 2.4 for parquets

[ https://issues.apache.org/jira/browse/SPARK-28558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936485#comment-16936485 ] Nicolas Laduguie commented on SPARK-28558: -- Hi [~spearson] [~holden] we are usi

[jira] [Resolved] (SPARK-29168) Use a unique color on selected item on timeline view

[ https://issues.apache.org/jira/browse/SPARK-29168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29168. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25846 [https://

[jira] [Commented] (SPARK-28558) DatasetWriter partitionBy is changing the group file permissions in 2.4 for parquets

[ https://issues.apache.org/jira/browse/SPARK-28558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936504#comment-16936504 ] Nicolas Laduguie commented on SPARK-28558: -- FYI here is a screenshot showing th

[jira] [Updated] (SPARK-28558) DatasetWriter partitionBy is changing the group file permissions in 2.4 for parquets

[ https://issues.apache.org/jira/browse/SPARK-28558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Laduguie updated SPARK-28558: - Attachment: image-2019-09-24-09-20-07-225.png > DatasetWriter partitionBy is changing th

[jira] [Created] (SPARK-29226) Upgrade jackson-databind to 2.9.10 and fix vulnerabilities.

jiaan.geng created SPARK-29226: -- Summary: Upgrade jackson-databind to 2.9.10 and fix vulnerabilities. Key: SPARK-29226 URL: https://issues.apache.org/jira/browse/SPARK-29226 Project: Spark Issu

[jira] [Commented] (SPARK-29211) Second invocation of custom UDF results in exception (when invoked from shell)

[ https://issues.apache.org/jira/browse/SPARK-29211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936529#comment-16936529 ] Dilip Biswal commented on SPARK-29211: -- [~dongjoon] I tried with 2.3.4 and see the

[jira] [Commented] (SPARK-29211) Second invocation of custom UDF results in exception (when invoked from shell)

[ https://issues.apache.org/jira/browse/SPARK-29211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936538#comment-16936538 ] angerszhu commented on SPARK-29211: --- [~dkbiswal] Set a local repo follow Apache/spark

[jira] [Updated] (SPARK-29226) Upgrade jackson-databind to 2.9.10 and fix vulnerabilities.

[ https://issues.apache.org/jira/browse/SPARK-29226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29226: Component/s: (was: Spark Core) Build > Upgrade jackson-databind to 2.9.10 and

[jira] [Updated] (SPARK-29226) Upgrade jackson-databind to 2.9.10 and fix vulnerabilities.

[ https://issues.apache.org/jira/browse/SPARK-29226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29226: Affects Version/s: (was: 2.4.5) > Upgrade jackson-databind to 2.9.10 and fix vulnerabilities.

[jira] [Created] (SPARK-29227) track rule info in optimization phase

wenxuanguan created SPARK-29227: --- Summary: track rule info in optimization phase Key: SPARK-29227 URL: https://issues.apache.org/jira/browse/SPARK-29227 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-29227) Track rule info in optimization phase

[ https://issues.apache.org/jira/browse/SPARK-29227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wenxuanguan updated SPARK-29227: Summary: Track rule info in optimization phase (was: track rule info in optimization phase ) >

[jira] [Updated] (SPARK-29227) Track rule info in optimization phase

[ https://issues.apache.org/jira/browse/SPARK-29227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wenxuanguan updated SPARK-29227: Description: Track timing info for each rule in optimization phase using QueryPlanningTracker in S

[jira] [Commented] (SPARK-29119) DEFAULT option is not supported in Spark

[ https://issues.apache.org/jira/browse/SPARK-29119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936596#comment-16936596 ] jiaan.geng commented on SPARK-29119: This is duplicate with https://issues.apache.or

[jira] [Assigned] (SPARK-28678) Specify that start index is 1-based in docstring of pyspark.sql.functions.slice

[ https://issues.apache.org/jira/browse/SPARK-28678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-28678: Assignee: Ting Yang > Specify that start index is 1-based in docstring of > pyspark.sql.

[jira] [Resolved] (SPARK-28678) Specify that start index is 1-based in docstring of pyspark.sql.functions.slice

[ https://issues.apache.org/jira/browse/SPARK-28678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28678. -- Fix Version/s: 3.0.0 2.4.5 Resolution: Fixed Issue resolved by pull

[jira] [Resolved] (SPARK-28292) Enable inject user-defined Hint

[ https://issues.apache.org/jira/browse/SPARK-28292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28292. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25746 [https://gith

[jira] [Assigned] (SPARK-28292) Enable inject user-defined Hint

[ https://issues.apache.org/jira/browse/SPARK-28292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-28292: --- Assignee: Xiao Li > Enable inject user-defined Hint > --- > >

[jira] [Created] (SPARK-29228) Adding the ability to add headers to the KafkaRecords published to Kafka from spark structured streaming

R JHS B created SPARK-29228: --- Summary: Adding the ability to add headers to the KafkaRecords published to Kafka from spark structured streaming Key: SPARK-29228 URL: https://issues.apache.org/jira/browse/SPARK-29228

[jira] [Commented] (SPARK-29221) Flaky test: SQLQueryTestSuite.sql (subquery/scalar-subquery/scalar-subquery-select.sql)

[ https://issues.apache.org/jira/browse/SPARK-29221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936713#comment-16936713 ] Yuming Wang commented on SPARK-29221: - https://amplab.cs.berkeley.edu/jenkins/job/sp

[jira] [Commented] (SPARK-29228) Adding the ability to add headers to the KafkaRecords published to Kafka from spark structured streaming

[ https://issues.apache.org/jira/browse/SPARK-29228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936715#comment-16936715 ] Randal Boyle commented on SPARK-29228: -- I [~rjhsb1989] am working on this :) > Add

[jira] [Commented] (SPARK-28558) DatasetWriter partitionBy is changing the group file permissions in 2.4 for parquets

[ https://issues.apache.org/jira/browse/SPARK-28558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936719#comment-16936719 ] Nicolas Laduguie commented on SPARK-28558: -- I debugged Spark and my understandi

[jira] [Created] (SPARK-29229) Change the additional remote repository in IsolatedClientLoader to google minor

Yuanjian Li created SPARK-29229: --- Summary: Change the additional remote repository in IsolatedClientLoader to google minor Key: SPARK-29229 URL: https://issues.apache.org/jira/browse/SPARK-29229 Project

[jira] [Commented] (SPARK-29197) Remove saveModeForDSV2 in DataFrameWriter

[ https://issues.apache.org/jira/browse/SPARK-29197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936731#comment-16936731 ] Ido Michael commented on SPARK-29197: - Great job Burak! And thanks for the quick re

[jira] [Commented] (SPARK-29166) Add parameters to limit the number of dynamic partitions for data source table

[ https://issues.apache.org/jira/browse/SPARK-29166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936732#comment-16936732 ] Ido Michael commented on SPARK-29166: - No problem [~cltlfcjin], great job! > Add pa

[jira] [Updated] (SPARK-29175) Make maven central repository in IsolatedClientLoader configurable

[ https://issues.apache.org/jira/browse/SPARK-29175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanjian Li updated SPARK-29175: Description: We need to connect a central repository in IsolatedClientLoader for downloading Hive

[jira] [Commented] (SPARK-29225) Spark SQL 'DESC FORMATTED TABLE' show different format with hive

[ https://issues.apache.org/jira/browse/SPARK-29225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936755#comment-16936755 ] Hyukjin Kwon commented on SPARK-29225: -- Can you show current & expected input/outpu

[jira] [Commented] (SPARK-29222) Flaky test: pyspark.mllib.tests.test_streaming_algorithms.StreamingLinearRegressionWithTests.test_parameter_convergence

[ https://issues.apache.org/jira/browse/SPARK-29222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936756#comment-16936756 ] Hyukjin Kwon commented on SPARK-29222: -- cc [~bryanc] > Flaky test: > pyspark.mlli

[jira] [Resolved] (SPARK-29217) How to read streaming output path by ignoring metadata log files

[ https://issues.apache.org/jira/browse/SPARK-29217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29217. -- Resolution: Invalid Please ask questions to mailing list or stackoverflow. > How to read stre

[jira] [Commented] (SPARK-29210) Spark 3.0 Migration guide should contain the details of to_utc_timestamp function and from_utc_timestamp

[ https://issues.apache.org/jira/browse/SPARK-29210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936760#comment-16936760 ] Hyukjin Kwon commented on SPARK-29210: -- Deprecations don't have to be documented in

[jira] [Resolved] (SPARK-29210) Spark 3.0 Migration guide should contain the details of to_utc_timestamp function and from_utc_timestamp

[ https://issues.apache.org/jira/browse/SPARK-29210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29210. -- Resolution: Won't Fix > Spark 3.0 Migration guide should contain the details of to_utc_timesta

[jira] [Commented] (SPARK-29205) Pyspark tests failed for suspected performance problem on ARM

[ https://issues.apache.org/jira/browse/SPARK-29205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936763#comment-16936763 ] Hyukjin Kwon commented on SPARK-29205: -- It seems to be flaky test, rather. and Yes,

[jira] [Resolved] (SPARK-29205) Pyspark tests failed for suspected performance problem on ARM

[ https://issues.apache.org/jira/browse/SPARK-29205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29205. -- Resolution: Duplicate Unfortunately, seems it's still flaky (although less flaky after the fix

[jira] [Commented] (SPARK-29195) Can't config orc.compress.size option for native ORC writer

[ https://issues.apache.org/jira/browse/SPARK-29195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936767#comment-16936767 ] Hyukjin Kwon commented on SPARK-29195: -- Can you narrow down if this is a bug in ORC

[jira] [Updated] (SPARK-29186) SubqueryAlias name value is null in Spark 2.4.3 Logical plan.

[ https://issues.apache.org/jira/browse/SPARK-29186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29186: - Target Version/s: (was: 2.4.3) > SubqueryAlias name value is null in Spark 2.4.3 Logical plan.

[jira] [Commented] (SPARK-29186) SubqueryAlias name value is null in Spark 2.4.3 Logical plan.

[ https://issues.apache.org/jira/browse/SPARK-29186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936770#comment-16936770 ] Hyukjin Kwon commented on SPARK-29186: -- Can you reformat the description with \{cod

[jira] [Updated] (SPARK-29185) Add new SaveMode types for Spark SQL jdbc datasource

[ https://issues.apache.org/jira/browse/SPARK-29185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29185: - Component/s: (was: Input/Output) SQL > Add new SaveMode types for Spark SQL

[jira] [Resolved] (SPARK-29184) I have installed pyspark and created code in notebook but when I am running it.its throwing error-Py4JJavaError: An error occurred while calling z:org.apache.spark.api.

[ https://issues.apache.org/jira/browse/SPARK-29184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29184. -- Resolution: Invalid Please ask questions to mailing list or stackoverflow. > I have installed

[jira] [Commented] (SPARK-29178) Improve shown message on sql show()

[ https://issues.apache.org/jira/browse/SPARK-29178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936772#comment-16936772 ] Hyukjin Kwon commented on SPARK-29178: -- In order to do that it requires to count al

[jira] [Updated] (SPARK-29225) Spark SQL 'DESC FORMATTED TABLE' show different format with hive

[ https://issues.apache.org/jira/browse/SPARK-29225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-29225: -- Description: Current `DESC FORMATTED TABLE` show different table desc format, this problem cause HUE

[jira] [Updated] (SPARK-29225) Spark SQL 'DESC FORMATTED TABLE' show different format with hive

[ https://issues.apache.org/jira/browse/SPARK-29225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-29225: -- Attachment: current saprk.jpg > Spark SQL 'DESC FORMATTED TABLE' show different format with hive > ---

[jira] [Resolved] (SPARK-29178) Improve shown message on sql show()

[ https://issues.apache.org/jira/browse/SPARK-29178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29178. -- Resolution: Won't Fix > Improve shown message on sql show() >

[jira] [Updated] (SPARK-29174) LOCAL is not supported in INSERT OVERWRITE DIRECTORY to data source

[ https://issues.apache.org/jira/browse/SPARK-29174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29174: - Description: *using does not work for insert overwrite when in local  but works when insert ove

[jira] [Updated] (SPARK-29174) LOCAL is not supported in INSERT OVERWRITE DIRECTORY to data source

[ https://issues.apache.org/jira/browse/SPARK-29174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29174: - Description: *using does not work for insert overwrite when in local  but works when insert ove

[jira] [Updated] (SPARK-29225) Spark SQL 'DESC FORMATTED TABLE' show different format with hive

[ https://issues.apache.org/jira/browse/SPARK-29225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-29225: -- Description: Current `DESC FORMATTED TABLE` show different table desc format, this problem cause HUE

[jira] [Updated] (SPARK-29225) Spark SQL 'DESC FORMATTED TABLE' show different format with hive

[ https://issues.apache.org/jira/browse/SPARK-29225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-29225: -- Description: Current `DESC FORMATTED TABLE` show different table desc format, this problem cause HUE

[jira] [Updated] (SPARK-29225) Spark SQL 'DESC FORMATTED TABLE' show different format with hive

[ https://issues.apache.org/jira/browse/SPARK-29225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-29225: -- Attachment: Screen Shot 2019-09-24 at 9.14.39 PM.png > Spark SQL 'DESC FORMATTED TABLE' show different

[jira] [Resolved] (SPARK-29156) Hive has appending data as part of cdc, In write mode we should be able to write only changes captured to teradata or datasource.

[ https://issues.apache.org/jira/browse/SPARK-29156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29156. -- Resolution: Invalid no feedback > Hive has appending data as part of cdc, In write mode we sh

[jira] [Commented] (SPARK-28990) SparkSQL invalid call to toAttribute on unresolved object, tree: *

[ https://issues.apache.org/jira/browse/SPARK-28990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936778#comment-16936778 ] fengchaoge commented on SPARK-28990: @[~726575...@qq.com] hello daile ,Can you send

[jira] [Commented] (SPARK-29225) Spark SQL 'DESC FORMATTED TABLE' show different format with hive

[ https://issues.apache.org/jira/browse/SPARK-29225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936781#comment-16936781 ] angerszhu commented on SPARK-29225: --- [~hyukjin.kwon] Sorry for forget to add desc. 

[jira] [Updated] (SPARK-29225) Spark SQL 'DESC FORMATTED TABLE' show different format with hive

[ https://issues.apache.org/jira/browse/SPARK-29225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-29225: -- Attachment: Screen Shot 2019-09-24 at 9.26.14 PM.png > Spark SQL 'DESC FORMATTED TABLE' show different

[jira] [Updated] (SPARK-29225) Spark SQL 'DESC FORMATTED TABLE' show different format with hive

[ https://issues.apache.org/jira/browse/SPARK-29225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-29225: -- Description: Current `DESC FORMATTED TABLE` show different table desc format, this problem cause HUE

[jira] [Updated] (SPARK-29225) Spark SQL 'DESC FORMATTED TABLE' show different format with hive

[ https://issues.apache.org/jira/browse/SPARK-29225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-29225: -- Description: Current `DESC FORMATTED TABLE` show different table desc format, this problem cause HUE

[jira] [Resolved] (SPARK-29095) add extractInstances

[ https://issues.apache.org/jira/browse/SPARK-29095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-29095. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25802 [https://github.c

[jira] [Assigned] (SPARK-29095) add extractInstances

[ https://issues.apache.org/jira/browse/SPARK-29095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-29095: - Assignee: zhengruifeng > add extractInstances > > > Key: S

[jira] [Created] (SPARK-29230) Fix NullPointerException in the test class ProcfsMetricsGetterSuite

Jiaqi Li created SPARK-29230: Summary: Fix NullPointerException in the test class ProcfsMetricsGetterSuite Key: SPARK-29230 URL: https://issues.apache.org/jira/browse/SPARK-29230 Project: Spark

[jira] [Updated] (SPARK-29230) Fix NullPointerException in the test class ProcfsMetricsGetterSuite

[ https://issues.apache.org/jira/browse/SPARK-29230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaqi Li updated SPARK-29230: - Description: When I use `ProcfsMetricsGetterSuite` testing, always throw out  `java.lang.NullPointerExce

[jira] [Commented] (SPARK-29231) Can not infer an additional set of constraints if contains CAST

[ https://issues.apache.org/jira/browse/SPARK-29231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936978#comment-16936978 ] Yuming Wang commented on SPARK-29231: - I'm working on. > Can not infer an additiona

[jira] [Created] (SPARK-29231) Can not infer an additional set of constraints if contains CAST

Yuming Wang created SPARK-29231: --- Summary: Can not infer an additional set of constraints if contains CAST Key: SPARK-29231 URL: https://issues.apache.org/jira/browse/SPARK-29231 Project: Spark

[jira] [Resolved] (SPARK-29228) Adding the ability to add headers to the KafkaRecords published to Kafka from spark structured streaming

[ https://issues.apache.org/jira/browse/SPARK-29228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randal Boyle resolved SPARK-29228. -- Resolution: Duplicate > Adding the ability to add headers to the KafkaRecords published to Kaf

[jira] [Resolved] (SPARK-29229) Change the additional remote repository in IsolatedClientLoader to google minor

[ https://issues.apache.org/jira/browse/SPARK-29229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29229. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25915 [https://gith

[jira] [Assigned] (SPARK-29229) Change the additional remote repository in IsolatedClientLoader to google minor

[ https://issues.apache.org/jira/browse/SPARK-29229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29229: --- Assignee: Yuanjian Li > Change the additional remote repository in IsolatedClientLoader to

[jira] [Updated] (SPARK-29229) Change the additional remote repository in IsolatedClientLoader to google minor

[ https://issues.apache.org/jira/browse/SPARK-29229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29229: -- Issue Type: Bug (was: Improvement) > Change the additional remote repository in IsolatedClien

[jira] [Updated] (SPARK-29229) Change the additional remote repository in IsolatedClientLoader to google minor

[ https://issues.apache.org/jira/browse/SPARK-29229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29229: -- Fix Version/s: 2.4.5 > Change the additional remote repository in IsolatedClientLoader to goog

[jira] [Commented] (SPARK-29229) Change the additional remote repository in IsolatedClientLoader to google minor

[ https://issues.apache.org/jira/browse/SPARK-29229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937057#comment-16937057 ] Dongjoon Hyun commented on SPARK-29229: --- This is backported to `branch-2.4` via h

[jira] [Resolved] (SPARK-29082) Spark driver cannot start with only delegation tokens

[ https://issues.apache.org/jira/browse/SPARK-29082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29082. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25901 [https://

[jira] [Created] (SPARK-29232) RandomForestRegressionModel does not update the parameter maps of the DecisionTreeRegressionModels underneath

Jiaqi Guo created SPARK-29232: - Summary: RandomForestRegressionModel does not update the parameter maps of the DecisionTreeRegressionModels underneath Key: SPARK-29232 URL: https://issues.apache.org/jira/browse/SPARK-

[jira] [Created] (SPARK-29233) Add regex expression checks for executorEnv in K8S mode

merrily01 created SPARK-29233: - Summary: Add regex expression checks for executorEnv in K8S mode Key: SPARK-29233 URL: https://issues.apache.org/jira/browse/SPARK-29233 Project: Spark Issue Type:

[jira] [Created] (SPARK-29234) bucketed table created by Spark SQL DataFrame is in SequenceFile format

Suchintak Patnaik created SPARK-29234: - Summary: bucketed table created by Spark SQL DataFrame is in SequenceFile format Key: SPARK-29234 URL: https://issues.apache.org/jira/browse/SPARK-29234 Pro

[jira] [Updated] (SPARK-29233) Add regex expression checks for executorEnv in K8S mode

[ https://issues.apache.org/jira/browse/SPARK-29233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] merrily01 updated SPARK-29233: -- Attachment: error.jpeg > Add regex expression checks for executorEnv in K8S mode > ---

[jira] [Updated] (SPARK-29234) bucketed table created by Spark SQL DataFrame is in SequenceFile format

[ https://issues.apache.org/jira/browse/SPARK-29234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suchintak Patnaik updated SPARK-29234: -- Description: When we create a bucketed table as follows, it's input and output format

[jira] [Updated] (SPARK-29233) Add regex expression checks for executorEnv in K8S mode

[ https://issues.apache.org/jira/browse/SPARK-29233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] merrily01 updated SPARK-29233: -- Attachment: podEnv.jpeg > Add regex expression checks for executorEnv in K8S mode > --

[jira] [Updated] (SPARK-29234) bucketed table created by Spark SQL DataFrame is in SequenceFile format

[ https://issues.apache.org/jira/browse/SPARK-29234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suchintak Patnaik updated SPARK-29234: -- Description: When we create a bucketed table as follows, it's input and output format

[jira] [Commented] (SPARK-29220) Flaky test: org.apache.spark.deploy.yarn.LocalityPlacementStrategySuite.handle large number of containers and tasks (SPARK-18750) [hadoop-3.2][java11]

[ https://issues.apache.org/jira/browse/SPARK-29220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937120#comment-16937120 ] shane knapp commented on SPARK-29220: - for https://amplab.cs.berkeley.edu/jenkins/j

[jira] [Issue Comment Deleted] (SPARK-29220) Flaky test: org.apache.spark.deploy.yarn.LocalityPlacementStrategySuite.handle large number of containers and tasks (SPARK-18750) [hadoop-3.2][java11]

[ https://issues.apache.org/jira/browse/SPARK-29220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shane knapp updated SPARK-29220: Comment: was deleted (was: for https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder

[jira] [Commented] (SPARK-29220) Flaky test: org.apache.spark.deploy.yarn.LocalityPlacementStrategySuite.handle large number of containers and tasks (SPARK-18750) [hadoop-3.2][java11]

[ https://issues.apache.org/jira/browse/SPARK-29220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937124#comment-16937124 ] shane knapp commented on SPARK-29220: - ARGH i just messed up and accidentally overwr

[jira] [Commented] (SPARK-29220) Flaky test: org.apache.spark.deploy.yarn.LocalityPlacementStrategySuite.handle large number of containers and tasks (SPARK-18750) [hadoop-3.2][java11]

[ https://issues.apache.org/jira/browse/SPARK-29220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937126#comment-16937126 ] shane knapp commented on SPARK-29220: - hmm, i see that https://github.com/apache/spa

[jira] [Updated] (SPARK-29211) Second invocation of custom UDF results in exception (when invoked from shell)

[ https://issues.apache.org/jira/browse/SPARK-29211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29211: -- Affects Version/s: 2.3.4 > Second invocation of custom UDF results in exception (when invoked

[jira] [Commented] (SPARK-29211) Second invocation of custom UDF results in exception (when invoked from shell)

[ https://issues.apache.org/jira/browse/SPARK-29211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937149#comment-16937149 ] Dongjoon Hyun commented on SPARK-29211: --- Thank you, [~dkbiswal]. That's enough. >

[jira] [Commented] (SPARK-29220) Flaky test: org.apache.spark.deploy.yarn.LocalityPlacementStrategySuite.handle large number of containers and tasks (SPARK-18750) [hadoop-3.2][java11]

[ https://issues.apache.org/jira/browse/SPARK-29220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937152#comment-16937152 ] Dongjoon Hyun commented on SPARK-29220: --- Thank you for investigation, [~shaneknapp

[jira] [Commented] (SPARK-29220) Flaky test: org.apache.spark.deploy.yarn.LocalityPlacementStrategySuite.handle large number of containers and tasks (SPARK-18750) [hadoop-3.2][java11]

[ https://issues.apache.org/jira/browse/SPARK-29220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937154#comment-16937154 ] Dongjoon Hyun commented on SPARK-29220: --- Since I'm monitoring Jenkins jobs, I'll p

[jira] [Commented] (SPARK-29220) Flaky test: org.apache.spark.deploy.yarn.LocalityPlacementStrategySuite.handle large number of containers and tasks (SPARK-18750) [hadoop-3.2][java11]

[ https://issues.apache.org/jira/browse/SPARK-29220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937155#comment-16937155 ] shane knapp commented on SPARK-29220: - it's ok... let's try and keep an eye on thes

[jira] [Created] (SPARK-29235) CrossValidatorModel.avgMetrics disappears after model is written/read again

Matthew Bedford created SPARK-29235: --- Summary: CrossValidatorModel.avgMetrics disappears after model is written/read again Key: SPARK-29235 URL: https://issues.apache.org/jira/browse/SPARK-29235 Pro

[jira] [Updated] (SPARK-29235) CrossValidatorModel.avgMetrics disappears after model is written/read again

[ https://issues.apache.org/jira/browse/SPARK-29235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Bedford updated SPARK-29235: Description:   Right after a CrossValidatorModel is trained, it has avgMetrics.  After th

[jira] [Updated] (SPARK-29235) CrossValidatorModel.avgMetrics disappears after model is written/read again

[ https://issues.apache.org/jira/browse/SPARK-29235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Bedford updated SPARK-29235: Description:   Right after a CrossValidatorModel is trained, it has avgMetrics.  After th

[jira] [Commented] (SPARK-29220) Flaky test: org.apache.spark.deploy.yarn.LocalityPlacementStrategySuite.handle large number of containers and tasks (SPARK-18750) [hadoop-3.2][java11]

[ https://issues.apache.org/jira/browse/SPARK-29220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937170#comment-16937170 ] Jungtaek Lim commented on SPARK-29220: -- Could we expose a way to store these unit-t

[jira] [Resolved] (SPARK-29230) Fix NullPointerException in the test class ProcfsMetricsGetterSuite

[ https://issues.apache.org/jira/browse/SPARK-29230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29230. --- Fix Version/s: 3.0.0 Resolution: Fixed This is resolved via https://github.com/apache

[jira] [Assigned] (SPARK-29230) Fix NullPointerException in the test class ProcfsMetricsGetterSuite

[ https://issues.apache.org/jira/browse/SPARK-29230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29230: - Assignee: Jiaqi Li > Fix NullPointerException in the test class ProcfsMetricsGetterSuit

[jira] [Updated] (SPARK-29230) Fix NullPointerException in the test class ProcfsMetricsGetterSuite

[ https://issues.apache.org/jira/browse/SPARK-29230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29230: -- Component/s: Tests > Fix NullPointerException in the test class ProcfsMetricsGetterSuite > ---

[jira] [Updated] (SPARK-29235) CrossValidatorModel.avgMetrics disappears after model is written/read again

[ https://issues.apache.org/jira/browse/SPARK-29235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Bedford updated SPARK-29235: Affects Version/s: 2.4.3 > CrossValidatorModel.avgMetrics disappears after model is writte

[jira] [Commented] (SPARK-29206) Number of shuffle Netty server threads should be a multiple of number of chunk fetch handler threads

[ https://issues.apache.org/jira/browse/SPARK-29206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937219#comment-16937219 ] Min Shen commented on SPARK-29206: -- [~irashid], Would appreciate your thoughts on this

[jira] [Updated] (SPARK-29235) CrossValidatorModel.avgMetrics disappears after model is written/read again

[ https://issues.apache.org/jira/browse/SPARK-29235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Bedford updated SPARK-29235: Description:   Right after a CrossValidatorModel is trained, it has avgMetrics.  After th

[jira] [Commented] (SPARK-28502) Error with struct conversion while using pandas_udf

[ https://issues.apache.org/jira/browse/SPARK-28502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937233#comment-16937233 ] Bryan Cutler commented on SPARK-28502: -- I was able to reproduce in Spark 2.4.3. The

[jira] [Resolved] (SPARK-29234) bucketed table created by Spark SQL DataFrame is in SequenceFile format

[ https://issues.apache.org/jira/browse/SPARK-29234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-29234. - Resolution: Duplicate Issue resolved by pull request 24486 [https://github.com/apache/spark/pull

[jira] [Closed] (SPARK-28915) The new keyword is not used when instantiating the WorkerOffer.

[ https://issues.apache.org/jira/browse/SPARK-28915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaqi Li closed SPARK-28915. > The new keyword is not used when instantiating the WorkerOffer. > --

[jira] [Created] (SPARK-29236) Access 'executorDataMap' out of 'DriverEndpoint' should be protected by lock

Xianyang Liu created SPARK-29236: Summary: Access 'executorDataMap' out of 'DriverEndpoint' should be protected by lock Key: SPARK-29236 URL: https://issues.apache.org/jira/browse/SPARK-29236 Project:

[jira] [Updated] (SPARK-29233) Add regex expression checks for executorEnv in K8S mode

[ https://issues.apache.org/jira/browse/SPARK-29233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] merrily01 updated SPARK-29233: -- Description: In k8s mode, there are some naming regular expression requirements and checks for the ke

  1   2   >