[jira] [Resolved] (SPARK-29512) REPAIR TABLE should look up catalog/table like v2 commands

[ https://issues.apache.org/jira/browse/SPARK-29512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-29512. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26168

[jira] [Assigned] (SPARK-29512) REPAIR TABLE should look up catalog/table like v2 commands

[ https://issues.apache.org/jira/browse/SPARK-29512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-29512: --- Assignee: Terry Kim > REPAIR TABLE should look up catalog/table like v2 commands >

[jira] [Created] (SPARK-29516) Test ThriftServerQueryTestSuite asynchronously

Yuming Wang created SPARK-29516: --- Summary: Test ThriftServerQueryTestSuite asynchronously Key: SPARK-29516 URL: https://issues.apache.org/jira/browse/SPARK-29516 Project: Spark Issue Type:

[jira] [Commented] (SPARK-29106) Add jenkins arm test for spark

[ https://issues.apache.org/jira/browse/SPARK-29106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955061#comment-16955061 ] zhao bo commented on SPARK-29106: - Thanks [~shaneknapp]. That's great that you will copy a full jenkins

[jira] [Updated] (SPARK-29494) ArrayOutOfBoundsException when converting from string to timestamp

[ https://issues.apache.org/jira/browse/SPARK-29494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-29494: - Fix Version/s: (was: 2.4.5) > ArrayOutOfBoundsException when converting from string to

[jira] [Resolved] (SPARK-29502) typed interval expression should fail for invalid format

[ https://issues.apache.org/jira/browse/SPARK-29502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-29502. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26151

[jira] [Resolved] (SPARK-29466) Show `Duration` for running drivers in Standalone master web UI

[ https://issues.apache.org/jira/browse/SPARK-29466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29466. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26113

[jira] [Assigned] (SPARK-29466) Show `Duration` for running drivers in Standalone master web UI

[ https://issues.apache.org/jira/browse/SPARK-29466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29466: - Assignee: Dongjoon Hyun > Show `Duration` for running drivers in Standalone master web

[jira] [Assigned] (SPARK-29494) ArrayOutOfBoundsException when converting from string to timestamp

[ https://issues.apache.org/jira/browse/SPARK-29494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-29494: Assignee: Rahul Shivu Mahadev > ArrayOutOfBoundsException when converting from string to

[jira] [Resolved] (SPARK-29494) ArrayOutOfBoundsException when converting from string to timestamp

[ https://issues.apache.org/jira/browse/SPARK-29494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-29494. -- Fix Version/s: 3.0.0 2.4.5 Resolution: Fixed Issue resolved by pull

[jira] [Updated] (SPARK-29494) ArrayOutOfBoundsException when converting from string to timestamp

[ https://issues.apache.org/jira/browse/SPARK-29494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-29494: - Priority: Minor (was: Major) > ArrayOutOfBoundsException when converting from string to

[jira] [Assigned] (SPARK-29515) MapStatuses SerDeser Benchmark

[ https://issues.apache.org/jira/browse/SPARK-29515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-29515: --- Assignee: DB Tsai > MapStatuses SerDeser Benchmark > -- > >

[jira] [Resolved] (SPARK-29515) MapStatuses SerDeser Benchmark

[ https://issues.apache.org/jira/browse/SPARK-29515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-29515. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26169

[jira] [Updated] (SPARK-29515) MapStatuses SerDeser Benchmark

[ https://issues.apache.org/jira/browse/SPARK-29515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-29515: Affects Version/s: (was: 3.0.0) 2.4.4 > MapStatuses SerDeser Benchmark >

[jira] [Created] (SPARK-29515) MapStatuses SerDeser Benchmark

DB Tsai created SPARK-29515: --- Summary: MapStatuses SerDeser Benchmark Key: SPARK-29515 URL: https://issues.apache.org/jira/browse/SPARK-29515 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-29106) Add jenkins arm test for spark

[ https://issues.apache.org/jira/browse/SPARK-29106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954912#comment-16954912 ] Shane Knapp commented on SPARK-29106: - also, i will be exploring the purchase of an ARM server for

[jira] [Commented] (SPARK-29106) Add jenkins arm test for spark

[ https://issues.apache.org/jira/browse/SPARK-29106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954904#comment-16954904 ] Shane Knapp commented on SPARK-29106: - i'm actually not going to use the script – the testing code

[jira] [Updated] (SPARK-29503) MapObjects doesn't copy Unsafe data when nested under Safe data

[ https://issues.apache.org/jira/browse/SPARK-29503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-29503: Description: In order for MapObjects to operate safely, it checks to see if the result of the

[jira] [Updated] (SPARK-29503) MapObjects doesn't copy Unsafe data when nested under Safe data

[ https://issues.apache.org/jira/browse/SPARK-29503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-29503: Description: *strong text*In order for MapObjects to operate safely, it checks to see if the

[jira] [Commented] (SPARK-29106) Add jenkins arm test for spark

[ https://issues.apache.org/jira/browse/SPARK-29106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954879#comment-16954879 ] zhao bo commented on SPARK-29106: - And if possible and you are free, could you please help us to make

[jira] [Updated] (SPARK-29076) Generalize the PVTestSuite to no longer need the minikube tag

[ https://issues.apache.org/jira/browse/SPARK-29076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29076: -- Priority: Major (was: Trivial) > Generalize the PVTestSuite to no longer need the minikube

[jira] [Commented] (SPARK-29106) Add jenkins arm test for spark

[ https://issues.apache.org/jira/browse/SPARK-29106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954842#comment-16954842 ] zhao bo commented on SPARK-29106: - Thanks @shane. Correct, the test dependency of pyspark and spark R

[jira] [Created] (SPARK-29514) String function: string_to_array

Kent Yao created SPARK-29514: Summary: String function: string_to_array Key: SPARK-29514 URL: https://issues.apache.org/jira/browse/SPARK-29514 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-29106) Add jenkins arm test for spark

[ https://issues.apache.org/jira/browse/SPARK-29106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954809#comment-16954809 ] Shane Knapp commented on SPARK-29106: - we're definitely going to have an issue w/the both R and

[jira] [Commented] (SPARK-29507) Support ALTER TABLE SET OWNER command

[ https://issues.apache.org/jira/browse/SPARK-29507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954805#comment-16954805 ] Kent Yao commented on SPARK-29507: -- I am working on it > Support ALTER TABLE SET OWNER command >

[jira] [Updated] (SPARK-27812) kubernetes client import non-daemon thread which block jvm exit.

[ https://issues.apache.org/jira/browse/SPARK-27812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27812: -- Fix Version/s: 2.4.5 > kubernetes client import non-daemon thread which block jvm exit. >

[jira] [Resolved] (SPARK-28420) Date/Time Functions: date_part for intervals

[ https://issues.apache.org/jira/browse/SPARK-28420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28420. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25981

[jira] [Assigned] (SPARK-28420) Date/Time Functions: date_part for intervals

[ https://issues.apache.org/jira/browse/SPARK-28420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-28420: --- Assignee: Maxim Gekk (was: Yuming Wang) > Date/Time Functions: date_part for intervals >

[jira] [Assigned] (SPARK-28420) Date/Time Functions: date_part for intervals

[ https://issues.apache.org/jira/browse/SPARK-28420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-28420: --- Assignee: Yuming Wang > Date/Time Functions: date_part for intervals >

[jira] [Commented] (SPARK-29106) Add jenkins arm test for spark

[ https://issues.apache.org/jira/browse/SPARK-29106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954705#comment-16954705 ] Shane Knapp commented on SPARK-29106: - re: real time logging -- yeah i noticed that. :) i'll look

[jira] [Commented] (SPARK-29481) all the commands should look up catalog/table like v2 commands

[ https://issues.apache.org/jira/browse/SPARK-29481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954688#comment-16954688 ] L. C. Hsieh commented on SPARK-29481: - Thanks for pinging me. Will take some in this weekend. > all

[jira] [Commented] (SPARK-29513) REFRESH TABLE should look up catalog/table like v2 commands

[ https://issues.apache.org/jira/browse/SPARK-29513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954675#comment-16954675 ] Terry Kim commented on SPARK-29513: --- I am working on this. > REFRESH TABLE should look up

[jira] [Created] (SPARK-29513) REFRESH TABLE should look up catalog/table like v2 commands

Terry Kim created SPARK-29513: - Summary: REFRESH TABLE should look up catalog/table like v2 commands Key: SPARK-29513 URL: https://issues.apache.org/jira/browse/SPARK-29513 Project: Spark Issue

[jira] [Created] (SPARK-29512) REPAIR TABLE should look up catalog/table like v2 commands

Terry Kim created SPARK-29512: - Summary: REPAIR TABLE should look up catalog/table like v2 commands Key: SPARK-29512 URL: https://issues.apache.org/jira/browse/SPARK-29512 Project: Spark Issue

[jira] [Commented] (SPARK-29512) REPAIR TABLE should look up catalog/table like v2 commands

[ https://issues.apache.org/jira/browse/SPARK-29512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954672#comment-16954672 ] Terry Kim commented on SPARK-29512: --- I am working on this. > REPAIR TABLE should look up

[jira] [Resolved] (SPARK-29014) DataSourceV2: Clean up current, default, and session catalog uses

[ https://issues.apache.org/jira/browse/SPARK-29014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29014. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26120

[jira] [Assigned] (SPARK-29014) DataSourceV2: Clean up current, default, and session catalog uses

[ https://issues.apache.org/jira/browse/SPARK-29014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29014: --- Assignee: Terry Kim > DataSourceV2: Clean up current, default, and session catalog uses >

[jira] [Created] (SPARK-29511) DataSourceV2: Support CREATE NAMESPACE

Terry Kim created SPARK-29511: - Summary: DataSourceV2: Support CREATE NAMESPACE Key: SPARK-29511 URL: https://issues.apache.org/jira/browse/SPARK-29511 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-29481) all the commands should look up catalog/table like v2 commands

[ https://issues.apache.org/jira/browse/SPARK-29481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954661#comment-16954661 ] Wenchen Fan commented on SPARK-29481: - I really appreciate it if more people can help on this

[jira] [Updated] (SPARK-29510) JobGroup ID is not set for the job submitted from Spark-SQL and Spark -Shell

[ https://issues.apache.org/jira/browse/SPARK-29510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ABHISHEK KUMAR GUPTA updated SPARK-29510: - Attachment: JobGroup2.png JobGroup3.png

[jira] [Commented] (SPARK-29510) JobGroup ID is not set for the job submitted from Spark-SQL and Spark -Shell

[ https://issues.apache.org/jira/browse/SPARK-29510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954645#comment-16954645 ] Ankit Raj Boudh commented on SPARK-29510: - I will start working in this issue > JobGroup ID is

[jira] [Created] (SPARK-29510) JobGroup ID is not set for the job submitted from Spark-SQL and Spark -Shell

ABHISHEK KUMAR GUPTA created SPARK-29510: Summary: JobGroup ID is not set for the job submitted from Spark-SQL and Spark -Shell Key: SPARK-29510 URL: https://issues.apache.org/jira/browse/SPARK-29510

[jira] [Created] (SPARK-29509) Deduplicate code blocks in Kafka data source

Jungtaek Lim created SPARK-29509: Summary: Deduplicate code blocks in Kafka data source Key: SPARK-29509 URL: https://issues.apache.org/jira/browse/SPARK-29509 Project: Spark Issue Type:

[jira] [Commented] (SPARK-29508) Implicitly cast strings in datetime arithmetic operations

[ https://issues.apache.org/jira/browse/SPARK-29508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954579#comment-16954579 ] Maxim Gekk commented on SPARK-29508: I am working on it > Implicitly cast strings in datetime

[jira] [Resolved] (SPARK-29482) ANALYZE TABLE should look up catalog/table like v2 commands

[ https://issues.apache.org/jira/browse/SPARK-29482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-29482. Resolution: Fixed The issue is resolved in https://github.com/apache/spark/pull/26129 >

[jira] [Resolved] (SPARK-29478) Improve tooltip information for AggregatedLogs Tab

[ https://issues.apache.org/jira/browse/SPARK-29478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ABHISHEK KUMAR GUPTA resolved SPARK-29478. -- Resolution: Invalid It is not supported by Open Source community > Improve

[jira] [Created] (SPARK-29508) Implicitly cast strings in datetime arithmetic operations

Maxim Gekk created SPARK-29508: -- Summary: Implicitly cast strings in datetime arithmetic operations Key: SPARK-29508 URL: https://issues.apache.org/jira/browse/SPARK-29508 Project: Spark Issue

[jira] [Assigned] (SPARK-29489) ml.evaluation support log-loss

[ https://issues.apache.org/jira/browse/SPARK-29489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng reassigned SPARK-29489: Assignee: zhengruifeng > ml.evaluation support log-loss > --

[jira] [Resolved] (SPARK-29489) ml.evaluation support log-loss

[ https://issues.apache.org/jira/browse/SPARK-29489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-29489. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26135

[jira] [Commented] (SPARK-29106) Add jenkins arm test for spark

[ https://issues.apache.org/jira/browse/SPARK-29106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954447#comment-16954447 ] zhao bo commented on SPARK-29106: - The reason to introducing the shell script is that I found if we call

[jira] [Commented] (SPARK-29106) Add jenkins arm test for spark

[ https://issues.apache.org/jira/browse/SPARK-29106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954445#comment-16954445 ] zhao bo commented on SPARK-29106: - Hi,  I create a pretty simple shell script in 

[jira] [Commented] (SPARK-29262) DataFrameWriter insertIntoPartition function

[ https://issues.apache.org/jira/browse/SPARK-29262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954424#comment-16954424 ] feiwang commented on SPARK-29262: - I'll try to implement it. > DataFrameWriter insertIntoPartition

[jira] [Resolved] (SPARK-28120) RocksDB state storage

[ https://issues.apache.org/jira/browse/SPARK-28120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Agrawal resolved SPARK-28120. Resolution: Later The implementation will be submitted to https://spark-packages.org. >

[jira] [Commented] (SPARK-28022) k8s pod affinity to achieve cloud native friendly autoscaling

[ https://issues.apache.org/jira/browse/SPARK-28022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954398#comment-16954398 ] Jiaxin Shan commented on SPARK-28022: - I don't quite understand the use case here. Sound like you

[jira] [Resolved] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29295. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25979

[jira] [Assigned] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29295: --- Assignee: L. C. Hsieh > Duplicate result when dropping partition of an external table and

[jira] [Created] (SPARK-29507) Support ALTER TABLE SET OWNER command

Kent Yao created SPARK-29507: Summary: Support ALTER TABLE SET OWNER command Key: SPARK-29507 URL: https://issues.apache.org/jira/browse/SPARK-29507 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-29405) Alter table / Insert statements should not change a table's ownership

[ https://issues.apache.org/jira/browse/SPARK-29405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29405: --- Assignee: Kent Yao > Alter table / Insert statements should not change a table's ownership

[jira] [Resolved] (SPARK-29405) Alter table / Insert statements should not change a table's ownership

[ https://issues.apache.org/jira/browse/SPARK-29405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29405. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26068

[jira] [Assigned] (SPARK-29444) Add configuration to support JacksonGenrator to keep fields with null values

[ https://issues.apache.org/jira/browse/SPARK-29444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29444: --- Assignee: Jackey Lee > Add configuration to support JacksonGenrator to keep fields with

[jira] [Resolved] (SPARK-29444) Add configuration to support JacksonGenrator to keep fields with null values

[ https://issues.apache.org/jira/browse/SPARK-29444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29444. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26098

[jira] [Resolved] (SPARK-29092) EXPLAIN FORMATTED does not work well with DPP

[ https://issues.apache.org/jira/browse/SPARK-29092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29092. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26042

[jira] [Assigned] (SPARK-29092) EXPLAIN FORMATTED does not work well with DPP

[ https://issues.apache.org/jira/browse/SPARK-29092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29092: --- Assignee: Dilip Biswal > EXPLAIN FORMATTED does not work well with DPP >

[jira] [Commented] (SPARK-29505) desc extended is case sensitive

[ https://issues.apache.org/jira/browse/SPARK-29505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954330#comment-16954330 ] Shivu Sondur commented on SPARK-29505: -- I am checking this issue > desc extended is case

[jira] [Created] (SPARK-29506) Use dynamicPartitionOverwrite in FileCommitProtocol when insert into hive table

L. C. Hsieh created SPARK-29506: --- Summary: Use dynamicPartitionOverwrite in FileCommitProtocol when insert into hive table Key: SPARK-29506 URL: https://issues.apache.org/jira/browse/SPARK-29506

[jira] [Created] (SPARK-29505) desc extended is case sensitive

ABHISHEK KUMAR GUPTA created SPARK-29505: Summary: desc extended is case sensitive Key: SPARK-29505 URL: https://issues.apache.org/jira/browse/SPARK-29505 Project: Spark Issue

[jira] [Commented] (SPARK-29106) Add jenkins arm test for spark

[ https://issues.apache.org/jira/browse/SPARK-29106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954320#comment-16954320 ] huangtianhua commented on SPARK-29106: -- [~shaneknapp], the build