[jira] [Updated] (SPARK-48791) Perf regression due to accumulator registration overhead using CopyOnWriteArrayList

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48791: --- Labels: pull-request-available (was: ) > Perf regression due to accumulator registration ov

[jira] [Created] (SPARK-48791) Perf regression due to accumulator registration overhead using CopyOnWriteArrayList

2024-07-02 Thread wuyi (Jira)
wuyi created SPARK-48791: Summary: Perf regression due to accumulator registration overhead using CopyOnWriteArrayList Key: SPARK-48791 URL: https://issues.apache.org/jira/browse/SPARK-48791 Project: Spark

[jira] [Updated] (SPARK-48790) Use checkDatasetUnorderly in DeprecatedDatasetAggregatorSuite

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48790: --- Labels: pull-request-available (was: ) > Use checkDatasetUnorderly in DeprecatedDatasetAggr

[jira] [Created] (SPARK-48790) Use checkDatasetUnorderly in DeprecatedDatasetAggregatorSuite

2024-07-02 Thread Rui Wang (Jira)
Rui Wang created SPARK-48790: Summary: Use checkDatasetUnorderly in DeprecatedDatasetAggregatorSuite Key: SPARK-48790 URL: https://issues.apache.org/jira/browse/SPARK-48790 Project: Spark Issue

[jira] [Resolved] (SPARK-48760) Introduce ALTER TABLE ... CLUSTER BY to change clustering columns

2024-07-02 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48760. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47156 [https://gith

[jira] [Assigned] (SPARK-48774) Use SparkSession in SQLImplicits

2024-07-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48774: Assignee: Hyukjin Kwon > Use SparkSession in SQLImplicits > -

[jira] [Resolved] (SPARK-48774) Use SparkSession in SQLImplicits

2024-07-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48774. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47173 [https://gi

[jira] [Updated] (SPARK-48774) Use SparkSession in SQLImplicits

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48774: --- Labels: pull-request-available (was: ) > Use SparkSession in SQLImplicits > ---

[jira] [Updated] (SPARK-48760) Introduce ALTER TABLE ... CLUSTER BY to change clustering columns

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48760: --- Labels: pull-request-available (was: ) > Introduce ALTER TABLE ... CLUSTER BY to change clu

[jira] [Updated] (SPARK-42051) Codegen Support for HiveGenericUDF

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-42051: --- Labels: pull-request-available (was: ) > Codegen Support for HiveGenericUDF > -

[jira] [Updated] (SPARK-48628) Add task peak on/off heap execution memory metrics

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48628: --- Labels: pull-request-available (was: ) > Add task peak on/off heap execution memory metrics

[jira] [Resolved] (SPARK-48785) Add a simple data source example in the user guide

2024-07-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48785. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47187 [https://gi

[jira] [Assigned] (SPARK-48785) Add a simple data source example in the user guide

2024-07-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48785: Assignee: Allison Wang > Add a simple data source example in the user guide > ---

[jira] [Assigned] (SPARK-48714) Implement df.mergeInto in PySpark

2024-07-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48714: Assignee: Pengfei Xu > Implement df.mergeInto in PySpark > --

[jira] [Updated] (SPARK-48714) Implement df.mergeInto in PySpark

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48714: --- Labels: pull-request-available (was: ) > Implement df.mergeInto in PySpark > --

[jira] [Resolved] (SPARK-48714) Implement df.mergeInto in PySpark

2024-07-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48714. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47086 [https://gi

[jira] [Updated] (SPARK-48777) Migrate Spark Connect Go to Bazel

2024-07-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-48777: - Fix Version/s: (was: 4.0.0) > Migrate Spark Connect Go to Bazel > --

[jira] [Resolved] (SPARK-48777) Migrate Spark Connect Go to Bazel

2024-07-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48777. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 23 [https://githu

[jira] [Updated] (SPARK-48710) Incompatibilities with NumPy 2.0

2024-07-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-48710: - Fix Version/s: 4.0.0 > Incompatibilities with NumPy 2.0 > > >

[jira] [Assigned] (SPARK-48710) Incompatibilities with NumPy 2.0

2024-07-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48710: Assignee: Patrick Marx > Incompatibilities with NumPy 2.0 > -

[jira] [Resolved] (SPARK-48710) Incompatibilities with NumPy 2.0

2024-07-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48710. -- Fix Version/s: 3.4.4 3.5.2 Resolution: Fixed Issue resolved by pull

[jira] [Updated] (SPARK-48628) Add task peak on/off heap execution memory metrics

2024-07-02 Thread Ziqi Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ziqi Liu updated SPARK-48628: - Description: Currently there is no task on/off heap execution memory metrics. There is a [peakExecution

[jira] [Created] (SPARK-48788) Accumulate task-level execution memory in stage and display in Spark UI

2024-07-02 Thread Ziqi Liu (Jira)
Ziqi Liu created SPARK-48788: Summary: Accumulate task-level execution memory in stage and display in Spark UI Key: SPARK-48788 URL: https://issues.apache.org/jira/browse/SPARK-48788 Project: Spark

[jira] [Updated] (SPARK-48787) Upgrade Kafka to 3.7.1

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48787: --- Labels: pull-request-available (was: ) > Upgrade Kafka to 3.7.1 > -- >

[jira] [Updated] (SPARK-48720) Clarify the command ALTER TABLE ... UNSET TBLPROPERTIES ... diff between v1 and v2 in doc

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48720: --- Labels: pull-request-available (was: ) > Clarify the command ALTER TABLE ... UNSET TBLPROPE

[jira] [Updated] (SPARK-48786) Define Conf Properties for Spark Operator Controller

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48786: --- Labels: pull-request-available (was: ) > Define Conf Properties for Spark Operator Controll

[jira] [Created] (SPARK-48786) Define Conf Properties for Spark Operator Controller

2024-07-02 Thread Zhou JIANG (Jira)
Zhou JIANG created SPARK-48786: -- Summary: Define Conf Properties for Spark Operator Controller Key: SPARK-48786 URL: https://issues.apache.org/jira/browse/SPARK-48786 Project: Spark Issue Type:

[jira] [Updated] (SPARK-48710) Incompatibilities with NumPy 2.0

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48710: --- Labels: pull-request-available (was: ) > Incompatibilities with NumPy 2.0 > ---

[jira] [Updated] (SPARK-48780) Make errors in NamedParametersSupport generic to handle functions and procedures

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48780: --- Labels: pull-request-available (was: ) > Make errors in NamedParametersSupport generic to h

[jira] [Updated] (SPARK-48780) Make errors in NamedParametersSupport generic to handle functions and procedures

2024-07-02 Thread Anton Okolnychyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Okolnychyi updated SPARK-48780: - Summary: Make errors in NamedParametersSupport generic to handle functions and procedure

[jira] [Updated] (SPARK-48772) State Data Source Read Change Feed

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48772: --- Labels: pull-request-available (was: ) > State Data Source Read Change Feed > -

[jira] [Updated] (SPARK-48772) State Data Source Read Change Feed

2024-07-02 Thread Yuchen Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuchen Liu updated SPARK-48772: --- Description: The current state reader can only return the entire state at a specific version. If an

[jira] [Updated] (SPARK-48785) Add a simple data source example in the user guide

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48785: --- Labels: pull-request-available (was: ) > Add a simple data source example in the user guide

[jira] [Created] (SPARK-48785) Add a simple data source example in the user guide

2024-07-02 Thread Allison Wang (Jira)
Allison Wang created SPARK-48785: Summary: Add a simple data source example in the user guide Key: SPARK-48785 URL: https://issues.apache.org/jira/browse/SPARK-48785 Project: Spark Issue Type

[jira] [Resolved] (SPARK-48770) Read operator metadata only once on driver for store read

2024-07-02 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-48770. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47167 [https://gi

[jira] [Assigned] (SPARK-48770) Read operator metadata only once on driver for store read

2024-07-02 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-48770: Assignee: Anish Shrigondekar > Read operator metadata only once on driver for store read

[jira] [Updated] (SPARK-48770) Read operator metadata only once on driver for store read

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48770: --- Labels: pull-request-available (was: ) > Read operator metadata only once on driver for sto

[jira] [Assigned] (SPARK-48589) Add option snapshotStartBatchId and snapshotPartitionId to state data source

2024-07-02 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-48589: Assignee: Yuchen Liu > Add option snapshotStartBatchId and snapshotPartitionId to state d

[jira] [Resolved] (SPARK-48589) Add option snapshotStartBatchId and snapshotPartitionId to state data source

2024-07-02 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-48589. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46944 [https://gi

[jira] [Updated] (SPARK-48784) Add ::: syntax as a shorthand for try_cast

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48784: --- Labels: pull-request-available (was: ) > Add ::: syntax as a shorthand for try_cast > -

[jira] [Created] (SPARK-48784) Add ::: syntax as a shorthand for try_cast

2024-07-02 Thread Gene Pang (Jira)
Gene Pang created SPARK-48784: - Summary: Add ::: syntax as a shorthand for try_cast Key: SPARK-48784 URL: https://issues.apache.org/jira/browse/SPARK-48784 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-48783) Update the table-valued function documentation

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48783: --- Labels: pull-request-available (was: ) > Update the table-valued function documentation > -

[jira] [Created] (SPARK-48783) Update the table-valued function documentation

2024-07-02 Thread Allison Wang (Jira)
Allison Wang created SPARK-48783: Summary: Update the table-valued function documentation Key: SPARK-48783 URL: https://issues.apache.org/jira/browse/SPARK-48783 Project: Spark Issue Type: Su

[jira] [Updated] (SPARK-48755) [Python State V2] Base implementation and ValueState support

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48755: --- Labels: pull-request-available (was: ) > [Python State V2] Base implementation and ValueSta

[jira] [Updated] (SPARK-48773) Document config "spark.default.parallelism"

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48773: --- Labels: pull-request-available (was: ) > Document config "spark.default.parallelism" >

[jira] [Updated] (SPARK-48776) XML, JSON and CSV default timestamp formatting fails for seconds offset

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48776: --- Labels: pull-request-available (was: ) > XML, JSON and CSV default timestamp formatting fai

[jira] [Resolved] (SPARK-48554) Use R 4.4.0 in `windows` R GitHub Action Windows job

2024-07-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48554. --- Fix Version/s: 4.0.0 Assignee: BingKun Pan Resolution: Fixed > Use R 4.4.0 i

[jira] [Resolved] (SPARK-48177) Upgrade `Parquet` to 1.14.1

2024-07-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48177. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46447 [https://

[jira] [Created] (SPARK-48782) Add support for loading stored procedures in catalogs

2024-07-02 Thread Anton Okolnychyi (Jira)
Anton Okolnychyi created SPARK-48782: Summary: Add support for loading stored procedures in catalogs Key: SPARK-48782 URL: https://issues.apache.org/jira/browse/SPARK-48782 Project: Spark

[jira] [Created] (SPARK-48781) Add Catalog APIs for loading stored procedures

2024-07-02 Thread Anton Okolnychyi (Jira)
Anton Okolnychyi created SPARK-48781: Summary: Add Catalog APIs for loading stored procedures Key: SPARK-48781 URL: https://issues.apache.org/jira/browse/SPARK-48781 Project: Spark Issue

[jira] [Created] (SPARK-48780) Make NamedParametersSupport generic to handle functions and procedures

2024-07-02 Thread Anton Okolnychyi (Jira)
Anton Okolnychyi created SPARK-48780: Summary: Make NamedParametersSupport generic to handle functions and procedures Key: SPARK-48780 URL: https://issues.apache.org/jira/browse/SPARK-48780 Projec

[jira] [Updated] (SPARK-48780) Make NamedParametersSupport generic to handle functions and procedures

2024-07-02 Thread Anton Okolnychyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Okolnychyi updated SPARK-48780: - Description: Functions and procedures are routines that support position and named-based

[jira] [Updated] (SPARK-48529) [M0] Support for labels

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48529: --- Labels: pull-request-available (was: ) > [M0] Support for labels > ---

[jira] [Updated] (SPARK-46625) IDENTIFIER clause does not work with CTE reference

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46625: --- Labels: pull-request-available (was: ) > IDENTIFIER clause does not work with CTE reference

[jira] [Updated] (SPARK-48742) Virtual Column Family for RocksDB

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48742: --- Labels: pull-request-available (was: ) > Virtual Column Family for RocksDB > --

[jira] [Updated] (SPARK-44718) High On-heap memory usage is detected while doing parquet-file reading with Off-Heap memory mode enabled on spark

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44718: --- Labels: pull-request-available (was: ) > High On-heap memory usage is detected while doing

[jira] [Updated] (SPARK-48769) ScalaUDF should support constant folding

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48769: --- Labels: pull-request-available (was: ) > ScalaUDF should support constant folding > ---

[jira] [Updated] (SPARK-48726) Create the StateSchemaV3 file format, and write this out for the TransformWithStateExec operator

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48726: --- Labels: pull-request-available (was: ) > Create the StateSchemaV3 file format, and write

[jira] [Updated] (SPARK-48735) Performance Improvement for BIN function

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48735: --- Labels: pull-request-available (was: ) > Performance Improvement for BIN function > ---

[jira] [Updated] (SPARK-48738) Correct since version for built-in func alias `random`, `position`, `mod`, `cardinality`, `current_schema`, `user`, `session_user`, `char_length`, `character_length`

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48738: --- Labels: pull-request-available (was: ) > Correct since version for built-in func alias `ran

[jira] [Updated] (SPARK-39901) Reconsider design of ignoreCorruptFiles feature

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-39901: --- Labels: pull-request-available (was: ) > Reconsider design of ignoreCorruptFiles feature >

[jira] [Updated] (SPARK-48699) Refine collation API

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48699: --- Labels: pull-request-available (was: ) > Refine collation API > > >

[jira] [Created] (SPARK-48779) Improve collation support testing - add golden files

2024-07-02 Thread Jira
Uroš Bojanić created SPARK-48779: Summary: Improve collation support testing - add golden files Key: SPARK-48779 URL: https://issues.apache.org/jira/browse/SPARK-48779 Project: Spark Issue Ty

[jira] [Created] (SPARK-48778) Improve collation support testing - add unit tests

2024-07-02 Thread Jira
Uroš Bojanić created SPARK-48778: Summary: Improve collation support testing - add unit tests Key: SPARK-48778 URL: https://issues.apache.org/jira/browse/SPARK-48778 Project: Spark Issue Type

[jira] [Updated] (SPARK-48715) UTF8String to String conversions should use Unicode replacement logic

2024-07-02 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-48715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-48715: - Summary: UTF8String to String conversions should use Unicode replacement logic (was: UTF8String

[jira] [Commented] (SPARK-46625) IDENTIFIER clause does not work with CTE reference

2024-07-02 Thread Nebojsa Savic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17862449#comment-17862449 ] Nebojsa Savic commented on SPARK-46625: --- Working on it. > IDENTIFIER clause does

[jira] [Commented] (SPARK-48776) XML, JSON and CSV default timestamp formatting fails for seconds offset

2024-07-02 Thread Nikita Awasthi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17862414#comment-17862414 ] Nikita Awasthi commented on SPARK-48776: User 'milastdbx' has created a pull req

[jira] [Created] (SPARK-48776) XML, JSON and CSV default timestamp formatting fails for seconds offset

2024-07-02 Thread Milan Stefanovic (Jira)
Milan Stefanovic created SPARK-48776: Summary: XML, JSON and CSV default timestamp formatting fails for seconds offset Key: SPARK-48776 URL: https://issues.apache.org/jira/browse/SPARK-48776 Proje

[jira] [Resolved] (SPARK-48764) Filtering out IPython frame from user stack

2024-07-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48764. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47159 [https://gi

[jira] [Assigned] (SPARK-48764) Filtering out IPython frame from user stack

2024-07-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48764: Assignee: Haejoon Lee > Filtering out IPython frame from user stack > ---

[jira] [Updated] (SPARK-48775) Replace SQLContext with SparkSession in STS

2024-07-02 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-48775: -- Summary: Replace SQLContext with SparkSession in STS (was: Replace SQLContext with SparkSession in ne

[jira] [Assigned] (SPARK-48774) Use SparkSession in SQLImplicits

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48774: -- Assignee: Apache Spark > Use SparkSession in SQLImplicits > -

[jira] [Assigned] (SPARK-48774) Use SparkSession in SQLImplicits

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48774: -- Assignee: Apache Spark > Use SparkSession in SQLImplicits > -

[jira] [Commented] (SPARK-48774) Use SparkSession in SQLImplicits

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17861364#comment-17861364 ] ASF GitHub Bot commented on SPARK-48774: User 'HyukjinKwon' has created a pull r

[jira] [Assigned] (SPARK-48774) Use SparkSession in SQLImplicits

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48774: -- Assignee: (was: Apache Spark) > Use SparkSession in SQLImplicits > --

[jira] [Updated] (SPARK-48710) Incompatibilities with NumPy 2.0

2024-07-02 Thread Patrick Marx (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Marx updated SPARK-48710: - Affects Version/s: 3.5.1 > Incompatibilities with NumPy 2.0 > >

[jira] [Assigned] (SPARK-48710) Incompatibilities with NumPy 2.0

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48710: -- Assignee: (was: Apache Spark) > Incompatibilities with NumPy 2.0 > --

[jira] [Assigned] (SPARK-48710) Incompatibilities with NumPy 2.0

2024-07-02 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48710: -- Assignee: Apache Spark > Incompatibilities with NumPy 2.0 > -

[jira] [Updated] (SPARK-48754) Improve code towards productionizing the go library

2024-07-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-48754: - Fix Version/s: (was: 4.0.0) > Improve code towards productionizing the go library >

[jira] [Assigned] (SPARK-48754) Improve code towards productionizing the go library

2024-07-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48754: Assignee: Peter Kristensen > Improve code towards productionizing the go library > --

[jira] [Resolved] (SPARK-48754) Improve code towards productionizing the go library

2024-07-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48754. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 20 [https://githu

[jira] [Resolved] (SPARK-48759) Add migration doc for CREATE TABLE AS SELECT behavior change behavior change since Spark 3.4

2024-07-02 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48759. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47152 [https://gith

[jira] [Updated] (SPARK-48720) Clarify the command ALTER TABLE ... UNSET TBLPROPERTIES ... diff between v1 and v2 in doc

2024-07-02 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-48720: Issue Type: Improvement (was: Bug) > Clarify the command ALTER TABLE ... UNSET TBLPROPERTIES ...

[jira] [Updated] (SPARK-48720) Clarify the command ALTER TABLE ... UNSET TBLPROPERTIES ... diff between v1 and v2 in doc

2024-07-02 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-48720: Summary: Clarify the command ALTER TABLE ... UNSET TBLPROPERTIES ... diff between v1 and v2 in doc

[jira] [Updated] (SPARK-48720) Clarify the command ALTER TABLE ... UNSET TBLPROPERTIES ... diff between v1 and v2 in doc

2024-07-02 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-48720: Priority: Minor (was: Major) > Clarify the command ALTER TABLE ... UNSET TBLPROPERTIES ... diff b