[jira] [Updated] (SPARK-42702) Support parameterized CTE

[ https://issues.apache.org/jira/browse/SPARK-42702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-42702: Fix Version/s: 3.4.0 (was: 3.4.1) > Support parameterized CTE > ---

[jira] [Commented] (SPARK-42809) Upgrade scala-maven-plugin from 4.8.0 to 4.8.1

[ https://issues.apache.org/jira/browse/SPARK-42809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702453#comment-17702453 ] Apache Spark commented on SPARK-42809: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-42787) in spark-py docker images, arrowkeys do not work in (scala) spark-shell

[ https://issues.apache.org/jira/browse/SPARK-42787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702434#comment-17702434 ] Yikun Jiang commented on SPARK-42787: - > while it works for pyspark, it does not for

[jira] [Commented] (SPARK-42788) spark binaries are not added to $PATH in public docker hub spark-py

[ https://issues.apache.org/jira/browse/SPARK-42788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702433#comment-17702433 ] Yikun Jiang commented on SPARK-42788: - Yes, I think we can add a line like: {code:j

[jira] [Comment Edited] (SPARK-42788) spark binaries are not added to $PATH in public docker hub spark-py

[ https://issues.apache.org/jira/browse/SPARK-42788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702433#comment-17702433 ] Yikun Jiang edited comment on SPARK-42788 at 3/20/23 4:04 AM:

[jira] [Created] (SPARK-42860) Add analysed logical mode in org.apache.spark.sql.execution.ExplainMode

arindam patra created SPARK-42860: - Summary: Add analysed logical mode in org.apache.spark.sql.execution.ExplainMode Key: SPARK-42860 URL: https://issues.apache.org/jira/browse/SPARK-42860 Project: Sp

[jira] [Created] (SPARK-42859) Basic support for pandas API on Spark

Haejoon Lee created SPARK-42859: --- Summary: Basic support for pandas API on Spark Key: SPARK-42859 URL: https://issues.apache.org/jira/browse/SPARK-42859 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-42859) Basic support for pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-42859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702423#comment-17702423 ] Haejoon Lee commented on SPARK-42859: - I'm working on this > Basic support for pand

[jira] [Created] (SPARK-42858) Refactor HiveGenericUDTF

BingKun Pan created SPARK-42858: --- Summary: Refactor HiveGenericUDTF Key: SPARK-42858 URL: https://issues.apache.org/jira/browse/SPARK-42858 Project: Spark Issue Type: Sub-task Compone

[jira] [Commented] (SPARK-42595) Support query inserted partitions after insert data into table when hive.exec.dynamic.partition=true

[ https://issues.apache.org/jira/browse/SPARK-42595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702418#comment-17702418 ] zhang haoyan commented on SPARK-42595: -- Gently ping [~maxgekk] for suggestions > S

[jira] [Updated] (SPARK-42857) Unable to CreateDataFrame from Decimal128

[ https://issues.apache.org/jira/browse/SPARK-42857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42857: - Description: To reproduce: {code:sh} >>> from decimal import Decimal >>> spark.createDataFrame

[jira] [Created] (SPARK-42857) Unable to CreateDataFrame from Decimal128

Xinrong Meng created SPARK-42857: Summary: Unable to CreateDataFrame from Decimal128 Key: SPARK-42857 URL: https://issues.apache.org/jira/browse/SPARK-42857 Project: Spark Issue Type: Story

[jira] [Assigned] (SPARK-42827) Support `functions#array_prepend`

[ https://issues.apache.org/jira/browse/SPARK-42827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42827: Assignee: (was: Apache Spark) > Support `functions#array_prepend` > -

[jira] [Assigned] (SPARK-42827) Support `functions#array_prepend`

[ https://issues.apache.org/jira/browse/SPARK-42827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42827: Assignee: Apache Spark > Support `functions#array_prepend` >

[jira] [Commented] (SPARK-42827) Support `functions#array_prepend`

[ https://issues.apache.org/jira/browse/SPARK-42827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702398#comment-17702398 ] Apache Spark commented on SPARK-42827: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-42508) Extract the common .ml classes to `mllib-common`

[ https://issues.apache.org/jira/browse/SPARK-42508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702396#comment-17702396 ] Apache Spark commented on SPARK-42508: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-42751) Pyspark.pandas.series.str.findall can't handle tuples that are returned by regex

[ https://issues.apache.org/jira/browse/SPARK-42751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702394#comment-17702394 ] Hyukjin Kwon commented on SPARK-42751: -- cc [~itholic] FYI > Pyspark.pandas.series.

[jira] [Commented] (SPARK-42787) in spark-py docker images, arrowkeys do not work in (scala) spark-shell

[ https://issues.apache.org/jira/browse/SPARK-42787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702393#comment-17702393 ] Hyukjin Kwon commented on SPARK-42787: -- cc [~yikun] FYI > in spark-py docker image

[jira] [Commented] (SPARK-42788) spark binaries are not added to $PATH in public docker hub spark-py

[ https://issues.apache.org/jira/browse/SPARK-42788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702392#comment-17702392 ] Hyukjin Kwon commented on SPARK-42788: -- cc [~yikun] FYI > spark binaries are not a

[jira] [Updated] (SPARK-42822) Ambiguous reference for case-insensitive grouping column in groupby. applyInPandas

[ https://issues.apache.org/jira/browse/SPARK-42822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42822: - Summary: Ambiguous reference for case-insensitive grouping column in groupby. applyInPandas (wa

[jira] [Commented] (SPARK-42825) setParams() only sets explicitly named params. Is this intentional or a bug?

[ https://issues.apache.org/jira/browse/SPARK-42825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702391#comment-17702391 ] Hyukjin Kwon commented on SPARK-42825: -- Should probably fix the docs? > setParams(

[jira] [Commented] (SPARK-42829) Added Identifier to the cached RDD operator on the Stages page

[ https://issues.apache.org/jira/browse/SPARK-42829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702390#comment-17702390 ] Hyukjin Kwon commented on SPARK-42829: -- Would be great to follow if there are some

[jira] [Comment Edited] (SPARK-42829) Added Identifier to the cached RDD operator on the Stages page

[ https://issues.apache.org/jira/browse/SPARK-42829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702390#comment-17702390 ] Hyukjin Kwon edited comment on SPARK-42829 at 3/20/23 1:15 AM: ---

[jira] [Updated] (SPARK-42837) spark-submit - issue when resolving dependencies hosted on a private repository in kubernetes cluster mode

[ https://issues.apache.org/jira/browse/SPARK-42837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42837: - Component/s: Kubernetes > spark-submit - issue when resolving dependencies hosted on a private

[jira] [Resolved] (SPARK-42852) Revert NamedLambdaVariable related changes from EquivalentExpressions

[ https://issues.apache.org/jira/browse/SPARK-42852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42852. -- Fix Version/s: 3.4.0 Assignee: Peter Toth Resolution: Fixed Fixed in https://g

[jira] [Resolved] (SPARK-42850) Remove duplicated rule CombineFilters in Optimizer

[ https://issues.apache.org/jira/browse/SPARK-42850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42850. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40471 [https://gi

[jira] [Resolved] (SPARK-41233) High-order function: array_prepend

[ https://issues.apache.org/jira/browse/SPARK-41233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41233. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 38947 [https://

[jira] [Assigned] (SPARK-42247) Standardize `returnType` property of UserDefinedFunction

[ https://issues.apache.org/jira/browse/SPARK-42247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42247: Assignee: Takuya Ueshin (was: Xinrong Meng) > Standardize `returnType` property of UserD

[jira] [Resolved] (SPARK-42247) Standardize `returnType` property of UserDefinedFunction

[ https://issues.apache.org/jira/browse/SPARK-42247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42247. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40472 [https://gi

[jira] [Assigned] (SPARK-42247) Standardize `returnType` property of UserDefinedFunction

[ https://issues.apache.org/jira/browse/SPARK-42247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42247: Assignee: Xinrong Meng > Standardize `returnType` property of UserDefinedFunction > -

[jira] [Resolved] (SPARK-42848) Implement DataFrame.registerTempTable

[ https://issues.apache.org/jira/browse/SPARK-42848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42848. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40469 [https://gi

[jira] [Assigned] (SPARK-42848) Implement DataFrame.registerTempTable

[ https://issues.apache.org/jira/browse/SPARK-42848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42848: Assignee: Takuya Ueshin > Implement DataFrame.registerTempTable > ---

[jira] [Resolved] (SPARK-42020) createDataFrame with UDT

[ https://issues.apache.org/jira/browse/SPARK-42020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42020. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40402 [https://gi

[jira] [Assigned] (SPARK-42020) createDataFrame with UDT

[ https://issues.apache.org/jira/browse/SPARK-42020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42020: Assignee: Takuya Ueshin > createDataFrame with UDT > > >

[jira] [Updated] (SPARK-42856) Inconsistent output from label propagation algorithm in Graph X due to tie-breaking logic in vertexProgram

[ https://issues.apache.org/jira/browse/SPARK-42856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammadreza Haghpanah updated SPARK-42856: --- Description: We are experiencing inconsistent output from the label propagat

[jira] [Updated] (SPARK-42856) Inconsistent output from label propagation algorithm in Graph X due to tie-breaking logic in vertexProgram

[ https://issues.apache.org/jira/browse/SPARK-42856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammadreza Haghpanah updated SPARK-42856: --- Description: We are experiencing inconsistent output from the label propagat

[jira] [Created] (SPARK-42856) Inconsistent output from label propagation algorithm in Graph X due to tie-breaking logic in vertexProgram

Mohammadreza Haghpanah created SPARK-42856: -- Summary: Inconsistent output from label propagation algorithm in Graph X due to tie-breaking logic in vertexProgram Key: SPARK-42856 URL: https://issues.apache

[jira] [Created] (SPARK-42855) Migrate nullability write checks to runtime for V2 tables

Anton Okolnychyi created SPARK-42855: Summary: Migrate nullability write checks to runtime for V2 tables Key: SPARK-42855 URL: https://issues.apache.org/jira/browse/SPARK-42855 Project: Spark

[jira] [Commented] (SPARK-42779) Allow V2 writes to indicate advisory partition size

[ https://issues.apache.org/jira/browse/SPARK-42779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702351#comment-17702351 ] Apache Spark commented on SPARK-42779: -- User 'aokolnychyi' has created a pull reque

[jira] [Resolved] (SPARK-42853) Update the Spark Doc to match the new website style

[ https://issues.apache.org/jira/browse/SPARK-42853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-42853. Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40269 [https:

[jira] [Assigned] (SPARK-42853) Update the Spark Doc to match the new website style

[ https://issues.apache.org/jira/browse/SPARK-42853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-42853: -- Assignee: Martin Grund > Update the Spark Doc to match the new website style > --

[jira] [Commented] (SPARK-42546) SPARK-42045 is incomplete in supporting ANSI_MODE fro round() and bround()

[ https://issues.apache.org/jira/browse/SPARK-42546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702337#comment-17702337 ] Serge Rielau commented on SPARK-42546: -- Can't speak for Wenchen, but +1 [~ddavies1]

[jira] [Comment Edited] (SPARK-42546) SPARK-42045 is incomplete in supporting ANSI_MODE fro round() and bround()

[ https://issues.apache.org/jira/browse/SPARK-42546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702337#comment-17702337 ] Serge Rielau edited comment on SPARK-42546 at 3/19/23 6:30 PM: ---

[jira] [Commented] (SPARK-42847) Assign a name to the error class _LEGACY_ERROR_TEMP_2013

[ https://issues.apache.org/jira/browse/SPARK-42847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702310#comment-17702310 ] Max Gekk commented on SPARK-42847: -- [~tyrantlucifer] Please, go ahead. The ticket will

[jira] [Commented] (SPARK-42838) Assign a name to the error class _LEGACY_ERROR_TEMP_2000

[ https://issues.apache.org/jira/browse/SPARK-42838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702309#comment-17702309 ] Max Gekk commented on SPARK-42838: -- [~breakpointsca] The ticket will be assigned to you

[jira] [Assigned] (SPARK-42838) Assign a name to the error class _LEGACY_ERROR_TEMP_2000

[ https://issues.apache.org/jira/browse/SPARK-42838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-42838: Assignee: (was: Max Gekk) > Assign a name to the error class _LEGACY_ERROR_TEMP_2000 > --

[jira] [Assigned] (SPARK-42838) Assign a name to the error class _LEGACY_ERROR_TEMP_2000

[ https://issues.apache.org/jira/browse/SPARK-42838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-42838: Assignee: Max Gekk > Assign a name to the error class _LEGACY_ERROR_TEMP_2000 > -

[jira] [Assigned] (SPARK-42805) 'Conflicting attributes' exception is thrown when joining checkpointed dataframe

[ https://issues.apache.org/jira/browse/SPARK-42805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42805: Assignee: (was: Apache Spark) > 'Conflicting attributes' exception is thrown when joi

[jira] [Commented] (SPARK-42805) 'Conflicting attributes' exception is thrown when joining checkpointed dataframe

[ https://issues.apache.org/jira/browse/SPARK-42805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702306#comment-17702306 ] Apache Spark commented on SPARK-42805: -- User 'ming95' has created a pull reques

[jira] [Commented] (SPARK-42805) 'Conflicting attributes' exception is thrown when joining checkpointed dataframe

[ https://issues.apache.org/jira/browse/SPARK-42805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702305#comment-17702305 ] Apache Spark commented on SPARK-42805: -- User 'ming95' has created a pull reques

[jira] [Assigned] (SPARK-42805) 'Conflicting attributes' exception is thrown when joining checkpointed dataframe

[ https://issues.apache.org/jira/browse/SPARK-42805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42805: Assignee: Apache Spark > 'Conflicting attributes' exception is thrown when joining checkp

[jira] [Commented] (SPARK-42847) Assign a name to the error class _LEGACY_ERROR_TEMP_2013

[ https://issues.apache.org/jira/browse/SPARK-42847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702301#comment-17702301 ] Chao Tian commented on SPARK-42847: --- I want to try it, could you please assign it to m

[jira] [Comment Edited] (SPARK-39722) Make Dataset.showString() public

[ https://issues.apache.org/jira/browse/SPARK-39722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702198#comment-17702198 ] Vindhya G edited comment on SPARK-39722 at 3/19/23 12:18 PM: -

[jira] [Comment Edited] (SPARK-39722) Make Dataset.showString() public

[ https://issues.apache.org/jira/browse/SPARK-39722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702198#comment-17702198 ] Vindhya G edited comment on SPARK-39722 at 3/19/23 11:45 AM: -

[jira] [Updated] (SPARK-42854) Jackson 2.15

[ https://issues.apache.org/jira/browse/SPARK-42854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated SPARK-42854: --- Description: I'm not advocating for an upgrade to [Jackson 2.15|https://github.com/FasterXML/jackso

[jira] [Updated] (SPARK-42854) Jackson 2.15

[ https://issues.apache.org/jira/browse/SPARK-42854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated SPARK-42854: --- Description: I'm not advocating for an upgrade to Jackson 2.15. 2.15.0-rc1 has just been released a

[jira] [Created] (SPARK-42854) Jackson 2.15

PJ Fanning created SPARK-42854: -- Summary: Jackson 2.15 Key: SPARK-42854 URL: https://issues.apache.org/jira/browse/SPARK-42854 Project: Spark Issue Type: Improvement Components: Input/

[jira] [Assigned] (SPARK-42853) Update the Spark Doc to match the new website style

[ https://issues.apache.org/jira/browse/SPARK-42853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42853: Assignee: (was: Apache Spark) > Update the Spark Doc to match the new website style >

[jira] [Assigned] (SPARK-42853) Update the Spark Doc to match the new website style

[ https://issues.apache.org/jira/browse/SPARK-42853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42853: Assignee: Apache Spark > Update the Spark Doc to match the new website style > --

[jira] [Commented] (SPARK-42853) Update the Spark Doc to match the new website style

[ https://issues.apache.org/jira/browse/SPARK-42853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702243#comment-17702243 ] Apache Spark commented on SPARK-42853: -- User 'grundprinzip' has created a pull requ

[jira] [Created] (SPARK-42853) Update the Spark Doc to match the new website style

Martin Grund created SPARK-42853: Summary: Update the Spark Doc to match the new website style Key: SPARK-42853 URL: https://issues.apache.org/jira/browse/SPARK-42853 Project: Spark Issue Typ