[jira] [Commented] (SPARK-41819) Implement Dataframe.rdd getNumPartitions

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653653#comment-17653653 ] Sandeep Singh commented on SPARK-41819: --- Should be moved under

[jira] [Commented] (SPARK-41818) Support DataFrameWriter.saveAsTable

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653654#comment-17653654 ] Sandeep Singh commented on SPARK-41818: --- Could be moved under

[jira] [Updated] (SPARK-41822) Setup Scala/JVM Client Connection

2023-01-02 Thread Venkata Sai Akhil Gudesa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Sai Akhil Gudesa updated SPARK-41822: - Summary: Setup Scala/JVM Client Connection (was: Setup Scala Client

[jira] [Updated] (SPARK-41836) Implement `transform_values` function

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41836: -- Summary: Implement `transform_values` function (was: CLONE - Implement `transform_values`

[jira] [Assigned] (SPARK-41804) InterpretedUnsafeProjection doesn't properly handle an array of UDTs

2023-01-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41804: Assignee: Bruce Robbins > InterpretedUnsafeProjection doesn't properly handle an array

[jira] [Resolved] (SPARK-41804) InterpretedUnsafeProjection doesn't properly handle an array of UDTs

2023-01-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41804. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39349

[jira] [Created] (SPARK-41847) DataFrame mapfield invalid type

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41847: - Summary: DataFrame mapfield invalid type Key: SPARK-41847 URL: https://issues.apache.org/jira/browse/SPARK-41847 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-41847) DataFrame mapfield invalid type

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41847: -- Description: {code:java} File

[jira] [Updated] (SPARK-41847) DataFrame mapfield,structlist invalid type

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41847: -- Summary: DataFrame mapfield,structlist invalid type (was: DataFrame mapfield invalid type)

[jira] [Created] (SPARK-41852) Fix `pmod` function

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41852: - Summary: Fix `pmod` function Key: SPARK-41852 URL: https://issues.apache.org/jira/browse/SPARK-41852 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-41852) Fix `pmod` function

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41852: -- Description: {code:java} File

[jira] [Created] (SPARK-41853) Use Map in place of SortedMap for ErrorClassesJsonReader

2023-01-02 Thread Ted Yu (Jira)
Ted Yu created SPARK-41853: -- Summary: Use Map in place of SortedMap for ErrorClassesJsonReader Key: SPARK-41853 URL: https://issues.apache.org/jira/browse/SPARK-41853 Project: Spark Issue Type:

[jira] [Updated] (SPARK-41848) Tasks are over-scheduled with TaskResourceProfile

2023-01-02 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-41848: - Priority: Blocker (was: Major) > Tasks are over-scheduled with TaskResourceProfile >

[jira] [Updated] (SPARK-41824) Implement DataFrame.explain format to be similar to PySpark

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41824: -- Description: {code:java} File

[jira] [Created] (SPARK-41824) Implement DataFrame.explain format to be similar to PySpark

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41824: - Summary: Implement DataFrame.explain format to be similar to PySpark Key: SPARK-41824 URL: https://issues.apache.org/jira/browse/SPARK-41824 Project: Spark

[jira] [Created] (SPARK-41826) Implement Dataframe.readStream

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41826: - Summary: Implement Dataframe.readStream Key: SPARK-41826 URL: https://issues.apache.org/jira/browse/SPARK-41826 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-41826) Implement Dataframe.readStream

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41826: -- Description: {code:java} File

[jira] [Created] (SPARK-41829) Implement Dataframe.sort ordering

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41829: - Summary: Implement Dataframe.sort ordering Key: SPARK-41829 URL: https://issues.apache.org/jira/browse/SPARK-41829 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-41832) DataFrame.unionByName output is wrong

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41832: - Summary: DataFrame.unionByName output is wrong Key: SPARK-41832 URL: https://issues.apache.org/jira/browse/SPARK-41832 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-41836) CLONE - Implement `transform_values` function

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41836: - Summary: CLONE - Implement `transform_values` function Key: SPARK-41836 URL: https://issues.apache.org/jira/browse/SPARK-41836 Project: Spark Issue Type:

[jira] [Created] (SPARK-41835) Implement `transform_keys` function

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41835: - Summary: Implement `transform_keys` function Key: SPARK-41835 URL: https://issues.apache.org/jira/browse/SPARK-41835 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-41839) Implement SparkSession.sparkContext

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41839: -- Description: (was: {code:java} File

[jira] [Created] (SPARK-41839) Implement SparkSession.sparkContext

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41839: - Summary: Implement SparkSession.sparkContext Key: SPARK-41839 URL: https://issues.apache.org/jira/browse/SPARK-41839 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-41841) Support PyPI packaging without JVM

2023-01-02 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41841: Summary: Support PyPI packaging without JVM Key: SPARK-41841 URL: https://issues.apache.org/jira/browse/SPARK-41841 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-41845) Fix `count(expr("*"))` function

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41845: - Summary: Fix `count(expr("*"))` function Key: SPARK-41845 URL: https://issues.apache.org/jira/browse/SPARK-41845 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-41844) Implement `intX2` function

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh resolved SPARK-41844. --- Resolution: Invalid > Implement `intX2` function > -- > >

[jira] [Commented] (SPARK-41814) Column.eqNullSafe fails on NaN comparison

2023-01-02 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653736#comment-17653736 ] Ruifeng Zheng commented on SPARK-41814: --- this issue is due to: 1, the conversion from rows to

[jira] [Created] (SPARK-41850) Fix DataFrameReader.isnan

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41850: - Summary: Fix DataFrameReader.isnan Key: SPARK-41850 URL: https://issues.apache.org/jira/browse/SPARK-41850 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-41850) Fix `isnan` function

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41850: -- Summary: Fix `isnan` function (was: Fix DataFrameReader.isnan) > Fix `isnan` function >

[jira] [Updated] (SPARK-41850) Fix `isnan` function

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41850: -- Description: {code:java} File

[jira] [Commented] (SPARK-41850) Fix `isnan` function

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653738#comment-17653738 ] Sandeep Singh commented on SPARK-41850: --- This should be moved under SPARK-41283 > Fix `isnan`

[jira] [Commented] (SPARK-41853) Use Map in place of SortedMap for ErrorClassesJsonReader

2023-01-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653741#comment-17653741 ] Apache Spark commented on SPARK-41853: -- User 'tedyu' has created a pull request for this issue:

[jira] [Created] (SPARK-41825) DataFrame.show formating int as double

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41825: - Summary: DataFrame.show formating int as double Key: SPARK-41825 URL: https://issues.apache.org/jira/browse/SPARK-41825 Project: Spark Issue Type:

[jira] [Updated] (SPARK-41825) DataFrame.show formating int as double

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41825: -- Description: {code:java} File

[jira] [Updated] (SPARK-41827) DataFrame.groupBy requires all cols be Column or str

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41827: -- Description: {code:java} File

[jira] [Created] (SPARK-41827) DataFrame.groupBy requires all cols be Column or str

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41827: - Summary: DataFrame.groupBy requires all cols be Column or str Key: SPARK-41827 URL: https://issues.apache.org/jira/browse/SPARK-41827 Project: Spark Issue

[jira] [Updated] (SPARK-41833) DataFrame.collect() output parity with pyspark

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41833: -- Description: {code:java} File

[jira] [Created] (SPARK-41833) DataFrame.collect() output parity with pyspark

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41833: - Summary: DataFrame.collect() output parity with pyspark Key: SPARK-41833 URL: https://issues.apache.org/jira/browse/SPARK-41833 Project: Spark Issue Type:

[jira] [Created] (SPARK-41837) DataFrame.createDataFrame datatype conversion error

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41837: - Summary: DataFrame.createDataFrame datatype conversion error Key: SPARK-41837 URL: https://issues.apache.org/jira/browse/SPARK-41837 Project: Spark Issue

[jira] [Updated] (SPARK-41837) DataFrame.createDataFrame datatype conversion error

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41837: -- Description: {code:java} File

[jira] [Assigned] (SPARK-41659) Enable doctests in pyspark.sql.connect.readwriter

2023-01-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41659: Assignee: Sandeep Singh (was: Hyukjin Kwon) > Enable doctests in

[jira] [Commented] (SPARK-41835) Implement `transform_keys` function

2023-01-02 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653722#comment-17653722 ] Ruifeng Zheng commented on SPARK-41835: --- this function was added > Implement `transform_keys`

[jira] [Updated] (SPARK-41843) Implement SparkSession.udf

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41843: -- Description: {code:java} File

[jira] [Commented] (SPARK-37677) spark on k8s, when the user want to push python3.6.6.zip to the pod , but no permission to execute

2023-01-02 Thread jingxiong zhong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653733#comment-17653733 ] jingxiong zhong commented on SPARK-37677: - At present, I have repaired Hadoop version 3.3.5, but

[jira] [Resolved] (SPARK-37521) insert overwrite table but the partition information stored in Metastore was not changed

2023-01-02 Thread jingxiong zhong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jingxiong zhong resolved SPARK-37521. - Resolution: Won't Fix > insert overwrite table but the partition information stored in

[jira] [Updated] (SPARK-41849) Implement DataFrameReader.text

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41849: -- Description: {code:java} File

[jira] [Created] (SPARK-41849) Implement DataFrameReader.text

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41849: - Summary: Implement DataFrameReader.text Key: SPARK-41849 URL: https://issues.apache.org/jira/browse/SPARK-41849 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-41820) DataFrame.createOrReplaceGlobalTempView - SparkConnectException: requirement failed

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41820: - Summary: DataFrame.createOrReplaceGlobalTempView - SparkConnectException: requirement failed Key: SPARK-41820 URL: https://issues.apache.org/jira/browse/SPARK-41820

[jira] [Updated] (SPARK-41821) Fix DataFrame.describe

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41821: -- Description: {code:java} File

[jira] [Created] (SPARK-41821) Fix DataFrame.describe

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41821: - Summary: Fix DataFrame.describe Key: SPARK-41821 URL: https://issues.apache.org/jira/browse/SPARK-41821 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-41828) Implement creating empty Dataframe

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41828: - Summary: Implement creating empty Dataframe Key: SPARK-41828 URL: https://issues.apache.org/jira/browse/SPARK-41828 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-41828) Implement creating empty Dataframe

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41828: -- Description: {code:java} File

[jira] [Commented] (SPARK-41311) Rewrite test RENAME_SRC_PATH_NOT_FOUND to trigger the error from user space

2023-01-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653701#comment-17653701 ] Apache Spark commented on SPARK-41311: -- User 'ibuder' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41311) Rewrite test RENAME_SRC_PATH_NOT_FOUND to trigger the error from user space

2023-01-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41311: Assignee: (was: Apache Spark) > Rewrite test RENAME_SRC_PATH_NOT_FOUND to trigger

[jira] [Commented] (SPARK-41311) Rewrite test RENAME_SRC_PATH_NOT_FOUND to trigger the error from user space

2023-01-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653702#comment-17653702 ] Apache Spark commented on SPARK-41311: -- User 'ibuder' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41311) Rewrite test RENAME_SRC_PATH_NOT_FOUND to trigger the error from user space

2023-01-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41311: Assignee: Apache Spark > Rewrite test RENAME_SRC_PATH_NOT_FOUND to trigger the error

[jira] [Created] (SPARK-41840) DataFrame.show(): 'Column' object is not callable

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41840: - Summary: DataFrame.show(): 'Column' object is not callable Key: SPARK-41840 URL: https://issues.apache.org/jira/browse/SPARK-41840 Project: Spark Issue

[jira] [Updated] (SPARK-41840) DataFrame.show(): 'Column' object is not callable

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41840: -- Description: {code:java} File

[jira] [Updated] (SPARK-41819) Implement Dataframe.rdd getNumPartitions

2023-01-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41819: - Parent: (was: SPARK-41281) Issue Type: Bug (was: Sub-task) > Implement

[jira] [Updated] (SPARK-41819) Implement Dataframe.rdd getNumPartitions

2023-01-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41819: - Parent: SPARK-41279 Issue Type: Sub-task (was: Bug) > Implement Dataframe.rdd

[jira] [Updated] (SPARK-41819) Implement Dataframe.rdd getNumPartitions

2023-01-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41819: - Epic Link: (was: SPARK-39375) > Implement Dataframe.rdd getNumPartitions >

[jira] [Resolved] (SPARK-41659) Enable doctests in pyspark.sql.connect.readwriter

2023-01-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41659. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39331

[jira] [Updated] (SPARK-41819) Implement Dataframe.rdd getNumPartitions

2023-01-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41819: - Epic Link: SPARK-39375 > Implement Dataframe.rdd getNumPartitions >

[jira] [Assigned] (SPARK-41659) Enable doctests in pyspark.sql.connect.readwriter

2023-01-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41659: Assignee: Hyukjin Kwon > Enable doctests in pyspark.sql.connect.readwriter >

[jira] [Updated] (SPARK-41847) DataFrame mapfield,structlist invalid type

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41847: -- Description: {code:java} File

[jira] [Assigned] (SPARK-41853) Use Map in place of SortedMap for ErrorClassesJsonReader

2023-01-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41853: Assignee: (was: Apache Spark) > Use Map in place of SortedMap for

[jira] [Commented] (SPARK-41853) Use Map in place of SortedMap for ErrorClassesJsonReader

2023-01-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653740#comment-17653740 ] Apache Spark commented on SPARK-41853: -- User 'tedyu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41853) Use Map in place of SortedMap for ErrorClassesJsonReader

2023-01-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41853: Assignee: Apache Spark > Use Map in place of SortedMap for ErrorClassesJsonReader >

[jira] [Updated] (SPARK-41831) DataFrame.transform: Only Column or String can be used for projections

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41831: -- Description: {code:java} File

[jira] [Created] (SPARK-41831) Fix DataFrame.transform: Only Column or String can be used for projections

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41831: - Summary: Fix DataFrame.transform: Only Column or String can be used for projections Key: SPARK-41831 URL: https://issues.apache.org/jira/browse/SPARK-41831

[jira] [Commented] (SPARK-41656) Enable doctests in pyspark.sql.connect.dataframe

2023-01-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653673#comment-17653673 ] Apache Spark commented on SPARK-41656: -- User 'techaddict' has created a pull request for this

[jira] [Assigned] (SPARK-41656) Enable doctests in pyspark.sql.connect.dataframe

2023-01-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41656: Assignee: Apache Spark > Enable doctests in pyspark.sql.connect.dataframe >

[jira] [Commented] (SPARK-41656) Enable doctests in pyspark.sql.connect.dataframe

2023-01-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653674#comment-17653674 ] Apache Spark commented on SPARK-41656: -- User 'techaddict' has created a pull request for this

[jira] [Assigned] (SPARK-41656) Enable doctests in pyspark.sql.connect.dataframe

2023-01-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41656: Assignee: (was: Apache Spark) > Enable doctests in pyspark.sql.connect.dataframe >

[jira] [Created] (SPARK-41844) Implement `intX2` function

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41844: - Summary: Implement `intX2` function Key: SPARK-41844 URL: https://issues.apache.org/jira/browse/SPARK-41844 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-41835) Implement `transform_keys` function

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41835: -- Description: {code:java} File

[jira] [Commented] (SPARK-41835) Implement `transform_keys` function

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653731#comment-17653731 ] Sandeep Singh commented on SPARK-41835: --- My bad, error is about expected input types. > Implement

[jira] [Updated] (SPARK-41844) Implement `intX2` function

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41844: -- Description: {code:java} File

[jira] [Updated] (SPARK-41847) DataFrame mapfield,structlist invalid type

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41847: -- Description: {code:java} File

[jira] [Created] (SPARK-41848) Tasks are over-scheduled with TaskResourceProfile

2023-01-02 Thread wuyi (Jira)
wuyi created SPARK-41848: Summary: Tasks are over-scheduled with TaskResourceProfile Key: SPARK-41848 URL: https://issues.apache.org/jira/browse/SPARK-41848 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-41832) DataFrame.unionByName output is wrong

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41832: -- Description: {code:java} File

[jira] [Updated] (SPARK-41833) DataFrame.collect() output parity with pyspark

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41833: -- Description: {code:java}

[jira] [Created] (SPARK-41838) DataFrame.show() fix map printing

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41838: - Summary: DataFrame.show() fix map printing Key: SPARK-41838 URL: https://issues.apache.org/jira/browse/SPARK-41838 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-41838) DataFrame.show() fix map printing

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41838: -- Description: {code:java} File

[jira] [Commented] (SPARK-41804) InterpretedUnsafeProjection doesn't properly handle an array of UDTs

2023-01-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653709#comment-17653709 ] Apache Spark commented on SPARK-41804: -- User 'bersprockets' has created a pull request for this

[jira] [Assigned] (SPARK-41804) InterpretedUnsafeProjection doesn't properly handle an array of UDTs

2023-01-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41804: Assignee: Apache Spark > InterpretedUnsafeProjection doesn't properly handle an array of

[jira] [Assigned] (SPARK-41804) InterpretedUnsafeProjection doesn't properly handle an array of UDTs

2023-01-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41804: Assignee: (was: Apache Spark) > InterpretedUnsafeProjection doesn't properly handle

[jira] [Updated] (SPARK-41818) Support DataFrameWriter.saveAsTable

2023-01-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41818: - Parent: (was: SPARK-41281) Issue Type: Bug (was: Sub-task) > Support

[jira] [Updated] (SPARK-41818) Support DataFrameWriter.saveAsTable

2023-01-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41818: - Epic Link: SPARK-39375 > Support DataFrameWriter.saveAsTable >

[jira] [Updated] (SPARK-41817) SparkSession.read support reading with schema

2023-01-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41817: - Parent: (was: SPARK-41281) Issue Type: Bug (was: Sub-task) > SparkSession.read

[jira] [Updated] (SPARK-41817) SparkSession.read support reading with schema

2023-01-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41817: - Epic Link: SPARK-39375 > SparkSession.read support reading with schema >

[jira] [Updated] (SPARK-41818) Support DataFrameWriter.saveAsTable

2023-01-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41818: - Epic Link: (was: SPARK-39375) > Support DataFrameWriter.saveAsTable >

[jira] [Updated] (SPARK-41818) Support DataFrameWriter.saveAsTable

2023-01-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41818: - Parent: SPARK-41284 Issue Type: Sub-task (was: Bug) > Support

[jira] [Updated] (SPARK-41817) SparkSession.read support reading with schema

2023-01-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41817: - Epic Link: (was: SPARK-39375) > SparkSession.read support reading with schema >

[jira] [Updated] (SPARK-41817) SparkSession.read support reading with schema

2023-01-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41817: - Parent: SPARK-41284 Issue Type: Sub-task (was: Bug) > SparkSession.read support

[jira] [Updated] (SPARK-41842) Support data type Timestamp(NANOSECOND, null)

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41842: -- Description: {code:java} File

[jira] [Commented] (SPARK-41842) Support data type Timestamp(NANOSECOND, null)

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653728#comment-17653728 ] Sandeep Singh commented on SPARK-41842: --- Not sure about the EPIC for this one. > Support data

[jira] [Created] (SPARK-41843) Implement SparkSession.udf

2023-01-02 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41843: - Summary: Implement SparkSession.udf Key: SPARK-41843 URL: https://issues.apache.org/jira/browse/SPARK-41843 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-41823) DataFrame.join creating ambiguous column names

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh resolved SPARK-41823. --- Resolution: Duplicate > DataFrame.join creating ambiguous column names >

[jira] [Updated] (SPARK-41847) DataFrame mapfield,structlist invalid type

2023-01-02 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41847: -- Description: {code:java} File

  1   2   3   >