[jira] [Commented] (SPARK-40844) Flip the default value of Kafka offset fetching config

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620051#comment-17620051 ] Apache Spark commented on SPARK-40844: -- User 'HeartSaVioR' has created a pull reque

[jira] [Assigned] (SPARK-40844) Flip the default value of Kafka offset fetching config

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40844: Assignee: Apache Spark > Flip the default value of Kafka offset fetching config > ---

[jira] [Assigned] (SPARK-40844) Flip the default value of Kafka offset fetching config

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40844: Assignee: (was: Apache Spark) > Flip the default value of Kafka offset fetching confi

[jira] [Commented] (SPARK-40823) Connect Proto should carry unparsed identifiers

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620067#comment-17620067 ] Apache Spark commented on SPARK-40823: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-40823) Connect Proto should carry unparsed identifiers

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620070#comment-17620070 ] Apache Spark commented on SPARK-40823: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-40823) Connect Proto should carry unparsed identifiers

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620072#comment-17620072 ] Apache Spark commented on SPARK-40823: -- User 'HyukjinKwon' has created a pull reque

[jira] [Created] (SPARK-40845) Add template support for SPARK_GPG_KEY

2022-10-19 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-40845: --- Summary: Add template support for SPARK_GPG_KEY Key: SPARK-40845 URL: https://issues.apache.org/jira/browse/SPARK-40845 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-40779) Fix `corrwith` to work properly with different anchor.

2022-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40779. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38292 [https://gi

[jira] [Assigned] (SPARK-40779) Fix `corrwith` to work properly with different anchor.

2022-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40779: Assignee: Haejoon Lee > Fix `corrwith` to work properly with different anchor. >

[jira] [Assigned] (SPARK-40839) [Python] Implement `DataFrame.sample`

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40839: Assignee: Apache Spark (was: Ruifeng Zheng) > [Python] Implement `DataFrame.sample` > --

[jira] [Assigned] (SPARK-40839) [Python] Implement `DataFrame.sample`

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40839: Assignee: Ruifeng Zheng (was: Apache Spark) > [Python] Implement `DataFrame.sample` > --

[jira] [Commented] (SPARK-40839) [Python] Implement `DataFrame.sample`

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620086#comment-17620086 ] Apache Spark commented on SPARK-40839: -- User 'zhengruifeng' has created a pull requ

[jira] [Resolved] (SPARK-40813) Add limit and offset to Connect DSL

2022-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40813. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38275 [https://gi

[jira] [Assigned] (SPARK-40813) Add limit and offset to Connect DSL

2022-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40813: Assignee: Rui Wang > Add limit and offset to Connect DSL > --

[jira] [Commented] (SPARK-40820) Creating StructType from Json

2022-10-19 Thread Anthony Wainer Cachay Guivin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620099#comment-17620099 ] Anthony Wainer Cachay Guivin commented on SPARK-40820: -- Here an exa

[jira] [Resolved] (SPARK-40753) Fix bug in test case for catalog directory operation

2022-10-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-40753. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38206 [https://gith

[jira] [Assigned] (SPARK-40753) Fix bug in test case for catalog directory operation

2022-10-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-40753: --- Assignee: xiaoping.huang > Fix bug in test case for catalog directory operation > -

[jira] [Updated] (SPARK-40734) KafkaMicroBatchV2SourceWithAdminSuite failed

2022-10-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-40734: - Description: - ensure stream-stream self-join generates only one offset in log and correct metrics ***

[jira] [Updated] (SPARK-40734) KafkaMicroBatchV2SourceWithAdminSuite failed

2022-10-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-40734: - Summary: KafkaMicroBatchV2SourceWithAdminSuite failed (was: KafkaMicroBatchSourceSuite failed) > Kafka

[jira] [Commented] (SPARK-40734) KafkaMicroBatchV2SourceWithAdminSuite failed

2022-10-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620167#comment-17620167 ] Yang Jie commented on SPARK-40734: -- It looks like flaky test,  re-run will succeed, bu

[jira] [Created] (SPARK-40846) GA test failed with Java 8u352

2022-10-19 Thread Yang Jie (Jira)
Yang Jie created SPARK-40846: Summary: GA test failed with Java 8u352 Key: SPARK-40846 URL: https://issues.apache.org/jira/browse/SPARK-40846 Project: Spark Issue Type: Improvement Comp

[jira] [Commented] (SPARK-40846) GA test failed with Java 8u352

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620260#comment-17620260 ] Apache Spark commented on SPARK-40846: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-40846) GA test failed with Java 8u352

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40846: Assignee: Apache Spark > GA test failed with Java 8u352 > --

[jira] [Assigned] (SPARK-40846) GA test failed with Java 8u352

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40846: Assignee: (was: Apache Spark) > GA test failed with Java 8u352 >

[jira] [Commented] (SPARK-40846) GA test failed with Java 8u352

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620272#comment-17620272 ] Apache Spark commented on SPARK-40846: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-40819) Parquet INT64 (TIMESTAMP(NANOS,true)) now throwing Illegal Parquet type instead of automatically converting to LongType

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620298#comment-17620298 ] Apache Spark commented on SPARK-40819: -- User 'awdavidson' has created a pull reques

[jira] [Assigned] (SPARK-40819) Parquet INT64 (TIMESTAMP(NANOS,true)) now throwing Illegal Parquet type instead of automatically converting to LongType

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40819: Assignee: Apache Spark > Parquet INT64 (TIMESTAMP(NANOS,true)) now throwing Illegal Parqu

[jira] [Assigned] (SPARK-40819) Parquet INT64 (TIMESTAMP(NANOS,true)) now throwing Illegal Parquet type instead of automatically converting to LongType

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40819: Assignee: (was: Apache Spark) > Parquet INT64 (TIMESTAMP(NANOS,true)) now throwing Il

[jira] [Created] (SPARK-40847) SPARK: Load Data from Dataframe or RDD to DynamoDB

2022-10-19 Thread Vivek Garg (Jira)
Vivek Garg created SPARK-40847: -- Summary: SPARK: Load Data from Dataframe or RDD to DynamoDB Key: SPARK-40847 URL: https://issues.apache.org/jira/browse/SPARK-40847 Project: Spark Issue Type: Q

[jira] [Updated] (SPARK-40847) SPARK: Load Data from Dataframe or RDD to DynamoDB

2022-10-19 Thread Vivek Garg (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vivek Garg updated SPARK-40847: --- Description: I am using spark 2.1 on EMR and i have a dataframe like this: ClientNum | Value_1 | Va

[jira] [Commented] (SPARK-40791) The semantics of `F` in `DateTimeFormatter` have changed

2022-10-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620487#comment-17620487 ] Dongjoon Hyun commented on SPARK-40791: --- According to the PR comment, Java 11 has

[jira] [Comment Edited] (SPARK-40791) The semantics of `F` in `DateTimeFormatter` have changed

2022-10-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620487#comment-17620487 ] Dongjoon Hyun edited comment on SPARK-40791 at 10/19/22 6:12 PM: -

[jira] [Commented] (SPARK-34827) Support fetching shuffle blocks in batch with i/o encryption

2022-10-19 Thread Pankaj Nagla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620511#comment-17620511 ] Pankaj Nagla commented on SPARK-34827: -- Thank you for sharing such good information

[jira] [Created] (SPARK-40848) Protobuf: Generate descriptor files at build time

2022-10-19 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-40848: Summary: Protobuf: Generate descriptor files at build time Key: SPARK-40848 URL: https://issues.apache.org/jira/browse/SPARK-40848 Project: Spark Issue Type:

[jira] [Commented] (SPARK-40658) Protobuf v2 & v3 support

2022-10-19 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620551#comment-17620551 ] Raghu Angadi commented on SPARK-40658: -- [~sanysand...@gmail.com] , [~mposdev21] I a

[jira] [Commented] (SPARK-40777) Use error classes for Protobuf exceptions

2022-10-19 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620553#comment-17620553 ] Raghu Angadi commented on SPARK-40777: -- I filed a separate ticket SPARK-40848 for g

[jira] [Commented] (SPARK-40656) Schema-registry support for Protobuf format

2022-10-19 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620556#comment-17620556 ] Raghu Angadi commented on SPARK-40656: -- Schema-support for Avro is not here. I will

[jira] [Commented] (SPARK-40659) Schema evolution for protobuf (and Avro too?)

2022-10-19 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620557#comment-17620557 ] Raghu Angadi commented on SPARK-40659: -- Schema-support for Avro is not in open-sour

[jira] [Resolved] (SPARK-40844) Flip the default value of Kafka offset fetching config

2022-10-19 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-40844. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38306 [https://gi

[jira] [Assigned] (SPARK-40844) Flip the default value of Kafka offset fetching config

2022-10-19 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-40844: Assignee: Jungtaek Lim > Flip the default value of Kafka offset fetching config > ---

[jira] [Updated] (SPARK-40844) Flip the default value of Kafka offset fetching config

2022-10-19 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-40844: - Labels: release-notes (was: ) > Flip the default value of Kafka offset fetching config > --

[jira] [Commented] (SPARK-40659) Schema evolution for protobuf (and Avro too?)

2022-10-19 Thread Sandish Kumar HN (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620581#comment-17620581 ] Sandish Kumar HN commented on SPARK-40659: -- [~rangadi] is the reason being conf

[jira] [Updated] (SPARK-40025) Project Lightspeed: Faster and Simpler Stream Processing with Apache Spark

2022-10-19 Thread Boyang Jerry Peng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Jerry Peng updated SPARK-40025: -- Description: Project Lightspeed is an umbrella project aimed at improving a couple of

[jira] [Created] (SPARK-40849) Async log purge

2022-10-19 Thread Boyang Jerry Peng (Jira)
Boyang Jerry Peng created SPARK-40849: - Summary: Async log purge Key: SPARK-40849 URL: https://issues.apache.org/jira/browse/SPARK-40849 Project: Spark Issue Type: New Feature C

[jira] [Updated] (SPARK-40025) Project Lightspeed: Faster and Simpler Stream Processing with Apache Spark

2022-10-19 Thread Boyang Jerry Peng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Jerry Peng updated SPARK-40025: -- Description: Project Lightspeed is an umbrella project aimed at improving a couple of

[jira] [Updated] (SPARK-40025) Project Lightspeed: Faster and Simpler Stream Processing with Apache Spark

2022-10-19 Thread Boyang Jerry Peng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Jerry Peng updated SPARK-40025: -- Description: Project Lightspeed is an umbrella project aimed at improving a couple of

[jira] [Updated] (SPARK-40849) Async log purge

2022-10-19 Thread Boyang Jerry Peng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Jerry Peng updated SPARK-40849: -- Description: Purging old entries in both the offset log and commit log will be done as

[jira] [Updated] (SPARK-40849) Async log purge

2022-10-19 Thread Boyang Jerry Peng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Jerry Peng updated SPARK-40849: -- Description: Purging old entries in both the offset log and commit log will be done a

[jira] [Assigned] (SPARK-40849) Async log purge

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40849: Assignee: (was: Apache Spark) > Async log purge > --- > >

[jira] [Assigned] (SPARK-40849) Async log purge

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40849: Assignee: Apache Spark > Async log purge > --- > > Key: SPARK

[jira] [Commented] (SPARK-40849) Async log purge

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620641#comment-17620641 ] Apache Spark commented on SPARK-40849: -- User 'jerrypeng' has created a pull request

[jira] [Commented] (SPARK-40658) Protobuf v2 & v3 support

2022-10-19 Thread Mohan Parthasarathy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620651#comment-17620651 ] Mohan Parthasarathy commented on SPARK-40658: - [~rangadi] I started adding s

[jira] [Resolved] (SPARK-40846) GA test failed with Java 8u352

2022-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40846. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38311 [https://gi

[jira] [Assigned] (SPARK-40846) GA test failed with Java 8u352

2022-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40846: Assignee: Yang Jie > GA test failed with Java 8u352 > -- > >

[jira] [Commented] (SPARK-40658) Protobuf v2 & v3 support

2022-10-19 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620655#comment-17620655 ] Raghu Angadi commented on SPARK-40658: -- [~mposdev21] more tests are good. Lets add

[jira] [Commented] (SPARK-40658) Protobuf v2 & v3 support

2022-10-19 Thread Mohan Parthasarathy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620657#comment-17620657 ] Mohan Parthasarathy commented on SPARK-40658: - [~rangadi]  I did base it off

[jira] [Commented] (SPARK-40658) Protobuf v2 & v3 support

2022-10-19 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620659#comment-17620659 ] Raghu Angadi commented on SPARK-40658: -- Thats awesome! Lets look at both and merge

[jira] [Commented] (SPARK-40659) Schema evolution for protobuf (and Avro too?)

2022-10-19 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620661#comment-17620661 ] Raghu Angadi commented on SPARK-40659: -- Right, I don't think that is the reason. Da

[jira] [Resolved] (SPARK-40656) Schema-registry support for Protobuf format

2022-10-19 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi resolved SPARK-40656. -- Resolution: Won't Do > Schema-registry support for Protobuf format > -

[jira] [Updated] (SPARK-40025) Project Lightspeed: Faster and Simpler Stream Processing with Apache Spark

2022-10-19 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-40025: - Description: Project Lightspeed is an umbrella project aimed at improving a couple of key aspec

[jira] [Resolved] (SPARK-40539) PySpark readwriter API parity for Spark Connect

2022-10-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-40539. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38086 [https://gith

[jira] [Resolved] (SPARK-39590) Python API Parity in Structure Streaming

2022-10-19 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-39590. -- Resolution: Duplicate Closed as duplicated. > Python API Parity in Structure Streaming >

[jira] [Assigned] (SPARK-40539) PySpark readwriter API parity for Spark Connect

2022-10-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-40539: --- Assignee: Rui Wang > PySpark readwriter API parity for Spark Connect >

[jira] [Resolved] (SPARK-40847) SPARK: Load Data from Dataframe or RDD to DynamoDB

2022-10-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40847. -- Resolution: Invalid Let's ask questions into Spark user mailing list before filing it as an is

[jira] [Created] (SPARK-40850) Tests for Spark SQL Intrepetered Queries may execute Codegen

2022-10-19 Thread Holden Karau (Jira)
Holden Karau created SPARK-40850: Summary: Tests for Spark SQL Intrepetered Queries may execute Codegen Key: SPARK-40850 URL: https://issues.apache.org/jira/browse/SPARK-40850 Project: Spark

[jira] [Created] (SPARK-40851) TimestampFormatter behavior changed when using the latest Java

2022-10-19 Thread Yang Jie (Jira)
Yang Jie created SPARK-40851: Summary: TimestampFormatter behavior changed when using the latest Java Key: SPARK-40851 URL: https://issues.apache.org/jira/browse/SPARK-40851 Project: Spark Issue

[jira] [Updated] (SPARK-40851) TimestampFormatter behavior changed when using the latest Java 8/11/17

2022-10-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-40851: - Summary: TimestampFormatter behavior changed when using the latest Java 8/11/17 (was: TimestampFormatte

[jira] [Commented] (SPARK-40813) Add limit and offset to Connect DSL

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620711#comment-17620711 ] Apache Spark commented on SPARK-40813: -- User 'amaliujia' has created a pull request

[jira] [Commented] (SPARK-40813) Add limit and offset to Connect DSL

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620712#comment-17620712 ] Apache Spark commented on SPARK-40813: -- User 'amaliujia' has created a pull request

[jira] [Commented] (SPARK-40768) Migrate type check failures of bloom_filter_agg() onto error classes

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620739#comment-17620739 ] Apache Spark commented on SPARK-40768: -- User 'lvshaokang' has created a pull reques

[jira] [Assigned] (SPARK-40768) Migrate type check failures of bloom_filter_agg() onto error classes

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40768: Assignee: (was: Apache Spark) > Migrate type check failures of bloom_filter_agg() ont

[jira] [Assigned] (SPARK-40768) Migrate type check failures of bloom_filter_agg() onto error classes

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40768: Assignee: Apache Spark > Migrate type check failures of bloom_filter_agg() onto error cla

[jira] [Commented] (SPARK-40768) Migrate type check failures of bloom_filter_agg() onto error classes

2022-10-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620741#comment-17620741 ] Apache Spark commented on SPARK-40768: -- User 'lvshaokang' has created a pull reques

[jira] [Created] (SPARK-40852) Implement `DataFrame.summary`

2022-10-19 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-40852: - Summary: Implement `DataFrame.summary` Key: SPARK-40852 URL: https://issues.apache.org/jira/browse/SPARK-40852 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-40852) Implement `DataFrame.summary`

2022-10-19 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-40852: - Assignee: Ruifeng Zheng > Implement `DataFrame.summary` > -

[jira] [Commented] (SPARK-40758) Upgrade Apache zookeeper to get rid of CVE-2020-10663

2022-10-19 Thread Bilna (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620765#comment-17620765 ] Bilna commented on SPARK-40758: --- https://issues.apache.org/jira/browse/ZOOKEEPER-3933 This

[jira] [Commented] (SPARK-40457) upgrade jackson data mapper to latest

2022-10-19 Thread Bilna (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620766#comment-17620766 ] Bilna commented on SPARK-40457: --- [~hyukjin.kwon] Understood. So I think I can mark this as

[jira] [Commented] (SPARK-40422) Upgrade hive to 4.0.0

2022-10-19 Thread Bilna (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620774#comment-17620774 ] Bilna commented on SPARK-40422: --- [~srowen] In the mvn dependency tree I can see google-gso