[jira] [Commented] (SPARK-37673) Implement `ps.timedelta_range` method

[ https://issues.apache.org/jira/browse/SPARK-37673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513179#comment-17513179 ] Apache Spark commented on SPARK-37673: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-37673) Implement `ps.timedelta_range` method

[ https://issues.apache.org/jira/browse/SPARK-37673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513178#comment-17513178 ] Apache Spark commented on SPARK-37673: -- User 'LuciferYang' has created a pull reque

[jira] [Resolved] (SPARK-38672) Kafka, spark data frame read fails when a message does not exist for the timestamp in all partitions

[ https://issues.apache.org/jira/browse/SPARK-38672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-38672. -- Resolution: Invalid > Kafka, spark data frame read fails when a message does not exist for the

[jira] [Commented] (SPARK-38672) Kafka, spark data frame read fails when a message does not exist for the timestamp in all partitions

[ https://issues.apache.org/jira/browse/SPARK-38672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513175#comment-17513175 ] Jungtaek Lim commented on SPARK-38672: -- Could you please add the option "startingOf

[jira] [Updated] (SPARK-38655) OffsetWindowFunctionFrameBase cannot find the offset row whose input is not null

[ https://issues.apache.org/jira/browse/SPARK-38655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-38655: Fix Version/s: 3.2.2 > OffsetWindowFunctionFrameBase cannot find the offset row whose input is not

[jira] [Resolved] (SPARK-38655) OffsetWindowFunctionFrameBase cannot find the offset row whose input is not null

[ https://issues.apache.org/jira/browse/SPARK-38655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38655. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35971 [https://gith

[jira] [Assigned] (SPARK-38655) OffsetWindowFunctionFrameBase cannot find the offset row whose input is not null

[ https://issues.apache.org/jira/browse/SPARK-38655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-38655: --- Assignee: jiaan.geng > OffsetWindowFunctionFrameBase cannot find the offset row whose input

[jira] [Created] (SPARK-38673) Replace java assert with Junit api in Java UTs

Yang Jie created SPARK-38673: Summary: Replace java assert with Junit api in Java UTs Key: SPARK-38673 URL: https://issues.apache.org/jira/browse/SPARK-38673 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-38623) Add more comments and tests for HashShuffleSpec

[ https://issues.apache.org/jira/browse/SPARK-38623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38623. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35937 [https://gith

[jira] [Assigned] (SPARK-38623) Add more comments and tests for HashShuffleSpec

[ https://issues.apache.org/jira/browse/SPARK-38623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-38623: --- Assignee: Wenchen Fan > Add more comments and tests for HashShuffleSpec > -

[jira] [Assigned] (SPARK-38671) Publish snapshot from branch-3.3

[ https://issues.apache.org/jira/browse/SPARK-38671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38671: Assignee: Apache Spark > Publish snapshot from branch-3.3 > -

[jira] [Commented] (SPARK-38671) Publish snapshot from branch-3.3

[ https://issues.apache.org/jira/browse/SPARK-38671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513151#comment-17513151 ] Apache Spark commented on SPARK-38671: -- User 'wangyum' has created a pull request f

[jira] [Assigned] (SPARK-38671) Publish snapshot from branch-3.3

[ https://issues.apache.org/jira/browse/SPARK-38671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38671: Assignee: (was: Apache Spark) > Publish snapshot from branch-3.3 > --

[jira] [Created] (SPARK-38672) Kafka, spark data frame read fails when a message does not exist for the timestamp in all partitions

gattu shivakumar created SPARK-38672: Summary: Kafka, spark data frame read fails when a message does not exist for the timestamp in all partitions Key: SPARK-38672 URL: https://issues.apache.org/jira/browse/S

[jira] [Resolved] (SPARK-38391) Datasource v2 supports partial topN push-down

[ https://issues.apache.org/jira/browse/SPARK-38391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38391. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35710 [https://gith

[jira] [Assigned] (SPARK-38391) Datasource v2 supports partial topN push-down

[ https://issues.apache.org/jira/browse/SPARK-38391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-38391: --- Assignee: jiaan.geng > Datasource v2 supports partial topN push-down >

[jira] [Commented] (SPARK-38671) Publish snapshot from branch-3.3

[ https://issues.apache.org/jira/browse/SPARK-38671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513142#comment-17513142 ] Yuming Wang commented on SPARK-38671: - https://github.com/apache/spark/pull/35986 >

[jira] [Created] (SPARK-38671) Publish snapshot from branch-3.3

Yuming Wang created SPARK-38671: --- Summary: Publish snapshot from branch-3.3 Key: SPARK-38671 URL: https://issues.apache.org/jira/browse/SPARK-38671 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-38616) Keep track of SQL query text in Catalyst TreeNode

[ https://issues.apache.org/jira/browse/SPARK-38616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-38616. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35936 [https:

[jira] [Assigned] (SPARK-38670) Add offset commit time to streaming query listener

[ https://issues.apache.org/jira/browse/SPARK-38670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38670: Assignee: (was: Apache Spark) > Add offset commit time to streaming query listener >

[jira] [Assigned] (SPARK-38670) Add offset commit time to streaming query listener

[ https://issues.apache.org/jira/browse/SPARK-38670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38670: Assignee: Apache Spark > Add offset commit time to streaming query listener > ---

[jira] [Commented] (SPARK-38670) Add offset commit time to streaming query listener

[ https://issues.apache.org/jira/browse/SPARK-38670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513112#comment-17513112 ] Apache Spark commented on SPARK-38670: -- User 'jerrypeng' has created a pull request

[jira] [Created] (SPARK-38670) Add offset commit time to streaming query listener

Boyang Jerry Peng created SPARK-38670: - Summary: Add offset commit time to streaming query listener Key: SPARK-38670 URL: https://issues.apache.org/jira/browse/SPARK-38670 Project: Spark

[jira] [Assigned] (SPARK-38669) Validate input dataset of ml.clustering

[ https://issues.apache.org/jira/browse/SPARK-38669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38669: Assignee: Apache Spark > Validate input dataset of ml.clustering > --

[jira] [Commented] (SPARK-38669) Validate input dataset of ml.clustering

[ https://issues.apache.org/jira/browse/SPARK-38669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513099#comment-17513099 ] Apache Spark commented on SPARK-38669: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-38669) Validate input dataset of ml.clustering

[ https://issues.apache.org/jira/browse/SPARK-38669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38669: Assignee: (was: Apache Spark) > Validate input dataset of ml.clustering > ---

[jira] [Created] (SPARK-38669) Validate input dataset of ml.clustering

zhengruifeng created SPARK-38669: Summary: Validate input dataset of ml.clustering Key: SPARK-38669 URL: https://issues.apache.org/jira/browse/SPARK-38669 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-38336) Catalyst changes for DEFAULT column support

[ https://issues.apache.org/jira/browse/SPARK-38336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513060#comment-17513060 ] Apache Spark commented on SPARK-38336: -- User 'dtenedor' has created a pull request

[jira] [Commented] (SPARK-38336) Catalyst changes for DEFAULT column support

[ https://issues.apache.org/jira/browse/SPARK-38336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513061#comment-17513061 ] Apache Spark commented on SPARK-38336: -- User 'dtenedor' has created a pull request

[jira] [Updated] (SPARK-38668) Spark on Kubernetes: add separate pod watcher service to reduce pressure on K8s API server

[ https://issues.apache.org/jira/browse/SPARK-38668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BoYang updated SPARK-38668: --- Summary: Spark on Kubernetes: add separate pod watcher service to reduce pressure on K8s API server (was: S

[jira] [Updated] (SPARK-38668) Spark on Kubernetes: support external pod watcher to reduce pressure on K8s API server

[ https://issues.apache.org/jira/browse/SPARK-38668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BoYang updated SPARK-38668: --- Description: Spark driver will listen to all pods events to manage its executor pods. This will cause press

[jira] [Created] (SPARK-38668) Spark on Kubernetes: support external pod watcher to reduce pressure on K8s API server

BoYang created SPARK-38668: -- Summary: Spark on Kubernetes: support external pod watcher to reduce pressure on K8s API server Key: SPARK-38668 URL: https://issues.apache.org/jira/browse/SPARK-38668 Project: S

[jira] [Updated] (SPARK-38667) Optimizer generates error when using inner join along with sequence

[ https://issues.apache.org/jira/browse/SPARK-38667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars updated SPARK-38667: - Description: This issue occurred in a more complex scenario, so I've broken it down into a simple case. {*}Ste

[jira] [Created] (SPARK-38667) Optimizer generates error when using inner join along with sequence

Lars created SPARK-38667: Summary: Optimizer generates error when using inner join along with sequence Key: SPARK-38667 URL: https://issues.apache.org/jira/browse/SPARK-38667 Project: Spark Issue Ty

[jira] [Resolved] (SPARK-38665) upgrade jackson due to CVE-2020-36518

[ https://issues.apache.org/jira/browse/SPARK-38665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38665. - Fix Version/s: 3.3.0 Assignee: Cheng Pan Resolution: Fixed Issue resolved by pul

[jira] [Commented] (SPARK-19335) Spark should support doing an efficient DataFrame Upsert via JDBC

[ https://issues.apache.org/jira/browse/SPARK-19335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17512998#comment-17512998 ] s4782 commented on SPARK-19335: --- Hi,  is there any update on when this will be added?  

[jira] [Updated] (SPARK-38664) Support compact EventLog when there are illegal characters in the path

[ https://issues.apache.org/jira/browse/SPARK-38664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] w l updated SPARK-38664: Summary: Support compact EventLog when there are illegal characters in the path (was: EventLog compact will fail

[jira] [Updated] (SPARK-38257) Upgrade rocksdbjni to 7.0.3

[ https://issues.apache.org/jira/browse/SPARK-38257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-38257: - Summary: Upgrade rocksdbjni to 7.0.3 (was: Upgrade rocksdbjni to 6.29.3) > Upgrade rocksdbjni to 7.0.3

[jira] [Commented] (SPARK-38652) K8S IT Test DepsTestsSuite blocks with PathIOException in hadoop-aws-3.3.2

[ https://issues.apache.org/jira/browse/SPARK-38652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17512846#comment-17512846 ] qian commented on SPARK-38652: -- [~ste...@apache.org] Hi, I run the same suite against an re