[jira] [Resolved] (SPARK-43075) Change gRPC to grpcio when it is not installed.

2023-04-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43075. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40716 [https://gi

[jira] [Assigned] (SPARK-43075) Change gRPC to grpcio when it is not installed.

2023-04-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43075: Assignee: Bjørn Jørgensen > Change gRPC to grpcio when it is not installed. > ---

[jira] [Comment Edited] (SPARK-43001) Spark last window dont flush in append mode

2023-04-08 Thread Vindhya G (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17709932#comment-17709932 ] Vindhya G edited comment on SPARK-43001 at 4/9/23 5:07 AM: --- Ag

[jira] [Commented] (SPARK-43001) Spark last window dont flush in append mode

2023-04-08 Thread Vindhya G (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17709932#comment-17709932 ] Vindhya G commented on SPARK-43001: --- Agree.I think there should be some way for users

[jira] [Commented] (SPARK-43074) Add the function without constant parameters of `SessionState#executePlan`

2023-04-08 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17709931#comment-17709931 ] Snoot.io commented on SPARK-43074: -- User 'ming95' has created a pull request for th

[jira] [Updated] (SPARK-43067) Error class resource file in Kafka connector is misplaced

2023-04-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-43067: - Fix Version/s: 3.5.0 > Error class resource file in Kafka connector is misplaced > -

[jira] [Resolved] (SPARK-36121) Write data loss caused by stage retry when enable v2 FileOutputCommitter

2023-04-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-36121. -- Resolution: Auto Closed I'll close this since the affected version is EOL and we gave the sugg

[jira] [Resolved] (SPARK-38078) Aggregation with Watermark in AppendMode is holding data beyong water mark boundary.

2023-04-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-38078. -- Resolution: Invalid It's not clear whether this is a real issue or user's misunderstanding. In

[jira] [Commented] (SPARK-38078) Aggregation with Watermark in AppendMode is holding data beyong water mark boundary.

2023-04-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17709925#comment-17709925 ] Jungtaek Lim commented on SPARK-38078: -- If you use time window, the boundary Spark

[jira] [Commented] (SPARK-43001) Spark last window dont flush in append mode

2023-04-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17709924#comment-17709924 ] Jungtaek Lim commented on SPARK-43001: -- This is a known issue. The current availabl

[jira] [Resolved] (SPARK-43017) Connect to multiple hive metastore using single sparkcontext (i.e without stopping it")

2023-04-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-43017. -- Resolution: Invalid Please use user mailing list for questions. > Connect to multiple hive me

[jira] [Resolved] (SPARK-42976) spark sql Disable vectorized faild

2023-04-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-42976. -- Resolution: Invalid Please use user mailing list for questions. Furthermore if you read from

[jira] [Updated] (SPARK-42976) spark sql Disable vectorized faild

2023-04-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-42976: - Issue Type: Question (was: Improvement) > spark sql Disable vectorized faild > ---

[jira] [Updated] (SPARK-42976) spark sql Disable vectorized faild

2023-04-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-42976: - Priority: Minor (was: Blocker) > spark sql Disable vectorized faild >

[jira] [Updated] (SPARK-42976) spark sql Disable vectorized faild

2023-04-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-42976: - Fix Version/s: (was: 3.3.2) > spark sql Disable vectorized faild >

[jira] [Updated] (SPARK-42860) Add analysed logical mode in org.apache.spark.sql.execution.ExplainMode

2023-04-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-42860: - Priority: Major (was: Blocker) > Add analysed logical mode in org.apache.spark.sql.execution.Ex

[jira] [Assigned] (SPARK-43067) Error class resource file in Kafka connector is misplaced

2023-04-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-43067: Assignee: Jungtaek Lim > Error class resource file in Kafka connector is misplaced >

[jira] [Resolved] (SPARK-43067) Error class resource file in Kafka connector is misplaced

2023-04-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-43067. -- Fix Version/s: 3.4.1 Resolution: Fixed Issue resolved by pull request 40705 [https://gi

[jira] [Commented] (SPARK-43029) PySpark Breaks with Pandas 2.0

2023-04-08 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-43029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17709917#comment-17709917 ] Bjørn Jørgensen commented on SPARK-43029: - [It is deprecated in 3.4.0|https://g

[jira] [Created] (SPARK-43075) Change gRPC to grpcio when it is not installed.

2023-04-08 Thread Jira
Bjørn Jørgensen created SPARK-43075: --- Summary: Change gRPC to grpcio when it is not installed. Key: SPARK-43075 URL: https://issues.apache.org/jira/browse/SPARK-43075 Project: Spark Issue T

[jira] [Resolved] (SPARK-42931) dropDuplicates within watermark

2023-04-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-42931. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40561 [https://gi

[jira] [Assigned] (SPARK-42931) dropDuplicates within watermark

2023-04-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-42931: Assignee: Jungtaek Lim > dropDuplicates within watermark > --

[jira] [Updated] (SPARK-43074) Add the function without constant parameters of `SessionState#executePlan`

2023-04-08 Thread zzzzming95 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ming95 updated SPARK-43074: --- Description:   {code:java} ``` df = spark.sql("select 1")  catalyst_plan = df._jdf.queryExecution().

[jira] [Created] (SPARK-43074) Add the function without constant parameters of `SessionState#executePlan`

2023-04-08 Thread zzzzming95 (Jira)
ming95 created SPARK-43074: -- Summary: Add the function without constant parameters of `SessionState#executePlan` Key: SPARK-43074 URL: https://issues.apache.org/jira/browse/SPARK-43074 Project: Spark

[jira] [Updated] (SPARK-42280) add spark.yarn.archive/jars similar option for spark on K8S

2023-04-08 Thread Leibniz Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leibniz Hu updated SPARK-42280: --- Description: For spark on yarn,  there are `spark.yarn.archive` and `spark.yarn.jars` to distribute

[jira] [Updated] (SPARK-43073) Add proto types constants

2023-04-08 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43073: -- Summary: Add proto types constants (was: Add proto types constant) > Add proto types constant

[jira] [Created] (SPARK-43073) Add proto types constant

2023-04-08 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-43073: - Summary: Add proto types constant Key: SPARK-43073 URL: https://issues.apache.org/jira/browse/SPARK-43073 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-43066) Add Java test for dropDuplicates in batch DataFrame

2023-04-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43066: Assignee: Jungtaek Lim > Add Java test for dropDuplicates in batch DataFrame > --

[jira] [Resolved] (SPARK-43066) Add Java test for dropDuplicates in batch DataFrame

2023-04-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43066. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40702 [https://gi

[jira] [Resolved] (SPARK-43062) Add options to lint-python to run each test separately

2023-04-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43062. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40698 [https://gi

[jira] [Assigned] (SPARK-43062) Add options to lint-python to run each test separately

2023-04-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43062: Assignee: Takuya Ueshin > Add options to lint-python to run each test separately > --

[jira] [Assigned] (SPARK-42859) Basic support for pandas API on Spark

2023-04-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42859: Assignee: Haejoon Lee > Basic support for pandas API on Spark > -

[jira] [Resolved] (SPARK-42859) Basic support for pandas API on Spark

2023-04-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42859. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40525 [https://gi