[jira] [Created] (SPARK-39977) Remove unnecessary guava exclusion from jackson-module-scala

2022-08-04 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-39977: - Summary: Remove unnecessary guava exclusion from jackson-module-scala Key: SPARK-39977 URL: https://issues.apache.org/jira/browse/SPARK-39977 Project: Spark Issue

[jira] [Commented] (SPARK-39977) Remove unnecessary guava exclusion from jackson-module-scala

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575072#comment-17575072 ] Apache Spark commented on SPARK-39977: -- User 'pan3793' has created a pull request f

[jira] [Assigned] (SPARK-39977) Remove unnecessary guava exclusion from jackson-module-scala

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39977: Assignee: Apache Spark > Remove unnecessary guava exclusion from jackson-module-scala > -

[jira] [Commented] (SPARK-39977) Remove unnecessary guava exclusion from jackson-module-scala

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575073#comment-17575073 ] Apache Spark commented on SPARK-39977: -- User 'pan3793' has created a pull request f

[jira] [Assigned] (SPARK-39977) Remove unnecessary guava exclusion from jackson-module-scala

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39977: Assignee: (was: Apache Spark) > Remove unnecessary guava exclusion from jackson-modul

[jira] [Commented] (SPARK-39971) ANALYZE TABLE makes some queries run forever

2022-08-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575097#comment-17575097 ] Yuming Wang commented on SPARK-39971: - [~felipepessoto] Could you provide the query

[jira] [Commented] (SPARK-39921) SkewJoin--Stream side skew in BroadcastJoin

2022-08-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575108#comment-17575108 ] Yuming Wang commented on SPARK-39921: - +1. Would you like to file a PR? > SkewJoin-

[jira] [Resolved] (SPARK-39913) Upgrade Arrow to 9.0.0

2022-08-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-39913. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37331 [https://

[jira] [Assigned] (SPARK-39913) Upgrade Arrow to 9.0.0

2022-08-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-39913: - Assignee: Yang Jie > Upgrade Arrow to 9.0.0 > -- > >

[jira] [Commented] (SPARK-39921) SkewJoin--Stream side skew in BroadcastJoin

2022-08-04 Thread wang-zhun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575175#comment-17575175 ] wang-zhun commented on SPARK-39921: --- Yes, PR is in preparation > SkewJoin--Stream sid

[jira] [Commented] (SPARK-39976) NULL check in ArrayIntersect adds extraneous null from first param

2022-08-04 Thread zhiming she (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575177#comment-17575177 ] zhiming she commented on SPARK-39976: - I can reproduce this case , i will try to fix

[jira] [Updated] (SPARK-39978) Make filtered distinct count more accurate

2022-08-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39978: Attachment: data_dim_001.snappy.parquet > Make filtered distinct count more accurate > ---

[jira] [Created] (SPARK-39978) Make filtered distinct count more accurate

2022-08-04 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39978: --- Summary: Make filtered distinct count more accurate Key: SPARK-39978 URL: https://issues.apache.org/jira/browse/SPARK-39978 Project: Spark Issue Type: Improvem

[jira] [Assigned] (SPARK-39921) SkewJoin--Stream side skew in BroadcastJoin

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39921: Assignee: Apache Spark > SkewJoin--Stream side skew in BroadcastJoin > --

[jira] [Assigned] (SPARK-39921) SkewJoin--Stream side skew in BroadcastJoin

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39921: Assignee: (was: Apache Spark) > SkewJoin--Stream side skew in BroadcastJoin > ---

[jira] [Commented] (SPARK-39921) SkewJoin--Stream side skew in BroadcastJoin

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575213#comment-17575213 ] Apache Spark commented on SPARK-39921: -- User 'wang-zhun' has created a pull request

[jira] [Commented] (SPARK-39921) SkewJoin--Stream side skew in BroadcastJoin

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575214#comment-17575214 ] Apache Spark commented on SPARK-39921: -- User 'wang-zhun' has created a pull request

[jira] [Commented] (SPARK-39934) takeRDD in R is slow

2022-08-04 Thread deshanxiao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575221#comment-17575221 ] deshanxiao commented on SPARK-39934: [~hyukjin.kwon] Hi, Maybe there is something wr

[jira] [Created] (SPARK-39979) IndexOutOfBoundsException on groupby action and apply a pandas grouped map udf function

2022-08-04 Thread yaniv oren (Jira)
yaniv oren created SPARK-39979: -- Summary: IndexOutOfBoundsException on groupby action and apply a pandas grouped map udf function Key: SPARK-39979 URL: https://issues.apache.org/jira/browse/SPARK-39979 P

[jira] [Updated] (SPARK-39979) IndexOutOfBoundsException on groupby + apply pandas grouped map udf function

2022-08-04 Thread yaniv oren (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yaniv oren updated SPARK-39979: --- Summary: IndexOutOfBoundsException on groupby + apply pandas grouped map udf function (was: IndexOu

[jira] [Updated] (SPARK-39979) IndexOutOfBoundsException on groupby + apply pandas grouped map udf function

2022-08-04 Thread yaniv oren (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yaniv oren updated SPARK-39979: --- Description: I'm grouping on relatively small subset of groups with big size groups. Working with p

[jira] [Updated] (SPARK-39979) IndexOutOfBoundsException on groupby + apply pandas grouped map udf function

2022-08-04 Thread yaniv oren (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yaniv oren updated SPARK-39979: --- Description: I'm grouping on relatively small subset of groups with big size groups. Working with p

[jira] [Created] (SPARK-39980) Change infra image to static tag

2022-08-04 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-39980: --- Summary: Change infra image to static tag Key: SPARK-39980 URL: https://issues.apache.org/jira/browse/SPARK-39980 Project: Spark Issue Type: Bug Comp

[jira] [Commented] (SPARK-32952) Test failure on IBM Z: CoalesceShufflePartitionsSuite: - determining the number of reducers: complex query 1

2022-08-04 Thread Vivian Kong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575252#comment-17575252 ] Vivian Kong commented on SPARK-32952: - The test is still failing on Spark v3.3.0 on

[jira] [Commented] (SPARK-35520) Spark-SQL test fails on IBM Z for certain config combinations.

2022-08-04 Thread Vivian Kong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575254#comment-17575254 ] Vivian Kong commented on SPARK-35520: - The test is still failing on Spark v3.3.0 on

[jira] [Assigned] (SPARK-39980) Change infra image to static tag

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39980: Assignee: (was: Apache Spark) > Change infra image to static tag > --

[jira] [Assigned] (SPARK-39980) Change infra image to static tag

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39980: Assignee: Apache Spark > Change infra image to static tag > -

[jira] [Commented] (SPARK-39980) Change infra image to static tag

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575256#comment-17575256 ] Apache Spark commented on SPARK-39980: -- User 'Yikun' has created a pull request for

[jira] [Commented] (SPARK-37321) Wrong size estimation leads to "Cannot broadcast the table that is larger than 8GB: 8 GB"

2022-08-04 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-37321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575307#comment-17575307 ] Igor Uchôa commented on SPARK-37321: I'm facing the same situation too. You can see

[jira] [Commented] (SPARK-39876) Unpivot / melt function for SQL

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575311#comment-17575311 ] Apache Spark commented on SPARK-39876: -- User 'EnricoMi' has created a pull request

[jira] [Assigned] (SPARK-39876) Unpivot / melt function for SQL

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39876: Assignee: (was: Apache Spark) > Unpivot / melt function for SQL > ---

[jira] [Assigned] (SPARK-39876) Unpivot / melt function for SQL

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39876: Assignee: Apache Spark > Unpivot / melt function for SQL > --

[jira] [Created] (SPARK-39981) CheckOverflowInTableInsert returns exception rather than throwing it

2022-08-04 Thread Jason Darrell Lowe (Jira)
Jason Darrell Lowe created SPARK-39981: -- Summary: CheckOverflowInTableInsert returns exception rather than throwing it Key: SPARK-39981 URL: https://issues.apache.org/jira/browse/SPARK-39981 Proj

[jira] [Resolved] (SPARK-39872) HeapByteBuffer#get(int) is a hotspot path when using BytePackerForLong#unpack8Values with ByteBuffer input API

2022-08-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-39872. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37293 [https://

[jira] [Assigned] (SPARK-39872) HeapByteBuffer#get(int) is a hotspot path when using BytePackerForLong#unpack8Values with ByteBuffer input API

2022-08-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-39872: - Assignee: Yang Jie > HeapByteBuffer#get(int) is a hotspot path when using > BytePacker

[jira] [Resolved] (SPARK-39961) DS V2 push-down translate Cast if the cast is safe

2022-08-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-39961. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37388 [https://

[jira] [Updated] (SPARK-39971) ANALYZE TABLE makes some queries run forever

2022-08-04 Thread Felipe (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felipe updated SPARK-39971: --- Description: I'm using TPCDS to run benchmarks, and after running ANALYZE TABLE (without the FOR ALL COLUMN

[jira] [Updated] (SPARK-39971) ANALYZE TABLE makes some queries run forever

2022-08-04 Thread Felipe (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felipe updated SPARK-39971: --- Description: I'm using TPCDS to run benchmarks, and after running ANALYZE TABLE (without the FOR ALL COLUMN

[jira] [Assigned] (SPARK-39961) DS V2 push-down translate Cast if the cast is safe

2022-08-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-39961: - Assignee: jiaan.geng > DS V2 push-down translate Cast if the cast is safe > ---

[jira] [Updated] (SPARK-39971) ANALYZE TABLE makes some queries run forever

2022-08-04 Thread Felipe (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felipe updated SPARK-39971: --- Attachment: AfterAnalyzeTable WITHOUT ForAllColumns.txt AfterAnalyzeTableForAllColumns.txt

[jira] [Commented] (SPARK-39976) NULL check in ArrayIntersect adds extraneous null from first param

2022-08-04 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575398#comment-17575398 ] Thomas Graves commented on SPARK-39976: --- [~cloud_fan]  [~angerszhuuu]  who worked

[jira] [Updated] (SPARK-39971) ANALYZE TABLE makes some queries run forever

2022-08-04 Thread Felipe (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felipe updated SPARK-39971: --- Description: I'm using TPCDS to run benchmarks, and after running ANALYZE TABLE (without the FOR ALL COLUMN

[jira] [Updated] (SPARK-39976) NULL check in ArrayIntersect adds extraneous null from first param

2022-08-04 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-39976: -- Labels: corr (was: ) > NULL check in ArrayIntersect adds extraneous null from first param > -

[jira] [Updated] (SPARK-39976) NULL check in ArrayIntersect adds extraneous null from first param

2022-08-04 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-39976: -- Priority: Blocker (was: Major) > NULL check in ArrayIntersect adds extraneous null from first

[jira] [Updated] (SPARK-39976) NULL check in ArrayIntersect adds extraneous null from first param

2022-08-04 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-39976: -- Labels: (was: corr) > NULL check in ArrayIntersect adds extraneous null from first param > -

[jira] [Updated] (SPARK-39971) ANALYZE TABLE makes some queries run forever

2022-08-04 Thread Felipe (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felipe updated SPARK-39971: --- Attachment: AfterAnalyzeTableForAllColumns-joinreorder-disabled.txt > ANALYZE TABLE makes some queries run f

[jira] [Updated] (SPARK-39971) ANALYZE TABLE makes some queries run forever

2022-08-04 Thread Felipe (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felipe updated SPARK-39971: --- Attachment: AfterAnalyzeTableForAllColumns-joinreorder-disabled.txt > ANALYZE TABLE makes some queries run f

[jira] [Updated] (SPARK-39971) ANALYZE TABLE makes some queries run forever

2022-08-04 Thread Felipe (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felipe updated SPARK-39971: --- Attachment: (was: AfterAnalyzeTableForAllColumns-joinreorder-disabled.txt) > ANALYZE TABLE makes some q

[jira] [Created] (SPARK-39982) StructType.fromJson method missing documentation

2022-08-04 Thread Khalid Mammadov (Jira)
Khalid Mammadov created SPARK-39982: --- Summary: StructType.fromJson method missing documentation Key: SPARK-39982 URL: https://issues.apache.org/jira/browse/SPARK-39982 Project: Spark Issue

[jira] [Assigned] (SPARK-39982) StructType.fromJson method missing documentation

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39982: Assignee: Apache Spark > StructType.fromJson method missing documentation > -

[jira] [Assigned] (SPARK-39982) StructType.fromJson method missing documentation

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39982: Assignee: (was: Apache Spark) > StructType.fromJson method missing documentation > --

[jira] [Commented] (SPARK-39982) StructType.fromJson method missing documentation

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575405#comment-17575405 ] Apache Spark commented on SPARK-39982: -- User 'khalidmammadov' has created a pull re

[jira] [Commented] (SPARK-39982) StructType.fromJson method missing documentation

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575406#comment-17575406 ] Apache Spark commented on SPARK-39982: -- User 'khalidmammadov' has created a pull re

[jira] [Created] (SPARK-39983) Should not cache unserialized broadcast relations on the driver

2022-08-04 Thread Alex Balikov (Jira)
Alex Balikov created SPARK-39983: Summary: Should not cache unserialized broadcast relations on the driver Key: SPARK-39983 URL: https://issues.apache.org/jira/browse/SPARK-39983 Project: Spark

[jira] [Assigned] (SPARK-39970) Introduce ThrottledLogger to prevent log message flooding caused by network issues

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39970: Assignee: Apache Spark > Introduce ThrottledLogger to prevent log message flooding caused

[jira] [Commented] (SPARK-39970) Introduce ThrottledLogger to prevent log message flooding caused by network issues

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575466#comment-17575466 ] Apache Spark commented on SPARK-39970: -- User 'kevin85421' has created a pull reques

[jira] [Assigned] (SPARK-39970) Introduce ThrottledLogger to prevent log message flooding caused by network issues

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39970: Assignee: (was: Apache Spark) > Introduce ThrottledLogger to prevent log message floo

[jira] [Commented] (SPARK-39970) Introduce ThrottledLogger to prevent log message flooding caused by network issues

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575467#comment-17575467 ] Apache Spark commented on SPARK-39970: -- User 'kevin85421' has created a pull reques

[jira] [Created] (SPARK-39984) Check workerLastHeartbeat with master before HeartbeatReceiver expires an executor

2022-08-04 Thread Kai-Hsun Chen (Jira)
Kai-Hsun Chen created SPARK-39984: - Summary: Check workerLastHeartbeat with master before HeartbeatReceiver expires an executor Key: SPARK-39984 URL: https://issues.apache.org/jira/browse/SPARK-39984

[jira] [Resolved] (SPARK-39974) Create separate static image tag for infra cache

2022-08-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39974. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37402 [https://gi

[jira] [Assigned] (SPARK-39974) Create separate static image tag for infra cache

2022-08-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39974: Assignee: Yikun Jiang > Create separate static image tag for infra cache > --

[jira] [Created] (SPARK-39985) Test DEFAULT column values with DataFrames

2022-08-04 Thread Daniel (Jira)
Daniel created SPARK-39985: -- Summary: Test DEFAULT column values with DataFrames Key: SPARK-39985 URL: https://issues.apache.org/jira/browse/SPARK-39985 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-38493) Improve the test coverage for pyspark/pandas module

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575494#comment-17575494 ] Apache Spark commented on SPARK-38493: -- User 'itholic' has created a pull request f

[jira] [Created] (SPARK-39986) Better example for Co-grouped Map

2022-08-04 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-39986: Summary: Better example for Co-grouped Map Key: SPARK-39986 URL: https://issues.apache.org/jira/browse/SPARK-39986 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-39984) Check workerLastHeartbeat with master before HeartbeatReceiver expires an executor

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575500#comment-17575500 ] Apache Spark commented on SPARK-39984: -- User 'kevin85421' has created a pull reques

[jira] [Commented] (SPARK-39986) Better example for Co-grouped Map

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575501#comment-17575501 ] Apache Spark commented on SPARK-39986: -- User 'xinrong-meng' has created a pull requ

[jira] [Assigned] (SPARK-39986) Better example for Co-grouped Map

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39986: Assignee: Apache Spark > Better example for Co-grouped Map >

[jira] [Assigned] (SPARK-39984) Check workerLastHeartbeat with master before HeartbeatReceiver expires an executor

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39984: Assignee: (was: Apache Spark) > Check workerLastHeartbeat with master before Heartbea

[jira] [Commented] (SPARK-39984) Check workerLastHeartbeat with master before HeartbeatReceiver expires an executor

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575499#comment-17575499 ] Apache Spark commented on SPARK-39984: -- User 'kevin85421' has created a pull reques

[jira] [Assigned] (SPARK-39986) Better example for Co-grouped Map

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39986: Assignee: (was: Apache Spark) > Better example for Co-grouped Map > -

[jira] [Assigned] (SPARK-39984) Check workerLastHeartbeat with master before HeartbeatReceiver expires an executor

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39984: Assignee: Apache Spark > Check workerLastHeartbeat with master before HeartbeatReceiver e

[jira] [Commented] (SPARK-39934) takeRDD in R is slow

2022-08-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575502#comment-17575502 ] Hyukjin Kwon commented on SPARK-39934: -- [~deshanxiao] do you have a reproducer? Dat

[jira] [Commented] (SPARK-39983) Should not cache unserialized broadcast relations on the driver

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575505#comment-17575505 ] Apache Spark commented on SPARK-39983: -- User 'alex-balikov' has created a pull requ

[jira] [Assigned] (SPARK-39983) Should not cache unserialized broadcast relations on the driver

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39983: Assignee: (was: Apache Spark) > Should not cache unserialized broadcast relations on

[jira] [Commented] (SPARK-39983) Should not cache unserialized broadcast relations on the driver

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575506#comment-17575506 ] Apache Spark commented on SPARK-39983: -- User 'alex-balikov' has created a pull requ

[jira] [Assigned] (SPARK-39983) Should not cache unserialized broadcast relations on the driver

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39983: Assignee: Apache Spark > Should not cache unserialized broadcast relations on the driver

[jira] [Commented] (SPARK-39981) CheckOverflowInTableInsert returns exception rather than throwing it

2022-08-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575508#comment-17575508 ] Hyukjin Kwon commented on SPARK-39981: -- will make a quick fix soon. Thanks for repo

[jira] [Commented] (SPARK-39979) IndexOutOfBoundsException on groupby + apply pandas grouped map udf function

2022-08-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575509#comment-17575509 ] Hyukjin Kwon commented on SPARK-39979: -- This is from error limitation, it has to be

[jira] [Updated] (SPARK-39976) NULL check in ArrayIntersect adds extraneous null from first param

2022-08-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39976: - Priority: Major (was: Blocker) > NULL check in ArrayIntersect adds extraneous null from first p

[jira] [Commented] (SPARK-39833) Filtered parquet data frame count() and show() produce inconsistent results when spark.sql.parquet.filterPushdown is true

2022-08-04 Thread Ivan Sadikov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575510#comment-17575510 ] Ivan Sadikov commented on SPARK-39833: -- It appears to be a bug in Parquet-Mr.  The

[jira] [Comment Edited] (SPARK-39833) Filtered parquet data frame count() and show() produce inconsistent results when spark.sql.parquet.filterPushdown is true

2022-08-04 Thread Ivan Sadikov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575032#comment-17575032 ] Ivan Sadikov edited comment on SPARK-39833 at 8/5/22 1:48 AM:

[jira] [Assigned] (SPARK-39981) CheckOverflowInTableInsert returns exception rather than throwing it

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39981: Assignee: (was: Apache Spark) > CheckOverflowInTableInsert returns exception rather t

[jira] [Commented] (SPARK-39981) CheckOverflowInTableInsert returns exception rather than throwing it

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575514#comment-17575514 ] Apache Spark commented on SPARK-39981: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-39981) CheckOverflowInTableInsert returns exception rather than throwing it

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39981: Assignee: Apache Spark > CheckOverflowInTableInsert returns exception rather than throwin

[jira] [Updated] (SPARK-39971) ANALYZE TABLE makes some queries run forever

2022-08-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39971: - Component/s: SQL > ANALYZE TABLE makes some queries run forever > --

[jira] [Commented] (SPARK-39953) Hudi spark-submits from EMR 5.33 to EMR 6.5

2022-08-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575515#comment-17575515 ] Hyukjin Kwon commented on SPARK-39953: -- [~lavak] can you reproduce this in Apache S

[jira] [Updated] (SPARK-39971) ANALYZE TABLE makes some queries run forever

2022-08-04 Thread Felipe (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felipe updated SPARK-39971: --- Attachment: (was: AfterAnalyzeTableForAllColumns.txt) > ANALYZE TABLE makes some queries run forever > -

[jira] [Updated] (SPARK-39971) ANALYZE TABLE makes some queries run forever

2022-08-04 Thread Felipe (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felipe updated SPARK-39971: --- Attachment: (was: AfterAnalyzeTable WITHOUT ForAllColumns.txt) > ANALYZE TABLE makes some queries run fo

[jira] [Updated] (SPARK-39971) ANALYZE TABLE makes some queries run forever

2022-08-04 Thread Felipe (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felipe updated SPARK-39971: --- Attachment: (was: AfterAnalyzeTableForAllColumns-joinreorder-disabled.txt) > ANALYZE TABLE makes some q

[jira] [Updated] (SPARK-39971) ANALYZE TABLE makes some queries run forever

2022-08-04 Thread Felipe (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felipe updated SPARK-39971: --- Attachment: (was: BeforeAnalyzeTable.txt) > ANALYZE TABLE makes some queries run forever > -

[jira] [Resolved] (SPARK-39986) Better example for Co-grouped Map

2022-08-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39986. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37412 [https://gi

[jira] [Updated] (SPARK-39971) ANALYZE TABLE makes some queries run forever

2022-08-04 Thread Felipe (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felipe updated SPARK-39971: --- Attachment: 2.2.AfterAnalyzeTable WITHOUT ForAllColumns-joinreorder-enabled.txt 3.1.AfterAna

[jira] [Assigned] (SPARK-39986) Better example for Co-grouped Map

2022-08-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39986: Assignee: Xinrong Meng > Better example for Co-grouped Map >

[jira] [Commented] (SPARK-39971) ANALYZE TABLE makes some queries run forever

2022-08-04 Thread Felipe (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575538#comment-17575538 ] Felipe commented on SPARK-39971: [~yumwang] I added the query plans to all the scenarios

[jira] [Commented] (SPARK-39743) Unable to set zstd compression level while writing parquet files

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575542#comment-17575542 ] Apache Spark commented on SPARK-39743: -- User 'ming95' has created a pull reques

[jira] [Resolved] (SPARK-39775) Regression due to AVRO-2035

2022-08-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39775. - Fix Version/s: 3.3.1 3.2.3 3.4.0 Resolution: Fixed

[jira] [Assigned] (SPARK-39775) Regression due to AVRO-2035

2022-08-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39775: --- Assignee: Yuming Wang > Regression due to AVRO-2035 > --- > >

[jira] [Assigned] (SPARK-33782) Place spark.files, spark.jars and spark.files under the current working directory on the driver in K8S cluster mode

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33782: Assignee: Apache Spark > Place spark.files, spark.jars and spark.files under the current

[jira] [Assigned] (SPARK-33782) Place spark.files, spark.jars and spark.files under the current working directory on the driver in K8S cluster mode

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33782: Assignee: (was: Apache Spark) > Place spark.files, spark.jars and spark.files under t

[jira] [Commented] (SPARK-33782) Place spark.files, spark.jars and spark.files under the current working directory on the driver in K8S cluster mode

2022-08-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575552#comment-17575552 ] Apache Spark commented on SPARK-33782: -- User 'pralabhkumar' has created a pull requ

  1   2   >