[jira] [Commented] (SPARK-43170) The spark sql like statement is pushed down to parquet for execution, but the data cannot be queried

[ https://issues.apache.org/jira/browse/SPARK-43170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17713855#comment-17713855 ] todd commented on SPARK-43170: -- [~yumwang]  The code only executes spark.sql("xxx"), but does not perform

[jira] [Commented] (SPARK-43170) The spark sql like statement is pushed down to parquet for execution, but the data cannot be queried

[ https://issues.apache.org/jira/browse/SPARK-43170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17713543#comment-17713543 ] todd commented on SPARK-43170: -- [~yumwang]  no cache > The spark sql like statement is pushed down to

[jira] [Commented] (SPARK-43170) The spark sql like statement is pushed down to parquet for execution, but the data cannot be queried

[ https://issues.apache.org/jira/browse/SPARK-43170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17713403#comment-17713403 ] todd commented on SPARK-43170: -- Spark3.2.x is currently used in production, and there is no plan to upgrade

[jira] [Updated] (SPARK-43170) The spark sql like statement is pushed down to parquet for execution, but the data cannot be queried

[ https://issues.apache.org/jira/browse/SPARK-43170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] todd updated SPARK-43170: - Description: --DDL CREATE TABLE `ecom_dwm`.`dwm_user_app_action_sum_all` (   `gaid` STRING COMMENT '',  

[jira] [Updated] (SPARK-43170) The spark sql like statement is pushed down to parquet for execution, but the data cannot be queried

[ https://issues.apache.org/jira/browse/SPARK-43170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] todd updated SPARK-43170: - Description: --DDL CREATE TABLE `ecom_dwm`.`dwm_user_app_action_sum_all` (   `gaid` STRING COMMENT '',  

[jira] [Updated] (SPARK-43170) The spark sql like statement is pushed down to parquet for execution, but the data cannot be queried

[ https://issues.apache.org/jira/browse/SPARK-43170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] todd updated SPARK-43170: - Attachment: image-2023-04-18-10-59-30-199.png > The spark sql like statement is pushed down to parquet for

[jira] [Created] (SPARK-43170) The spark sql like statement is pushed down to parquet for execution, but the data cannot be queried

todd created SPARK-43170: Summary: The spark sql like statement is pushed down to parquet for execution, but the data cannot be queried Key: SPARK-43170 URL: https://issues.apache.org/jira/browse/SPARK-43170

[jira] [Commented] (SPARK-40298) shuffle data recovery on the reused PVCs no effect

[ https://issues.apache.org/jira/browse/SPARK-40298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601225#comment-17601225 ] todd commented on SPARK-40298: -- [~dongjoon]  I use aws spot cluster to terminate the instance where the

[jira] (SPARK-35593) Support shuffle data recovery on the reused PVCs

[ https://issues.apache.org/jira/browse/SPARK-35593 ] todd deleted comment on SPARK-35593: -- was (Author: todd5167): [~dongjoon] [~apachespark]     Can you take a look at this question：  https://issues.apache.org/jira/browse/SPARK-40298 > Support

[jira] [Reopened] (SPARK-40298) shuffle data recovery on the reused PVCs no effect

[ https://issues.apache.org/jira/browse/SPARK-40298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] todd reopened SPARK-40298: -- > shuffle data recovery on the reused PVCs no effect > --- > >

[jira] [Updated] (SPARK-40298) shuffle data recovery on the reused PVCs no effect

[ https://issues.apache.org/jira/browse/SPARK-40298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] todd updated SPARK-40298: - Priority: Blocker (was: Major) > shuffle data recovery on the reused PVCs no effect >

[jira] [Commented] (SPARK-35593) Support shuffle data recovery on the reused PVCs

[ https://issues.apache.org/jira/browse/SPARK-35593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17598758#comment-17598758 ] todd commented on SPARK-35593: -- [~dongjoon] [~apachespark]     Can you take a look at this question： 

[jira] [Updated] (SPARK-40298) shuffle data recovery on the reused PVCs no effect

[ https://issues.apache.org/jira/browse/SPARK-40298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] todd updated SPARK-40298: - Description: I use spark3.2.2 to test the [ Support shuffle data recovery on the reused PVCs (SPARK-35593) ]

[jira] [Updated] (SPARK-40298) shuffle data recovery on the reused PVCs no effect

[ https://issues.apache.org/jira/browse/SPARK-40298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] todd updated SPARK-40298: - Attachment: 1662002808396.jpg 1662002822097.jpg > shuffle data recovery on the reused PVCs no

[jira] [Created] (SPARK-40298) shuffle data recovery on the reused PVCs no effect

todd created SPARK-40298: Summary: shuffle data recovery on the reused PVCs no effect Key: SPARK-40298 URL: https://issues.apache.org/jira/browse/SPARK-40298 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17397898#comment-17397898 ] Cameron Todd commented on SPARK-18105: -- Oh sorry, I meant just a portion of the code can be

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17397603#comment-17397603 ] Cameron Todd commented on SPARK-18105: -- Good to hear. Also the count of 136,935,074 is right. 

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17397301#comment-17397301 ] Cameron Todd commented on SPARK-18105: -- Ok I added the zip file on this public S3 bucket, it holds

[jira] [Comment Edited] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395953#comment-17395953 ] Cameron Todd edited comment on SPARK-18105 at 8/9/21, 10:31 AM: Yep I

[jira] [Comment Edited] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395953#comment-17395953 ] Cameron Todd edited comment on SPARK-18105 at 8/9/21, 10:29 AM: Yep I

[jira] [Issue Comment Deleted] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cameron Todd updated SPARK-18105: - Comment: was deleted (was: [^hashed_data.zip]) > LZ4 failed to decompress a stream of shuffled

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395954#comment-17395954 ] Cameron Todd commented on SPARK-18105: -- [^hashed_data.zip] > LZ4 failed to decompress a stream of

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395953#comment-17395953 ] Cameron Todd commented on SPARK-18105: -- Yep I understand. I have attached my hashed data keeping

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17392971#comment-17392971 ] Cameron Todd commented on SPARK-18105: -- Let me know if that's enough info. From my tests if I

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17392968#comment-17392968 ] Cameron Todd commented on SPARK-18105: -- I'll attach a portion of the code that is not proprietary

[jira] [Updated] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cameron Todd updated SPARK-18105: - Attachment: TestWeightedGraph.java > LZ4 failed to decompress a stream of shuffled data >

[jira] [Comment Edited] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17392205#comment-17392205 ] Cameron Todd edited comment on SPARK-18105 at 8/3/21, 9:44 AM: --- I'm also

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17392205#comment-17392205 ] Cameron Todd commented on SPARK-18105: -- I'm also facing this same error when scaling up my project