We haven't seen many of these, but we have seen it a couple of times --
there is ongoing work under SPARK-26089 to address the issue we know about,
namely that we don't detect corruption in large shuffle blocks.
Do you believe the cases you have match that -- does it appear to be
corruption in
I/We have seen this error before on 1.6 but ever since we upgraded to 2.1
two years ago we haven't seen it
On Tue, Mar 12, 2019 at 2:19 AM wangfei wrote:
> Hi all,
> Non-deterministic FAILED_TO_UNCOMPRESS(5) or ’Stream is corrupted’
> errors
> may occur during shuffle read, described as
Hi all,
Non-deterministic FAILED_TO_UNCOMPRESS(5) or ’Stream is corrupted’ errors
may occur during shuffle read, described as this
JIRA(https://issues.apache.org/jira/browse/SPARK-4105).
There is not new comment for a long time in this JIRA. So, Is there
anyone seen these errors in
I am seeing the same issue with Spark 1.3.1.
I see this issue when reading sequence file stored in Sequence File format
(SEQorg.apache.hadoop.io.Textorg.apache.hadoop.io.Text'org.apache.hadoop.io.compress.GzipCodec?v?
)
All i do is
sc.sequenceFile(dwTable, classOf[Text],
Hi,
Can you please share your compression etc settings, which you are using.
Thanks,
Twinkle
On Wed, May 6, 2015 at 4:15 PM, Jianshi Huang jianshi.hu...@gmail.com
wrote:
I'm facing this error in Spark 1.3.1
https://issues.apache.org/jira/browse/SPARK-4105
Anyone knows what's the
I'm using the default settings.
Jianshi
On Wed, May 6, 2015 at 7:05 PM, twinkle sachdeva twinkle.sachd...@gmail.com
wrote:
Hi,
Can you please share your compression etc settings, which you are using.
Thanks,
Twinkle
On Wed, May 6, 2015 at 4:15 PM, Jianshi Huang jianshi.hu...@gmail.com
