zuston opened a new issue, #198:
URL: https://github.com/apache/incubator-uniffle/issues/198
I found some tasks of spark jobs will throw the exceptions that the
inconsistent blocks number. The stacktrace is as follows
```
22/09/03 15:29:21 ERROR Executor: Exception in task 330.0 in stage 9.0 (TID
59001)
org.apache.uniffle.common.exception.RssException: Blocks read inconsistent:
expected 30000 blocks, actual 15636 blocks
at
org.apache.uniffle.client.impl.ShuffleReadClientImpl.checkProcessedBlockIds(ShuffleReadClientImpl.java:215)
at
org.apache.spark.shuffle.reader.RssShuffleDataIterator.hasNext(RssShuffleDataIterator.java:135)
at
org.apache.spark.util.CompletionIterator.hasNext(CompletionIterator.scala:31)
```
I didn't find any error/warn log in shuffle server which stored the
corresponding partition data.
We dont set any replica config and directly use the MEMORY_LOCALFILE
storageType. Does this exception caused by the disk error?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@uniffle.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
