[ https://issues.apache.org/jira/browse/SPARK-43221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Qiang Yang updated SPARK-43221: ------------------------------- Attachment: image-2023-04-21-00-19-58-021.png > Executor obtained error information > ------------------------------------ > > Key: SPARK-43221 > URL: https://issues.apache.org/jira/browse/SPARK-43221 > Project: Spark > Issue Type: Bug > Components: Block Manager > Affects Versions: 3.1.1, 3.2.0, 3.3.0 > Reporter: Qiang Yang > Priority: Major > Attachments: image-2023-04-21-00-19-58-021.png > > Original Estimate: 24h > Remaining Estimate: 24h > > Spark on Yarn Cluster > When multiple executors exist on a node, and the same block exists on both > executors, with some in memory and some on disk. > Probabilistically, the executor failed to obtain the block,throw Exception: > java.lang.ArrayIndexOutofBoundsException: 0 > at > org.apache.spark.broadcast.TorrentBroadcast.$anonfun$readBlocks$1(TorrentBroadcast.scala:183) > > Next, I will replay the process of the problem occurring: > step 1: > The executor requests the driver to obtain block > information(locationsAndStatusOption). The input parameters are BlockId and > the host of its own node. Please note that it does not carry port information > code: !image-2023-04-21-00-19-58-021.png! > step 2: > -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org