[
https://issues.apache.org/jira/browse/SPARK-32691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214516#comment-17214516
]
zhengruifeng commented on SPARK-32691:
--------------------------------------
[~huangtianhua] It looks like that this is an already existing issue before
SPARK-32517, which is triggered by changing #executors. ({{val clusterUrl =
"local-cluster[2,1,1024]"}} -> {{val clusterUrl = "local-cluster[3,1,1024]"}}).
in the attached failure.log, block {{rdd_0_0}} was added three time, but it
should be added only twice:
{code:java}
...
20/08/31 20:39:21.361 dispatcher-BlockManagerMaster INFO BlockManagerInfo:
Added rdd_0_0 in memory on 192.168.1.225:36305 (size: 416.0 B, free: 546.3 MiB)
...
20/08/31 20:41:21.844 dispatcher-BlockManagerMaster INFO BlockManagerInfo:
Added rdd_0_0 in memory on 192.168.1.225:35957 (size: 416.0 B, free: 546.3 MiB)
...
20/08/31 20:41:27.771 dispatcher-BlockManagerMaster INFO BlockManagerInfo:
Added rdd_0_0 in memory on 192.168.1.225:34623 (size: 416.0 B, free: 546.3 MiB)
...
{code}
IIUC, the result of {{testCaching}} in {{DistributedSuite}} should be
irrelevant to the number of executor? [~dongjoon]
> Test org.apache.spark.DistributedSuite failed on arm64 jenkins
> --------------------------------------------------------------
>
> Key: SPARK-32691
> URL: https://issues.apache.org/jira/browse/SPARK-32691
> Project: Spark
> Issue Type: Bug
> Components: Spark Core, Tests
> Affects Versions: 3.1.0
> Environment: ARM64
> Reporter: huangtianhua
> Priority: Major
> Attachments: Screen Shot 2020-09-28 at 8.49.04 AM.png, failure.log,
> success.log
>
>
> Tests of org.apache.spark.DistributedSuite are failed on arm64 jenkins:
> https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-arm/
> - caching in memory and disk, replicated (encryption = on) (with
> replication as stream) *** FAILED ***
> 3 did not equal 2; got 3 replicas instead of 2 (DistributedSuite.scala:191)
> - caching in memory and disk, serialized, replicated (encryption = on)
> (with replication as stream) *** FAILED ***
> 3 did not equal 2; got 3 replicas instead of 2 (DistributedSuite.scala:191)
> - caching in memory, serialized, replicated (encryption = on) (with
> replication as stream) *** FAILED ***
> 3 did not equal 2; got 3 replicas instead of 2 (DistributedSuite.scala:191)
> .......
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
