[
https://issues.apache.org/jira/browse/SOLR-15473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17362407#comment-17362407
]
Robert Muir commented on SOLR-15473:
------------------------------------
Here's the writeup I sent to the dev list. I think there are two separate
issues here:
1. some solr tests don't obey their sandbox and fail with tests.workDir (if it
is set in the user's build.properties). These tests try to access wrong parts
of the filesystem which can cause tests to meddle with each other. obeying the
test sandbox (tests.workDir) is important, it is how I prevent these tests from
destroying my SSDs.
2. some solr HDFS tests will falsely fail if they "think" disk space is low
(even when it is not running out). They dump megabytes of output, but this part
is the key:
{noformat}
[junit4] 2> 1000960 WARN (IPC Server handler 3 on 33951) [ ]
o.a.h.h.s.b.BlockPlacementPolicy Failed to place enough replicas, still in need
of 2 to reach 2 (unavailableStorages=[],
storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK],
creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) For more
information, please enable DEBUG log level on
org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy and
org.apache.hadoop.net.NetworkTopology
[junit4] 2> 1000960 WARN (IPC Server handler 3 on 33951) [ ]
o.a.h.h.p.BlockStoragePolicy Failed to place enough replicas: expected size is
2 but only 0 storage types can be selected (replication=2, selected=[],
unavailable=[DISK], removed=[DISK, DISK], policy=BlockStoragePolicy{HOT:7,
storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]})
[junit4] 2> 1000960 WARN (IPC Server handler 3 on 33951) [ ]
o.a.h.h.s.b.BlockPlacementPolicy Failed to place enough replicas, still in need
of 2 to reach 2 (unavailableStorages=[DISK],
storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK],
creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) All
required storage types are unavailable: unavailableStorages=[DISK],
storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK],
creationFallbacks=[], replicationFallbacks=[ARCHIVE]}
[junit4] 2> 1000961 WARN (Thread-2642) [ ] o.a.h.h.DataStreamer
DataStreamer Exception
[junit4] 2> =>
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /testfile
could only be written to 0 of the 1 minReplication nodes. There are 2
datanode(s) running and 2 node(s) are excluded in this operation.
{noformat}
So I think these tests should be tweaked to not require gigabytes of free space
to pass. (fix the threshold or whatever, or add an assume or something). I
worked around the situation by temporarily repartitioning and giving them
another gigabyte (!). In no event was there ever any danger of running out of
space! They just falsely fail even when there are hundreds of MB available.
Seems they have some kind of bogus threshold in the algorithm (e.g. inspecting
percentages or something).
> solr tests fail from smoketester
> --------------------------------
>
> Key: SOLR-15473
> URL: https://issues.apache.org/jira/browse/SOLR-15473
> Project: Solr
> Issue Type: Task
> Security Level: Public(Default Security Level. Issues are Public)
> Affects Versions: 8.9
> Reporter: Robert Muir
> Priority: Blocker
> Attachments: test.log.gz, test2.log.gz
>
>
> Whenever I run the solr tests on my linux thinkpad, they fail.
> I see stuff like this:
> {noformat}
> [junit4] Tests with failures [seed: FB8461777F8A9E7D] (first 10 out of 26):
> [junit4] -
> org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore
> (suite)
> [junit4] - org.apache.solr.update.TestHdfsUpdateLog (suite)
> [junit4] - org.apache.solr.store.hdfs.HdfsDirectoryTest (suite)
> [junit4] - org.apache.solr.cloud.SolrCLIZkUtilsTest.testDownconfig
> [junit4] - org.apache.solr.cloud.SolrCLIZkUtilsTest.testLs
> [junit4] - org.apache.solr.cloud.SolrCLIZkUtilsTest.testRm
> [junit4] - org.apache.solr.cloud.SolrCLIZkUtilsTest.testCp
> [junit4] - org.apache.solr.cloud.SolrCLIZkUtilsTest.testMv
> [junit4] - org.apache.solr.cloud.SolrCLIZkUtilsTest.testUpconfig
> [junit4] -
> org.apache.solr.cloud.api.collections.HdfsCloudIncrementalBackupTest
> (suite)
> {noformat}
> It creates a multi-megabyte amount of logs that floods my scrollback buffer.
> But as the smoketester tees the output to a specific file, I will upload it
> here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
