Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/598/

6 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.MultiThreadedOCPTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
        at __randomizedtesting.SeedInfo.seed([936281279AE7F682]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.MultiThreadedOCPTest

Error Message:
Captured an uncaught exception in thread: Thread[id=328, name=qtp149694054-328, 
state=RUNNABLE, group=TGRP-MultiThreadedOCPTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=328, name=qtp149694054-328, state=RUNNABLE, 
group=TGRP-MultiThreadedOCPTest]
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
        at __randomizedtesting.SeedInfo.seed([936281279AE7F682]:0)
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:714)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1047)
        at 
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
        at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
        at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
        at 
org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:665)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.MultiThreadedOCPTest

Error Message:
Captured an uncaught exception in thread: Thread[id=232, 
name=qtp1627420012-232, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=232, name=qtp1627420012-232, state=RUNNABLE, 
group=TGRP-MultiThreadedOCPTest]
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
        at __randomizedtesting.SeedInfo.seed([936281279AE7F682]:0)
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:714)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1047)
        at 
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
        at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
        at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
        at 
org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:665)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  
junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.handler.TestReplicationHandlerBackup:     1) Thread[id=3209, 
name=Thread-1278, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]      
   at java.net.PlainSocketImpl.socketConnect(Native Method)         at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)    
     at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
         at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)      
   at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)         at 
java.net.Socket.connect(Socket.java:579)         at 
java.net.Socket.connect(Socket.java:528)         at 
sun.net.NetworkClient.doConnect(NetworkClient.java:180)         at 
sun.net.www.http.HttpClient.openServer(HttpClient.java:432)         at 
sun.net.www.http.HttpClient.openServer(HttpClient.java:527)         at 
sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)         at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
         at java.net.URL.openStream(URL.java:1037)         at 
org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.handler.TestReplicationHandlerBackup: 
   1) Thread[id=3209, name=Thread-1278, state=RUNNABLE, 
group=TGRP-TestReplicationHandlerBackup]
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:579)
        at java.net.Socket.connect(Socket.java:528)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
        at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
        at java.net.URL.openStream(URL.java:1037)
        at 
org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)
        at __randomizedtesting.SeedInfo.seed([936281279AE7F682]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

Error Message:
There are still zombie threads that couldn't be terminated:    1) 
Thread[id=3209, name=Thread-1278, state=RUNNABLE, 
group=TGRP-TestReplicationHandlerBackup]         at 
java.net.PlainSocketImpl.socketConnect(Native Method)         at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)    
     at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
         at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)      
   at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)         at 
java.net.Socket.connect(Socket.java:579)         at 
java.net.Socket.connect(Socket.java:528)         at 
sun.net.NetworkClient.doConnect(NetworkClient.java:180)         at 
sun.net.www.http.HttpClient.openServer(HttpClient.java:432)         at 
sun.net.www.http.HttpClient.openServer(HttpClient.java:527)         at 
sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)         at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
         at java.net.URL.openStream(URL.java:1037)         at 
org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=3209, name=Thread-1278, state=RUNNABLE, 
group=TGRP-TestReplicationHandlerBackup]
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:579)
        at java.net.Socket.connect(Socket.java:528)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
        at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
        at java.net.URL.openStream(URL.java:1037)
        at 
org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)
        at __randomizedtesting.SeedInfo.seed([936281279AE7F682]:0)


FAILED:  org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
        at __randomizedtesting.SeedInfo.seed([936281279AE7F682]:0)




Build Log:
[...truncated 11528 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandlerBackup
   [junit4]   2> Creating dataDir: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/init-core-data-001
   [junit4]   2> 1720889 T3189 oas.SolrTestCaseJ4.setUp ###Starting doTestBackup
   [junit4]   2> 1720907 T3189 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1720910 T3189 oejs.AbstractConnector.doStart Started 
SocketConnector@127.0.0.1:60236
   [junit4]   2> 1720911 T3189 oass.SolrDispatchFilter.init 
SolrDispatchFilter.init()
   [junit4]   2> 1720911 T3189 oasc.SolrResourceLoader.locateSolrHome JNDI not 
configured for solr (NoInitialContextEx)
   [junit4]   2> 1720911 T3189 oasc.SolrResourceLoader.locateSolrHome using 
system property solr.solr.home: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001
   [junit4]   2> 1720912 T3189 oasc.SolrResourceLoader.<init> new 
SolrResourceLoader for directory: 
'/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/'
   [junit4]   2> 1720956 T3189 oasc.ConfigSolr.fromFile Loading container 
configuration from 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/solr.xml
   [junit4]   2> 1721034 T3189 oasc.CoreContainer.<init> New CoreContainer 
1708505120
   [junit4]   2> 1721034 T3189 oasc.CoreContainer.load Loading cores into 
CoreContainer 
[instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/]
   [junit4]   2> 1721035 T3189 oashc.HttpShardHandlerFactory.getParameter 
Setting socketTimeout to: 90000
   [junit4]   2> 1721036 T3189 oashc.HttpShardHandlerFactory.getParameter 
Setting urlScheme to: 
   [junit4]   2> 1721036 T3189 oashc.HttpShardHandlerFactory.getParameter 
Setting connTimeout to: 15000
   [junit4]   2> 1721036 T3189 oashc.HttpShardHandlerFactory.getParameter 
Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1721037 T3189 oashc.HttpShardHandlerFactory.getParameter 
Setting corePoolSize to: 0
   [junit4]   2> 1721037 T3189 oashc.HttpShardHandlerFactory.getParameter 
Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1721037 T3189 oashc.HttpShardHandlerFactory.getParameter 
Setting maxThreadIdleTime to: 5
   [junit4]   2> 1721038 T3189 oashc.HttpShardHandlerFactory.getParameter 
Setting sizeOfQueue to: -1
   [junit4]   2> 1721038 T3189 oashc.HttpShardHandlerFactory.getParameter 
Setting fairnessPolicy to: false
   [junit4]   2> 1721039 T3189 oasu.UpdateShardHandler.<init> Creating 
UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 1721039 T3189 oasl.LogWatcher.createWatcher SLF4J impl is 
org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1721039 T3189 oasl.LogWatcher.newRegisteredLogWatcher 
Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1721040 T3189 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1721045 T3199 oasc.SolrResourceLoader.<init> new 
SolrResourceLoader for directory: 
'/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/'
   [junit4]   2> 1721088 T3199 oasc.SolrConfig.<init> Using Lucene 
MatchVersion: LUCENE_5_0
   [junit4]   2> 1721126 T3199 oasc.SolrConfig.<init> Loaded SolrConfig: 
solrconfig.xml
   [junit4]   2> 1721127 T3199 oass.IndexSchema.readSchema Reading Solr Schema 
from schema.xml
   [junit4]   2> 1721134 T3199 oass.IndexSchema.readSchema [collection1] Schema 
name=test
   [junit4]   2> 1721150 T3199 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1721151 T3199 oasc.CoreContainer.create Creating SolrCore 
'collection1' using configuration from instancedir 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/
   [junit4]   2> 1721152 T3199 oasc.SolrCore.initDirectoryFactory 
org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1721152 T3199 oasc.SolrCore.<init> [collection1] Opening new 
SolrCore at 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/,
 
dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data/
   [junit4]   2> 1721152 T3199 oasc.SolrCore.<init> JMX monitoring not detected 
for core: collection1
   [junit4]   2> 1721154 T3199 oasc.CachingDirectoryFactory.get return new 
directory for 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data
   [junit4]   2> 1721155 T3199 oasc.SolrCore.getNewIndexDir New index directory 
detected: old=null 
new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data/index/
   [junit4]   2> 1721155 T3199 oasc.SolrCore.initIndex WARN [collection1] Solr 
index directory 
'/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data/index'
 doesn't exist. Creating new index...
   [junit4]   2> 1721156 T3199 oasc.CachingDirectoryFactory.get return new 
directory for 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data/index
   [junit4]   2> 1721156 T3199 oasu.RandomMergePolicy.<init> RandomMergePolicy 
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=10, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=71.3056640625, 
floorSegmentMB=0.4873046875, forceMergeDeletesPctAllowed=15.706223013408033, 
segmentsPerTier=32.0, maxCFSSegmentSizeMB=0.7490234375, noCFSRatio=0.0
   [junit4]   2> 1721157 T3199 oasc.SolrDeletionPolicy.onCommit 
SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2>                
commit{dir=MockDirectoryWrapper(RAMDirectory@8eba254 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@71968fe0),segFN=segments_1,generation=1}
   [junit4]   2> 1721157 T3199 oasc.SolrDeletionPolicy.updateCommits newest 
commit generation = 1
   [junit4]   2> 1721159 T3199 oasc.SolrCore.loadUpdateProcessorChains no 
updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1721159 T3199 oasc.RequestHandlers.initHandlersFromConfig 
created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1721159 T3199 oasc.RequestHandlers.initHandlersFromConfig 
created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1721160 T3199 oasc.RequestHandlers.initHandlersFromConfig 
created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1721160 T3199 oasc.RequestHandlers.initHandlersFromConfig 
created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1721160 T3199 oasc.RequestHandlers.initHandlersFromConfig 
created standard: solr.StandardRequestHandler
   [junit4]   2> 1721160 T3199 oasc.RequestHandlers.initHandlersFromConfig 
created /replication: solr.ReplicationHandler
   [junit4]   2> 1721161 T3199 oasc.RequestHandlers.initHandlersFromConfig WARN 
Multiple requestHandler registered to the same name: /update ignoring: 
org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1721161 T3199 oasc.RequestHandlers.initHandlersFromConfig 
created /update: solr.UpdateRequestHandler
   [junit4]   2> 1721180 T3199 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1721182 T3199 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1721183 T3199 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1721185 T3199 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1721187 T3199 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1721189 T3199 oasu.CommitTracker.<init> Hard AutoCommit: 
disabled
   [junit4]   2> 1721190 T3199 oasu.CommitTracker.<init> Soft AutoCommit: 
disabled
   [junit4]   2> 1721190 T3199 oasu.RandomMergePolicy.<init> RandomMergePolicy 
wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: 
minMergeSize=1000, mergeFactor=19, maxMergeSize=9223372036854775807, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.6265002443195115]
   [junit4]   2> 1721191 T3199 oasc.SolrDeletionPolicy.onInit 
SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2>                
commit{dir=MockDirectoryWrapper(RAMDirectory@8eba254 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@71968fe0),segFN=segments_1,generation=1}
   [junit4]   2> 1721191 T3199 oasc.SolrDeletionPolicy.updateCommits newest 
commit generation = 1
   [junit4]   2> 1721191 T3199 oass.SolrIndexSearcher.<init> Opening 
Searcher@59f64efa[collection1] main
   [junit4]   2> 1721192 T3199 
oasr.ManagedResourceStorage$FileStorageIO.configure File-based storage 
initialized to use dir: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/conf
   [junit4]   2> 1721192 T3199 oasr.RestManager.init Initializing RestManager 
with initArgs: 
{storageDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/conf}
   [junit4]   2> 1721192 T3199 oasr.ManagedResourceStorage.load Reading 
_rest_managed.json using 
file:dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/conf
   [junit4]   2> 1721193 T3199 oasr.ManagedResource.reloadFromStorage WARN No 
stored data found for /rest/managed
   [junit4]   2> 1721194 T3199 oasr.ManagedResourceStorage$JsonStorage.store 
Saved JSON object to path _rest_managed.json using 
file:dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/conf
   [junit4]   2> 1721194 T3199 oasr.RestManager.init Initializing 0 registered 
ManagedResources
   [junit4]   2> 1721194 T3199 oash.ReplicationHandler.inform Replication 
enabled for following config files: schema-replication2.xml:schema.xml
   [junit4]   2> 1721195 T3199 oash.ReplicationHandler.inform Commits will be 
reserved for  10000
   [junit4]   2> 1721195 T3200 oasc.SolrCore.registerSearcher [collection1] 
Registered new searcher Searcher@59f64efa[collection1] 
main{UninvertingDirectoryReader()}
   [junit4]   2> 1721195 T3199 oasc.CoreContainer.registerCore registering 
core: collection1
   [junit4]   2> 1721196 T3189 oass.SolrDispatchFilter.init 
user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
   [junit4]   2> 1721196 T3189 oass.SolrDispatchFilter.init 
SolrDispatchFilter.init() done
   [junit4]   2> ASYNC  NEW_CORE C1234 name=collection1 
org.apache.solr.core.SolrCore@7d1d22be
   [junit4]   2> 1721198 T3192 C1234 oasu.DirectUpdateHandler2.deleteAll 
[collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 1721199 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{deleteByQuery=*:*} 0 1
   [junit4]   2> 1721202 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[0]} 
0 0
   [junit4]   2> 1721205 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1]} 
0 0
   [junit4]   2> 1721207 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[2]} 
0 0
   [junit4]   2> 1721209 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[3]} 
0 0
   [junit4]   2> 1721211 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[4]} 
0 0
   [junit4]   2> 1721213 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[5]} 
0 0
   [junit4]   2> 1721215 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[6]} 
0 0
   [junit4]   2> 1721217 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[7]} 
0 0
   [junit4]   2> 1721219 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[8]} 
0 0
   [junit4]   2> 1721221 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[9]} 
0 0
   [junit4]   2> 1721223 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[10]} 0 0
   [junit4]   2> 1721225 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[11]} 0 0
   [junit4]   2> 1721227 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[12]} 0 0
   [junit4]   2> 1721229 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[13]} 0 0
   [junit4]   2> 1721231 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[14]} 0 0
   [junit4]   2> 1721233 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[15]} 0 0
   [junit4]   2> 1721235 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[16]} 0 0
   [junit4]   2> 1721237 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[17]} 0 0
   [junit4]   2> 1721239 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[18]} 0 0
   [junit4]   2> 1721241 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[19]} 0 0
   [junit4]   2> 1721243 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[20]} 0 0
   [junit4]   2> 1721245 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[21]} 0 0
   [junit4]   2> 1721247 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[22]} 0 0
   [junit4]   2> 1721249 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[23]} 0 0
   [junit4]   2> 1721251 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[24]} 0 0
   [junit4]   2> 1721253 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[25]} 0 0
   [junit4]   2> 1721255 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[26]} 0 0
   [junit4]   2> 1721257 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[27]} 0 0
   [junit4]   2> 1721259 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[28]} 0 0
   [junit4]   2> 1721261 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[29]} 0 0
   [junit4]   2> 1721263 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[30]} 0 0
   [junit4]   2> 1721265 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[31]} 0 0
   [junit4]   2> 1721267 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[32]} 0 0
   [junit4]   2> 1721269 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[33]} 0 0
   [junit4]   2> 1721271 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[34]} 0 0
   [junit4]   2> 1721273 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[35]} 0 0
   [junit4]   2> 1721275 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[36]} 0 0
   [junit4]   2> 1721277 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[37]} 0 0
   [junit4]   2> 1721279 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[38]} 0 0
   [junit4]   2> 1721281 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[39]} 0 0
   [junit4]   2> 1721283 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[40]} 0 0
   [junit4]   2> 1721285 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[41]} 0 0
   [junit4]   2> 1721287 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[42]} 0 0
   [junit4]   2> 1721289 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[43]} 0 0
   [junit4]   2> 1721291 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[44]} 0 0
   [junit4]   2> 1721293 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[45]} 0 0
   [junit4]   2> 1721295 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[46]} 0 0
   [junit4]   2> 1721297 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[47]} 0 0
   [junit4]   2> 1721299 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[48]} 0 0
   [junit4]   2> 1721301 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[49]} 0 0
   [junit4]   2> 1721303 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[50]} 0 0
   [junit4]   2> 1721305 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[51]} 0 0
   [junit4]   2> 1721307 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[52]} 0 0
   [junit4]   2> 1721309 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[53]} 0 0
   [junit4]   2> 1721311 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[54]} 0 0
   [junit4]   2> 1721313 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[55]} 0 0
   [junit4]   2> 1721315 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[56]} 0 0
   [junit4]   2> 1721317 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[57]} 0 0
   [junit4]   2> 1721319 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[58]} 0 0
   [junit4]   2> 1721321 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[59]} 0 0
   [junit4]   2> 1721323 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[60]} 0 0
   [junit4]   2> 1721325 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[61]} 0 0
   [junit4]   2> 1721327 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[62]} 0 0
   [junit4]   2> 1721329 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[63]} 0 0
   [junit4]   2> 1721331 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[64]} 0 0
   [junit4]   2> 1721333 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[65]} 0 0
   [junit4]   2> 1721335 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[66]} 0 0
   [junit4]   2> 1721337 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[67]} 0 0
   [junit4]   2> 1721339 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[68]} 0 0
   [junit4]   2> 1721341 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[69]} 0 0
   [junit4]   2> 1721343 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[70]} 0 0
   [junit4]   2> 1721345 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[71]} 0 0
   [junit4]   2> 1721347 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[72]} 0 0
   [junit4]   2> 1721349 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[73]} 0 0
   [junit4]   2> 1721351 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[74]} 0 0
   [junit4]   2> 1721353 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[75]} 0 0
   [junit4]   2> 1721355 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[76]} 0 0
   [junit4]   2> 1721357 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[77]} 0 0
   [junit4]   2> 1721359 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[78]} 0 0
   [junit4]   2> 1721361 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[79]} 0 0
   [junit4]   2> 1721363 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[80]} 0 0
   [junit4]   2> 1721365 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[81]} 0 0
   [junit4]   2> 1721367 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[82]} 0 0
   [junit4]   2> 1721369 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update params={wt=javabin&version=2} 
{add=[83]} 0 0
   [junit4]   2> 1721371 T3192 C1234 oasu.DirectUpdateHandler2.commit start 
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1721373 T3192 C1234 oasc.SolrDeletionPolicy.onCommit 
SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2>                
commit{dir=MockDirectoryWrapper(RAMDirectory@8eba254 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@71968fe0),segFN=segments_1,generation=1}
   [junit4]   2>                
commit{dir=MockDirectoryWrapper(RAMDirectory@8eba254 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@71968fe0),segFN=segments_2,generation=2}
   [junit4]   2> 1721374 T3192 C1234 oasc.SolrDeletionPolicy.updateCommits 
newest commit generation = 2
   [junit4]   2> 1721376 T3192 C1234 oass.SolrIndexSearcher.<init> Opening 
Searcher@7effa51f[collection1] main
   [junit4]   2> 1721377 T3192 C1234 oasu.DirectUpdateHandler2.commit 
end_commit_flush
   [junit4]   2> 1721377 T3200 oasc.SolrCore.registerSearcher [collection1] 
Registered new searcher Searcher@7effa51f[collection1] 
main{UninvertingDirectoryReader(Uninverting(_0(5.0):c84))}
   [junit4]   2> 1721377 T3192 C1234 oasup.LogUpdateProcessor.finish 
[collection1] webapp=/solr path=/update 
params={waitSearcher=true&softCommit=false&commit=true&wt=javabin&version=2} 
{commit=} 0 6
   [junit4]   2> 1721382 T3194 C1234 oasc.SolrCore.execute [collection1] 
webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 1721395 T3193 C1234 oasc.SolrCore.execute [collection1] 
webapp=/solr path=/replication params={name=vdookyxjtgmxqws&command=backup} 
status=0 QTime=14 
   [junit4]   2> 1721395 T3204 oash.SnapShooter.createSnapshot Creating backup 
snapshot...
   [junit4]   2> 1721408 T3204 oash.SnapShooter.createSnapshot Done creating 
backup snapshot: vdookyxjtgmxqws
   [junit4]   2> 1721584 T3193 C1234 oasc.SolrCore.execute [collection1] 
webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 1722593 T3193 C1234 oasc.SolrCore.execute [collection1] 
webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 1722594 T3194 C1234 oasc.SolrCore.execute [collection1] 
webapp=/solr path=/replication params={name=ribyuvugqsowycnhiq&command=backup} 
status=0 QTime=2 
   [junit4]   2> 1722594 T3206 oash.SnapShooter.createSnapshot Creating backup 
snapshot...
   [junit4]   2> 1722602 T3206 oash.SnapShooter.createSnapshot Done creating 
backup snapshot: ribyuvugqsowycnhiq
   [junit4]   2> 1722795 T3194 C1234 oasc.SolrCore.execute [collection1] 
webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> ASYNC  NEW_CORE C1235 name=collection1 
org.apache.solr.core.SolrCore@7d1d22be
   [junit4]   2> 1722809 T3193 C1235 oasc.SolrCore.execute [collection1] 
webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 1722814 T3194 C1235 oasc.SolrCore.execute [collection1] 
webapp=/solr path=/replication 
params={name=vdookyxjtgmxqws&command=deletebackup} status=0 QTime=6 
   [junit4]   2> 1722814 T3208 oash.SnapShooter.deleteNamedSnapshot Deleting 
snapshot: vdookyxjtgmxqws
   [junit4]   2> 1723011 T3194 C1235 oasc.SolrCore.execute [collection1] 
webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 1723013 T3194 C1235 oasc.SolrCore.execute [collection1] 
webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 1723013 T3189 oas.SolrTestCaseJ4.tearDown ###Ending 
doTestBackup
   [junit4]   2> 1723013 T3210 oash.SnapShooter.deleteNamedSnapshot Deleting 
snapshot: ribyuvugqsowycnhiq
   [junit4]   2> 1723013 T3193 C1235 oasc.SolrCore.execute [collection1] 
webapp=/solr path=/replication 
params={name=ribyuvugqsowycnhiq&command=deletebackup} status=0 QTime=0 
   [junit4]   2> 1723014 T3189 oasc.CoreContainer.shutdown Shutting down 
CoreContainer instance=1708505120
   [junit4]   2> 1723015 T3189 oejsh.ContextHandler.doStop stopped 
o.e.j.s.ServletContextHandler{/solr,null}
   [junit4]   2> 1723016 T3193 C1235 oasc.SolrException.log ERROR 
null:org.eclipse.jetty.io.EofException
   [junit4]   2>                at 
org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:142)
   [junit4]   2>                at 
org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:107)
   [junit4]   2>                at 
sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
   [junit4]   2>                at 
sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
   [junit4]   2>                at 
sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
   [junit4]   2>                at 
sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
   [junit4]   2>                at 
java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
   [junit4]   2>                at 
org.apache.solr.util.FastWriter.flush(FastWriter.java:137)
   [junit4]   2>                at 
org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:769)
   [junit4]   2>                at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:428)
   [junit4]   2>                at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2>                at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
   [junit4]   2>                at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
   [junit4]   2>                at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
   [junit4]   2>                at 
org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
   [junit4]   2>                at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
   [junit4]   2>                at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
   [junit4]   2>                at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
   [junit4]   2>                at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
   [junit4]   2>                at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
   [junit4]   2>                at 
org.eclipse.jetty.server.Server.handle(Server.java:368)
   [junit4]   2>                at 
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
   [junit4]   2>                at 
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
   [junit4]   2>                at 
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
   [junit4]   2>                at 
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
   [junit4]   2>                at 
org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
   [junit4]   2>                at 
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
   [junit4]   2>                at 
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
   [junit4]   2>                at 
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>        
   [junit4]   2> 1723017 T3193 C1235 oasc.SolrException.log ERROR 
null:org.eclipse.jetty.io.EofException
   [junit4]   2>                at 
org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:142)
   [junit4]   2>                at 
org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:107)
   [junit4]   2>                at 
sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
   [junit4]   2>                at 
sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
   [junit4]   2>                at 
sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
   [junit4]   2>                at 
sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
   [junit4]   2>                at 
java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
   [junit4]   2>                at 
org.apache.solr.util.FastWriter.flush(FastWriter.java:137)
   [junit4]   2>                at 
org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:769)
   [junit4]   2>                at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:428)
   [junit4]   2>                at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2>                at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
   [junit4]   2>                at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
   [junit4]   2>                at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
   [junit4]   2>                at 
org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
   [junit4]   2>                at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
   [junit4]   2>                at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
   [junit4]   2>                at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
   [junit4]   2>                at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
   [junit4]   2>                at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
   [junit4]   2>                at 
org.eclipse.jetty.server.Server.handle(Server.java:368)
   [junit4]   2>                at 
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
   [junit4]   2>                at 
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
   [junit4]   2>                at 
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
   [junit4]   2>                at 
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
   [junit4]   2>                at 
org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
   [junit4]   2>                at 
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
   [junit4]   2>                at 
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
   [junit4]   2>                at 
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>        
   [junit4]   2> 1723019 T3193 C1235 oasc.SolrCore.close [collection1]  CLOSING 
SolrCore org.apache.solr.core.SolrCore@7d1d22be
   [junit4]   2> 1723019 T3193 C1235 oasu.DirectUpdateHandler2.close closing 
DirectUpdateHandler2{commits=1,autocommits=0,soft 
autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=84,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
   [junit4]   2> 1723020 T3193 C1235 oasu.SolrCoreState.decrefSolrCoreState 
Closing SolrCoreState
   [junit4]   2> 1723020 T3193 C1235 oasu.DefaultSolrCoreState.closeIndexWriter 
SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 1723020 T3193 C1235 oasu.DefaultSolrCoreState.closeIndexWriter 
closing IndexWriter with IndexWriterCloser
   [junit4]   2> 1723021 T3193 C1235 oasc.SolrCore.closeSearcher [collection1] 
Closing main searcher on request.
   [junit4]   2> 1723021 T3193 C1235 oasc.CachingDirectoryFactory.close Closing 
MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 1723021 T3193 C1235 
oasc.CachingDirectoryFactory.closeCacheValue looking to close 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data/index
 
[CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data/index;done=false>>]
   [junit4]   2> 1723022 T3193 C1235 oasc.CachingDirectoryFactory.close Closing 
directory: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data/index
   [junit4]   2> 1723022 T3193 C1235 
oasc.CachingDirectoryFactory.closeCacheValue looking to close 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data
 
[CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data;done=false>>]
   [junit4]   2> 1723022 T3193 C1235 oasc.CachingDirectoryFactory.close Closing 
directory: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data
   [junit4]   2> 1728026 T3189 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 7147 T3188 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will 
linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> 27233 T3188 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 1 
thread leaked from SUITE scope at 
org.apache.solr.handler.TestReplicationHandlerBackup: 
   [junit4]   2>           1) Thread[id=3209, name=Thread-1278, state=RUNNABLE, 
group=TGRP-TestReplicationHandlerBackup]
   [junit4]   2>                at 
java.net.PlainSocketImpl.socketConnect(Native Method)
   [junit4]   2>                at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
   [junit4]   2>                at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
   [junit4]   2>                at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
   [junit4]   2>                at 
java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
   [junit4]   2>                at java.net.Socket.connect(Socket.java:579)
   [junit4]   2>                at java.net.Socket.connect(Socket.java:528)
   [junit4]   2>                at 
sun.net.NetworkClient.doConnect(NetworkClient.java:180)
   [junit4]   2>                at 
sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
   [junit4]   2>                at 
sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
   [junit4]   2>                at 
sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
   [junit4]   2>                at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
   [junit4]   2>                at java.net.URL.openStream(URL.java:1037)
   [junit4]   2>                at 
org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)
   [junit4]   2> 27234 T3188 ccr.ThreadLeakControl.tryToInterruptAll Starting 
to interrupt leaked threads:
   [junit4]   2>           1) Thread[id=3209, name=Thread-1278, state=RUNNABLE, 
group=TGRP-TestReplicationHandlerBackup]
   [junit4]   2> 30240 T3188 ccr.ThreadLeakControl.tryToInterruptAll SEVERE 
There are still zombie threads that couldn't be terminated:
   [junit4]   2>           1) Thread[id=3209, name=Thread-1278, state=RUNNABLE, 
group=TGRP-TestReplicationHandlerBackup]
   [junit4]   2>                at 
java.net.PlainSocketImpl.socketConnect(Native Method)
   [junit4]   2>                at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
   [junit4]   2>                at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
   [junit4]   2>                at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
   [junit4]   2>                at 
java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
   [junit4]   2>                at java.net.Socket.connect(Socket.java:579)
   [junit4]   2>                at java.net.Socket.connect(Socket.java:528)
   [junit4]   2>                at 
sun.net.NetworkClient.doConnect(NetworkClient.java:180)
   [junit4]   2>                at 
sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
   [junit4]   2>                at 
sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
   [junit4]   2>                at 
sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
   [junit4]   2>                at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
   [junit4]   2>                at java.net.URL.openStream(URL.java:1037)
   [junit4]   2>                at 
org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)
   [junit4]   2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, 
locale=hu, timezone=Asia/Kashgar
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 
(64-bit)/cpus=16,threads=2,free=219235216,total=311427072
   [junit4]   2> NOTE: All tests run in this JVM: [TestCloudInspectUtil, 
SpellPossibilityIteratorTest, TestJmxIntegration, FastVectorHighlighterTest, 
XsltUpdateRequestHandlerTest, MigrateRouteKeyTest, 
TestDefaultSimilarityFactory, TestRandomDVFaceting, TermsComponentTest, 
BadIndexSchemaTest, SolrXmlInZkTest, BlockCacheTest, PeerSyncTest, 
HdfsBasicDistributedZk2Test, TestManagedSynonymFilterFactory, ZkControllerTest, 
ExternalFileFieldSortTest, SpellCheckComponentTest, 
ParsingFieldUpdateProcessorsTest, SuggestComponentTest, 
ChaosMonkeyNothingIsSafeTest, TestRangeQuery, TestIntervalFaceting, 
DistributedDebugComponentTest, TestSchemaSimilarityResource, 
TestDFRSimilarityFactory, TestClassNameShortening, TestArbitraryIndexDir, 
TestDynamicFieldResource, TestFieldResource, TestDocBasedVersionConstraints, 
SolrCoreCheckLockOnStartupTest, SolrCmdDistributorTest, 
TestCopyFieldCollectionResource, DirectSolrSpellCheckerTest, ZkNodePropsTest, 
ClusterStateUpdateTest, PreAnalyzedFieldTest, TestSolrJ, HighlighterConfigTest, 
TestReplicationHandler, IndexSchemaTest, TestSearcherReuse, TestCodecSupport, 
BasicFunctionalityTest, ZkCLITest, StatelessScriptUpdateProcessorFactoryTest, 
PreAnalyzedUpdateProcessorTest, TestReloadAndDeleteDocs, EchoParamsTest, 
OverseerCollectionProcessorTest, TestSolrXmlPersistence, HighlighterTest, 
AnalysisErrorHandlingTest, LeaderElectionTest, TestSchemaVersionResource, 
TestSolrCoreProperties, TestSolrDeletionPolicy1, WordBreakSolrSpellCheckerTest, 
FileUtilsTest, TestTolerantSearch, ResponseLogComponentTest, 
TestStressRecovery, FunctionTest, TestHighlightDedupGrouping, 
TestSolrDeletionPolicy2, QueryEqualityTest, TestReplicationHandlerBackup]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 
'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestReplicationHandlerBackup -Dtests.seed=936281279AE7F682 
-Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
-Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt 
-Dtests.locale=hu -Dtests.timezone=Asia/Kashgar -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J0 | TestReplicationHandlerBackup (suite) <<<
   [junit4]    > Throwable #1: 
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.handler.TestReplicationHandlerBackup: 
   [junit4]    >    1) Thread[id=3209, name=Thread-1278, state=RUNNABLE, 
group=TGRP-TestReplicationHandlerBackup]
   [junit4]    >         at java.net.PlainSocketImpl.socketConnect(Native 
Method)
   [junit4]    >         at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
   [junit4]    >         at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
   [junit4]    >         at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
   [junit4]    >         at 
java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
   [junit4]    >         at java.net.Socket.connect(Socket.java:579)
   [junit4]    >         at java.net.Socket.connect(Socket.java:528)
   [junit4]    >         at 
sun.net.NetworkClient.doConnect(NetworkClient.java:180)
   [junit4]    >         at 
sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
   [junit4]    >         at 
sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
   [junit4]    >         at 
sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
   [junit4]    >         at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
   [junit4]    >         at java.net.URL.openStream(URL.java:1037)
   [junit4]    >         at 
org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([936281279AE7F682]:0)Throwable #2: 
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=3209, name=Thread-1278, state=RUNNABLE, 
group=TGRP-TestReplicationHandlerBackup]
   [junit4]    >         at java.net.PlainSocketImpl.socketConnect(Native 
Method)
   [junit4]    >         at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
   [junit4]    >         at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
   [junit4]    >         at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
   [junit4]    >         at 
java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
   [junit4]    >         at java.net.Socket.connect(Socket.java:579)
   [junit4]    >         at java.net.Socket.connect(Socket.java:528)
   [junit4]    >         at 
sun.net.NetworkClient.doConnect(NetworkClient.java:180)
   [junit4]    >         at 
sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
   [junit4]    >         at 
sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
   [junit4]    >         at 
sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
   [junit4]    >         at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
   [junit4]    >         at java.net.URL.openStream(URL.java:1037)
   [junit4]    >         at 
org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([936281279AE7F682]:0)
   [junit4] Completed on J0 in 30.28s, 1 test, 2 errors <<< FAILURES!

[...truncated 818 lines...]
   [junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest
   [junit4]   2> log4j:WARN No such property [conversionPattern] in 
org.apache.solr.util.SolrLogLayout.
   [junit4]   2> Creating dataDir: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/init-core-data-001
   [junit4]   2> 3859 T11 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl 
(true) and clientAuth (true)
   [junit4]   2> 3962 T11 oas.BaseDistributedSearchTestCase.initHostContext 
Setting hostContext system property: /
   [junit4]   2> 4017 T11 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 4026 T11 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 4035 T12 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting 
server
   [junit4]   2> 4336 T11 oasc.ZkTestServer.run start zk server on port:61636
   [junit4]   2> 4613 T11 oascc.ConnectionManager.waitForConnected Waiting for 
client to connect to ZooKeeper
   [junit4]   2> 4790 T18 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@6f9bc97 name:ZooKeeperConnection 
Watcher:127.0.0.1:61636 got event WatchedEvent state:SyncConnected type:None 
path:null path:null type:None
   [junit4]   2> 4792 T11 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 4799 T11 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 4926 T11 oascc.ConnectionManager.waitForConnected Waiting for 
client to connect to ZooKeeper
   [junit4]   2> 4929 T20 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@7a429aa2 
name:ZooKeeperConnection Watcher:127.0.0.1:61636/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4930 T11 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 4955 T11 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 4960 T11 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/shards
   [junit4]   2> 4965 T11 oascc.SolrZkClient.makePath makePath: 
/collections/control_collection
   [junit4]   2> 4970 T11 oascc.SolrZkClient.makePath makePath: 
/collections/control_collection/shards
   [junit4]   2> 4976 T11 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 5046 T11 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/solrconfig.xml
   [junit4]   2> 5053 T11 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 5054 T11 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/schema.xml
   [junit4]   2> 5059 T11 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 5061 T11 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 5066 T11 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 5067 T11 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/stopwords.txt
   [junit4]   2> 5072 T11 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 5074 T11 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/protwords.txt
   [junit4]   2> 5078 T11 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 5080 T11 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/currency.xml
   [junit4]   2> 5084 T11 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 5086 T11 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/enumsConfig.xml
   [junit4]   2> 5091 T11 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 5092 T11 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/open-exchange-rates.json
   [junit4]   2> 5097 T11 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 5099 T11 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 5103 T11 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 5105 T11 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/old_synonyms.txt
   [junit4]   2> 5109 T11 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 5111 T11 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/synonyms.txt
   [junit4]   2> 5129 T11 oascc.ConnectionManager.waitForConnected Waiting for 
client to connect to ZooKeeper
   [junit4]   2> 5132 T22 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@302c28cc 
name:ZooKeeperConnection Watcher:127.0.0.1:61636/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 5133 T11 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 8789 T11 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 9359 T11 oejus.SslContextFactory.doStart Enabled Protocols 
[SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, 
TLSv1.1, TLSv1.2]
   [junit4]   2> 9407 T11 oejs.AbstractConnector.doStart Started 
SslSocketConnector@127.0.0.1:32014
   [junit4]   2> 10281 T11 oass.SolrDispatchFilter.init 
SolrDispatchFilter.init()
   [junit4]   2> 10437 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not 
configured for solr (NoInitialContextEx)
   [junit4]   2> 10438 T11 oasc.SolrResourceLoader.locateSolrHome using system 
property solr.solr.home: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-002
   [junit4]   2> 10439 T11 oasc.SolrResourceLoader.<init> new 
SolrResourceLoader for directory: 
'/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-002/'
   [junit4]   2> 11442 T11 oasc.ConfigSolr.fromFile Loading container 
configuration from 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-002/solr.xml
   [junit4]   2> 11751 T11 oasc.CoreContainer.<init> New CoreContainer 788197590
   [junit4]   2> 11759 T11 oasc.CoreContainer.load Loading cores into 
CoreContainer 
[instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-002/]
   [junit4]   2> 11911 T11 oashc.HttpShardHandlerFactory.getParameter Setting 
socketTimeout to: 90000
   [junit4]   2> 11912 T11 oashc.HttpShardHandlerFactory.getParameter Setting 
urlScheme to: 
   [junit4]   2> 11912 T11 oashc.HttpShardHandlerFactory.getParameter Setting 
connTimeout to: 15000
   [junit4]   2> 11913 T11 oashc.HttpShardHandlerFactory.getParameter Setting 
maxConnectionsPerHost to: 20
   [junit4]   2> 11914 T11 oashc.HttpShardHandlerFactory.getParameter Setting 
corePoolSize to: 0
   [junit4]   2> 11914 T11 oashc.HttpShardHandlerFactory.getParameter Setting 
maximumPoolSize to: 2147483647
   [junit4]   2> 11915 T11 oashc.HttpShardHandlerFactory.getParameter Setting 
maxThreadIdleTime to: 5
   [junit4]   2> 11915 T11 oashc.HttpShardHandlerFactory.getParameter Setting 
sizeOfQueue to: -1
   [junit4]   2> 11916 T11 oashc.HttpShardHandlerFactory.getParameter Setting 
fairnessPolicy to: false
   [junit4]   2> 12066 T11 oasu.UpdateShardHandler.<init> Creating 
UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 12091 T11 oasl.LogWatcher.createWatcher SLF4J impl is 
org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 12098 T11 oasl.LogWatcher.newRegisteredLogWatcher Registering 
Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 12108 T11 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 12109 T11 oasc.ZkContainer.initZooKeeper Zookeeper 
client=127.0.0.1:61636/solr
   [junit4]   2> 12153 T11 oasc.ZkController.checkChrootPath zkHost includes 
chroot
   [junit4]   2> 12154 T11 oascc.ConnectionManager.waitForConnected Waiting for 
client to connect to ZooKeeper
   [junit4]   2> 12159 T34 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@77eea508 
name:ZooKeeperConnection Watcher:127.0.0.1:61636 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 12160 T11 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 12178 T11 oascc.ConnectionManager.waitForConnected Waiting for 
client to connect to ZooKeeper
   [junit4]   2> 12180 T36 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@67b8e751 
name:ZooKeeperConnection Watcher:127.0.0.1:61636/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 12181 T11 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 12204 T11 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 12220 T11 oascc.SolrZkClient.makePath makePath: 
/overseer/collection-queue-work
   [junit4]   2> 12230 T11 oascc.SolrZkClient.makePath makePath: 
/overseer/collection-map-running
   [junit4]   2> 12235 T11 oascc.SolrZkClient.makePath makePath: 
/overseer/collection-map-completed
   [junit4]   2> 12241 T11 oascc.SolrZkClient.makePath makePath: 
/overseer/collection-map-failure
   [junit4]   2> 12284 T11 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 12287 T11 oasc.ZkController.createEphemeralLiveNode Register 
node as live in ZooKeeper:/live_nodes/127.0.0.1:32014_
   [junit4]   2> 12295 T11 oascc.SolrZkClient.makePath makePath: 
/live_nodes/127.0.0.1:32014_
   [junit4]   2> 12306 T11 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 12310 T11 oascc.SolrZkClient.makePath makePath: 
/overseer_elect/election
   [junit4]   2> 12332 T11 oasc.OverseerElectionContext.runLeaderProcess I am 
going to be the leader 127.0.0.1:32014_
   [junit4]   2> 12333 T11 oascc.SolrZkClient.makePath makePath: 
/overseer_elect/leader
   [junit4]   2> 12337 T11 oasc.Overseer.start Overseer 
(id=92228166816432132-127.0.0.1:32014_-n_0000000000) starting
   [junit4]   2> 12363 T11 oascc.SolrZkClient.makePath makePath: 
/overseer/queue-work
   [junit4]   2> 12469 T38 oasc.OverseerCollectionProcessor.run Process current 
queue of collection creations
   [junit4]   2> 12469 T11 oascc.SolrZkClient.makePath makePath: 
/clusterstate.json
   [junit4]   2> 12473 T11 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 12475 T11 
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state 
from ZooKeeper... 
   [junit4]   2> 12562 T37 oasc.Overseer$ClusterStateUpdater.run Starting to 
work on the main queue
   [junit4]   2> 12906 T39 oasc.ZkController.publish publishing 
core=collection1 state=down collection=control_collection
   [junit4]   2> 12906 T39 oasc.ZkController.publish numShards not found on 
descriptor - reading it from system property
   [junit4]   2> 12919 T39 oasc.ZkController.waitForCoreNodeName look for our 
core node name
   [junit4]   2> 12928 T40 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 12939 T37 oascc.ZkStateReader.updateClusterState Updating 
cloud state from ZooKeeper... 
   [junit4]   2> 12941 T37 oasc.Overseer$ClusterStateUpdater.updateState Update 
state numShards=1 message={
   [junit4]   2>          "operation":"state",
   [junit4]   2>          "state":"down",
   [junit4]   2>          "base_url":"https://127.0.0.1:32014";,
   [junit4]   2>          "core":"collection1",
   [junit4]   2>          "roles":null,
   [junit4]   2>          "node_name":"127.0.0.1:32014_",
   [junit4]   2>          "shard":null,
   [junit4]   2>          "collection":"control_collection",
   [junit4]   2>          "numShards":"1",
   [junit4]   2>          "core_node_name":null}
   [junit4]   2> 12941 T37 oasc.Overseer$ClusterStateUpdater.createCollection 
Create collection control_collection with shards [shard1]
   [junit4]   2> 13003 T37 oasc.Overseer$ClusterStateUpdater.updateState 
Assigning new node to shard shard=shard1
   [junit4]   2> 13008 T40 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 13030 T40 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 13921 T39 oasc.ZkController.waitForShardId waiting to find 
shard id in clusterstate for collection1
   [junit4]   2> 13921 T39 oasc.ZkController.createCollectionZkNode Check for 
collection zkNode:control_collection
   [junit4]   2> 13923 T39 oasc.ZkController.createCollectionZkNode Collection 
zkNode exists
   [junit4]   2> 13924 T39 oascc.ZkStateReader.readConfigName Load collection 
config from:/collections/control_collection
   [junit4]   2> 13926 T39 oascc.ZkStateReader.readConfigName 
path=/collections/control_collection configName=conf1 specified config exists 
in ZooKeeper
   [junit4]   2> 13927 T39 oasc.SolrResourceLoader.<init> new 
SolrResourceLoader for directory: 
'/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-002/collection1/'
   [junit4]   2> 13930 T39 oasc.SolrResourceLoader.replaceClassLoader Adding 
'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-002/collection1/lib/classes/'
 to classloader
   [junit4]   2> 13932 T39 oasc.SolrResourceLoader.replaceClassLoader Adding 
'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-002/collection1/lib/README'
 to classloader
   [junit4]   2> 13933 T39 oasc.SolrResourceLoader.replaceClassLoader Adding 
'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-002/collection1/lib/.svn/'
 to classloader
   [junit4]   2> 14282 T39 oasc.SolrConfig.<init> Using Lucene MatchVersion: 
LUCENE_5_0
   [junit4]   2> 14706 T39 oasc.SolrConfig.<init> Loaded SolrConfig: 
solrconfig.xml
   [junit4]   2> 14770 T39 oass.IndexSchema.readSchema Reading Solr Schema from 
schema.xml
   [junit4]   2> 14973 T39 oass.IndexSchema.readSchema [collection1] Schema 
name=test
   [junit4]   2> 17235 T39 oass.OpenExchangeRatesOrgProvider.init Initialized 
with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 17290 T39 oass.IndexSchema.readSchema default search field in 
schema is text
   [junit4]   2> 17293 T39 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 18101 T39 oass.FileExchangeRateProvider.reload Reloading 
exchange rates from file currency.xml
   [junit4]   2> 18126 T39 oass.FileExchangeRateProvider.reload Reloading 
exchange rates from file currency.xml
   [junit4]   2> 18132 T39 oass.OpenExchangeRatesOrgProvider.reload Reloading 
exchange rates from open-exchange-rates.json
   [junit4]   2> 18138 T39 
oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key 
IMPORTANT NOTE
   [junit4]   2> 18139 T39 
oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, 
got STRING
   [junit4]   2> 18139 T39 oass.OpenExchangeRatesOrgProvider.reload Reloading 
exchange rates from open-exchange-rates.json
   [junit4]   2> 18141 T39 
oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key 
IMPORTANT NOTE
   [junit4]   2> 18142 T39 
oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, 
got STRING
   [junit4]   2> 18146 T39 oasc.CoreContainer.create Creating SolrCore 
'collection1' using configuration from collection control_collection
   [junit4]   2> 18265 T39 oasc.SolrCore.initDirectoryFactory 
org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 18302 T39 oasc.SolrCore.<init> [collection1] Opening new 
SolrCore at 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-002/collection1/,
 
dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-001/control/data/
   [junit4]   2> 18311 T39 oasc.JmxMonitoredMap.<init> JMX monitoring is 
enabled. Adding Solr mbeans to JMX Server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@5d2c041a
   [junit4]   2> 18527 T39 oasc.CachingDirectoryFactory.get return new 
directory for 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-001/control/data
   [junit4]   2> 18527 T39 oasc.SolrCore.getNewIndexDir New index directory 
detected: old=null 
new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-001/control/data/index/
   [junit4]   2> 18528 T39 oasc.SolrCore.initIndex WARN [collection1] Solr 
index directory 
'/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-001/control/data/index'
 doesn't exist. Creating new index...
   [junit4]   2> 18560 T39 oasc.CachingDirectoryFactory.get return new 
directory for 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-001/control/data/index
   [junit4]   2> 18563 T39 oasu.RandomMergePolicy.<init> RandomMergePolicy 
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=23, maxMergeAtOnceExplicit=37, maxMergedSegmentMB=20.2890625, 
floorSegmentMB=1.1396484375, forceMergeDeletesPctAllowed=22.888894824493903, 
segmentsPerTier=43.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.3218127037386259
   [junit4]   2> 18763 T39 oasc.SolrDeletionPolicy.onCommit 
SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2>                
commit{dir=MockDirectoryWrapper(RAMDirectory@3ffd1bed 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@19292ff8),segFN=segments_1,generation=1}
   [junit4]   2> 18767 T39 oasc.SolrDeletionPolicy.updateCommits newest commit 
generation = 1
   [junit4]   2> 19708 T39 oasup.UpdateRequestProcessorChain.init creating 
updateRequestProcessorChain "nodistrib"
   [junit4]   2> 19877 T39 oasup.UpdateRequestProcessorChain.init creating 
updateRequestProcessorChain "dedupe"
   [junit4]   2> 19897 T39 oasup.UpdateRequestProcessorChain.init inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 19897 T39 oasup.UpdateRequestProcessorChain.init creating 
updateRequestProcessorChain "stored_sig"
   [junit4]   2> 19898 T39 oasup.UpdateRequestProcessorChain.init inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 19898 T39 oasup.UpdateRequestProcessorChain.init creating 
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 20069 T39 oasup.UpdateRequestProcessorChain.init creating 
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 20070 T39 oasup.UpdateRequestProcessorChain.init inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain 
"distrib-dup-test-chain-implicit"
   [junit4]   2> 20070 T39 oasc.SolrCore.loadUpdateProcessorChains no 
updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 20105 T39 oasc.RequestHandlers.initHandlersFromConfig created 
/update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 20106 T39 oasc.RequestHandlers.initHandlersFromConfig created 
/update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 20107 T39 oasc.RequestHandlers.initHandlersFromConfig created 
/update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 20109 T39 oasc.RequestHandlers.initHandlersFromConfig created 
/update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 20149 T39 oasc.RequestHandlers.initHandlersFromConfig created 
standard: solr.StandardRequestHandler
   [junit4]   2> 20180 T39 oasc.RequestHandlers.initHandlersFromConfig created 
/get: solr.RealTimeGetHandler
   [junit4]   2> 20180 T39 oasc.RequestHandlers.initHandlersFromConfig adding 
lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 20191 T39 oasc.RequestHandlers.initHandlersFromConfig created 
/replication: solr.ReplicationHandler
   [junit4]   2> 20219 T39 oasc.RequestHandlers.initHandlersFromConfig WARN 
Multiple requestHandler registered to the same name: /update ignoring: 
org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 20220 T39 oasc.RequestHandlers.initHandlersFromConfig created 
/update: solr.UpdateRequestHandler
   [junit4]   2> 20225 T39 oasc.RequestHandlers.initHandlersFromConfig created 
/admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 20265 T39 oasc.RequestHandlers.initHandlersFromConfig created 
/admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 20303 T39 oasc.RequestHandlers.initHandlersFromConfig created 
/admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 20474 T39 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 20502 T39 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 20504 T39 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 20506 T39 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 20508 T39 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 20626 T39 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 20627 T39 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 20629 T39 oasu.RandomMergePolicy.<init> RandomMergePolicy 
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=26, maxMergeAtOnceExplicit=13, maxMergedSegmentMB=21.779296875, 
floorSegmentMB=1.29296875, forceMergeDeletesPctAllowed=25.258018587109554, 
segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.6165322401803253
   [junit4]   2> 20645 T39 oasc.SolrDeletionPolicy.onInit 
SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2>                
commit{dir=MockDirectoryWrapper(RAMDirectory@3ffd1bed 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@19292ff8),segFN=segments_1,generation=1}
   [junit4]   2> 20645 T39 oasc.SolrDeletionPolicy.updateCommits newest commit 
generation = 1
   [junit4]   2> 20741 T39 oass.SolrIndexSearcher.<init> Opening 
Searcher@57944a64[collection1] main
   [junit4]   2> 20790 T39 oascc.ZkStateReader.readConfigName Load collection 
config from:/collections/control_collection
   [junit4]   2> 20792 T39 oascc.ZkStateReader.readConfigName 
path=/collections/control_collection configName=conf1 specified config exists 
in ZooKeeper
   [junit4]   2> 20792 T39 oasr.ManagedResourceStorage.newStorageIO Setting up 
ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 20808 T39 
oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured 
ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 20809 T39 oasr.RestManager.init Initializing RestManager with 
initArgs: {}
   [junit4]   2> 20935 T39 oasr.ManagedResourceStorage.load Reading 
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 20936 T39 
oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found 
for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 20937 T39 oasr.ManagedResourceStorage.load Loaded null at path 
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 20938 T39 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/_rest_managed.json
   [junit4]   2> 20941 T39 
oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new 
znode /configs/conf1/_rest_managed.json
   [junit4]   2> 20942 T39 oasr.ManagedResourceStorage$JsonStorage.store Saved 
JSON object to path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 20942 T39 oasr.RestManager.init Initializing 0 registered 
ManagedResources
   [junit4]   2> 21177 T41 oasc.SolrCore.registerSearcher [collection1] 
Registered new searcher Searcher@57944a64[collection1] 
main{UninvertingDirectoryReader()}
   [junit4]   2> 21192 T39 oasc.CoreContainer.registerCore registering core: 
collection1
   [junit4]   2> 21199 T44 oasc.ZkController.register Register replica - 
core:collection1 address:https://127.0.0.1:32014 collection:control_collection 
shard:shard1
   [junit4]   2> 21200 T11 oass.SolrDispatchFilter.init 
user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 21201 T11 oass.SolrDispatchFilter.init 
SolrDispatchFilter.init() done
   [junit4]   2> 21234 T44 oascc.SolrZkClient.makePath makePath: 
/collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 21241 T44 oasc.ShardLeaderElectionContext.runLeaderProcess 
Running the leader process for shard shard1
   [junit4]   2> 21244 T40 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 21244 T44 
oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found 
to continue.
   [junit4]   2> 21245 T44 oasc.ShardLeaderElectionContext.runLeaderProcess I 
may be the new leader - try and sync
   [junit4]   2> 21246 T37 oascc.ZkStateReader.updateClusterState Updating 
cloud state from ZooKeeper... 
   [junit4]   2> 21258 T40 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 21259 T40 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> ASYNC  NEW_CORE C0 name=collection1 
org.apache.solr.core.SolrCore@2154cbb6 url=https://127.0.0.1:32014/collection1 
node=127.0.0.1:32014_ C0_STATE=coll:control_collection core:collection1 
props:{state=down, base_url=https://127.0.0.1:32014, core=collection1, 
node_name=127.0.0.1:32014_}
   [junit4]   2> 21259 T44 C0 P32014 oasc.SyncStrategy.sync Sync replicas to 
https://127.0.0.1:32014/collection1/
   [junit4]   2> 21266 T44 C0 P32014 oasc.SyncStrategy.syncReplicas Sync 
Success - now sync replicas to me
   [junit4]   2> 21266 T44 C0 P32014 oasc.SyncStrategy.syncToMe 
https://127.0.0.1:32014/collection1/ has no replicas
   [junit4]   2> 21267 T44 oasc.ShardLeaderElectionContext.runLeaderProcess I 
am the new leader: https://127.0.0.1:32014/collection1/ shard1
   [junit4]   2> 21278 T44 oascc.SolrZkClient.makePath makePath: 
/collections/control_collection/leaders/shard1
   [junit4]   2> 21284 T40 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 21286 T37 oascc.ZkStateReader.updateClusterState Updating 
cloud state from ZooKeeper... 
   [junit4]   2> 21290 T40 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 21336 T11 oascc.ConnectionManager.waitForConnected Waiting for 
client to connect to ZooKeeper
   [junit4]   2> 21337 T46 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@356ebe7 name:ZooKeeperConnection 
Watcher:127.0.0.1:61636/solr got event WatchedEvent state:SyncConnected 
type:None path:null path:null type:None
   [junit4]   2> 21338 T11 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 21339 T11 
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state 
from ZooKeeper... 
   [junit4]   2> 21345 T11 oasc.ChaosMonkey.monkeyLog monkey: init - expire 
sessions:false cause connection loss:false
   [junit4]   2> 21393 T40 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 21393 T47 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 21439 T44 oasc.ZkController.register We are 
https://127.0.0.1:32014/collection1/ and leader is 
https://127.0.0.1:32014/collection1/
   [junit4]   2> 21446 T44 oasc.ZkController.register No LogReplay needed for 
core=collection1 baseURL=https://127.0.0.1:32014
   [junit4]   2> 21446 T44 oasc.ZkController.checkRecovery I am the leader, no 
recovery necessary
   [junit4]   2> 21447 T44 oasc.ZkController.publish publishing 
core=collection1 state=active collection=control_collection
   [junit4]   2> 21447 T44 oasc.ZkController.publish numShards not found on 
descriptor - reading it from system property
   [junit4]   2> 21449 T40 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 21450 T49 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 21450 T44 oascc.ZkStateReader.updateClusterState Updating 
cloud state from ZooKeeper... 
   [junit4]   2> 21450 T48 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 21452 T37 oascc.ZkStateReader.updateClusterState Updating 
cloud state from ZooKeeper... 
   [junit4]   2> 21453 T37 oasc.Overseer$ClusterStateUpdater.updateState Update 
state numShards=2 message={
   [junit4]  

[...truncated too long message...]

t]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>                at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>          83) Thread[id=336, 
name=OverseerThreadFactory-4-thread-5, state=WAITING, group=Overseer collection 
creation process.]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>                at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458)
   [junit4]   2>                at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]   2>                at 
java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:925)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>          84) Thread[id=200, 
name=parallelCoreAdminExecutor-32-thread-1, state=WAITING, 
group=TGRP-MultiThreadedOCPTest]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>                at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>          85) Thread[id=286, name=searcherExecutor-74-thread-1, 
state=WAITING, group=TGRP-MultiThreadedOCPTest]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>                at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>          86) Thread[id=158, 
name=OverseerThreadFactory-4-thread-1, state=WAITING, group=Overseer collection 
creation process.]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>                at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458)
   [junit4]   2>                at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]   2>                at 
java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:925)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Lucene41, 
sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=cs, 
timezone=Atlantic/Faroe
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 
(64-bit)/cpus=16,threads=90,free=143068944,total=310378496
   [junit4]   2> NOTE: All tests run in this JVM: [MultiThreadedOCPTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 
'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=MultiThreadedOCPTest -Dtests.seed=936281279AE7F682 
-Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
-Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt 
-Dtests.locale=cs -Dtests.timezone=Atlantic/Faroe -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J1 | MultiThreadedOCPTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 
7200000 msec).
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([936281279AE7F682]:0)Throwable #2: 
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=328, name=qtp149694054-328, state=RUNNABLE, 
group=TGRP-MultiThreadedOCPTest]
   [junit4]    > Caused by: java.lang.OutOfMemoryError: unable to create new 
native thread
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([936281279AE7F682]:0)
   [junit4]    >        at java.lang.Thread.start0(Native Method)
   [junit4]    >        at java.lang.Thread.start(Thread.java:714)
   [junit4]    >        at 
sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1047)
   [junit4]    >        at 
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
   [junit4]    >        at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
   [junit4]    >        at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
   [junit4]    >        at 
org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:665)
   [junit4]    >        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]    >        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]    >        at java.lang.Thread.run(Thread.java:745)Throwable #3: 
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=232, name=qtp1627420012-232, state=RUNNABLE, 
group=TGRP-MultiThreadedOCPTest]
   [junit4]    > Caused by: java.lang.OutOfMemoryError: unable to create new 
native thread
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([936281279AE7F682]:0)
   [junit4]    >        at java.lang.Thread.start0(Native Method)
   [junit4]    >        at java.lang.Thread.start(Thread.java:714)
   [junit4]    >        at 
sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1047)
   [junit4]    >        at 
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
   [junit4]    >        at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
   [junit4]    >        at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
   [junit4]    >        at 
org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:665)
   [junit4]    >        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]    >        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]    >        at java.lang.Thread.run(Thread.java:745)
   [junit4] Completed on J1 in 7226.10s, 1 test, 4 errors <<< FAILURES!

[...truncated 323 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:474:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:447:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:45:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:37:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:189:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:496:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1298:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:922:
 There were test failures: 422 suites, 1757 tests, 5 suite-level errors, 1 
error, 1275 ignored (3 assumptions)

Total time: 251 minutes 57 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-trunk #592
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 184747043 bytes
Compression is 0.0%
Took 33 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to