Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/598/
6 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.cloud.MultiThreadedOCPTest Error Message: Suite timeout exceeded (>= 7200000 msec). Stack Trace: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec). at __randomizedtesting.SeedInfo.seed([936281279AE7F682]:0) FAILED: junit.framework.TestSuite.org.apache.solr.cloud.MultiThreadedOCPTest Error Message: Captured an uncaught exception in thread: Thread[id=328, name=qtp149694054-328, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest] Stack Trace: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=328, name=qtp149694054-328, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest] Caused by: java.lang.OutOfMemoryError: unable to create new native thread at __randomizedtesting.SeedInfo.seed([936281279AE7F682]:0) at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:714) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1047) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323) at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:665) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:745) FAILED: junit.framework.TestSuite.org.apache.solr.cloud.MultiThreadedOCPTest Error Message: Captured an uncaught exception in thread: Thread[id=232, name=qtp1627420012-232, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest] Stack Trace: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=232, name=qtp1627420012-232, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest] Caused by: java.lang.OutOfMemoryError: unable to create new native thread at __randomizedtesting.SeedInfo.seed([936281279AE7F682]:0) at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:714) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1047) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323) at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:665) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:745) FAILED: junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup Error Message: 1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup: 1) Thread[id=3209, name=Thread-1278, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at java.net.Socket.connect(Socket.java:528) at sun.net.NetworkClient.doConnect(NetworkClient.java:180) at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323) at java.net.URL.openStream(URL.java:1037) at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314) Stack Trace: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup: 1) Thread[id=3209, name=Thread-1278, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at java.net.Socket.connect(Socket.java:528) at sun.net.NetworkClient.doConnect(NetworkClient.java:180) at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323) at java.net.URL.openStream(URL.java:1037) at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314) at __randomizedtesting.SeedInfo.seed([936281279AE7F682]:0) FAILED: junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup Error Message: There are still zombie threads that couldn't be terminated: 1) Thread[id=3209, name=Thread-1278, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at java.net.Socket.connect(Socket.java:528) at sun.net.NetworkClient.doConnect(NetworkClient.java:180) at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323) at java.net.URL.openStream(URL.java:1037) at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314) Stack Trace: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated: 1) Thread[id=3209, name=Thread-1278, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at java.net.Socket.connect(Socket.java:528) at sun.net.NetworkClient.doConnect(NetworkClient.java:180) at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323) at java.net.URL.openStream(URL.java:1037) at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314) at __randomizedtesting.SeedInfo.seed([936281279AE7F682]:0) FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch Error Message: Test abandoned because suite timeout was reached. Stack Trace: java.lang.Exception: Test abandoned because suite timeout was reached. at __randomizedtesting.SeedInfo.seed([936281279AE7F682]:0) Build Log: [...truncated 11528 lines...] [junit4] Suite: org.apache.solr.handler.TestReplicationHandlerBackup [junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/init-core-data-001 [junit4] 2> 1720889 T3189 oas.SolrTestCaseJ4.setUp ###Starting doTestBackup [junit4] 2> 1720907 T3189 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 1720910 T3189 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:60236 [junit4] 2> 1720911 T3189 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 1720911 T3189 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 1720911 T3189 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001 [junit4] 2> 1720912 T3189 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/' [junit4] 2> 1720956 T3189 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/solr.xml [junit4] 2> 1721034 T3189 oasc.CoreContainer.<init> New CoreContainer 1708505120 [junit4] 2> 1721034 T3189 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/] [junit4] 2> 1721035 T3189 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 1721036 T3189 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 1721036 T3189 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 1721036 T3189 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 1721037 T3189 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 1721037 T3189 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 1721037 T3189 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 1721038 T3189 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 1721038 T3189 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 1721039 T3189 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 1721039 T3189 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1721039 T3189 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1721040 T3189 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 1721045 T3199 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/' [junit4] 2> 1721088 T3199 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0 [junit4] 2> 1721126 T3199 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 1721127 T3199 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml [junit4] 2> 1721134 T3199 oass.IndexSchema.readSchema [collection1] Schema name=test [junit4] 2> 1721150 T3199 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 1721151 T3199 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/ [junit4] 2> 1721152 T3199 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 1721152 T3199 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data/ [junit4] 2> 1721152 T3199 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1 [junit4] 2> 1721154 T3199 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data [junit4] 2> 1721155 T3199 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data/index/ [junit4] 2> 1721155 T3199 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data/index' doesn't exist. Creating new index... [junit4] 2> 1721156 T3199 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data/index [junit4] 2> 1721156 T3199 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=71.3056640625, floorSegmentMB=0.4873046875, forceMergeDeletesPctAllowed=15.706223013408033, segmentsPerTier=32.0, maxCFSSegmentSizeMB=0.7490234375, noCFSRatio=0.0 [junit4] 2> 1721157 T3199 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@8eba254 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@71968fe0),segFN=segments_1,generation=1} [junit4] 2> 1721157 T3199 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 1721159 T3199 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 1721159 T3199 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 1721159 T3199 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 1721160 T3199 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 1721160 T3199 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 1721160 T3199 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler [junit4] 2> 1721160 T3199 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler [junit4] 2> 1721161 T3199 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 1721161 T3199 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler [junit4] 2> 1721180 T3199 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1721182 T3199 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1721183 T3199 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1721185 T3199 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1721187 T3199 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1721189 T3199 oasu.CommitTracker.<init> Hard AutoCommit: disabled [junit4] 2> 1721190 T3199 oasu.CommitTracker.<init> Soft AutoCommit: disabled [junit4] 2> 1721190 T3199 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=19, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6265002443195115] [junit4] 2> 1721191 T3199 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@8eba254 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@71968fe0),segFN=segments_1,generation=1} [junit4] 2> 1721191 T3199 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 1721191 T3199 oass.SolrIndexSearcher.<init> Opening Searcher@59f64efa[collection1] main [junit4] 2> 1721192 T3199 oasr.ManagedResourceStorage$FileStorageIO.configure File-based storage initialized to use dir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/conf [junit4] 2> 1721192 T3199 oasr.RestManager.init Initializing RestManager with initArgs: {storageDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/conf} [junit4] 2> 1721192 T3199 oasr.ManagedResourceStorage.load Reading _rest_managed.json using file:dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/conf [junit4] 2> 1721193 T3199 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed [junit4] 2> 1721194 T3199 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using file:dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/conf [junit4] 2> 1721194 T3199 oasr.RestManager.init Initializing 0 registered ManagedResources [junit4] 2> 1721194 T3199 oash.ReplicationHandler.inform Replication enabled for following config files: schema-replication2.xml:schema.xml [junit4] 2> 1721195 T3199 oash.ReplicationHandler.inform Commits will be reserved for 10000 [junit4] 2> 1721195 T3200 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@59f64efa[collection1] main{UninvertingDirectoryReader()} [junit4] 2> 1721195 T3199 oasc.CoreContainer.registerCore registering core: collection1 [junit4] 2> 1721196 T3189 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0 [junit4] 2> 1721196 T3189 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> ASYNC NEW_CORE C1234 name=collection1 org.apache.solr.core.SolrCore@7d1d22be [junit4] 2> 1721198 T3192 C1234 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX [junit4] 2> 1721199 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 1 [junit4] 2> 1721202 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[0]} 0 0 [junit4] 2> 1721205 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1]} 0 0 [junit4] 2> 1721207 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[2]} 0 0 [junit4] 2> 1721209 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[3]} 0 0 [junit4] 2> 1721211 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[4]} 0 0 [junit4] 2> 1721213 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[5]} 0 0 [junit4] 2> 1721215 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[6]} 0 0 [junit4] 2> 1721217 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[7]} 0 0 [junit4] 2> 1721219 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[8]} 0 0 [junit4] 2> 1721221 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[9]} 0 0 [junit4] 2> 1721223 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[10]} 0 0 [junit4] 2> 1721225 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[11]} 0 0 [junit4] 2> 1721227 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[12]} 0 0 [junit4] 2> 1721229 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[13]} 0 0 [junit4] 2> 1721231 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[14]} 0 0 [junit4] 2> 1721233 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[15]} 0 0 [junit4] 2> 1721235 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[16]} 0 0 [junit4] 2> 1721237 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[17]} 0 0 [junit4] 2> 1721239 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[18]} 0 0 [junit4] 2> 1721241 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[19]} 0 0 [junit4] 2> 1721243 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[20]} 0 0 [junit4] 2> 1721245 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[21]} 0 0 [junit4] 2> 1721247 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[22]} 0 0 [junit4] 2> 1721249 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[23]} 0 0 [junit4] 2> 1721251 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[24]} 0 0 [junit4] 2> 1721253 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[25]} 0 0 [junit4] 2> 1721255 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[26]} 0 0 [junit4] 2> 1721257 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[27]} 0 0 [junit4] 2> 1721259 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[28]} 0 0 [junit4] 2> 1721261 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[29]} 0 0 [junit4] 2> 1721263 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[30]} 0 0 [junit4] 2> 1721265 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[31]} 0 0 [junit4] 2> 1721267 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[32]} 0 0 [junit4] 2> 1721269 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[33]} 0 0 [junit4] 2> 1721271 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[34]} 0 0 [junit4] 2> 1721273 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[35]} 0 0 [junit4] 2> 1721275 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[36]} 0 0 [junit4] 2> 1721277 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[37]} 0 0 [junit4] 2> 1721279 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[38]} 0 0 [junit4] 2> 1721281 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[39]} 0 0 [junit4] 2> 1721283 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[40]} 0 0 [junit4] 2> 1721285 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[41]} 0 0 [junit4] 2> 1721287 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[42]} 0 0 [junit4] 2> 1721289 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[43]} 0 0 [junit4] 2> 1721291 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[44]} 0 0 [junit4] 2> 1721293 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[45]} 0 0 [junit4] 2> 1721295 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[46]} 0 0 [junit4] 2> 1721297 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[47]} 0 0 [junit4] 2> 1721299 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[48]} 0 0 [junit4] 2> 1721301 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[49]} 0 0 [junit4] 2> 1721303 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[50]} 0 0 [junit4] 2> 1721305 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[51]} 0 0 [junit4] 2> 1721307 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[52]} 0 0 [junit4] 2> 1721309 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[53]} 0 0 [junit4] 2> 1721311 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[54]} 0 0 [junit4] 2> 1721313 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[55]} 0 0 [junit4] 2> 1721315 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[56]} 0 0 [junit4] 2> 1721317 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[57]} 0 0 [junit4] 2> 1721319 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[58]} 0 0 [junit4] 2> 1721321 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[59]} 0 0 [junit4] 2> 1721323 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[60]} 0 0 [junit4] 2> 1721325 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[61]} 0 0 [junit4] 2> 1721327 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[62]} 0 0 [junit4] 2> 1721329 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[63]} 0 0 [junit4] 2> 1721331 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[64]} 0 0 [junit4] 2> 1721333 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[65]} 0 0 [junit4] 2> 1721335 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[66]} 0 0 [junit4] 2> 1721337 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[67]} 0 0 [junit4] 2> 1721339 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[68]} 0 0 [junit4] 2> 1721341 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[69]} 0 0 [junit4] 2> 1721343 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[70]} 0 0 [junit4] 2> 1721345 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[71]} 0 0 [junit4] 2> 1721347 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[72]} 0 0 [junit4] 2> 1721349 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[73]} 0 0 [junit4] 2> 1721351 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[74]} 0 0 [junit4] 2> 1721353 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[75]} 0 0 [junit4] 2> 1721355 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[76]} 0 0 [junit4] 2> 1721357 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[77]} 0 0 [junit4] 2> 1721359 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[78]} 0 0 [junit4] 2> 1721361 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[79]} 0 0 [junit4] 2> 1721363 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[80]} 0 0 [junit4] 2> 1721365 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[81]} 0 0 [junit4] 2> 1721367 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[82]} 0 0 [junit4] 2> 1721369 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[83]} 0 0 [junit4] 2> 1721371 T3192 C1234 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false} [junit4] 2> 1721373 T3192 C1234 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@8eba254 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@71968fe0),segFN=segments_1,generation=1} [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@8eba254 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@71968fe0),segFN=segments_2,generation=2} [junit4] 2> 1721374 T3192 C1234 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2 [junit4] 2> 1721376 T3192 C1234 oass.SolrIndexSearcher.<init> Opening Searcher@7effa51f[collection1] main [junit4] 2> 1721377 T3192 C1234 oasu.DirectUpdateHandler2.commit end_commit_flush [junit4] 2> 1721377 T3200 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7effa51f[collection1] main{UninvertingDirectoryReader(Uninverting(_0(5.0):c84))} [junit4] 2> 1721377 T3192 C1234 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&softCommit=false&commit=true&wt=javabin&version=2} {commit=} 0 6 [junit4] 2> 1721382 T3194 C1234 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 [junit4] 2> 1721395 T3193 C1234 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={name=vdookyxjtgmxqws&command=backup} status=0 QTime=14 [junit4] 2> 1721395 T3204 oash.SnapShooter.createSnapshot Creating backup snapshot... [junit4] 2> 1721408 T3204 oash.SnapShooter.createSnapshot Done creating backup snapshot: vdookyxjtgmxqws [junit4] 2> 1721584 T3193 C1234 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 [junit4] 2> 1722593 T3193 C1234 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 [junit4] 2> 1722594 T3194 C1234 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={name=ribyuvugqsowycnhiq&command=backup} status=0 QTime=2 [junit4] 2> 1722594 T3206 oash.SnapShooter.createSnapshot Creating backup snapshot... [junit4] 2> 1722602 T3206 oash.SnapShooter.createSnapshot Done creating backup snapshot: ribyuvugqsowycnhiq [junit4] 2> 1722795 T3194 C1234 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 [junit4] 2> ASYNC NEW_CORE C1235 name=collection1 org.apache.solr.core.SolrCore@7d1d22be [junit4] 2> 1722809 T3193 C1235 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 [junit4] 2> 1722814 T3194 C1235 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={name=vdookyxjtgmxqws&command=deletebackup} status=0 QTime=6 [junit4] 2> 1722814 T3208 oash.SnapShooter.deleteNamedSnapshot Deleting snapshot: vdookyxjtgmxqws [junit4] 2> 1723011 T3194 C1235 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 [junit4] 2> 1723013 T3194 C1235 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 [junit4] 2> 1723013 T3189 oas.SolrTestCaseJ4.tearDown ###Ending doTestBackup [junit4] 2> 1723013 T3210 oash.SnapShooter.deleteNamedSnapshot Deleting snapshot: ribyuvugqsowycnhiq [junit4] 2> 1723013 T3193 C1235 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={name=ribyuvugqsowycnhiq&command=deletebackup} status=0 QTime=0 [junit4] 2> 1723014 T3189 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1708505120 [junit4] 2> 1723015 T3189 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null} [junit4] 2> 1723016 T3193 C1235 oasc.SolrException.log ERROR null:org.eclipse.jetty.io.EofException [junit4] 2> at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:142) [junit4] 2> at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:107) [junit4] 2> at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) [junit4] 2> at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291) [junit4] 2> at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295) [junit4] 2> at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141) [junit4] 2> at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229) [junit4] 2> at org.apache.solr.util.FastWriter.flush(FastWriter.java:137) [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:769) [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:428) [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) [junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) [junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229) [junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) [junit4] 2> at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301) [junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) [junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) [junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) [junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) [junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) [junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:368) [junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) [junit4] 2> at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) [junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942) [junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004) [junit4] 2> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) [junit4] 2> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) [junit4] 2> at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) [junit4] 2> at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> [junit4] 2> 1723017 T3193 C1235 oasc.SolrException.log ERROR null:org.eclipse.jetty.io.EofException [junit4] 2> at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:142) [junit4] 2> at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:107) [junit4] 2> at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) [junit4] 2> at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291) [junit4] 2> at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295) [junit4] 2> at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141) [junit4] 2> at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229) [junit4] 2> at org.apache.solr.util.FastWriter.flush(FastWriter.java:137) [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:769) [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:428) [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) [junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) [junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229) [junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) [junit4] 2> at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301) [junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) [junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) [junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) [junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) [junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) [junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:368) [junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) [junit4] 2> at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) [junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942) [junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004) [junit4] 2> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) [junit4] 2> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) [junit4] 2> at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) [junit4] 2> at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> [junit4] 2> 1723019 T3193 C1235 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@7d1d22be [junit4] 2> 1723019 T3193 C1235 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=84,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0} [junit4] 2> 1723020 T3193 C1235 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState [junit4] 2> 1723020 T3193 C1235 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter [junit4] 2> 1723020 T3193 C1235 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser [junit4] 2> 1723021 T3193 C1235 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request. [junit4] 2> 1723021 T3193 C1235 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked [junit4] 2> 1723021 T3193 C1235 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data/index [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data/index;done=false>>] [junit4] 2> 1723022 T3193 C1235 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data/index [junit4] 2> 1723022 T3193 C1235 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data;done=false>>] [junit4] 2> 1723022 T3193 C1235 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.handler.TestReplicationHandlerBackup-936281279AE7F682-001/solr-instance-001/collection1/data [junit4] 2> 1728026 T3189 oas.SolrTestCaseJ4.deleteCore ###deleteCore [junit4] 2> 7147 T3188 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s). [junit4] 2> 27233 T3188 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup: [junit4] 2> 1) Thread[id=3209, name=Thread-1278, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup] [junit4] 2> at java.net.PlainSocketImpl.socketConnect(Native Method) [junit4] 2> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [junit4] 2> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [junit4] 2> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [junit4] 2> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [junit4] 2> at java.net.Socket.connect(Socket.java:579) [junit4] 2> at java.net.Socket.connect(Socket.java:528) [junit4] 2> at sun.net.NetworkClient.doConnect(NetworkClient.java:180) [junit4] 2> at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) [junit4] 2> at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) [junit4] 2> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652) [junit4] 2> at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323) [junit4] 2> at java.net.URL.openStream(URL.java:1037) [junit4] 2> at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314) [junit4] 2> 27234 T3188 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads: [junit4] 2> 1) Thread[id=3209, name=Thread-1278, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup] [junit4] 2> 30240 T3188 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated: [junit4] 2> 1) Thread[id=3209, name=Thread-1278, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup] [junit4] 2> at java.net.PlainSocketImpl.socketConnect(Native Method) [junit4] 2> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [junit4] 2> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [junit4] 2> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [junit4] 2> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [junit4] 2> at java.net.Socket.connect(Socket.java:579) [junit4] 2> at java.net.Socket.connect(Socket.java:528) [junit4] 2> at sun.net.NetworkClient.doConnect(NetworkClient.java:180) [junit4] 2> at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) [junit4] 2> at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) [junit4] 2> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652) [junit4] 2> at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323) [junit4] 2> at java.net.URL.openStream(URL.java:1037) [junit4] 2> at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314) [junit4] 2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, locale=hu, timezone=Asia/Kashgar [junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=2,free=219235216,total=311427072 [junit4] 2> NOTE: All tests run in this JVM: [TestCloudInspectUtil, SpellPossibilityIteratorTest, TestJmxIntegration, FastVectorHighlighterTest, XsltUpdateRequestHandlerTest, MigrateRouteKeyTest, TestDefaultSimilarityFactory, TestRandomDVFaceting, TermsComponentTest, BadIndexSchemaTest, SolrXmlInZkTest, BlockCacheTest, PeerSyncTest, HdfsBasicDistributedZk2Test, TestManagedSynonymFilterFactory, ZkControllerTest, ExternalFileFieldSortTest, SpellCheckComponentTest, ParsingFieldUpdateProcessorsTest, SuggestComponentTest, ChaosMonkeyNothingIsSafeTest, TestRangeQuery, TestIntervalFaceting, DistributedDebugComponentTest, TestSchemaSimilarityResource, TestDFRSimilarityFactory, TestClassNameShortening, TestArbitraryIndexDir, TestDynamicFieldResource, TestFieldResource, TestDocBasedVersionConstraints, SolrCoreCheckLockOnStartupTest, SolrCmdDistributorTest, TestCopyFieldCollectionResource, DirectSolrSpellCheckerTest, ZkNodePropsTest, ClusterStateUpdateTest, PreAnalyzedFieldTest, TestSolrJ, HighlighterConfigTest, TestReplicationHandler, IndexSchemaTest, TestSearcherReuse, TestCodecSupport, BasicFunctionalityTest, ZkCLITest, StatelessScriptUpdateProcessorFactoryTest, PreAnalyzedUpdateProcessorTest, TestReloadAndDeleteDocs, EchoParamsTest, OverseerCollectionProcessorTest, TestSolrXmlPersistence, HighlighterTest, AnalysisErrorHandlingTest, LeaderElectionTest, TestSchemaVersionResource, TestSolrCoreProperties, TestSolrDeletionPolicy1, WordBreakSolrSpellCheckerTest, FileUtilsTest, TestTolerantSearch, ResponseLogComponentTest, TestStressRecovery, FunctionTest, TestHighlightDedupGrouping, TestSolrDeletionPolicy2, QueryEqualityTest, TestReplicationHandlerBackup] [junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory. [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestReplicationHandlerBackup -Dtests.seed=936281279AE7F682 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=hu -Dtests.timezone=Asia/Kashgar -Dtests.file.encoding=US-ASCII [junit4] ERROR 0.00s J0 | TestReplicationHandlerBackup (suite) <<< [junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup: [junit4] > 1) Thread[id=3209, name=Thread-1278, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup] [junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method) [junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [junit4] > at java.net.Socket.connect(Socket.java:579) [junit4] > at java.net.Socket.connect(Socket.java:528) [junit4] > at sun.net.NetworkClient.doConnect(NetworkClient.java:180) [junit4] > at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) [junit4] > at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) [junit4] > at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652) [junit4] > at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323) [junit4] > at java.net.URL.openStream(URL.java:1037) [junit4] > at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314) [junit4] > at __randomizedtesting.SeedInfo.seed([936281279AE7F682]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated: [junit4] > 1) Thread[id=3209, name=Thread-1278, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup] [junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method) [junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [junit4] > at java.net.Socket.connect(Socket.java:579) [junit4] > at java.net.Socket.connect(Socket.java:528) [junit4] > at sun.net.NetworkClient.doConnect(NetworkClient.java:180) [junit4] > at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) [junit4] > at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) [junit4] > at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652) [junit4] > at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323) [junit4] > at java.net.URL.openStream(URL.java:1037) [junit4] > at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314) [junit4] > at __randomizedtesting.SeedInfo.seed([936281279AE7F682]:0) [junit4] Completed on J0 in 30.28s, 1 test, 2 errors <<< FAILURES! [...truncated 818 lines...] [junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest [junit4] 2> log4j:WARN No such property [conversionPattern] in org.apache.solr.util.SolrLogLayout. [junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/init-core-data-001 [junit4] 2> 3859 T11 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true) [junit4] 2> 3962 T11 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: / [junit4] 2> 4017 T11 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch [junit4] 2> 4026 T11 oasc.ZkTestServer.run STARTING ZK TEST SERVER [junit4] 1> client port:0.0.0.0/0.0.0.0:0 [junit4] 2> 4035 T12 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server [junit4] 2> 4336 T11 oasc.ZkTestServer.run start zk server on port:61636 [junit4] 2> 4613 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 4790 T18 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6f9bc97 name:ZooKeeperConnection Watcher:127.0.0.1:61636 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 4792 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 4799 T11 oascc.SolrZkClient.makePath makePath: /solr [junit4] 2> 4926 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 4929 T20 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7a429aa2 name:ZooKeeperConnection Watcher:127.0.0.1:61636/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 4930 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 4955 T11 oascc.SolrZkClient.makePath makePath: /collections/collection1 [junit4] 2> 4960 T11 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards [junit4] 2> 4965 T11 oascc.SolrZkClient.makePath makePath: /collections/control_collection [junit4] 2> 4970 T11 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards [junit4] 2> 4976 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml [junit4] 2> 5046 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml [junit4] 2> 5053 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml [junit4] 2> 5054 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml [junit4] 2> 5059 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 5061 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 5066 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt [junit4] 2> 5067 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt [junit4] 2> 5072 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt [junit4] 2> 5074 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt [junit4] 2> 5078 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml [junit4] 2> 5080 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml [junit4] 2> 5084 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml [junit4] 2> 5086 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml [junit4] 2> 5091 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json [junit4] 2> 5092 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json [junit4] 2> 5097 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 5099 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 5103 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt [junit4] 2> 5105 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt [junit4] 2> 5109 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt [junit4] 2> 5111 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt [junit4] 2> 5129 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 5132 T22 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@302c28cc name:ZooKeeperConnection Watcher:127.0.0.1:61636/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 5133 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 8789 T11 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 9359 T11 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 9407 T11 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:32014 [junit4] 2> 10281 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 10437 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 10438 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-002 [junit4] 2> 10439 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-002/' [junit4] 2> 11442 T11 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-002/solr.xml [junit4] 2> 11751 T11 oasc.CoreContainer.<init> New CoreContainer 788197590 [junit4] 2> 11759 T11 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-002/] [junit4] 2> 11911 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 11912 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 11912 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 11913 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 11914 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 11914 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 11915 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 11915 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 11916 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 12066 T11 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 12091 T11 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 12098 T11 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 12108 T11 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 12109 T11 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:61636/solr [junit4] 2> 12153 T11 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 12154 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 12159 T34 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@77eea508 name:ZooKeeperConnection Watcher:127.0.0.1:61636 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 12160 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 12178 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 12180 T36 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@67b8e751 name:ZooKeeperConnection Watcher:127.0.0.1:61636/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 12181 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 12204 T11 oascc.SolrZkClient.makePath makePath: /overseer/queue [junit4] 2> 12220 T11 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work [junit4] 2> 12230 T11 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running [junit4] 2> 12235 T11 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed [junit4] 2> 12241 T11 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure [junit4] 2> 12284 T11 oascc.SolrZkClient.makePath makePath: /live_nodes [junit4] 2> 12287 T11 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32014_ [junit4] 2> 12295 T11 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:32014_ [junit4] 2> 12306 T11 oascc.SolrZkClient.makePath makePath: /overseer_elect [junit4] 2> 12310 T11 oascc.SolrZkClient.makePath makePath: /overseer_elect/election [junit4] 2> 12332 T11 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:32014_ [junit4] 2> 12333 T11 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader [junit4] 2> 12337 T11 oasc.Overseer.start Overseer (id=92228166816432132-127.0.0.1:32014_-n_0000000000) starting [junit4] 2> 12363 T11 oascc.SolrZkClient.makePath makePath: /overseer/queue-work [junit4] 2> 12469 T38 oasc.OverseerCollectionProcessor.run Process current queue of collection creations [junit4] 2> 12469 T11 oascc.SolrZkClient.makePath makePath: /clusterstate.json [junit4] 2> 12473 T11 oascc.SolrZkClient.makePath makePath: /aliases.json [junit4] 2> 12475 T11 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 12562 T37 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue [junit4] 2> 12906 T39 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection [junit4] 2> 12906 T39 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 12919 T39 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 12928 T40 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 12939 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4] 2> 12941 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={ [junit4] 2> "operation":"state", [junit4] 2> "state":"down", [junit4] 2> "base_url":"https://127.0.0.1:32014", [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "node_name":"127.0.0.1:32014_", [junit4] 2> "shard":null, [junit4] 2> "collection":"control_collection", [junit4] 2> "numShards":"1", [junit4] 2> "core_node_name":null} [junit4] 2> 12941 T37 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1] [junit4] 2> 13003 T37 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1 [junit4] 2> 13008 T40 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 13030 T40 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 13921 T39 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1 [junit4] 2> 13921 T39 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection [junit4] 2> 13923 T39 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 13924 T39 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection [junit4] 2> 13926 T39 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper [junit4] 2> 13927 T39 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-002/collection1/' [junit4] 2> 13930 T39 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-002/collection1/lib/classes/' to classloader [junit4] 2> 13932 T39 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-002/collection1/lib/README' to classloader [junit4] 2> 13933 T39 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-002/collection1/lib/.svn/' to classloader [junit4] 2> 14282 T39 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0 [junit4] 2> 14706 T39 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 14770 T39 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml [junit4] 2> 14973 T39 oass.IndexSchema.readSchema [collection1] Schema name=test [junit4] 2> 17235 T39 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 17290 T39 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 17293 T39 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 18101 T39 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 18126 T39 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 18132 T39 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 18138 T39 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 18139 T39 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 18139 T39 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 18141 T39 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 18142 T39 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 18146 T39 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection [junit4] 2> 18265 T39 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 18302 T39 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-002/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-001/control/data/ [junit4] 2> 18311 T39 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5d2c041a [junit4] 2> 18527 T39 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-001/control/data [junit4] 2> 18527 T39 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-001/control/data/index/ [junit4] 2> 18528 T39 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-001/control/data/index' doesn't exist. Creating new index... [junit4] 2> 18560 T39 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-936281279AE7F682-001/tempDir-001/control/data/index [junit4] 2> 18563 T39 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=23, maxMergeAtOnceExplicit=37, maxMergedSegmentMB=20.2890625, floorSegmentMB=1.1396484375, forceMergeDeletesPctAllowed=22.888894824493903, segmentsPerTier=43.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3218127037386259 [junit4] 2> 18763 T39 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@3ffd1bed lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@19292ff8),segFN=segments_1,generation=1} [junit4] 2> 18767 T39 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 19708 T39 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib" [junit4] 2> 19877 T39 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe" [junit4] 2> 19897 T39 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4] 2> 19897 T39 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig" [junit4] 2> 19898 T39 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4] 2> 19898 T39 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit" [junit4] 2> 20069 T39 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 20070 T39 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 20070 T39 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 20105 T39 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 20106 T39 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 20107 T39 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 20109 T39 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 20149 T39 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler [junit4] 2> 20180 T39 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler [junit4] 2> 20180 T39 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler [junit4] 2> 20191 T39 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler [junit4] 2> 20219 T39 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 20220 T39 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler [junit4] 2> 20225 T39 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers [junit4] 2> 20265 T39 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler [junit4] 2> 20303 T39 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler [junit4] 2> 20474 T39 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 20502 T39 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 20504 T39 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 20506 T39 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 20508 T39 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 20626 T39 oasu.CommitTracker.<init> Hard AutoCommit: disabled [junit4] 2> 20627 T39 oasu.CommitTracker.<init> Soft AutoCommit: disabled [junit4] 2> 20629 T39 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=13, maxMergedSegmentMB=21.779296875, floorSegmentMB=1.29296875, forceMergeDeletesPctAllowed=25.258018587109554, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6165322401803253 [junit4] 2> 20645 T39 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@3ffd1bed lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@19292ff8),segFN=segments_1,generation=1} [junit4] 2> 20645 T39 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 20741 T39 oass.SolrIndexSearcher.<init> Opening Searcher@57944a64[collection1] main [junit4] 2> 20790 T39 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection [junit4] 2> 20792 T39 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper [junit4] 2> 20792 T39 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1 [junit4] 2> 20808 T39 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 [junit4] 2> 20809 T39 oasr.RestManager.init Initializing RestManager with initArgs: {} [junit4] 2> 20935 T39 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 20936 T39 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json [junit4] 2> 20937 T39 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 20938 T39 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json [junit4] 2> 20941 T39 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json [junit4] 2> 20942 T39 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 20942 T39 oasr.RestManager.init Initializing 0 registered ManagedResources [junit4] 2> 21177 T41 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@57944a64[collection1] main{UninvertingDirectoryReader()} [junit4] 2> 21192 T39 oasc.CoreContainer.registerCore registering core: collection1 [junit4] 2> 21199 T44 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:32014 collection:control_collection shard:shard1 [junit4] 2> 21200 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1 [junit4] 2> 21201 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 21234 T44 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election [junit4] 2> 21241 T44 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1 [junit4] 2> 21244 T40 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 21244 T44 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue. [junit4] 2> 21245 T44 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync [junit4] 2> 21246 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4] 2> 21258 T40 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 21259 T40 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> ASYNC NEW_CORE C0 name=collection1 org.apache.solr.core.SolrCore@2154cbb6 url=https://127.0.0.1:32014/collection1 node=127.0.0.1:32014_ C0_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:32014, core=collection1, node_name=127.0.0.1:32014_} [junit4] 2> 21259 T44 C0 P32014 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:32014/collection1/ [junit4] 2> 21266 T44 C0 P32014 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me [junit4] 2> 21266 T44 C0 P32014 oasc.SyncStrategy.syncToMe https://127.0.0.1:32014/collection1/ has no replicas [junit4] 2> 21267 T44 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:32014/collection1/ shard1 [junit4] 2> 21278 T44 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1 [junit4] 2> 21284 T40 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 21286 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4] 2> 21290 T40 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 21336 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 21337 T46 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@356ebe7 name:ZooKeeperConnection Watcher:127.0.0.1:61636/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 21338 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 21339 T11 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 21345 T11 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false [junit4] 2> 21393 T40 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 21393 T47 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 21439 T44 oasc.ZkController.register We are https://127.0.0.1:32014/collection1/ and leader is https://127.0.0.1:32014/collection1/ [junit4] 2> 21446 T44 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:32014 [junit4] 2> 21446 T44 oasc.ZkController.checkRecovery I am the leader, no recovery necessary [junit4] 2> 21447 T44 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection [junit4] 2> 21447 T44 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 21449 T40 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 21450 T49 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 21450 T44 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4] 2> 21450 T48 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 21452 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4] 2> 21453 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] [...truncated too long message...] t] [junit4] 2> at sun.misc.Unsafe.park(Native Method) [junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) [junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) [junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> 83) Thread[id=336, name=OverseerThreadFactory-4-thread-5, state=WAITING, group=Overseer collection creation process.] [junit4] 2> at sun.misc.Unsafe.park(Native Method) [junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) [junit4] 2> at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) [junit4] 2> at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359) [junit4] 2> at java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:925) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> 84) Thread[id=200, name=parallelCoreAdminExecutor-32-thread-1, state=WAITING, group=TGRP-MultiThreadedOCPTest] [junit4] 2> at sun.misc.Unsafe.park(Native Method) [junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) [junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) [junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> 85) Thread[id=286, name=searcherExecutor-74-thread-1, state=WAITING, group=TGRP-MultiThreadedOCPTest] [junit4] 2> at sun.misc.Unsafe.park(Native Method) [junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) [junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) [junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> 86) Thread[id=158, name=OverseerThreadFactory-4-thread-1, state=WAITING, group=Overseer collection creation process.] [junit4] 2> at sun.misc.Unsafe.park(Native Method) [junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) [junit4] 2> at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) [junit4] 2> at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359) [junit4] 2> at java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:925) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> NOTE: test params are: codec=Lucene41, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=cs, timezone=Atlantic/Faroe [junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=90,free=143068944,total=310378496 [junit4] 2> NOTE: All tests run in this JVM: [MultiThreadedOCPTest] [junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory. [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=MultiThreadedOCPTest -Dtests.seed=936281279AE7F682 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=cs -Dtests.timezone=Atlantic/Faroe -Dtests.file.encoding=US-ASCII [junit4] ERROR 0.00s J1 | MultiThreadedOCPTest (suite) <<< [junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec). [junit4] > at __randomizedtesting.SeedInfo.seed([936281279AE7F682]:0)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=328, name=qtp149694054-328, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest] [junit4] > Caused by: java.lang.OutOfMemoryError: unable to create new native thread [junit4] > at __randomizedtesting.SeedInfo.seed([936281279AE7F682]:0) [junit4] > at java.lang.Thread.start0(Native Method) [junit4] > at java.lang.Thread.start(Thread.java:714) [junit4] > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1047) [junit4] > at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312) [junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339) [junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323) [junit4] > at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:665) [junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) [junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) [junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=232, name=qtp1627420012-232, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest] [junit4] > Caused by: java.lang.OutOfMemoryError: unable to create new native thread [junit4] > at __randomizedtesting.SeedInfo.seed([936281279AE7F682]:0) [junit4] > at java.lang.Thread.start0(Native Method) [junit4] > at java.lang.Thread.start(Thread.java:714) [junit4] > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1047) [junit4] > at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312) [junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339) [junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323) [junit4] > at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:665) [junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) [junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) [junit4] > at java.lang.Thread.run(Thread.java:745) [junit4] Completed on J1 in 7226.10s, 1 test, 4 errors <<< FAILURES! [...truncated 323 lines...] BUILD FAILED /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:474: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:447: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:45: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:37: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:189: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:496: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1298: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:922: There were test failures: 422 suites, 1757 tests, 5 suite-level errors, 1 error, 1275 ignored (3 assumptions) Total time: 251 minutes 57 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Sending artifact delta relative to Lucene-Solr-NightlyTests-trunk #592 Archived 3 artifacts Archive block size is 32768 Received 0 blocks and 184747043 bytes Compression is 0.0% Took 33 sec Recording test results Email was triggered for: Failure Sending email for trigger: Failure
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org