Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/210/

5 tests failed.
FAILED:  org.apache.lucene.index.TestIndexSorting.testNumericAlreadySorted

Error Message:
Captured an uncaught exception in thread: Thread[id=8689, name=Lucene Merge 
Thread #481, state=RUNNABLE, group=TGRP-TestIndexSorting]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=8689, name=Lucene Merge Thread #481, 
state=RUNNABLE, group=TGRP-TestIndexSorting]
Caused by: org.apache.lucene.index.MergePolicy$MergeException: 
java.lang.IllegalStateException: this writer hit an unrecoverable error; cannot 
complete merge
        at __randomizedtesting.SeedInfo.seed([5E6DA42196675E92]:0)
        at 
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
        at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
Caused by: java.lang.IllegalStateException: this writer hit an unrecoverable 
error; cannot complete merge
        at 
org.apache.lucene.index.IndexWriter.commitMerge(IndexWriter.java:3705)
        at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4469)
        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3897)
        at 
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
        at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
Caused by: java.lang.OutOfMemoryError: Java heap space
        at 
org.apache.lucene.util.packed.Packed8ThreeBlocks.<init>(Packed8ThreeBlocks.java:41)
        at 
org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:963)
        at 
org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:939)
        at 
org.apache.lucene.util.packed.GrowableWriter.ensureCapacity(GrowableWriter.java:80)
        at 
org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:88)
        at 
org.apache.lucene.util.packed.AbstractPagedMutable.set(AbstractPagedMutable.java:98)
        at org.apache.lucene.util.fst.NodeHash.addNew(NodeHash.java:152)
        at org.apache.lucene.util.fst.NodeHash.rehash(NodeHash.java:169)
        at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:133)
        at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:214)
        at org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:310)
        at org.apache.lucene.util.fst.Builder.add(Builder.java:414)
        at 
org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.writeFST(MemoryDocValuesConsumer.java:367)
        at 
org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.addSortedField(MemoryDocValuesConsumer.java:404)
        at 
org.apache.lucene.codecs.DocValuesConsumer.mergeSortedField(DocValuesConsumer.java:653)
        at 
org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:204)
        at 
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.merge(PerFieldDocValuesFormat.java:153)
        at 
org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:167)
        at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:111)
        at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4320)
        ... 3 more


FAILED:  
org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1480111658207,generation=2,filelist=[_su.fdt, _su.fdx, 
_su.fnm, _su.nvd, _su.nvm, _su.si, _su_FSTOrd50_0.doc, _su_FSTOrd50_0.tbk, 
_su_FSTOrd50_0.tix, _t3.fdt, _t3.fdx, _t3.fnm, _t3.nvd, _t3.nvm, _t3.si, 
_t3_FSTOrd50_0.doc, _t3_FSTOrd50_0.tbk, _t3_FSTOrd50_0.tix, _t7.cfe, _t7.cfs, 
_t7.si, _ta.cfe, _ta.cfs, _ta.si, _tb.fdt, _tb.fdx, _tb.fnm, _tb.nvd, _tb.nvm, 
_tb.si, _tb_FSTOrd50_0.doc, _tb_FSTOrd50_0.tbk, _tb_FSTOrd50_0.tix, _tc.fdt, 
_tc.fdx, _tc.fnm, _tc.nvd, _tc.nvm, _tc.si, _tc_FSTOrd50_0.doc, 
_tc_FSTOrd50_0.tbk, _tc_FSTOrd50_0.tix, _td.fdt, _td.fdx, _td.fnm, _td.nvd, 
_td.nvm, _td.si, _td_FSTOrd50_0.doc, _td_FSTOrd50_0.tbk, _td_FSTOrd50_0.tix, 
_te.fdt, _te.fdx, _te.fnm, _te.nvd, _te.nvm, _te.si, _te_FSTOrd50_0.doc, 
_te_FSTOrd50_0.tbk, _te_FSTOrd50_0.tix, _tf.fdt, _tf.fdx, _tf.fnm, _tf.nvd, 
_tf.nvm, _tf.si, _tf_FSTOrd50_0.doc, _tf_FSTOrd50_0.tbk, _tf_FSTOrd50_0.tix, 
_tg.fdt, _tg.fdx, _tg.fnm, _tg.nvd, _tg.nvm, _tg.si, _tg_FSTOrd50_0.doc, 
_tg_FSTOrd50_0.tbk, _tg_FSTOrd50_0.tix, _th.fdt, _th.fdx, _th.fnm, _th.nvd, 
_th.nvm, _th.si, _th_FSTOrd50_0.doc, _th_FSTOrd50_0.tbk, _th_FSTOrd50_0.tix, 
_ti.fdt, _ti.fdx, _ti.fnm, _ti.nvd, _ti.nvm, _ti.si, _ti_FSTOrd50_0.doc, 
_ti_FSTOrd50_0.tbk, _ti_FSTOrd50_0.tix, _tj.fdt, _tj.fdx, _tj.fnm, _tj.nvd, 
_tj.nvm, _tj.si, _tj_FSTOrd50_0.doc, _tj_FSTOrd50_0.tbk, _tj_FSTOrd50_0.tix, 
_tk.fdt, _tk.fdx, _tk.fnm, _tk.nvd, _tk.nvm, _tk.si, _tk_FSTOrd50_0.doc, 
_tk_FSTOrd50_0.tbk, _tk_FSTOrd50_0.tix, _tl.fdt, _tl.fdx, _tl.fnm, _tl.nvd, 
_tl.nvm, _tl.si, _tl_FSTOrd50_0.doc, _tl_FSTOrd50_0.tbk, _tl_FSTOrd50_0.tix, 
_tm.fdt, _tm.fdx, _tm.fnm, _tm.nvd, _tm.nvm, _tm.si, _tm_FSTOrd50_0.doc, 
_tm_FSTOrd50_0.tbk, _tm_FSTOrd50_0.tix, _tn.fdt, _tn.fdx, _tn.fnm, _tn.nvd, 
_tn.nvm, _tn.si, _tn_FSTOrd50_0.doc, _tn_FSTOrd50_0.tbk, _tn_FSTOrd50_0.tix, 
_to.fdt, _to.fdx, _to.fnm, _to.nvd, _to.nvm, _to.si, _to_FSTOrd50_0.doc, 
_to_FSTOrd50_0.tbk, _to_FSTOrd50_0.tix, _tp.fdt, _tp.fdx, _tp.fnm, _tp.nvd, 
_tp.nvm, _tp.si, _tp_FSTOrd50_0.doc, _tp_FSTOrd50_0.tbk, _tp_FSTOrd50_0.tix, 
segments_2]}]> but 
was:<[{indexVersion=1480111658207,generation=2,filelist=[_su.fdt, _su.fdx, 
_su.fnm, _su.nvd, _su.nvm, _su.si, _su_FSTOrd50_0.doc, _su_FSTOrd50_0.tbk, 
_su_FSTOrd50_0.tix, _t3.fdt, _t3.fdx, _t3.fnm, _t3.nvd, _t3.nvm, _t3.si, 
_t3_FSTOrd50_0.doc, _t3_FSTOrd50_0.tbk, _t3_FSTOrd50_0.tix, _t7.cfe, _t7.cfs, 
_t7.si, _ta.cfe, _ta.cfs, _ta.si, _tb.fdt, _tb.fdx, _tb.fnm, _tb.nvd, _tb.nvm, 
_tb.si, _tb_FSTOrd50_0.doc, _tb_FSTOrd50_0.tbk, _tb_FSTOrd50_0.tix, _tc.fdt, 
_tc.fdx, _tc.fnm, _tc.nvd, _tc.nvm, _tc.si, _tc_FSTOrd50_0.doc, 
_tc_FSTOrd50_0.tbk, _tc_FSTOrd50_0.tix, _td.fdt, _td.fdx, _td.fnm, _td.nvd, 
_td.nvm, _td.si, _td_FSTOrd50_0.doc, _td_FSTOrd50_0.tbk, _td_FSTOrd50_0.tix, 
_te.fdt, _te.fdx, _te.fnm, _te.nvd, _te.nvm, _te.si, _te_FSTOrd50_0.doc, 
_te_FSTOrd50_0.tbk, _te_FSTOrd50_0.tix, _tf.fdt, _tf.fdx, _tf.fnm, _tf.nvd, 
_tf.nvm, _tf.si, _tf_FSTOrd50_0.doc, _tf_FSTOrd50_0.tbk, _tf_FSTOrd50_0.tix, 
_tg.fdt, _tg.fdx, _tg.fnm, _tg.nvd, _tg.nvm, _tg.si, _tg_FSTOrd50_0.doc, 
_tg_FSTOrd50_0.tbk, _tg_FSTOrd50_0.tix, _th.fdt, _th.fdx, _th.fnm, _th.nvd, 
_th.nvm, _th.si, _th_FSTOrd50_0.doc, _th_FSTOrd50_0.tbk, _th_FSTOrd50_0.tix, 
_ti.fdt, _ti.fdx, _ti.fnm, _ti.nvd, _ti.nvm, _ti.si, _ti_FSTOrd50_0.doc, 
_ti_FSTOrd50_0.tbk, _ti_FSTOrd50_0.tix, _tj.fdt, _tj.fdx, _tj.fnm, _tj.nvd, 
_tj.nvm, _tj.si, _tj_FSTOrd50_0.doc, _tj_FSTOrd50_0.tbk, _tj_FSTOrd50_0.tix, 
_tk.fdt, _tk.fdx, _tk.fnm, _tk.nvd, _tk.nvm, _tk.si, _tk_FSTOrd50_0.doc, 
_tk_FSTOrd50_0.tbk, _tk_FSTOrd50_0.tix, _tl.fdt, _tl.fdx, _tl.fnm, _tl.nvd, 
_tl.nvm, _tl.si, _tl_FSTOrd50_0.doc, _tl_FSTOrd50_0.tbk, _tl_FSTOrd50_0.tix, 
_tm.fdt, _tm.fdx, _tm.fnm, _tm.nvd, _tm.nvm, _tm.si, _tm_FSTOrd50_0.doc, 
_tm_FSTOrd50_0.tbk, _tm_FSTOrd50_0.tix, _tn.fdt, _tn.fdx, _tn.fnm, _tn.nvd, 
_tn.nvm, _tn.si, _tn_FSTOrd50_0.doc, _tn_FSTOrd50_0.tbk, _tn_FSTOrd50_0.tix, 
_to.fdt, _to.fdx, _to.fnm, _to.nvd, _to.nvm, _to.si, _to_FSTOrd50_0.doc, 
_to_FSTOrd50_0.tbk, _to_FSTOrd50_0.tix, _tp.fdt, _tp.fdx, _tp.fnm, _tp.nvd, 
_tp.nvm, _tp.si, _tp_FSTOrd50_0.doc, _tp_FSTOrd50_0.tbk, _tp_FSTOrd50_0.tix, 
segments_2]}, {indexVersion=1480111658207,generation=3,filelist=[_su.fdt, 
_su.fdx, _su.fnm, _su.nvd, _su.nvm, _su.si, _su_FSTOrd50_0.doc, 
_su_FSTOrd50_0.tbk, _su_FSTOrd50_0.tix, _t3.fdt, _t3.fdx, _t3.fnm, _t3.nvd, 
_t3.nvm, _t3.si, _t3_FSTOrd50_0.doc, _t3_FSTOrd50_0.tbk, _t3_FSTOrd50_0.tix, 
_t7.cfe, _t7.cfs, _t7.si, _ta.cfe, _ta.cfs, _ta.si, _tc.fdt, _tc.fdx, _tc.fnm, 
_tc.nvd, _tc.nvm, _tc.si, _tc_FSTOrd50_0.doc, _tc_FSTOrd50_0.tbk, 
_tc_FSTOrd50_0.tix, _td.fdt, _td.fdx, _td.fnm, _td.nvd, _td.nvm, _td.si, 
_td_FSTOrd50_0.doc, _td_FSTOrd50_0.tbk, _td_FSTOrd50_0.tix, _te.fdt, _te.fdx, 
_te.fnm, _te.nvd, _te.nvm, _te.si, _te_FSTOrd50_0.doc, _te_FSTOrd50_0.tbk, 
_te_FSTOrd50_0.tix, _tf.fdt, _tf.fdx, _tf.fnm, _tf.nvd, _tf.nvm, _tf.si, 
_tf_FSTOrd50_0.doc, _tf_FSTOrd50_0.tbk, _tf_FSTOrd50_0.tix, _tg.fdt, _tg.fdx, 
_tg.fnm, _tg.nvd, _tg.nvm, _tg.si, _tg_FSTOrd50_0.doc, _tg_FSTOrd50_0.tbk, 
_tg_FSTOrd50_0.tix, _th.fdt, _th.fdx, _th.fnm, _th.nvd, _th.nvm, _th.si, 
_th_FSTOrd50_0.doc, _th_FSTOrd50_0.tbk, _th_FSTOrd50_0.tix, _ti.fdt, _ti.fdx, 
_ti.fnm, _ti.nvd, _ti.nvm, _ti.si, _ti_FSTOrd50_0.doc, _ti_FSTOrd50_0.tbk, 
_ti_FSTOrd50_0.tix, _tj.fdt, _tj.fdx, _tj.fnm, _tj.nvd, _tj.nvm, _tj.si, 
_tj_FSTOrd50_0.doc, _tj_FSTOrd50_0.tbk, _tj_FSTOrd50_0.tix, _tl.fdt, _tl.fdx, 
_tl.fnm, _tl.nvd, _tl.nvm, _tl.si, _tl_FSTOrd50_0.doc, _tl_FSTOrd50_0.tbk, 
_tl_FSTOrd50_0.tix, _tm.fdt, _tm.fdx, _tm.fnm, _tm.nvd, _tm.nvm, _tm.si, 
_tm_FSTOrd50_0.doc, _tm_FSTOrd50_0.tbk, _tm_FSTOrd50_0.tix, _tn.fdt, _tn.fdx, 
_tn.fnm, _tn.nvd, _tn.nvm, _tn.si, _tn_FSTOrd50_0.doc, _tn_FSTOrd50_0.tbk, 
_tn_FSTOrd50_0.tix, _to.fdt, _to.fdx, _to.fnm, _to.nvd, _to.nvm, _to.si, 
_to_FSTOrd50_0.doc, _to_FSTOrd50_0.tbk, _to_FSTOrd50_0.tix, _tp.fdt, _tp.fdx, 
_tp.fnm, _tp.nvd, _tp.nvm, _tp.si, _tp_FSTOrd50_0.doc, _tp_FSTOrd50_0.tbk, 
_tp_FSTOrd50_0.tix, _tq.cfe, _tq.cfs, _tq.si, segments_3]}]>

Stack Trace:
java.lang.AssertionError: 
expected:<[{indexVersion=1480111658207,generation=2,filelist=[_su.fdt, _su.fdx, 
_su.fnm, _su.nvd, _su.nvm, _su.si, _su_FSTOrd50_0.doc, _su_FSTOrd50_0.tbk, 
_su_FSTOrd50_0.tix, _t3.fdt, _t3.fdx, _t3.fnm, _t3.nvd, _t3.nvm, _t3.si, 
_t3_FSTOrd50_0.doc, _t3_FSTOrd50_0.tbk, _t3_FSTOrd50_0.tix, _t7.cfe, _t7.cfs, 
_t7.si, _ta.cfe, _ta.cfs, _ta.si, _tb.fdt, _tb.fdx, _tb.fnm, _tb.nvd, _tb.nvm, 
_tb.si, _tb_FSTOrd50_0.doc, _tb_FSTOrd50_0.tbk, _tb_FSTOrd50_0.tix, _tc.fdt, 
_tc.fdx, _tc.fnm, _tc.nvd, _tc.nvm, _tc.si, _tc_FSTOrd50_0.doc, 
_tc_FSTOrd50_0.tbk, _tc_FSTOrd50_0.tix, _td.fdt, _td.fdx, _td.fnm, _td.nvd, 
_td.nvm, _td.si, _td_FSTOrd50_0.doc, _td_FSTOrd50_0.tbk, _td_FSTOrd50_0.tix, 
_te.fdt, _te.fdx, _te.fnm, _te.nvd, _te.nvm, _te.si, _te_FSTOrd50_0.doc, 
_te_FSTOrd50_0.tbk, _te_FSTOrd50_0.tix, _tf.fdt, _tf.fdx, _tf.fnm, _tf.nvd, 
_tf.nvm, _tf.si, _tf_FSTOrd50_0.doc, _tf_FSTOrd50_0.tbk, _tf_FSTOrd50_0.tix, 
_tg.fdt, _tg.fdx, _tg.fnm, _tg.nvd, _tg.nvm, _tg.si, _tg_FSTOrd50_0.doc, 
_tg_FSTOrd50_0.tbk, _tg_FSTOrd50_0.tix, _th.fdt, _th.fdx, _th.fnm, _th.nvd, 
_th.nvm, _th.si, _th_FSTOrd50_0.doc, _th_FSTOrd50_0.tbk, _th_FSTOrd50_0.tix, 
_ti.fdt, _ti.fdx, _ti.fnm, _ti.nvd, _ti.nvm, _ti.si, _ti_FSTOrd50_0.doc, 
_ti_FSTOrd50_0.tbk, _ti_FSTOrd50_0.tix, _tj.fdt, _tj.fdx, _tj.fnm, _tj.nvd, 
_tj.nvm, _tj.si, _tj_FSTOrd50_0.doc, _tj_FSTOrd50_0.tbk, _tj_FSTOrd50_0.tix, 
_tk.fdt, _tk.fdx, _tk.fnm, _tk.nvd, _tk.nvm, _tk.si, _tk_FSTOrd50_0.doc, 
_tk_FSTOrd50_0.tbk, _tk_FSTOrd50_0.tix, _tl.fdt, _tl.fdx, _tl.fnm, _tl.nvd, 
_tl.nvm, _tl.si, _tl_FSTOrd50_0.doc, _tl_FSTOrd50_0.tbk, _tl_FSTOrd50_0.tix, 
_tm.fdt, _tm.fdx, _tm.fnm, _tm.nvd, _tm.nvm, _tm.si, _tm_FSTOrd50_0.doc, 
_tm_FSTOrd50_0.tbk, _tm_FSTOrd50_0.tix, _tn.fdt, _tn.fdx, _tn.fnm, _tn.nvd, 
_tn.nvm, _tn.si, _tn_FSTOrd50_0.doc, _tn_FSTOrd50_0.tbk, _tn_FSTOrd50_0.tix, 
_to.fdt, _to.fdx, _to.fnm, _to.nvd, _to.nvm, _to.si, _to_FSTOrd50_0.doc, 
_to_FSTOrd50_0.tbk, _to_FSTOrd50_0.tix, _tp.fdt, _tp.fdx, _tp.fnm, _tp.nvd, 
_tp.nvm, _tp.si, _tp_FSTOrd50_0.doc, _tp_FSTOrd50_0.tbk, _tp_FSTOrd50_0.tix, 
segments_2]}]> but 
was:<[{indexVersion=1480111658207,generation=2,filelist=[_su.fdt, _su.fdx, 
_su.fnm, _su.nvd, _su.nvm, _su.si, _su_FSTOrd50_0.doc, _su_FSTOrd50_0.tbk, 
_su_FSTOrd50_0.tix, _t3.fdt, _t3.fdx, _t3.fnm, _t3.nvd, _t3.nvm, _t3.si, 
_t3_FSTOrd50_0.doc, _t3_FSTOrd50_0.tbk, _t3_FSTOrd50_0.tix, _t7.cfe, _t7.cfs, 
_t7.si, _ta.cfe, _ta.cfs, _ta.si, _tb.fdt, _tb.fdx, _tb.fnm, _tb.nvd, _tb.nvm, 
_tb.si, _tb_FSTOrd50_0.doc, _tb_FSTOrd50_0.tbk, _tb_FSTOrd50_0.tix, _tc.fdt, 
_tc.fdx, _tc.fnm, _tc.nvd, _tc.nvm, _tc.si, _tc_FSTOrd50_0.doc, 
_tc_FSTOrd50_0.tbk, _tc_FSTOrd50_0.tix, _td.fdt, _td.fdx, _td.fnm, _td.nvd, 
_td.nvm, _td.si, _td_FSTOrd50_0.doc, _td_FSTOrd50_0.tbk, _td_FSTOrd50_0.tix, 
_te.fdt, _te.fdx, _te.fnm, _te.nvd, _te.nvm, _te.si, _te_FSTOrd50_0.doc, 
_te_FSTOrd50_0.tbk, _te_FSTOrd50_0.tix, _tf.fdt, _tf.fdx, _tf.fnm, _tf.nvd, 
_tf.nvm, _tf.si, _tf_FSTOrd50_0.doc, _tf_FSTOrd50_0.tbk, _tf_FSTOrd50_0.tix, 
_tg.fdt, _tg.fdx, _tg.fnm, _tg.nvd, _tg.nvm, _tg.si, _tg_FSTOrd50_0.doc, 
_tg_FSTOrd50_0.tbk, _tg_FSTOrd50_0.tix, _th.fdt, _th.fdx, _th.fnm, _th.nvd, 
_th.nvm, _th.si, _th_FSTOrd50_0.doc, _th_FSTOrd50_0.tbk, _th_FSTOrd50_0.tix, 
_ti.fdt, _ti.fdx, _ti.fnm, _ti.nvd, _ti.nvm, _ti.si, _ti_FSTOrd50_0.doc, 
_ti_FSTOrd50_0.tbk, _ti_FSTOrd50_0.tix, _tj.fdt, _tj.fdx, _tj.fnm, _tj.nvd, 
_tj.nvm, _tj.si, _tj_FSTOrd50_0.doc, _tj_FSTOrd50_0.tbk, _tj_FSTOrd50_0.tix, 
_tk.fdt, _tk.fdx, _tk.fnm, _tk.nvd, _tk.nvm, _tk.si, _tk_FSTOrd50_0.doc, 
_tk_FSTOrd50_0.tbk, _tk_FSTOrd50_0.tix, _tl.fdt, _tl.fdx, _tl.fnm, _tl.nvd, 
_tl.nvm, _tl.si, _tl_FSTOrd50_0.doc, _tl_FSTOrd50_0.tbk, _tl_FSTOrd50_0.tix, 
_tm.fdt, _tm.fdx, _tm.fnm, _tm.nvd, _tm.nvm, _tm.si, _tm_FSTOrd50_0.doc, 
_tm_FSTOrd50_0.tbk, _tm_FSTOrd50_0.tix, _tn.fdt, _tn.fdx, _tn.fnm, _tn.nvd, 
_tn.nvm, _tn.si, _tn_FSTOrd50_0.doc, _tn_FSTOrd50_0.tbk, _tn_FSTOrd50_0.tix, 
_to.fdt, _to.fdx, _to.fnm, _to.nvd, _to.nvm, _to.si, _to_FSTOrd50_0.doc, 
_to_FSTOrd50_0.tbk, _to_FSTOrd50_0.tix, _tp.fdt, _tp.fdx, _tp.fnm, _tp.nvd, 
_tp.nvm, _tp.si, _tp_FSTOrd50_0.doc, _tp_FSTOrd50_0.tbk, _tp_FSTOrd50_0.tix, 
segments_2]}, {indexVersion=1480111658207,generation=3,filelist=[_su.fdt, 
_su.fdx, _su.fnm, _su.nvd, _su.nvm, _su.si, _su_FSTOrd50_0.doc, 
_su_FSTOrd50_0.tbk, _su_FSTOrd50_0.tix, _t3.fdt, _t3.fdx, _t3.fnm, _t3.nvd, 
_t3.nvm, _t3.si, _t3_FSTOrd50_0.doc, _t3_FSTOrd50_0.tbk, _t3_FSTOrd50_0.tix, 
_t7.cfe, _t7.cfs, _t7.si, _ta.cfe, _ta.cfs, _ta.si, _tc.fdt, _tc.fdx, _tc.fnm, 
_tc.nvd, _tc.nvm, _tc.si, _tc_FSTOrd50_0.doc, _tc_FSTOrd50_0.tbk, 
_tc_FSTOrd50_0.tix, _td.fdt, _td.fdx, _td.fnm, _td.nvd, _td.nvm, _td.si, 
_td_FSTOrd50_0.doc, _td_FSTOrd50_0.tbk, _td_FSTOrd50_0.tix, _te.fdt, _te.fdx, 
_te.fnm, _te.nvd, _te.nvm, _te.si, _te_FSTOrd50_0.doc, _te_FSTOrd50_0.tbk, 
_te_FSTOrd50_0.tix, _tf.fdt, _tf.fdx, _tf.fnm, _tf.nvd, _tf.nvm, _tf.si, 
_tf_FSTOrd50_0.doc, _tf_FSTOrd50_0.tbk, _tf_FSTOrd50_0.tix, _tg.fdt, _tg.fdx, 
_tg.fnm, _tg.nvd, _tg.nvm, _tg.si, _tg_FSTOrd50_0.doc, _tg_FSTOrd50_0.tbk, 
_tg_FSTOrd50_0.tix, _th.fdt, _th.fdx, _th.fnm, _th.nvd, _th.nvm, _th.si, 
_th_FSTOrd50_0.doc, _th_FSTOrd50_0.tbk, _th_FSTOrd50_0.tix, _ti.fdt, _ti.fdx, 
_ti.fnm, _ti.nvd, _ti.nvm, _ti.si, _ti_FSTOrd50_0.doc, _ti_FSTOrd50_0.tbk, 
_ti_FSTOrd50_0.tix, _tj.fdt, _tj.fdx, _tj.fnm, _tj.nvd, _tj.nvm, _tj.si, 
_tj_FSTOrd50_0.doc, _tj_FSTOrd50_0.tbk, _tj_FSTOrd50_0.tix, _tl.fdt, _tl.fdx, 
_tl.fnm, _tl.nvd, _tl.nvm, _tl.si, _tl_FSTOrd50_0.doc, _tl_FSTOrd50_0.tbk, 
_tl_FSTOrd50_0.tix, _tm.fdt, _tm.fdx, _tm.fnm, _tm.nvd, _tm.nvm, _tm.si, 
_tm_FSTOrd50_0.doc, _tm_FSTOrd50_0.tbk, _tm_FSTOrd50_0.tix, _tn.fdt, _tn.fdx, 
_tn.fnm, _tn.nvd, _tn.nvm, _tn.si, _tn_FSTOrd50_0.doc, _tn_FSTOrd50_0.tbk, 
_tn_FSTOrd50_0.tix, _to.fdt, _to.fdx, _to.fnm, _to.nvd, _to.nvm, _to.si, 
_to_FSTOrd50_0.doc, _to_FSTOrd50_0.tbk, _to_FSTOrd50_0.tix, _tp.fdt, _tp.fdx, 
_tp.fnm, _tp.nvd, _tp.nvm, _tp.si, _tp_FSTOrd50_0.doc, _tp_FSTOrd50_0.tbk, 
_tp_FSTOrd50_0.tix, _tq.cfe, _tq.cfs, _tq.si, segments_3]}]>
        at 
__randomizedtesting.SeedInfo.seed([B140AD8EB85F37B7:9497B6BEC81739B4]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:147)
        at 
org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1229)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.index.TestIndexSorting.testRandom3

Error Message:
this IndexWriter is closed

Stack Trace:
org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
        at 
__randomizedtesting.SeedInfo.seed([5E6DA42196675E92:FCB5EAFBF2957794]:0)
        at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:748)
        at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:762)
        at 
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1566)
        at 
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1315)
        at 
org.apache.lucene.index.TestIndexSorting.testRandom3(TestIndexSorting.java:2208)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
        at 
org.apache.lucene.util.packed.Packed8ThreeBlocks.<init>(Packed8ThreeBlocks.java:41)
        at 
org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:963)
        at 
org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:939)
        at 
org.apache.lucene.util.packed.GrowableWriter.ensureCapacity(GrowableWriter.java:80)
        at 
org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:88)
        at 
org.apache.lucene.util.packed.AbstractPagedMutable.set(AbstractPagedMutable.java:98)
        at org.apache.lucene.util.fst.NodeHash.addNew(NodeHash.java:152)
        at org.apache.lucene.util.fst.NodeHash.rehash(NodeHash.java:169)
        at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:133)
        at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:214)
        at org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:310)
        at org.apache.lucene.util.fst.Builder.add(Builder.java:414)
        at 
org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.writeFST(MemoryDocValuesConsumer.java:367)
        at 
org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.addSortedField(MemoryDocValuesConsumer.java:404)
        at 
org.apache.lucene.codecs.DocValuesConsumer.mergeSortedField(DocValuesConsumer.java:653)
        at 
org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:204)
        at 
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.merge(PerFieldDocValuesFormat.java:153)
        at 
org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:167)
        at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:111)
        at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4320)
        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3897)
        at 
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
        at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)


FAILED:  
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI

Error Message:
Expected to see collection awhollynewcollection_0 null Last available state: 
DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/2)={
   "replicationFactor":"1",   "shards":{"shard1":{       
"range":"80000000-7fffffff",       "state":"active",       "replicas":{}}},   
"router":{"name":"compositeId"},   "maxShardsPerNode":"1",   
"autoAddReplicas":"false"}

Stack Trace:
java.lang.AssertionError: Expected to see collection awhollynewcollection_0
null
Last available state: 
DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/2)={
  "replicationFactor":"1",
  "shards":{"shard1":{
      "range":"80000000-7fffffff",
      "state":"active",
      "replicas":{}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false"}
        at 
__randomizedtesting.SeedInfo.seed([B140AD8EB85F37B7:F935D93ABE6C1822]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:236)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:496)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets

Error Message:
KeeperErrorCode = NoNode for /collections/withconfigset2

Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode 
for /collections/withconfigset2
        at 
__randomizedtesting.SeedInfo.seed([B140AD8EB85F37B7:9C3EE2D44F7B8DBB]:0)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:356)
        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:353)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets(CollectionsAPIDistributedZkTest.java:425)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 1501 lines...]
   [junit4] Suite: org.apache.lucene.index.TestIndexSorting
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 
'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestIndexSorting 
-Dtests.method=testRandom3 -Dtests.seed=5E6DA42196675E92 -Dtests.multiplier=2 
-Dtests.nightly=true -Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt
 -Dtests.locale=sr-CS -Dtests.timezone=Europe/Berlin -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] ERROR    494s J2 | TestIndexSorting.testRandom3 <<<
   [junit4]    > Throwable #1: org.apache.lucene.store.AlreadyClosedException: 
this IndexWriter is closed
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([5E6DA42196675E92:FCB5EAFBF2957794]:0)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:748)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:762)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1566)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1315)
   [junit4]    >        at 
org.apache.lucene.index.TestIndexSorting.testRandom3(TestIndexSorting.java:2208)
   [junit4]    >        at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]    >        at 
org.apache.lucene.util.packed.Packed8ThreeBlocks.<init>(Packed8ThreeBlocks.java:41)
   [junit4]    >        at 
org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:963)
   [junit4]    >        at 
org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:939)
   [junit4]    >        at 
org.apache.lucene.util.packed.GrowableWriter.ensureCapacity(GrowableWriter.java:80)
   [junit4]    >        at 
org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:88)
   [junit4]    >        at 
org.apache.lucene.util.packed.AbstractPagedMutable.set(AbstractPagedMutable.java:98)
   [junit4]    >        at 
org.apache.lucene.util.fst.NodeHash.addNew(NodeHash.java:152)
   [junit4]    >        at 
org.apache.lucene.util.fst.NodeHash.rehash(NodeHash.java:169)
   [junit4]    >        at 
org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:133)
   [junit4]    >        at 
org.apache.lucene.util.fst.Builder.compileNode(Builder.java:214)
   [junit4]    >        at 
org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:310)
   [junit4]    >        at 
org.apache.lucene.util.fst.Builder.add(Builder.java:414)
   [junit4]    >        at 
org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.writeFST(MemoryDocValuesConsumer.java:367)
   [junit4]    >        at 
org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.addSortedField(MemoryDocValuesConsumer.java:404)
   [junit4]    >        at 
org.apache.lucene.codecs.DocValuesConsumer.mergeSortedField(DocValuesConsumer.java:653)
   [junit4]    >        at 
org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:204)
   [junit4]    >        at 
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.merge(PerFieldDocValuesFormat.java:153)
   [junit4]    >        at 
org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:167)
   [junit4]    >        at 
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:111)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4320)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3897)
   [junit4]    >        at 
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]    >        at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
   [junit4]   2> нов 25, 2016 7:57:09 PM 
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
 uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Lucene Merge 
Thread #481,5,TGRP-TestIndexSorting]
   [junit4]   2> org.apache.lucene.index.MergePolicy$MergeException: 
java.lang.IllegalStateException: this writer hit an unrecoverable error; cannot 
complete merge
   [junit4]   2>        at 
__randomizedtesting.SeedInfo.seed([5E6DA42196675E92]:0)
   [junit4]   2>        at 
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
   [junit4]   2>        at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
   [junit4]   2> Caused by: java.lang.IllegalStateException: this writer hit an 
unrecoverable error; cannot complete merge
   [junit4]   2>        at 
org.apache.lucene.index.IndexWriter.commitMerge(IndexWriter.java:3705)
   [junit4]   2>        at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4469)
   [junit4]   2>        at 
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3897)
   [junit4]   2>        at 
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]   2>        at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
   [junit4]   2> Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]   2>        at 
org.apache.lucene.util.packed.Packed8ThreeBlocks.<init>(Packed8ThreeBlocks.java:41)
   [junit4]   2>        at 
org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:963)
   [junit4]   2>        at 
org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:939)
   [junit4]   2>        at 
org.apache.lucene.util.packed.GrowableWriter.ensureCapacity(GrowableWriter.java:80)
   [junit4]   2>        at 
org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:88)
   [junit4]   2>        at 
org.apache.lucene.util.packed.AbstractPagedMutable.set(AbstractPagedMutable.java:98)
   [junit4]   2>        at 
org.apache.lucene.util.fst.NodeHash.addNew(NodeHash.java:152)
   [junit4]   2>        at 
org.apache.lucene.util.fst.NodeHash.rehash(NodeHash.java:169)
   [junit4]   2>        at 
org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:133)
   [junit4]   2>        at 
org.apache.lucene.util.fst.Builder.compileNode(Builder.java:214)
   [junit4]   2>        at 
org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:310)
   [junit4]   2>        at 
org.apache.lucene.util.fst.Builder.add(Builder.java:414)
   [junit4]   2>        at 
org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.writeFST(MemoryDocValuesConsumer.java:367)
   [junit4]   2>        at 
org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.addSortedField(MemoryDocValuesConsumer.java:404)
   [junit4]   2>        at 
org.apache.lucene.codecs.DocValuesConsumer.mergeSortedField(DocValuesConsumer.java:653)
   [junit4]   2>        at 
org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:204)
   [junit4]   2>        at 
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.merge(PerFieldDocValuesFormat.java:153)
   [junit4]   2>        at 
org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:167)
   [junit4]   2>        at 
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:111)
   [junit4]   2>        at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4320)
   [junit4]   2>        ... 3 more
   [junit4]   2> 
   [junit4]   2> нов 25, 2016 7:57:09 PM 
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
 uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Lucene Merge 
Thread #444,5,TGRP-TestIndexSorting]
   [junit4]   2> org.apache.lucene.index.MergePolicy$MergeException: 
java.lang.OutOfMemoryError: Java heap space
   [junit4]   2>        at 
__randomizedtesting.SeedInfo.seed([5E6DA42196675E92]:0)
   [junit4]   2>        at 
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
   [junit4]   2>        at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
   [junit4]   2> Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]   2>        at 
org.apache.lucene.util.packed.Packed8ThreeBlocks.<init>(Packed8ThreeBlocks.java:41)
   [junit4]   2>        at 
org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:963)
   [junit4]   2>        at 
org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:939)
   [junit4]   2>        at 
org.apache.lucene.util.packed.GrowableWriter.ensureCapacity(GrowableWriter.java:80)
   [junit4]   2>        at 
org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:88)
   [junit4]   2>        at 
org.apache.lucene.util.packed.AbstractPagedMutable.set(AbstractPagedMutable.java:98)
   [junit4]   2>        at 
org.apache.lucene.util.fst.NodeHash.addNew(NodeHash.java:152)
   [junit4]   2>        at 
org.apache.lucene.util.fst.NodeHash.rehash(NodeHash.java:169)
   [junit4]   2>        at 
org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:133)
   [junit4]   2>        at 
org.apache.lucene.util.fst.Builder.compileNode(Builder.java:214)
   [junit4]   2>        at 
org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:310)
   [junit4]   2>        at 
org.apache.lucene.util.fst.Builder.add(Builder.java:414)
   [junit4]   2>        at 
org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.writeFST(MemoryDocValuesConsumer.java:367)
   [junit4]   2>        at 
org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.addSortedField(MemoryDocValuesConsumer.java:404)
   [junit4]   2>        at 
org.apache.lucene.codecs.DocValuesConsumer.mergeSortedField(DocValuesConsumer.java:653)
   [junit4]   2>        at 
org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:204)
   [junit4]   2>        at 
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.merge(PerFieldDocValuesFormat.java:153)
   [junit4]   2>        at 
org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:167)
   [junit4]   2>        at 
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:111)
   [junit4]   2>        at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4320)
   [junit4]   2>        at 
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3897)
   [junit4]   2>        at 
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]   2>        at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 
'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestIndexSorting 
-Dtests.method=testNumericAlreadySorted -Dtests.seed=5E6DA42196675E92 
-Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt
 -Dtests.locale=sr-CS -Dtests.timezone=Europe/Berlin -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.42s J2 | TestIndexSorting.testNumericAlreadySorted <<<
   [junit4]    > Throwable #1: 
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=8689, name=Lucene Merge Thread #481, 
state=RUNNABLE, group=TGRP-TestIndexSorting]
   [junit4]    > Caused by: org.apache.lucene.index.MergePolicy$MergeException: 
java.lang.IllegalStateException: this writer hit an unrecoverable error; cannot 
complete merge
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([5E6DA42196675E92]:0)
   [junit4]    >        at 
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
   [junit4]    >        at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
   [junit4]    > Caused by: java.lang.IllegalStateException: this writer hit an 
unrecoverable error; cannot complete merge
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.commitMerge(IndexWriter.java:3705)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4469)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3897)
   [junit4]    >        at 
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]    >        at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]    >        at 
org.apache.lucene.util.packed.Packed8ThreeBlocks.<init>(Packed8ThreeBlocks.java:41)
   [junit4]    >        at 
org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:963)
   [junit4]    >        at 
org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:939)
   [junit4]    >        at 
org.apache.lucene.util.packed.GrowableWriter.ensureCapacity(GrowableWriter.java:80)
   [junit4]    >        at 
org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:88)
   [junit4]    >        at 
org.apache.lucene.util.packed.AbstractPagedMutable.set(AbstractPagedMutable.java:98)
   [junit4]    >        at 
org.apache.lucene.util.fst.NodeHash.addNew(NodeHash.java:152)
   [junit4]    >        at 
org.apache.lucene.util.fst.NodeHash.rehash(NodeHash.java:169)
   [junit4]    >        at 
org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:133)
   [junit4]    >        at 
org.apache.lucene.util.fst.Builder.compileNode(Builder.java:214)
   [junit4]    >        at 
org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:310)
   [junit4]    >        at 
org.apache.lucene.util.fst.Builder.add(Builder.java:414)
   [junit4]    >        at 
org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.writeFST(MemoryDocValuesConsumer.java:367)
   [junit4]    >        at 
org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.addSortedField(MemoryDocValuesConsumer.java:404)
   [junit4]    >        at 
org.apache.lucene.codecs.DocValuesConsumer.mergeSortedField(DocValuesConsumer.java:653)
   [junit4]    >        at 
org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:204)
   [junit4]    >        at 
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.merge(PerFieldDocValuesFormat.java:153)
   [junit4]    >        at 
org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:167)
   [junit4]    >        at 
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:111)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4320)
   [junit4]    >        ... 3 moreThrowable #2: 
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=8652, name=Lucene Merge Thread #444, 
state=RUNNABLE, group=TGRP-TestIndexSorting]
   [junit4]    > Caused by: org.apache.lucene.index.MergePolicy$MergeException: 
java.lang.OutOfMemoryError: Java heap space
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([5E6DA42196675E92]:0)
   [junit4]    >        at 
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
   [junit4]    >        at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]    >        at 
org.apache.lucene.util.packed.Packed8ThreeBlocks.<init>(Packed8ThreeBlocks.java:41)
   [junit4]    >        at 
org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:963)
   [junit4]    >        at 
org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:939)
   [junit4]    >        at 
org.apache.lucene.util.packed.GrowableWriter.ensureCapacity(GrowableWriter.java:80)
   [junit4]    >        at 
org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:88)
   [junit4]    >        at 
org.apache.lucene.util.packed.AbstractPagedMutable.set(AbstractPagedMutable.java:98)
   [junit4]    >        at 
org.apache.lucene.util.fst.NodeHash.addNew(NodeHash.java:152)
   [junit4]    >        at 
org.apache.lucene.util.fst.NodeHash.rehash(NodeHash.java:169)
   [junit4]    >        at 
org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:133)
   [junit4]    >        at 
org.apache.lucene.util.fst.Builder.compileNode(Builder.java:214)
   [junit4]    >        at 
org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:310)
   [junit4]    >        at 
org.apache.lucene.util.fst.Builder.add(Builder.java:414)
   [junit4]    >        at 
org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.writeFST(MemoryDocValuesConsumer.java:367)
   [junit4]    >        at 
org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.addSortedField(MemoryDocValuesConsumer.java:404)
   [junit4]    >        at 
org.apache.lucene.codecs.DocValuesConsumer.mergeSortedField(DocValuesConsumer.java:653)
   [junit4]    >        at 
org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:204)
   [junit4]    >        at 
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.merge(PerFieldDocValuesFormat.java:153)
   [junit4]    >        at 
org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:167)
   [junit4]    >        at 
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:111)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4320)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3897)
   [junit4]    >        at 
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]    >        at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestIndexSorting_5E6DA42196675E92-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): 
{docs=Lucene50(blocksize=128), norms=Lucene50(blocksize=128), 
positions=PostingsFormat(name=Asserting), id=Lucene50(blocksize=128), 
term_vectors=Lucene50(blocksize=128)}, 
docValues:{multi_valued_long=DocValuesFormat(name=Lucene54), 
double=DocValuesFormat(name=Memory), foo=DocValuesFormat(name=Lucene54), 
numeric=DocValuesFormat(name=Memory), positions=DocValuesFormat(name=Lucene54), 
float=DocValuesFormat(name=Lucene54), 
multi_valued_numeric=DocValuesFormat(name=Memory), 
int=DocValuesFormat(name=Memory), long=DocValuesFormat(name=Lucene54), 
points=DocValuesFormat(name=Direct), sorted=DocValuesFormat(name=Memory), 
multi_valued_double=DocValuesFormat(name=Memory), 
docs=DocValuesFormat(name=Memory), 
multi_valued_string=DocValuesFormat(name=Memory), 
norms=DocValuesFormat(name=Memory), bytes=DocValuesFormat(name=Memory), 
binary=DocValuesFormat(name=Memory), id=DocValuesFormat(name=Memory), 
multi_valued_int=DocValuesFormat(name=Direct), 
multi_valued_bytes=DocValuesFormat(name=Memory), 
multi_valued_float=DocValuesFormat(name=Lucene54), 
term_vectors=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=1731, 
maxMBSortInHeap=6.806785275263592, 
sim=RandomSimilarity(queryNorm=true,coord=crazy): {positions=IB SPL-DZ(0.3), 
id=DFR GB3(800.0), term_vectors=DFR I(n)B2}, locale=sr-CS, 
timezone=Europe/Berlin
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 
1.8.0_102 (64-bit)/cpus=4,threads=1,free=115671944,total=441974784
   [junit4]   2> NOTE: All tests run in this JVM: [TestSortedSetDocValues, 
TestBufferedIndexInput, Test2BBinaryDocValues, TestIntroSelector, 
TestCharTermAttributeImpl, TestApproximationSearchEquivalence, TestAssertions, 
TestLazyProxSkipping, TestIndexWriterMerging, TestGeoEncodingUtils, 
TestTimeLimitingCollector, TestPayloadsOnVectors, TestPositionIncrement, 
TestIndexWriterOutOfFileDescriptors, TestPackedTokenAttributeImpl, 
TestMultiThreadTermVectors, TestSparseFixedBitDocIdSet, TestMaxTermFrequency, 
TestIndexWriterNRTIsCurrent, TestNumericRangeQuery32, TestOperations, 
TestTermVectors, TestIndexWriterReader, TestMixedCodecs, TestBooleanSimilarity, 
TestWildcard, FiniteStringsIteratorTest, TestNoMergePolicy, 
TestIndexFileDeleter, TestParallelTermEnum, TestConcurrentMergeScheduler, 
TestDocIdSetBuilder, TestMultiLevelSkipList, TestMergeSchedulerExternal, 
TestDirectoryReader, TestTermsEnum2, TestTopDocsCollector, 
TestSameScoresWithThreads, TestBlockPostingsFormat3, TestNativeFSLockFactory, 
TestLongBitSet, TestDuelingCodecsAtNight, TestSpanBoostQuery, 
TestBagOfPositions, TestCharacterUtils, TestMutableValues, TestIntsRef, 
TestAtomicUpdate, TestIndexWriterForceMerge, TestCollectionUtil, TestBKD, 
TestSumDocFreq, TestLucene60FieldInfoFormat, TestMaxPosition, TestNorms, 
TestRoaringDocIdSet, TestDocumentWriter, TestLucene50StoredFieldsFormat, 
TestSimpleFSLockFactory, TestEarlyTermination, TestNamedSPILoader, 
TestFrequencyTrackingRingBuffer, TestStopFilter, TestTryDelete, 
TestTieredMergePolicy, TestReadOnlyIndex, TestComplexExplanationsOfNonMatches, 
TestBooleanOr, TestSpanExplanations, TestSpanExplanationsOfNonMatches, 
TestCharsRef, TestAutomatonQuery, TestVirtualMethod, TestFieldReuse, 
TestCodecHoldsOpenFiles, MultiCollectorTest, TestDocBoost, 
TestSimpleAttributeImpl, TestTermdocPerf, TestIndexCommit, Test2BPostingsBytes, 
TestIOUtils, TestByteArrayDataInput, Test2BPagedBytes, TestCharArraySet, 
TestDelegatingAnalyzerWrapper, TestReusableStringReader, TestWordlistLoader, 
TestCodecUtil, TestGrowableByteArrayDataOutput, TestBlockPostingsFormat, 
TestLucene50TermVectorsFormat, TestLucene53NormsFormat, 
TestLucene60PointsFormat, Test2BSortedDocValuesFixedSorted, 
Test2BSortedDocValuesOrds, TestAllFilesHaveChecksumFooter, TestDocIDMerger, 
TestDocValues, TestExitableDirectoryReader, TestFilterDirectoryReader, 
TestIndexSorting]
   [junit4] Completed [378/442 (1!)] on J2 in 530.92s, 47 tests, 2 errors <<< 
FAILURES!

[...truncated 229 lines...]
   [junit4] JVM J2: stdout was not empty, see: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/build/core/test/temp/junit4-J2-20161125_180049_954.sysout
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/heapdumps/java_pid14871.hprof
 ...
   [junit4] Heap dump file created [429408687 bytes in 2.285 secs]
   [junit4] <<< JVM J2: EOF ----

[...truncated 9903 lines...]
   [junit4] Suite: 
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B140AD8EB85F37B7-001/init-core-data-001
   [junit4]   2> 633376 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B140AD8EB85F37B7]-worker) [    
] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: 
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 633377 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B140AD8EB85F37B7]-worker) [    
] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B140AD8EB85F37B7-001/tempDir-001
   [junit4]   2> 633378 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B140AD8EB85F37B7]-worker) [    
] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 633378 INFO  (Thread-5110) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 633378 INFO  (Thread-5110) [    ] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 633478 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B140AD8EB85F37B7]-worker) [    
] o.a.s.c.ZkTestServer start zk server on port:57830
   [junit4]   2> 633492 INFO  (jetty-launcher-995-thread-1) [    ] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 633492 INFO  (jetty-launcher-995-thread-2) [    ] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 633493 INFO  (jetty-launcher-995-thread-3) [    ] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 633494 INFO  (jetty-launcher-995-thread-1) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@799ff47c{/solr,null,AVAILABLE}
   [junit4]   2> 633495 INFO  (jetty-launcher-995-thread-4) [    ] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 633497 INFO  (jetty-launcher-995-thread-2) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@4a004b84{/solr,null,AVAILABLE}
   [junit4]   2> 633500 INFO  (jetty-launcher-995-thread-3) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@68e7925c{/solr,null,AVAILABLE}
   [junit4]   2> 633502 INFO  (jetty-launcher-995-thread-2) [    ] 
o.e.j.s.ServerConnector Started ServerConnector@7b600255{SSL,[ssl, 
http/1.1]}{127.0.0.1:43683}
   [junit4]   2> 633502 INFO  (jetty-launcher-995-thread-2) [    ] 
o.e.j.s.Server Started @638206ms
   [junit4]   2> 633502 INFO  (jetty-launcher-995-thread-2) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=43683}
   [junit4]   2> 633502 ERROR (jetty-launcher-995-thread-2) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 633502 INFO  (jetty-launcher-995-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
6.4.0
   [junit4]   2> 633503 INFO  (jetty-launcher-995-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 633503 INFO  (jetty-launcher-995-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 633503 INFO  (jetty-launcher-995-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-11-25T21:37:19.968Z
   [junit4]   2> 633506 INFO  (jetty-launcher-995-thread-1) [    ] 
o.e.j.s.ServerConnector Started ServerConnector@4799d161{SSL,[ssl, 
http/1.1]}{127.0.0.1:54927}
   [junit4]   2> 633506 INFO  (jetty-launcher-995-thread-3) [    ] 
o.e.j.s.ServerConnector Started ServerConnector@25d338c3{SSL,[ssl, 
http/1.1]}{127.0.0.1:51856}
   [junit4]   2> 633506 INFO  (jetty-launcher-995-thread-1) [    ] 
o.e.j.s.Server Started @638209ms
   [junit4]   2> 633506 INFO  (jetty-launcher-995-thread-3) [    ] 
o.e.j.s.Server Started @638209ms
   [junit4]   2> 633506 INFO  (jetty-launcher-995-thread-1) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=54927}
   [junit4]   2> 633506 INFO  (jetty-launcher-995-thread-3) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=51856}
   [junit4]   2> 633506 ERROR (jetty-launcher-995-thread-1) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 633506 ERROR (jetty-launcher-995-thread-3) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 633506 INFO  (jetty-launcher-995-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
6.4.0
   [junit4]   2> 633506 INFO  (jetty-launcher-995-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
6.4.0
   [junit4]   2> 633506 INFO  (jetty-launcher-995-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 633506 INFO  (jetty-launcher-995-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 633506 INFO  (jetty-launcher-995-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 633506 INFO  (jetty-launcher-995-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 633506 INFO  (jetty-launcher-995-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-11-25T21:37:19.971Z
   [junit4]   2> 633506 INFO  (jetty-launcher-995-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-11-25T21:37:19.971Z
   [junit4]   2> 633512 INFO  (jetty-launcher-995-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 633513 INFO  (jetty-launcher-995-thread-4) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@7d49f821{/solr,null,AVAILABLE}
   [junit4]   2> 633515 INFO  (jetty-launcher-995-thread-4) [    ] 
o.e.j.s.ServerConnector Started ServerConnector@a505076{SSL,[ssl, 
http/1.1]}{127.0.0.1:51640}
   [junit4]   2> 633515 INFO  (jetty-launcher-995-thread-4) [    ] 
o.e.j.s.Server Started @638219ms
   [junit4]   2> 633515 INFO  (jetty-launcher-995-thread-4) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=51640}
   [junit4]   2> 633515 ERROR (jetty-launcher-995-thread-4) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 633515 INFO  (jetty-launcher-995-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
6.4.0
   [junit4]   2> 633515 INFO  (jetty-launcher-995-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 633516 INFO  (jetty-launcher-995-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 633516 INFO  (jetty-launcher-995-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-11-25T21:37:19.981Z
   [junit4]   2> 633520 INFO  (jetty-launcher-995-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 633521 INFO  (jetty-launcher-995-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 633522 INFO  (jetty-launcher-995-thread-3) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 633523 INFO  (jetty-launcher-995-thread-3) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57830/solr
   [junit4]   2> 633540 INFO  (jetty-launcher-995-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 633543 INFO  (jetty-launcher-995-thread-1) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 633543 INFO  (jetty-launcher-995-thread-1) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57830/solr
   [junit4]   2> 633548 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from 
client sessionid 0x1589d6bd5e40001, likely client has closed socket
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 633551 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from 
client sessionid 0x1589d6bd5e40004, likely client has closed socket
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 633553 INFO  (jetty-launcher-995-thread-2) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 633553 INFO  (jetty-launcher-995-thread-4) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 633567 INFO  (jetty-launcher-995-thread-4) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57830/solr
   [junit4]   2> 633568 INFO  (jetty-launcher-995-thread-2) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57830/solr
   [junit4]   2> 633593 INFO  (jetty-launcher-995-thread-1) 
[n:127.0.0.1:54927_solr    ] o.a.s.c.OverseerElectionContext I am going to be 
the leader 127.0.0.1:54927_solr
   [junit4]   2> 633600 INFO  (jetty-launcher-995-thread-2) 
[n:127.0.0.1:43683_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:43683_solr
   [junit4]   2> 633606 INFO  (jetty-launcher-995-thread-1) 
[n:127.0.0.1:54927_solr    ] o.a.s.c.Overseer Overseer 
(id=97000478464016394-127.0.0.1:54927_solr-n_0000000000) starting
   [junit4]   2> 633607 INFO  
(zkCallback-1015-thread-1-processing-n:127.0.0.1:54927_solr) 
[n:127.0.0.1:54927_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 633613 INFO  
(zkCallback-1016-thread-1-processing-n:127.0.0.1:43683_solr) 
[n:127.0.0.1:43683_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 633620 INFO  
(zkCallback-1007-thread-1-processing-n:127.0.0.1:51856_solr) 
[n:127.0.0.1:51856_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 633624 INFO  (jetty-launcher-995-thread-4) 
[n:127.0.0.1:51640_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 633626 INFO  (jetty-launcher-995-thread-3) 
[n:127.0.0.1:51856_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:51856_solr
   [junit4]   2> 633628 INFO  
(zkCallback-1016-thread-1-processing-n:127.0.0.1:43683_solr) 
[n:127.0.0.1:43683_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (1) -> (2)
   [junit4]   2> 633629 INFO  
(zkCallback-1015-thread-1-processing-n:127.0.0.1:54927_solr) 
[n:127.0.0.1:54927_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (1) -> (2)
   [junit4]   2> 633630 INFO  (jetty-launcher-995-thread-4) 
[n:127.0.0.1:51640_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:51640_solr
   [junit4]   2> 633630 INFO  
(zkCallback-1007-thread-1-processing-n:127.0.0.1:51856_solr) 
[n:127.0.0.1:51856_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (1) -> (2)
   [junit4]   2> 633635 INFO  
(zkCallback-1016-thread-1-processing-n:127.0.0.1:43683_solr) 
[n:127.0.0.1:43683_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (3)
   [junit4]   2> 633636 INFO  
(zkCallback-1015-thread-1-processing-n:127.0.0.1:54927_solr) 
[n:127.0.0.1:54927_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (3)
   [junit4]   2> 633644 INFO  
(zkCallback-1007-thread-1-processing-n:127.0.0.1:51856_solr) 
[n:127.0.0.1:51856_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (3)
   [junit4]   2> 633663 INFO  (jetty-launcher-995-thread-1) 
[n:127.0.0.1:54927_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:54927_solr
   [junit4]   2> 633665 INFO  
(zkCallback-1017-thread-1-processing-n:127.0.0.1:51640_solr) 
[n:127.0.0.1:51640_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (1) -> (4)
   [junit4]   2> 633676 INFO  
(zkCallback-1015-thread-1-processing-n:127.0.0.1:54927_solr) 
[n:127.0.0.1:54927_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 633688 INFO  
(zkCallback-1016-thread-1-processing-n:127.0.0.1:43683_solr) 
[n:127.0.0.1:43683_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 633688 INFO  
(zkCallback-1007-thread-1-processing-n:127.0.0.1:51856_solr) 
[n:127.0.0.1:51856_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 633703 INFO  (jetty-launcher-995-thread-3) 
[n:127.0.0.1:51856_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B140AD8EB85F37B7-001/tempDir-001/node3/.
   [junit4]   2> 633716 INFO  (jetty-launcher-995-thread-4) 
[n:127.0.0.1:51640_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B140AD8EB85F37B7-001/tempDir-001/node4/.
   [junit4]   2> 633730 INFO  (jetty-launcher-995-thread-1) 
[n:127.0.0.1:54927_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B140AD8EB85F37B7-001/tempDir-001/node1/.
   [junit4]   2> 633774 INFO  (jetty-launcher-995-thread-2) 
[n:127.0.0.1:43683_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B140AD8EB85F37B7-001/tempDir-001/node2/.
   [junit4]   2> 633782 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B140AD8EB85F37B7]-worker) [    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 633783 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B140AD8EB85F37B7]-worker) [    
] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:57830/solr ready
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 633856 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B140AD8EB85F37B7]-worker) [    
] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried 
hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 633866 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B140AD8EB85F37B7]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 633873 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B140AD8EB85F37B7]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 633894 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B140AD8EB85F37B7]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs
 to ./temp/Jetty_localhost_55062_hdfs____.swolca/webapp
   [junit4]   2> 634273 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B140AD8EB85F37B7]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:55062
   [junit4]   2> 634340 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B140AD8EB85F37B7]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 634342 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B140AD8EB85F37B7]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 634355 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B140AD8EB85F37B7]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_52743_datanode____33gyow/webapp
   [junit4]   2> 634756 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B140AD8EB85F37B7]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:52743
   [junit4]   2> 634955 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B140AD8EB85F37B7]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 634957 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B140AD8EB85F37B7]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 634990 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B140AD8EB85F37B7]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_49185_datanode____czs88g/webapp
   [junit4]   2> 635197 INFO  (IPC Server handler 3 on 57636) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-3ce20260-3ff3-4305-be82-843fe5b27f4b node 
DatanodeRegistration(127.0.0.1:33152, 
datanodeUuid=7cc9957e-4136-4a18-a247-46eea8ce185f, infoPort=47048, 
infoSecurePort=0, ipcPort=34189, 
storageInfo=lv=-56;cid=testClusterID;nsid=178918076;c=0), blocks: 0, 
hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 635197 INFO  (IPC Server handler 3 on 57636) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-8d5f91c3-4b6c-4ed9-b820-f1bb9b7956f5 node 
DatanodeRegistration(127.0.0.1:33152, 
datanodeUuid=7cc9957e-4136-4a18-a247-46eea8ce185f, infoPort=47048, 
infoSecurePort=0, ipcPort=34189, 
storageInfo=lv=-56;cid=testClusterID;nsid=178918076;c=0), blocks: 0, 
hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 635467 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B140AD8EB85F37B7]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:49185
   [junit4]   2> 635571 INFO  (IPC Server handler 4 on 57636) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-0b3f8de0-4abb-469c-9b4f-718ff7281aac node 
DatanodeRegistration(127.0.0.1:50054, 
datanodeUuid=1db72cdd-a548-4ca4-9901-4d837628f98d, infoPort=49876, 
infoSecurePort=0, ipcPort=43012, 
storageInfo=lv=-56;cid=testClusterID;nsid=178918076;c=0), blocks: 0, 
hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 635571 INFO  (IPC Server handler 4 on 57636) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-26e3fc30-0880-4075-b4a3-263ef5d1b3a9 node 
DatanodeRegistration(127.0.0.1:50054, 
datanodeUuid=1db72cdd-a548-4ca4-9901-4d837628f98d, infoPort=49876, 
infoSecurePort=0, ipcPort=43012, 
storageInfo=lv=-56;cid=testClusterID;nsid=178918076;c=0), blocks: 0, 
hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 635815 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testCreationAndDeletion-seed#[B140AD8EB85F37B7])
 [    ] o.a.s.SolrTestCaseJ4 ###Starting testCreationAndDeletion
   [junit4]   2> 635820 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testCreationAndDeletion-seed#[B140AD8EB85F37B7])
 [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 635883 INFO  (qtp2040721350-9407) [n:127.0.0.1:51640_solr    ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params 
replicationFactor=1&collection.configName=conf&name=created_and_deleted&action=CREATE&numShards=1&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 635886 INFO  
(OverseerThreadFactory-2336-thread-1-processing-n:127.0.0.1:54927_solr) 
[n:127.0.0.1:54927_solr    ] o.a.s.c.CreateCollectionCmd Create collection 
created_and_deleted
   [junit4]   2> 636061 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=conf&newCollection=true&name=created_and_deleted_shard1_replica1&action=CREATE&numShards=1&collection=created_and_deleted&shard=shard1&wt=javabin&version=2
   [junit4]   2> 636169 INFO  
(zkCallback-1015-thread-3-processing-n:127.0.0.1:54927_solr) 
[n:127.0.0.1:54927_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/created_and_deleted/state.json] for collection 
[created_and_deleted] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 637075 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
   [junit4]   2> 637084 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.s.IndexSchema [created_and_deleted_shard1_replica1] Schema name=minimal
   [junit4]   2> 637087 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 637088 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.CoreContainer Creating SolrCore 'created_and_deleted_shard1_replica1' 
using configuration from collection created_and_deleted
   [junit4]   2> 637112 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.c.HdfsDirectoryFactory 
solr.hdfs.home=hdfs://localhost:57636/data
   [junit4]   2> 637112 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Solr 
Kerberos Authentication disabled
   [junit4]   2> 637113 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.c.SolrCore 
[[created_and_deleted_shard1_replica1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B140AD8EB85F37B7-001/tempDir-001/node1/created_and_deleted_shard1_replica1],
 dataDir=[hdfs://localhost:57636/data/created_and_deleted/core_node1/data/]
   [junit4]   2> 637114 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating 
directory factory for path 
hdfs://localhost:57636/data/created_and_deleted/core_node1/data/snapshot_metadata
   [junit4]   2> 637134 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of 
slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 637134 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache 
target memory usage, slab size of [16777216] will allocate [1] slabs and use 
~[16777216] bytes
   [junit4]   2> 637134 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating 
new global HDFS BlockCache
   [junit4]   2> 638974 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on 
write is disabled
   [junit4]   2> 638984 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating 
directory factory for path 
hdfs://localhost:57636/data/created_and_deleted/core_node1/data
   [junit4]   2> 639032 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating 
directory factory for path 
hdfs://localhost:57636/data/created_and_deleted/core_node1/data/index
   [junit4]   2> 639040 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of 
slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 639040 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache 
target memory usage, slab size of [16777216] will allocate [1] slabs and use 
~[16777216] bytes
   [junit4]   2> 639047 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on 
write is disabled
   [junit4]   2> 639104 INFO  (IPC Server handler 1 on 57636) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:33152 is 
added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, 
primaryNodeIndex=-1, 
replicas=[ReplicaUC[[DISK]DS-0b3f8de0-4abb-469c-9b4f-718ff7281aac:NORMAL:127.0.0.1:50054|RBW],
 
ReplicaUC[[DISK]DS-8d5f91c3-4b6c-4ed9-b820-f1bb9b7956f5:NORMAL:127.0.0.1:33152|FINALIZED]]}
 size 0
   [junit4]   2> 639107 INFO  (IPC Server handler 3 on 57636) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50054 is 
added to blk_1073741825_1001 size 71
   [junit4]   2> 639160 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog 
implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 639160 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.u.UpdateLog Initializing 
UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 
maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 639160 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.u.HdfsUpdateLog Initializing 
HdfsUpdateLog: tlogDfsReplication=3
   [junit4]   2> 639175 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: 
disabled
   [junit4]   2> 639175 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: 
disabled
   [junit4]   2> 639295 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening 
[Searcher@4bf0564e[created_and_deleted_shard1_replica1] main]
   [junit4]   2> 639300 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.r.ManagedResourceStorage 
Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 639301 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded 
null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 639307 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.h.ReplicationHandler Commits will 
be reserved for  10000
   [junit4]   2> 639307 INFO  
(searcherExecutor-2345-thread-1-processing-n:127.0.0.1:54927_solr 
x:created_and_deleted_shard1_replica1 s:shard1 c:created_and_deleted 
r:core_node1) [n:127.0.0.1:54927_solr c:created_and_deleted s:shard1 
r:core_node1 x:created_and_deleted_shard1_replica1] o.a.s.c.SolrCore 
[created_and_deleted_shard1_replica1] Registered new searcher 
Searcher@4bf0564e[created_and_deleted_shard1_replica1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 639307 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.u.UpdateLog Could not find max 
version in index or recent updates, using new clock 1552007661640220672
   [junit4]   2> 639330 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.c.ShardLeaderElectionContext 
Enough replicas found to continue.
   [junit4]   2> 639332 INFO  
(zkCallback-1015-thread-2-processing-n:127.0.0.1:54927_solr) 
[n:127.0.0.1:54927_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/created_and_deleted/state.json] for collection 
[created_and_deleted] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 639332 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may 
be the new leader - try and sync
   [junit4]   2> 639333 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to 
https://127.0.0.1:54927/solr/created_and_deleted_shard1_replica1/
   [junit4]   2> 639333 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.c.SyncStrategy Sync Success - now 
sync replicas to me
   [junit4]   2> 639333 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.c.SyncStrategy 
https://127.0.0.1:54927/solr/created_and_deleted_shard1_replica1/ has no 
replicas
   [junit4]   2> 639346 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am 
the new leader: 
https://127.0.0.1:54927/solr/created_and_deleted_shard1_replica1/ shard1
   [junit4]   2> 639450 INFO  
(zkCallback-1015-thread-2-processing-n:127.0.0.1:54927_solr) 
[n:127.0.0.1:54927_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/created_and_deleted/state.json] for collection 
[created_and_deleted] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 639501 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.c.ZkController I am the leader, no 
recovery necessary
   [junit4]   2> 639503 INFO  (qtp53240269-9369) [n:127.0.0.1:54927_solr 
c:created_and_deleted s:shard1 r:core_node1 
x:created_and_deleted_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null 
path=/admin/cores 
params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=created_and_deleted_shard1_replica1&action=CREATE&numShards=1&collection=created_and_deleted&shard=shard1&wt=javabin&version=2}
 status=0 QTime=3443
   [junit4]   2> 639507 INFO  (qtp2040721350-9407) [n:127.0.0.1:51640_solr    ] 
o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 
30 seconds. Check all shard replicas
   [junit4]   2> 639611 INFO  
(zkCallback-1015-thread-3-processing-n:127.0.0.1:54927_s

[...truncated too long message...]

vd, _tc.nvm, _tc.si, _tc_FSTOrd50_0.doc, _tc_FSTOrd50_0.tbk, 
_tc_FSTOrd50_0.tix, _td.fdt, _td.fdx, _td.fnm, _td.nvd, _td.nvm, _td.si, 
_td_FSTOrd50_0.doc, _td_FSTOrd50_0.tbk, _td_FSTOrd50_0.tix, _te.fdt, _te.fdx, 
_te.fnm, _te.nvd, _te.nvm, _te.si, _te_FSTOrd50_0.doc, _te_FSTOrd50_0.tbk, 
_te_FSTOrd50_0.tix, _tf.fdt, _tf.fdx, _tf.fnm, _tf.nvd, _tf.nvm, _tf.si, 
_tf_FSTOrd50_0.doc, _tf_FSTOrd50_0.tbk, _tf_FSTOrd50_0.tix, _tg.fdt, _tg.fdx, 
_tg.fnm, _tg.nvd, _tg.nvm, _tg.si, _tg_FSTOrd50_0.doc, _tg_FSTOrd50_0.tbk, 
_tg_FSTOrd50_0.tix, _th.fdt, _th.fdx, _th.fnm, _th.nvd, _th.nvm, _th.si, 
_th_FSTOrd50_0.doc, _th_FSTOrd50_0.tbk, _th_FSTOrd50_0.tix, _ti.fdt, _ti.fdx, 
_ti.fnm, _ti.nvd, _ti.nvm, _ti.si, _ti_FSTOrd50_0.doc, _ti_FSTOrd50_0.tbk, 
_ti_FSTOrd50_0.tix, _tj.fdt, _tj.fdx, _tj.fnm, _tj.nvd, _tj.nvm, _tj.si, 
_tj_FSTOrd50_0.doc, _tj_FSTOrd50_0.tbk, _tj_FSTOrd50_0.tix, _tk.fdt, _tk.fdx, 
_tk.fnm, _tk.nvd, _tk.nvm, _tk.si, _tk_FSTOrd50_0.doc, _tk_FSTOrd50_0.tbk, 
_tk_FSTOrd50_0.tix, _tl.fdt, _tl.fdx, _tl.fnm, _tl.nvd, _tl.nvm, _tl.si, 
_tl_FSTOrd50_0.doc, _tl_FSTOrd50_0.tbk, _tl_FSTOrd50_0.tix, _tm.fdt, _tm.fdx, 
_tm.fnm, _tm.nvd, _tm.nvm, _tm.si, _tm_FSTOrd50_0.doc, _tm_FSTOrd50_0.tbk, 
_tm_FSTOrd50_0.tix, _tn.fdt, _tn.fdx, _tn.fnm, _tn.nvd, _tn.nvm, _tn.si, 
_tn_FSTOrd50_0.doc, _tn_FSTOrd50_0.tbk, _tn_FSTOrd50_0.tix, _to.fdt, _to.fdx, 
_to.fnm, _to.nvd, _to.nvm, _to.si, _to_FSTOrd50_0.doc, _to_FSTOrd50_0.tbk, 
_to_FSTOrd50_0.tix, _tp.fdt, _tp.fdx, _tp.fnm, _tp.nvd, _tp.nvm, _tp.si, 
_tp_FSTOrd50_0.doc, _tp_FSTOrd50_0.tbk, _tp_FSTOrd50_0.tix, segments_2]}]> but 
was:<[{indexVersion=1480111658207,generation=2,filelist=[_su.fdt, _su.fdx, 
_su.fnm, _su.nvd, _su.nvm, _su.si, _su_FSTOrd50_0.doc, _su_FSTOrd50_0.tbk, 
_su_FSTOrd50_0.tix, _t3.fdt, _t3.fdx, _t3.fnm, _t3.nvd, _t3.nvm, _t3.si, 
_t3_FSTOrd50_0.doc, _t3_FSTOrd50_0.tbk, _t3_FSTOrd50_0.tix, _t7.cfe, _t7.cfs, 
_t7.si, _ta.cfe, _ta.cfs, _ta.si, _tb.fdt, _tb.fdx, _tb.fnm, _tb.nvd, _tb.nvm, 
_tb.si, _tb_FSTOrd50_0.doc, _tb_FSTOrd50_0.tbk, _tb_FSTOrd50_0.tix, _tc.fdt, 
_tc.fdx, _tc.fnm, _tc.nvd, _tc.nvm, _tc.si, _tc_FSTOrd50_0.doc, 
_tc_FSTOrd50_0.tbk, _tc_FSTOrd50_0.tix, _td.fdt, _td.fdx, _td.fnm, _td.nvd, 
_td.nvm, _td.si, _td_FSTOrd50_0.doc, _td_FSTOrd50_0.tbk, _td_FSTOrd50_0.tix, 
_te.fdt, _te.fdx, _te.fnm, _te.nvd, _te.nvm, _te.si, _te_FSTOrd50_0.doc, 
_te_FSTOrd50_0.tbk, _te_FSTOrd50_0.tix, _tf.fdt, _tf.fdx, _tf.fnm, _tf.nvd, 
_tf.nvm, _tf.si, _tf_FSTOrd50_0.doc, _tf_FSTOrd50_0.tbk, _tf_FSTOrd50_0.tix, 
_tg.fdt, _tg.fdx, _tg.fnm, _tg.nvd, _tg.nvm, _tg.si, _tg_FSTOrd50_0.doc, 
_tg_FSTOrd50_0.tbk, _tg_FSTOrd50_0.tix, _th.fdt, _th.fdx, _th.fnm, _th.nvd, 
_th.nvm, _th.si, _th_FSTOrd50_0.doc, _th_FSTOrd50_0.tbk, _th_FSTOrd50_0.tix, 
_ti.fdt, _ti.fdx, _ti.fnm, _ti.nvd, _ti.nvm, _ti.si, _ti_FSTOrd50_0.doc, 
_ti_FSTOrd50_0.tbk, _ti_FSTOrd50_0.tix, _tj.fdt, _tj.fdx, _tj.fnm, _tj.nvd, 
_tj.nvm, _tj.si, _tj_FSTOrd50_0.doc, _tj_FSTOrd50_0.tbk, _tj_FSTOrd50_0.tix, 
_tk.fdt, _tk.fdx, _tk.fnm, _tk.nvd, _tk.nvm, _tk.si, _tk_FSTOrd50_0.doc, 
_tk_FSTOrd50_0.tbk, _tk_FSTOrd50_0.tix, _tl.fdt, _tl.fdx, _tl.fnm, _tl.nvd, 
_tl.nvm, _tl.si, _tl_FSTOrd50_0.doc, _tl_FSTOrd50_0.tbk, _tl_FSTOrd50_0.tix, 
_tm.fdt, _tm.fdx, _tm.fnm, _tm.nvd, _tm.nvm, _tm.si, _tm_FSTOrd50_0.doc, 
_tm_FSTOrd50_0.tbk, _tm_FSTOrd50_0.tix, _tn.fdt, _tn.fdx, _tn.fnm, _tn.nvd, 
_tn.nvm, _tn.si, _tn_FSTOrd50_0.doc, _tn_FSTOrd50_0.tbk, _tn_FSTOrd50_0.tix, 
_to.fdt, _to.fdx, _to.fnm, _to.nvd, _to.nvm, _to.si, _to_FSTOrd50_0.doc, 
_to_FSTOrd50_0.tbk, _to_FSTOrd50_0.tix, _tp.fdt, _tp.fdx, _tp.fnm, _tp.nvd, 
_tp.nvm, _tp.si, _tp_FSTOrd50_0.doc, _tp_FSTOrd50_0.tbk, _tp_FSTOrd50_0.tix, 
segments_2]}, {indexVersion=1480111658207,generation=3,filelist=[_su.fdt, 
_su.fdx, _su.fnm, _su.nvd, _su.nvm, _su.si, _su_FSTOrd50_0.doc, 
_su_FSTOrd50_0.tbk, _su_FSTOrd50_0.tix, _t3.fdt, _t3.fdx, _t3.fnm, _t3.nvd, 
_t3.nvm, _t3.si, _t3_FSTOrd50_0.doc, _t3_FSTOrd50_0.tbk, _t3_FSTOrd50_0.tix, 
_t7.cfe, _t7.cfs, _t7.si, _ta.cfe, _ta.cfs, _ta.si, _tc.fdt, _tc.fdx, _tc.fnm, 
_tc.nvd, _tc.nvm, _tc.si, _tc_FSTOrd50_0.doc, _tc_FSTOrd50_0.tbk, 
_tc_FSTOrd50_0.tix, _td.fdt, _td.fdx, _td.fnm, _td.nvd, _td.nvm, _td.si, 
_td_FSTOrd50_0.doc, _td_FSTOrd50_0.tbk, _td_FSTOrd50_0.tix, _te.fdt, _te.fdx, 
_te.fnm, _te.nvd, _te.nvm, _te.si, _te_FSTOrd50_0.doc, _te_FSTOrd50_0.tbk, 
_te_FSTOrd50_0.tix, _tf.fdt, _tf.fdx, _tf.fnm, _tf.nvd, _tf.nvm, _tf.si, 
_tf_FSTOrd50_0.doc, _tf_FSTOrd50_0.tbk, _tf_FSTOrd50_0.tix, _tg.fdt, _tg.fdx, 
_tg.fnm, _tg.nvd, _tg.nvm, _tg.si, _tg_FSTOrd50_0.doc, _tg_FSTOrd50_0.tbk, 
_tg_FSTOrd50_0.tix, _th.fdt, _th.fdx, _th.fnm, _th.nvd, _th.nvm, _th.si, 
_th_FSTOrd50_0.doc, _th_FSTOrd50_0.tbk, _th_FSTOrd50_0.tix, _ti.fdt, _ti.fdx, 
_ti.fnm, _ti.nvd, _ti.nvm, _ti.si, _ti_FSTOrd50_0.doc, _ti_FSTOrd50_0.tbk, 
_ti_FSTOrd50_0.tix, _tj.fdt, _tj.fdx, _tj.fnm, _tj.nvd, _tj.nvm, _tj.si, 
_tj_FSTOrd50_0.doc, _tj_FSTOrd50_0.tbk, _tj_FSTOrd50_0.tix, _tl.fdt, _tl.fdx, 
_tl.fnm, _tl.nvd, _tl.nvm, _tl.si, _tl_FSTOrd50_0.doc, _tl_FSTOrd50_0.tbk, 
_tl_FSTOrd50_0.tix, _tm.fdt, _tm.fdx, _tm.fnm, _tm.nvd, _tm.nvm, _tm.si, 
_tm_FSTOrd50_0.doc, _tm_FSTOrd50_0.tbk, _tm_FSTOrd50_0.tix, _tn.fdt, _tn.fdx, 
_tn.fnm, _tn.nvd, _tn.nvm, _tn.si, _tn_FSTOrd50_0.doc, _tn_FSTOrd50_0.tbk, 
_tn_FSTOrd50_0.tix, _to.fdt, _to.fdx, _to.fnm, _to.nvd, _to.nvm, _to.si, 
_to_FSTOrd50_0.doc, _to_FSTOrd50_0.tbk, _to_FSTOrd50_0.tix, _tp.fdt, _tp.fdx, 
_tp.fnm, _tp.nvd, _tp.nvm, _tp.si, _tp_FSTOrd50_0.doc, _tp_FSTOrd50_0.tbk, 
_tp_FSTOrd50_0.tix, _tq.cfe, _tq.cfs, _tq.si, segments_3]}]>
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([B140AD8EB85F37B7:9497B6BEC81739B4]:0)
   [junit4]    >        at 
org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1229)
   [junit4]    >        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2453875 INFO  
(SUITE-TestReplicationHandler-seed#[B140AD8EB85F37B7]-worker) [    ] 
o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_B140AD8EB85F37B7-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): 
{name=FSTOrd50, id=FSTOrd50, newname=FSTOrd50}, docValues:{}, 
maxPointsInLeafNode=894, maxMBSortInHeap=5.33029763703903, 
sim=RandomSimilarity(queryNorm=true,coord=yes): {}, locale=sr-ME, 
timezone=Africa/Mogadishu
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 
1.8.0_102 (64-bit)/cpus=4,threads=1,free=283504600,total=419954688
   [junit4]   2> NOTE: All tests run in this JVM: 
[TestSuggestSpellingConverter, TestSearchPerf, 
DistributedFacetPivotLongTailTest, LoggingHandlerTest, 
IndexBasedSpellCheckerTest, AnalyticsMergeStrategyTest, DeleteStatusTest, 
TestTrieFacet, WordBreakSolrSpellCheckerTest, DistributedExpandComponentTest, 
URLClassifyProcessorTest, TestSimpleQParserPlugin, 
UpdateRequestProcessorFactoryTest, TestMiniSolrCloudCluster, HighlighterTest, 
TestSchemaSimilarityResource, ClusterStateUpdateTest, 
TestLeaderInitiatedRecoveryThread, BooleanFieldTest, ZkStateReaderTest, 
SampleTest, CollectionsAPIAsyncDistributedZkTest, ExplicitHLLTest, 
TestConfigSetImmutable, TestBM25SimilarityFactory, DistanceUnitsTest, 
TestTolerantUpdateProcessorCloud, RAMDirectoryFactoryTest, FileUtilsTest, 
TestNamedUpdateProcessors, TestCharFilters, HdfsThreadLeakTest, 
OverseerTaskQueueTest, TestSolrQueryParserDefaultOperatorResource, 
TestSolrConfigHandler, TestWordDelimiterFilterFactory, SaslZkACLProviderTest, 
TestSQLHandler, TestMacroExpander, TestPartialUpdateDeduplication, 
DistributedFacetPivotWhiteBoxTest, TestFastOutputStream, 
TestSolrCloudSnapshots, LeaderFailoverAfterPartitionTest, 
SegmentsInfoRequestHandlerTest, IndexSchemaTest, SmileWriterTest, 
GraphQueryTest, AddSchemaFieldsUpdateProcessorFactoryTest, 
TestConfigSetsAPIZkFailure, TestAddFieldRealTimeGet, TestFiltering, 
SearchHandlerTest, TestRequestStatusCollectionAPI, BlockJoinFacetDistribTest, 
DocValuesMissingTest, AliasIntegrationTest, TestExceedMaxTermLength, 
SortByFunctionTest, TestSchemaManager, TestCorePropertiesReload, 
DistributedIntervalFacetingTest, TestImplicitCoreProperties, ConfigSetsAPITest, 
SolrInfoMBeanTest, TestTrie, MultiTermTest, TestDefaultSearchFieldResource, 
SpatialFilterTest, QueryResultKeyTest, TestSurroundQueryParser, 
ZkSolrClientTest, HdfsCollectionsAPIDistributedZkTest, SoftAutoCommitTest, 
SolrIndexConfigTest, TestSolrConfigHandlerConcurrent, SpellCheckComponentTest, 
TestCSVResponseWriter, CSVRequestHandlerTest, TestExactStatsCache, 
TestCloudPseudoReturnFields, TestCollectionAPI, TestFuzzyAnalyzedSuggestions, 
TestJsonRequest, TestReplicationHandlerBackup, SliceStateTest, 
DataDrivenBlockJoinTest, TestOnReconnectListenerSupport, TestBlobHandler, 
DirectSolrSpellCheckerTest, SimplePostToolTest, JsonLoaderTest, 
CloneFieldUpdateProcessorFactoryTest, TestInitQParser, TestDistributedGrouping, 
TestRTGBase, BasicFunctionalityTest, CopyFieldTest, TestSolrCLIRunExample, 
TriLevelCompositeIdRoutingTest, SolrIndexSplitterTest, 
TestHighFrequencyDictionaryFactory, TestPerFieldSimilarityClassic, 
TestSolrFieldCacheMBean, IndexSchemaRuntimeFieldTest, TestJettySolrRunner, 
ChaosMonkeySafeLeaderTest, CollectionsAPIDistributedZkTest, 
OpenCloseCoreStressTest, ShardRoutingTest, TestReplicationHandler]
   [junit4] Completed [377/657 (2!)] on J1 in 783.97s, 15 tests, 1 failure, 1 
skipped <<< FAILURES!

[...truncated 11989 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/build.xml:773:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/build.xml:717:
 Some of the tests produced a heap dump, but did not fail. Maybe a suppressed 
OutOfMemoryError? Dumps created:
* java_pid14871.hprof

Total time: 294 minutes 48 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to