mikemccand commented on PR #12624:
URL: https://github.com/apache/lucene/pull/12624#issuecomment-1828548480
Hmm I'm running `Test2BFSTs` on this patch and noticed it seems to take very
much longer during the `TEST: now verify` step where it confirms the built FST
accepts all the inputs it just compiled into it.
I `jstack`'d it and found this:
```
java.lang.Thread.State: RUNNABLE
at
java.nio.ByteBuffer.position([email protected]/ByteBuffer.java:1516)
at java.nio.ByteBuffer.position([email protected]/ByteBuffer.java:267)
at java.nio.Buffer.<init>([email protected]/Buffer.java:246)
at java.nio.ByteBuffer.<init>([email protected]/ByteBuffer.java:288)
at
java.nio.HeapByteBuffer.<init>([email protected]/HeapByteBuffer.java:95)
at
java.nio.HeapByteBufferR.<init>([email protected]/HeapByteBufferR.java:102)
at
java.nio.HeapByteBufferR.duplicate([email protected]/HeapByteBufferR.java:135)
at
java.nio.HeapByteBufferR.asReadOnlyBuffer([email protected]/HeapByteBufferR.java:148)
at
org.apache.lucene.store.ByteBuffersDataInput.<init>(ByteBuffersDataInput.java:60)
at
org.apache.lucene.store.ByteBuffersDataOutput.toDataInput(ByteBuffersDataOutput.java:279)
at
org.apache.lucene.util.fst.ReadWriteDataOutput.getReverseBytesReader(ReadWriteDataOutput.java:52)
at org.apache.lucene.util.fst.FST.getBytesReader(FST.java:1181)
at org.apache.lucene.util.fst.Util.get(Util.java:50)
at org.apache.lucene.util.fst.Test2BFST.test(Test2BFST.java:113)
```
It looks like getting the reversed reader has maybe become quite a bit more
expensive than `BytesStore` was?
Note that `Test2BFSTs` is quite silly -- it uses `Util.get` for every
lookup, instead of pulling and reusing a `ReverseBytesReader` like most "real"
usages of FST (e.g. terms dict) will do. So perhaps this performance change
doesn't really matter in practice? Though I wonder if all consumers of FST are
re-using their reversed readers?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
