Re: [JENKINS] Lucene » Lucene-NightlyTests-9.x - Build # 665 - Unstable!
OK I opened https://github.com/apache/lucene/pull/12535 Mike McCandless http://blog.mikemccandless.com On Sat, Sep 2, 2023 at 7:17 AM Michael McCandless wrote: > > The code is just good old socket accept loop as we have all learned it > in school when we were fighting to write a small echo server with C. > > LOL this is all my fault from lng ago, showing my poor understanding > of sockets/networking/C echo servers!! > > So it sounds like the client was just super slow in starting up and didn't > connect to the server within the timeout. > > So maybe we just remove the timeout entirely (client will eventually start > up?), and remove the pointless SO_REUSADDR? I'll try to whip up a PR. > > Mike McCandless > > http://blog.mikemccandless.com > > > On Sat, Sep 2, 2023 at 6:53 AM Uwe Schindler wrote: > >> Let's fix this issue with bogus socket reuse. I am not sure why it is >> there. We touched the code last time around 2012 >> >> Why does it has a timeout in setver at all? Normally the accept() call >> should have no timeout. If the client does not start fast enough, of course >> it runs into timeout. >> >> The code is just good old socket accept loop as we have all learned it in >> school when we were fighting to write a small echo server with C. The bug >> here is the timeout. A timeout should only be in the client and not in the >> waiting call. >> >> Uwe >> >> Uwe >> >> >> Am 31. August 2023 14:53:44 MESZ schrieb Robert Muir : >> >>> I looked at this lockverifyserver and would say its probably just the >>> craziness of this code. >>> >>> it sets 30 second socket timeout and intentionally calls accept() when >>> there is nothing yet to accept... well no wonder we see this issue. >>> >>> p.s. why does it set SO_REUSEADDR? no reason to do this leniency when >>> binding to port 0. nuke it. >>> >>> On Thu, Aug 31, 2023 at 8:46 AM Robert Muir wrote: >>> probably a bug in some jvm sockets code that called accept() in its default blocking mode, when there wasn't any connection to accept? in that case accept() call will just block and wait for someone to make a new connection. On Thu, Aug 31, 2023 at 8:16 AM Dawid Weiss wrote: > > > > https://ge.apache.org/s/orksynljk2yp6/tests/task/:lucene:core:test/details/org.apache.lucene.store.TestStressLockFactories/testSimpleFSLockFactory?top-execution=1 > > This test took 31 seconds... An extremely slow vm, perhaps? I don't know > what the default connection timeouts are... it does look weird though. > > Dawid > > On Thu, Aug 31, 2023 at 1:08 PM Michael McCandless > wrote: > >> >> Good grief -- why are we getting SocketTimeoutException in our >> LockVerifyServer's attempt to accept an incoming connection!? These are >> all processes running on the same host ... >> >> Mike McCandless >> >> http://blog.mikemccandless.com >> >> >> On Tue, Aug 29, 2023 at 11:17 PM Apache Jenkins Server >> wrote: >> >>> >>> Build: >>> https://ci-builds.apache.org/job/Lucene/job/Lucene-NightlyTests-9.x/665/ >>> >>> 2 tests failed. >>> FAILED: >>> org.apache.lucene.store.TestStressLockFactories.testSimpleFSLockFactory >>> >>> Error Message: >>> java.net.SocketTimeoutException: Accept timed out >>> >>> Stack Trace: >>> java.net.SocketTimeoutException: Accept timed out >>> at >>> __randomizedtesting.SeedInfo.seed([E1AD0D2AD68BA993:F325FE2A6E367AC7]:0) >>> at java.base/java.net.PlainSocketImpl.socketAccept(Native >>> Method) >>> at >>> java.base/java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:474) >>> at >>> java.base/java.net.ServerSocket.implAccept(ServerSocket.java:565) >>> at >>> java.base/java.net.ServerSocket.accept(ServerSocket.java:533) >>> at >>> org.apache.lucene.store.LockVerifyServer.run(LockVerifyServer.java:62) >>> at >>> org.apache.lucene.store.TestStressLockFactories.runImpl(TestStressLockFactories.java:53) >>> at >>> org.apache.lucene.store.TestStressLockFactories.testSimpleFSLockFactory(TestStressLockFactories.java:104) >>> at >>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native >>> Method) >>> at >>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>> at >>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> at java.base/java.lang.reflect.Method.invoke(Method.java:566) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) >>> at >>>
Re: [JENKINS] Lucene » Lucene-NightlyTests-9.x - Build # 665 - Unstable!
> The code is just good old socket accept loop as we have all learned it in school when we were fighting to write a small echo server with C. LOL this is all my fault from lng ago, showing my poor understanding of sockets/networking/C echo servers!! So it sounds like the client was just super slow in starting up and didn't connect to the server within the timeout. So maybe we just remove the timeout entirely (client will eventually start up?), and remove the pointless SO_REUSADDR? I'll try to whip up a PR. Mike McCandless http://blog.mikemccandless.com On Sat, Sep 2, 2023 at 6:53 AM Uwe Schindler wrote: > Let's fix this issue with bogus socket reuse. I am not sure why it is > there. We touched the code last time around 2012 > > Why does it has a timeout in setver at all? Normally the accept() call > should have no timeout. If the client does not start fast enough, of course > it runs into timeout. > > The code is just good old socket accept loop as we have all learned it in > school when we were fighting to write a small echo server with C. The bug > here is the timeout. A timeout should only be in the client and not in the > waiting call. > > Uwe > > Uwe > > > Am 31. August 2023 14:53:44 MESZ schrieb Robert Muir : > >> I looked at this lockverifyserver and would say its probably just the >> craziness of this code. >> >> it sets 30 second socket timeout and intentionally calls accept() when >> there is nothing yet to accept... well no wonder we see this issue. >> >> p.s. why does it set SO_REUSEADDR? no reason to do this leniency when >> binding to port 0. nuke it. >> >> On Thu, Aug 31, 2023 at 8:46 AM Robert Muir wrote: >> >>> >>> probably a bug in some jvm sockets code that called accept() in its >>> default blocking mode, when there wasn't any connection to accept? in >>> that case accept() call will just block and wait for someone to make a >>> new connection. >>> >>> On Thu, Aug 31, 2023 at 8:16 AM Dawid Weiss wrote: >>> https://ge.apache.org/s/orksynljk2yp6/tests/task/:lucene:core:test/details/org.apache.lucene.store.TestStressLockFactories/testSimpleFSLockFactory?top-execution=1 This test took 31 seconds... An extremely slow vm, perhaps? I don't know what the default connection timeouts are... it does look weird though. Dawid On Thu, Aug 31, 2023 at 1:08 PM Michael McCandless wrote: > > Good grief -- why are we getting SocketTimeoutException in our > LockVerifyServer's attempt to accept an incoming connection!? These are > all processes running on the same host ... > > Mike McCandless > > http://blog.mikemccandless.com > > > On Tue, Aug 29, 2023 at 11:17 PM Apache Jenkins Server > wrote: > >> >> Build: >> https://ci-builds.apache.org/job/Lucene/job/Lucene-NightlyTests-9.x/665/ >> >> 2 tests failed. >> FAILED: >> org.apache.lucene.store.TestStressLockFactories.testSimpleFSLockFactory >> >> Error Message: >> java.net.SocketTimeoutException: Accept timed out >> >> Stack Trace: >> java.net.SocketTimeoutException: Accept timed out >> at >> __randomizedtesting.SeedInfo.seed([E1AD0D2AD68BA993:F325FE2A6E367AC7]:0) >> at java.base/java.net.PlainSocketImpl.socketAccept(Native >> Method) >> at >> java.base/java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:474) >> at >> java.base/java.net.ServerSocket.implAccept(ServerSocket.java:565) >> at java.base/java.net.ServerSocket.accept(ServerSocket.java:533) >> at >> org.apache.lucene.store.LockVerifyServer.run(LockVerifyServer.java:62) >> at >> org.apache.lucene.store.TestStressLockFactories.runImpl(TestStressLockFactories.java:53) >> at >> org.apache.lucene.store.TestStressLockFactories.testSimpleFSLockFactory(TestStressLockFactories.java:104) >> at >> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native >> Method) >> at >> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at >> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.base/java.lang.reflect.Method.invoke(Method.java:566) >> at >> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) >> at >> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) >> at >> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) >> at >> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) >> at >>
Re: [JENKINS] Lucene » Lucene-NightlyTests-9.x - Build # 665 - Unstable!
Let's fix this issue with bogus socket reuse. I am not sure why it is there. We touched the code last time around 2012 Why does it has a timeout in setver at all? Normally the accept() call should have no timeout. If the client does not start fast enough, of course it runs into timeout. The code is just good old socket accept loop as we have all learned it in school when we were fighting to write a small echo server with C. The bug here is the timeout. A timeout should only be in the client and not in the waiting call. Uwe Uwe Am 31. August 2023 14:53:44 MESZ schrieb Robert Muir : >I looked at this lockverifyserver and would say its probably just the >craziness of this code. > >it sets 30 second socket timeout and intentionally calls accept() when >there is nothing yet to accept... well no wonder we see this issue. > >p.s. why does it set SO_REUSEADDR? no reason to do this leniency when >binding to port 0. nuke it. > >On Thu, Aug 31, 2023 at 8:46 AM Robert Muir wrote: >> >> probably a bug in some jvm sockets code that called accept() in its >> default blocking mode, when there wasn't any connection to accept? in >> that case accept() call will just block and wait for someone to make a >> new connection. >> >> On Thu, Aug 31, 2023 at 8:16 AM Dawid Weiss wrote: >> > >> > >> > https://ge.apache.org/s/orksynljk2yp6/tests/task/:lucene:core:test/details/org.apache.lucene.store.TestStressLockFactories/testSimpleFSLockFactory?top-execution=1 >> > >> > This test took 31 seconds... An extremely slow vm, perhaps? I don't know >> > what the default connection timeouts are... it does look weird though. >> > >> > Dawid >> > >> > On Thu, Aug 31, 2023 at 1:08 PM Michael McCandless >> > wrote: >> >> >> >> Good grief -- why are we getting SocketTimeoutException in our >> >> LockVerifyServer's attempt to accept an incoming connection!? These are >> >> all processes running on the same host ... >> >> >> >> Mike McCandless >> >> >> >> http://blog.mikemccandless.com >> >> >> >> >> >> On Tue, Aug 29, 2023 at 11:17 PM Apache Jenkins Server >> >> wrote: >> >>> >> >>> Build: >> >>> https://ci-builds.apache.org/job/Lucene/job/Lucene-NightlyTests-9.x/665/ >> >>> >> >>> 2 tests failed. >> >>> FAILED: >> >>> org.apache.lucene.store.TestStressLockFactories.testSimpleFSLockFactory >> >>> >> >>> Error Message: >> >>> java.net.SocketTimeoutException: Accept timed out >> >>> >> >>> Stack Trace: >> >>> java.net.SocketTimeoutException: Accept timed out >> >>> at >> >>> __randomizedtesting.SeedInfo.seed([E1AD0D2AD68BA993:F325FE2A6E367AC7]:0) >> >>> at java.base/java.net.PlainSocketImpl.socketAccept(Native Method) >> >>> at >> >>> java.base/java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:474) >> >>> at >> >>> java.base/java.net.ServerSocket.implAccept(ServerSocket.java:565) >> >>> at java.base/java.net.ServerSocket.accept(ServerSocket.java:533) >> >>> at >> >>> org.apache.lucene.store.LockVerifyServer.run(LockVerifyServer.java:62) >> >>> at >> >>> org.apache.lucene.store.TestStressLockFactories.runImpl(TestStressLockFactories.java:53) >> >>> at >> >>> org.apache.lucene.store.TestStressLockFactories.testSimpleFSLockFactory(TestStressLockFactories.java:104) >> >>> at >> >>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native >> >>> Method) >> >>> at >> >>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> >>> at >> >>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> >>> at java.base/java.lang.reflect.Method.invoke(Method.java:566) >> >>> at >> >>> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) >> >>> at >> >>> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) >> >>> at >> >>> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) >> >>> at >> >>> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) >> >>> at >> >>> org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) >> >>> at >> >>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) >> >>> at >> >>> org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) >> >>> at >> >>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) >> >>> at >> >>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) >> >>> at org.junit.rules.RunRules.evaluate(RunRules.java:20) >> >>> at >> >>>
Re: Vector Search with OpenAI Embeddings: Lucene Is All You Need
Hey, Very nice article! Looks like lots of manual work to look at search results in those examples. Great work! Do you have a DOI name for the article? Uwe Am 1. September 2023 07:22:09 MESZ schrieb Kent Fitch : >My testing shows Lucene's HNSW in a very positive light. The ability to >perform blended searches (vector/semantic and text) is valuable, even with >high quality embeddings, and helps when the searcher's intent is to search >for specific words or phrases (such as a name, or exact concepts) which get >blurred-out by semantics. I discussed blended searching using Lucene in >this Code4Lib article: https://journal.code4lib.org/articles/17443 > >And regarding performance, I have benchmarked Lucene's HNSW (circa Jan2023 >snapshot) on a test index of 192 million vectors of 1536 dimensions, >reduced by PQ coding to 512 bytes and stored in HNSW. Building this index >was slow (lots of time merging...) but once it was built, it did fit >entirely in memory (core i7-9800x (8 cores) with 128gb DDR4 memory running >at 2400 MT/s) so no IO was required at search time. (I modified the lucene >similarity code to support expansion of each of the 512 PQ byte codes back >to 3 floats for the distance calculation.) I havent updated this to take >advantage of the latest SIMD capability, but even so, once the HNSW >structure is in memory, a single-threaded topK=10 search thread achieves >2.4 queries/second. Two threads: 4.9 q/s, 4 threads: 7.2q/s, maxing out at >8 threads: 9.4 q/s. I guess the non-linear scaling with threads is due to >competition for memory bandwidth and cache. Curiously, I'm not getting >nearly as good performance out of the box using Milvus 2.3's diskANN, but I >need to find out why before condemning it. > >Kent Fitch > >On Thu, Aug 31, 2023 at 7:53 PM Michael McCandless < >luc...@mikemccandless.com> wrote: > >> Thanks Michael, very interesting! I of course agree that Lucene is all >> you need, heh ;) >> >> Jimmy Lin also tweeted about the strength of Lucene's HNSW: >> https://twitter.com/lintool/status/1681333664431460353?s=20 >> >> Mike McCandless >> >> http://blog.mikemccandless.com >> >> >> On Thu, Aug 31, 2023 at 3:31 AM Michael Wechner >> wrote: >> >>> Hi Together >>> >>> You might be interesed in this paper / article >>> >>> https://arxiv.org/abs/2308.14963 >>> >>> Thanks >>> >>> Michael >>> >>> - >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: dev-h...@lucene.apache.org >>> >>> -- Uwe Schindler Achterdiek 19, 28357 Bremen https://www.thetaphi.de
Re: [JENKINS] Lucene-9.x-Linux (64bit/openj9/jdk-17.0.5) - Build # 12560 - Unstable!
Hi, Let me update the version of openj9 and let's see. We should open bug report of it persists. Uwe Am 28. August 2023 09:28:11 MESZ schrieb Dawid Weiss : >The real reason for this is buried in other stack traces from >barrier-broken threads, it's this assertion: > >Caused by: >java.lang.AssertionError >at __randomizedtesting.SeedInfo.seed([F7B4CD7A5624D5EC]:0) >at app//org.junit.Assert.fail(Assert.java:87) >at app//org.junit.Assert.assertTrue(Assert.java:42) >at app//org.junit.Assert.assertTrue(Assert.java:53) >at > app//org.apache.lucene.index.TestIndexWriterThreadsToSegments$CheckSegmentCount.run(TestIndexWriterThreadsToSegments.java:150) >at > java.base@17.0.5/java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:222) >at > java.base@17.0.5/java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:364) >at > app//org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:236) > > > >On Mon, Aug 28, 2023 at 2:04 AM Policeman Jenkins Server < >jenk...@thetaphi.de> wrote: > >> Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/12560/ >> Java: 64bit/openj9/jdk-17.0.5 -XX:+UseCompressedOops -Xgcpolicy:balanced >> >> 2 tests failed. >> FAILED: >> org.apache.lucene.index.TestIndexWriterThreadsToSegments.testSegmentCountOnFlushRandom >> >> Error Message: >> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an >> uncaught exception in thread: Thread[id=1089, name=Thread-814, >> state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments] >> >> Stack Trace: >> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an >> uncaught exception in thread: Thread[id=1089, name=Thread-814, >> state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments] >> Caused by: java.lang.RuntimeException: >> java.util.concurrent.BrokenBarrierException >> at __randomizedtesting.SeedInfo.seed([F7B4CD7A5624D5EC]:0) >> at >> app//org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:239) >> Caused by: java.util.concurrent.BrokenBarrierException >> at java.base@17.0.5 >> /java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:252) >> at java.base@17.0.5 >> /java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:364) >> at >> app//org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:236) >> >> >> FAILED: >> org.apache.lucene.index.TestIndexWriterThreadsToSegments.classMethod >> >> Error Message: >> java.lang.AssertionError: The test or suite printed 8227 bytes to stdout >> and stderr, even though the limit was set to 8192 bytes. Increase the limit >> with @Limit, ignore it completely with @SuppressSysoutChecks or run with >> -Dtests.verbose=true >> >> Stack Trace: >> java.lang.AssertionError: The test or suite printed 8227 bytes to stdout >> and stderr, even though the limit was set to 8192 bytes. Increase the limit >> with @Limit, ignore it completely with @SuppressSysoutChecks or run with >> -Dtests.verbose=true >> at __randomizedtesting.SeedInfo.seed([F7B4CD7A5624D5EC]:0) >> at >> app//org.apache.lucene.tests.util.TestRuleLimitSysouts.afterIfSuccessful(TestRuleLimitSysouts.java:283) >> at >> app//com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterIfSuccessful(TestRuleAdapter.java:36) >> at >> app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:37) >> at >> app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) >> at >> app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) >> at >> app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) >> at >> app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) >> at >> app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) >> at app//org.junit.rules.RunRules.evaluate(RunRules.java:20) >> at >> app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >> at >> app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) >> at >> app//com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) >> at java.base@17.0.5/java.lang.Thread.run(Thread.java:857) >> >> - >> To unsubscribe, e-mail: builds-unsubscr...@lucene.apache.org >> For additional commands, e-mail: builds-h...@lucene.apache.org -- Uwe Schindler Achterdiek 19, 28357 Bremen
