Re: [JENKINS-EA] Lucene-jdk18panama-Linux (64bit/jdk-18-ea+26) - Build # 53 - Still Unstable!
Hi Uwe, The panama branch no longer reads unsafe so the new test on main fails after the merge: org.apache.lucene.core.tests.TestMMap.testUnmapSupported I've allowed myself to correct it on your branch to shut up jenkins, but please revert or tune to your liking. D. On Wed, Dec 22, 2021 at 7:23 AM Policeman Jenkins Server < jenk...@thetaphi.de> wrote: > Build: https://jenkins.thetaphi.de/job/Lucene-jdk18panama-Linux/53/ > Java: 64bit/jdk-18-ea+26 -XX:-UseCompressedOops -XX:+UseG1GC > > 1 tests failed. > FAILED: org.apache.lucene.core.tests.TestMMap.testUnmapSupported > > Error Message: > java.lang.AssertionError: Lucene Core can't read 'jdk.unsupported' module > > Stack Trace: > java.lang.AssertionError: Lucene Core can't read 'jdk.unsupported' module > at junit@4.13.1/org.junit.Assert.fail(Assert.java:89) > at junit@4.13.1/org.junit.Assert.assertTrue(Assert.java:42) > at org.apache.lucene.core.tests@10.0.0-SNAPSHOT > /org.apache.lucene.core.tests.TestMMap.testUnmapSupported(TestMMap.java:28) > at > java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) > at java.base/java.lang.reflect.Method.invoke(Method.java:577) > at junit@4.13.1 > /org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at junit@4.13.1 > /org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at junit@4.13.1 > /org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at junit@4.13.1 > /org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at junit@4.13.1 > /org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) > at junit@4.13.1 > /org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > at junit@4.13.1 > /org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) > at junit@4.13.1 > /org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > at junit@4.13.1 > /org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > at junit@4.13.1 > /org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) > at junit@4.13.1 > /org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) > at junit@4.13.1 > /org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) > at junit@4.13.1 > /org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) > at junit@4.13.1 > /org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) > at junit@4.13.1 > /org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) > at junit@4.13.1 > /org.junit.runners.ParentRunner.run(ParentRunner.java:413) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38) > at > org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62) > at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51) > at > java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) > at java.base/java.lang.reflect.Method.invoke(Method.java:577) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > at > org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33) > at > org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94) > at jdk.proxy1/jdk.proxy1.$Proxy2.processTestClass(Unknown Source) > at > org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:176) > at > org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129) > at > org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100) > at > org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60) > at > org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56) > at > org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133) > at >
Re: Searching Lucene FAQ with Lucene
Am 21.12.21 um 18:49 schrieb Michael Sokolov: interesting -- it always matches *something* I guess? yes, but this is something I would like to improve, that it knows when it does not know :-) I understand Lucene provides a score, but just defining a threshold doesn't really solve the problem, or do I misunderstand this? It seems to me one has to implement some kind of "understanding / reasoning" in order to solve this. Or what would be your approach? It might be helpful to show not only the answer, but also the question that was matched? yes, definitely, whereas the Katie frontend already provides this functionality https://ukatie.com/#/faq/9f206aec-5223-4e03-a2fc-c16e4b885ef8/en but I have to enhance the Javascript client used at https://lucene-faq.ukatie.com/ Thanks Michael On Mon, Dec 20, 2021 at 5:05 AM Michael Wechner wrote: Hi I am working on a webapp called "Katie" in order to detect duplicated questions https://ukatie.com/ As a test case I have imported the Lucene FAQ https://cwiki.apache.org/confluence/display/LUCENE/LuceneFAQ to https://ukatie.com/#/faq/9f206aec-5223-4e03-a2fc-c16e4b885ef8/en and made them available at https://lucene-faq.ukatie.com/ whereas the FAQ are loaded as JSON from the REST interface of Katie https://ukatie.com/swagger-ui/?urls.primaryName=API%20V2#/faq-controller-v-2/getFAQUsingGET_1 and the Javascript can be found at https://github.com/wyona/katie-4-faq I am currently "experimenting" with different search algorithms, e.g. Lucene only SentenceBERT- Lucene Vector Search SentenceBERT only Weaviate The goal is to find the right answer with "similar" questions, e.g. - "Are there mailing lists?" - "How can I ask questions re Lucene?" independent whether the question was trained/indexed or not or the answer contains keywords of the question whereas the answer in this particular case is https://ukatie.com/#/read-answer?domain-id=9f206aec-5223-4e03-a2fc-c16e4b885ef8=e19b6f48-62ac-427a-9d5e-d4e4eb110769 and another meaningful answer could be https://ukatie.com/#/read-answer?domain-id=9f206aec-5223-4e03-a2fc-c16e4b885ef8=154d9aa7-29e6-457e-a2ad-315b1a67599f There is still a lot to be improved :-) but it is lot of fun to use Lucene for this! Any feedback is very welcome or if you want to know more about the implementation details. Thanks Michael - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Searching Lucene FAQ with Lucene
interesting -- it always matches *something* I guess? It might be helpful to show not only the answer, but also the question that was matched? On Mon, Dec 20, 2021 at 5:05 AM Michael Wechner wrote: > > Hi > > I am working on a webapp called "Katie" in order to detect duplicated > questions > > https://ukatie.com/ > > As a test case I have imported the Lucene FAQ > > https://cwiki.apache.org/confluence/display/LUCENE/LuceneFAQ > > to > > https://ukatie.com/#/faq/9f206aec-5223-4e03-a2fc-c16e4b885ef8/en > > and made them available at > > https://lucene-faq.ukatie.com/ > > whereas the FAQ are loaded as JSON from the REST interface of Katie > > https://ukatie.com/swagger-ui/?urls.primaryName=API%20V2#/faq-controller-v-2/getFAQUsingGET_1 > > and the Javascript can be found at > > https://github.com/wyona/katie-4-faq > > I am currently "experimenting" with different search algorithms, e.g. > > Lucene only > SentenceBERT- Lucene Vector Search > SentenceBERT only > Weaviate > > The goal is to find the right answer with "similar" questions, e.g. > > - "Are there mailing lists?" > - "How can I ask questions re Lucene?" > > independent whether the question was trained/indexed or not or the > answer contains keywords of the question > > whereas the answer in this particular case is > > https://ukatie.com/#/read-answer?domain-id=9f206aec-5223-4e03-a2fc-c16e4b885ef8=e19b6f48-62ac-427a-9d5e-d4e4eb110769 > > and another meaningful answer could be > > https://ukatie.com/#/read-answer?domain-id=9f206aec-5223-4e03-a2fc-c16e4b885ef8=154d9aa7-29e6-457e-a2ad-315b1a67599f > > There is still a lot to be improved :-) but it is lot of fun to use > Lucene for this! > > Any feedback is very welcome or if you want to know more about the > implementation details. > > Thanks > > Michael > > > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [Heads up] Test framework package rename
> I wouldn't have thought to do such a change in a minor release, but I suppose > for tests it's fine. I think the benefits may outweigh the inconvenience in this case. It'd be a killer for us in the long run - all the tests would require a manual touch in backports. I don't see how this can be made to work in any other way (*)... D. (*) I'm fairly sure Uwe can write a script that will process all the test framework classes via asmlib and repackage it to the old name space but I don't think we should do it. ;) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [Heads up] Test framework package rename
I wouldn't have thought to do such a change in a minor release, but I suppose for tests it's fine. ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Tue, Dec 21, 2021 at 9:10 AM Uwe Schindler wrote: > Hi, > TortoiseGit's Viewer for diffs also helps well. But I just quicky skimmed > through it and only stopped at classes where different changes to imports > were done. > > Thanks, > Uwe > > Am 21. Dezember 2021 14:01:15 UTC schrieb Robert Muir : >> >> On Tue, Dec 21, 2021 at 8:20 AM Mark Jens wrote: >> >>> >>> You can use >>> https://patch-diff.githubusercontent.com/raw/apache/lucene/pull/551.diff to >>> render plain text and keep the browser responsive. >>> Another option is >>> https://patch-diff.githubusercontent.com/raw/apache/lucene/pull/551.patch >>> to see each commit separately. >>> >>> >> Thank you Mark. >> >> This does work easily for a large diff, and I get colored diff if I >> load it in vim (syntax on/set background=dark). >> But personally, especially for such a large diff, I get a much better >> color scheme for reviewing via 'git diff'. >> Seems I need to tweak the vim colors better for "downloaded diff" to >> be more useful... >> >> In any case, fetching Dawid's remote branch and testing it out myself >> was worth the trouble. It found a bug >> (https://issues.apache.org/jira/browse/LUCENE-10331) >> -- >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> >> -- > Uwe Schindler > Achterdiek 19, 28357 Bremen > https://www.thetaphi.de >
Re: [Heads up] Test framework package rename
Hi, TortoiseGit's Viewer for diffs also helps well. But I just quicky skimmed through it and only stopped at classes where different changes to imports were done. Thanks, Uwe Am 21. Dezember 2021 14:01:15 UTC schrieb Robert Muir : >On Tue, Dec 21, 2021 at 8:20 AM Mark Jens wrote: >> >> You can use >> https://patch-diff.githubusercontent.com/raw/apache/lucene/pull/551.diff to >> render plain text and keep the browser responsive. >> Another option is >> https://patch-diff.githubusercontent.com/raw/apache/lucene/pull/551.patch to >> see each commit separately. >> > >Thank you Mark. > >This does work easily for a large diff, and I get colored diff if I >load it in vim (syntax on/set background=dark). >But personally, especially for such a large diff, I get a much better >color scheme for reviewing via 'git diff'. >Seems I need to tweak the vim colors better for "downloaded diff" to >be more useful... > >In any case, fetching Dawid's remote branch and testing it out myself >was worth the trouble. It found a bug >(https://issues.apache.org/jira/browse/LUCENE-10331) > >- >To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >For additional commands, e-mail: dev-h...@lucene.apache.org > -- Uwe Schindler Achterdiek 19, 28357 Bremen https://www.thetaphi.de
Re: [Heads up] Test framework package rename
On Tue, Dec 21, 2021 at 8:20 AM Mark Jens wrote: > > You can use > https://patch-diff.githubusercontent.com/raw/apache/lucene/pull/551.diff to > render plain text and keep the browser responsive. > Another option is > https://patch-diff.githubusercontent.com/raw/apache/lucene/pull/551.patch to > see each commit separately. > Thank you Mark. This does work easily for a large diff, and I get colored diff if I load it in vim (syntax on/set background=dark). But personally, especially for such a large diff, I get a much better color scheme for reviewing via 'git diff'. Seems I need to tweak the vim colors better for "downloaded diff" to be more useful... In any case, fetching Dawid's remote branch and testing it out myself was worth the trouble. It found a bug (https://issues.apache.org/jira/browse/LUCENE-10331) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [Heads up] Test framework package rename
On Tue, 21 Dec 2021 at 15:19, Mark Jens wrote: > Hi, > > On Mon, 20 Dec 2021 at 17:24, Robert Muir wrote: > >> Thanks for doing this!! Give me a few minutes, I'll verify everything >> passes on linux, and review it locally. And I agree, sooner than later >> is better as it will conflict with anything and everything. >> >> Due to the expected noise of the import statements, the change is so >> large that it makes the Github UI unresponsive, so I'll review the >> diff locally while check runs. >> > > You can use > https://patch-diff.githubusercontent.com/raw/apache/lucene/pull/551.diff > to render plain text and keep the browser responsive. > Another option is > https://patch-diff.githubusercontent.com/raw/apache/lucene/pull/551.patch > to see each commit separately. > To make it more clear: just append '.diff' or '.patch' to https://github.com/apache/lucene/pull/551 and it will redirect to the above links. > > >> >> On Mon, Dec 20, 2021 at 9:55 AM Dawid Weiss >> wrote: >> > >> > >> > Hello everyone, >> > >> > I've completed the task of getting the test framework to not share any >> packages with Lucene core - this is here: >> > >> > https://issues.apache.org/jira/browse/LUCENE-10301 >> > https://github.com/apache/lucene/pull/551 >> > >> > Basically everything remains the same, except for the changed package >> prefix. The patch is rather large because it cuts across all of the code >> (imports, mostly). There are also some minor tweaks to expose >> package-private internals in the core to the test framework, now residing >> in a different package. >> > >> > Tests pass for me but I could use a pair of eyes on the patch. This >> will be rather annoying for backports (if you change anything in the test >> framework itself) so I'd like to apply it to 9x and main. Soon-ish, if >> there are no objections. >> > >> > Dawid >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> >>
Re: [Heads up] Test framework package rename
Hi, On Mon, 20 Dec 2021 at 17:24, Robert Muir wrote: > Thanks for doing this!! Give me a few minutes, I'll verify everything > passes on linux, and review it locally. And I agree, sooner than later > is better as it will conflict with anything and everything. > > Due to the expected noise of the import statements, the change is so > large that it makes the Github UI unresponsive, so I'll review the > diff locally while check runs. > You can use https://patch-diff.githubusercontent.com/raw/apache/lucene/pull/551.diff to render plain text and keep the browser responsive. Another option is https://patch-diff.githubusercontent.com/raw/apache/lucene/pull/551.patch to see each commit separately. > > On Mon, Dec 20, 2021 at 9:55 AM Dawid Weiss wrote: > > > > > > Hello everyone, > > > > I've completed the task of getting the test framework to not share any > packages with Lucene core - this is here: > > > > https://issues.apache.org/jira/browse/LUCENE-10301 > > https://github.com/apache/lucene/pull/551 > > > > Basically everything remains the same, except for the changed package > prefix. The patch is rather large because it cuts across all of the code > (imports, mostly). There are also some minor tweaks to expose > package-private internals in the core to the test framework, now residing > in a different package. > > > > Tests pass for me but I could use a pair of eyes on the patch. This will > be rather annoying for backports (if you change anything in the test > framework itself) so I'd like to apply it to 9x and main. Soon-ish, if > there are no objections. > > > > Dawid > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > >
Re: [JENKINS-EA] Lucene-9.x-Linux (64bit/jdk-18-ea+26) - Build # 496 - Failure!
This looks interesting. I will open bug report and contact them directly. Looks like the new long indexed loop opts are causing this. They were added to support 64 bit loop variables by Roland Westrelin. Uwe Am 21. Dezember 2021 08:28:14 UTC schrieb Dawid Weiss : >This is a JVM failure: > ># ># A fatal error has been detected by the Java Runtime Environment: ># ># SIGSEGV (0xb) at pc=0x7fc59e51c84b, pid=1722975, tid=1723672 ># ># JRE version: OpenJDK Runtime Environment (18.0+26) (build 18-ea+26-1787) ># Java VM: OpenJDK 64-Bit Server VM (18-ea+26-1787, mixed mode, >sharing, tiered, compressed class ptrs, parallel gc, linux-amd64) ># Problematic frame: ># V [libjvm.so+0xaba84b] >PhaseIdealLoop::build_loop_late_post_work(Node*, bool)+0xdb ># ># No core dump will be written. Core dumps have been disabled. To >enable core dumping, try "ulimit -c unlimited" before starting Java >again ># ># An error report file with more information is saved as: ># >/home/jenkins/workspace/Lucene-9.x-Linux/lucene/facet/build/tmp/tests-cwd/hs_err_pid1722975.log ># ># Compiler replay data is saved as: ># >/home/jenkins/workspace/Lucene-9.x-Linux/lucene/facet/build/tmp/tests-cwd/replay_pid1722975.log ># ># If you would like to submit a bug report, please visit: ># https://bugreport.java.com/bugreport/crash.jsp ># > >On Tue, Dec 21, 2021 at 12:48 AM Policeman Jenkins Server > wrote: >> >> Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/496/ >> Java: 64bit/jdk-18-ea+26 -XX:-UseCompressedOops -XX:+UseParallelGC >> >> All tests passed >> >> - >> To unsubscribe, e-mail: builds-unsubscr...@lucene.apache.org >> For additional commands, e-mail: builds-h...@lucene.apache.org > >- >To unsubscribe, e-mail: builds-unsubscr...@lucene.apache.org >For additional commands, e-mail: builds-h...@lucene.apache.org > -- Uwe Schindler Achterdiek 19, 28357 Bremen https://www.thetaphi.de
Re: [JENKINS-EA] Lucene-9.x-Linux (64bit/jdk-18-ea+26) - Build # 496 - Failure!
This is a JVM failure: # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x7fc59e51c84b, pid=1722975, tid=1723672 # # JRE version: OpenJDK Runtime Environment (18.0+26) (build 18-ea+26-1787) # Java VM: OpenJDK 64-Bit Server VM (18-ea+26-1787, mixed mode, sharing, tiered, compressed class ptrs, parallel gc, linux-amd64) # Problematic frame: # V [libjvm.so+0xaba84b] PhaseIdealLoop::build_loop_late_post_work(Node*, bool)+0xdb # # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again # # An error report file with more information is saved as: # /home/jenkins/workspace/Lucene-9.x-Linux/lucene/facet/build/tmp/tests-cwd/hs_err_pid1722975.log # # Compiler replay data is saved as: # /home/jenkins/workspace/Lucene-9.x-Linux/lucene/facet/build/tmp/tests-cwd/replay_pid1722975.log # # If you would like to submit a bug report, please visit: # https://bugreport.java.com/bugreport/crash.jsp # On Tue, Dec 21, 2021 at 12:48 AM Policeman Jenkins Server wrote: > > Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/496/ > Java: 64bit/jdk-18-ea+26 -XX:-UseCompressedOops -XX:+UseParallelGC > > All tests passed > > - > To unsubscribe, e-mail: builds-unsubscr...@lucene.apache.org > For additional commands, e-mail: builds-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
