[jira] Created: (LUCENE-903) FilteredQuery explanation inaccuracy with boost
FilteredQuery explanation inaccuracy with boost --- Key: LUCENE-903 URL: https://issues.apache.org/jira/browse/LUCENE-903 Project: Lucene - Java Issue Type: Bug Components: Search Affects Versions: 2.2 Reporter: Doron Cohen Assignee: Doron Cohen Priority: Minor Fix For: 2.2 The value of explanation is different than the product of its part if boost 1. This is exposed after tightening the explanation check (part of LUCENE-446). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Updated: (LUCENE-903) FilteredQuery explanation inaccuracy with boost
[ https://issues.apache.org/jira/browse/LUCENE-903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doron Cohen updated LUCENE-903: --- Attachment: lucene-903.patch Attached lucene-903.patch change the explanation test in CheckHits to verify that the explanation value matches the explanation details. This new test makes assumptions on the description of an explanation that has multi sub-details, which might be controversial: the description is assumed to comply with one of: - end with product of: - end with sum of: - end with sum of: - contain max plus x times others (where x is float) While this seems to cover all the current scoring combinations, future scorers might not fit in. We can always expand this list of course. With this check, TestSimpleExplanations fails for FilteredQuery - testFQ6(). Fix included. Also failing is TestBoostingTermQuery, just because its explanation had no product of:. Fix included. All tests pass. FilteredQuery explanation inaccuracy with boost --- Key: LUCENE-903 URL: https://issues.apache.org/jira/browse/LUCENE-903 Project: Lucene - Java Issue Type: Bug Components: Search Affects Versions: 2.2 Reporter: Doron Cohen Assignee: Doron Cohen Priority: Minor Fix For: 2.2 Attachments: lucene-903.patch The value of explanation is different than the product of its part if boost 1. This is exposed after tightening the explanation check (part of LUCENE-446). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Updated: (LUCENE-903) FilteredQuery explanation inaccuracy with boost
[ https://issues.apache.org/jira/browse/LUCENE-903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doron Cohen updated LUCENE-903: --- Lucene Fields: [Patch Available] (was: [New]) FilteredQuery explanation inaccuracy with boost --- Key: LUCENE-903 URL: https://issues.apache.org/jira/browse/LUCENE-903 Project: Lucene - Java Issue Type: Bug Components: Search Affects Versions: 2.2 Reporter: Doron Cohen Assignee: Doron Cohen Priority: Minor Fix For: 2.2 Attachments: lucene-903.patch The value of explanation is different than the product of its part if boost 1. This is exposed after tightening the explanation check (part of LUCENE-446). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[EMAIL PROTECTED]: Project lucene-java (in module lucene-java) failed
To whom it may engage... This is an automated request, but not an unsolicited one. For more information please visit http://gump.apache.org/nagged.html, and/or contact the folk at [EMAIL PROTECTED] Project lucene-java has an issue affecting its community integration. This issue affects 4 projects, and has been outstanding for 6 runs. The current state of this project is 'Failed', with reason 'Build Failed'. For reference only, the following projects are affected by this: - eyebrowse : Web-based mail archive browsing - jakarta-lucene : Java Based Search Engine - jakarta-slide : Content Management System based on WebDAV technology - lucene-java : Java Based Search Engine Full details are available at: http://vmgump.apache.org/gump/public/lucene-java/lucene-java/index.html That said, some information snippets are provided here. The following annotations (debug/informational/warning/error messages) were provided: -DEBUG- Sole output [lucene-core-03062007.jar] identifier set to project name -DEBUG- Dependency on javacc exists, no need to add for property javacc.home. -INFO- Failed with reason build failed -INFO- Failed to extract fallback artifacts from Gump Repository The following work was performed: http://vmgump.apache.org/gump/public/lucene-java/lucene-java/gump_work/build_lucene-java_lucene-java.html Work Name: build_lucene-java_lucene-java (Type: Build) Work ended in a state of : Failed Elapsed: 20 secs Command Line: /opt/jdk1.5/bin/java -Djava.awt.headless=true -Xbootclasspath/p:/usr/local/gump/public/workspace/xml-commons/java/external/build/xml-apis.jar:/usr/local/gump/public/workspace/xml-xerces2/build/xercesImpl.jar org.apache.tools.ant.Main -Dgump.merge=/x1/gump/public/gump/work/merge.xml -Dbuild.sysclasspath=only -Dversion=03062007 -Djavacc.home=/usr/local/gump/packages/javacc-3.1 package [Working Directory: /usr/local/gump/public/workspace/lucene-java] CLASSPATH: /opt/jdk1.5/lib/tools.jar:/usr/local/gump/public/workspace/lucene-java/build/classes/java:/usr/local/gump/public/workspace/lucene-java/build/classes/demo:/usr/local/gump/public/workspace/lucene-java/build/classes/test:/usr/local/gump/public/workspace/ant/dist/lib/ant-jmf.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-swing.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-apache-resolver.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-trax.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-junit.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-launcher.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-nodeps.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant.jar:/usr/local/gump/packages/junit3.8.1/junit.jar:/usr/local/gump/public/workspace/xml-commons/java/build/resolver.jar:/usr/local/gump/packages/je-1.7.1/lib/je.jar:/usr/local/gump/public/workspace/jakarta-commons/digester/dist/commons-digester.jar:/usr/local/gump/public/workspace/jakarta-regexp/build/jakarta-regexp-03062007.jar:/usr/local/gump/packages/javacc-3.1/bin/lib/javacc.jar:/usr/local/gump/public/workspace/jline/target/jline-0.9.92-SNAPSHOT.jar:/usr/local/gump/packages/jtidy-04aug2000r7-dev/build/Tidy.jar:/usr/local/gump/public/workspace/junit/dist/junit-03062007.jar:/usr/local/gump/public/workspace/xml-commons/java/external/build/xml-apis-ext.jar - [javac] location: class org.apache.lucene.store.db.DbDirectory [javac] DatabaseEntry key = new DatabaseEntry(new byte[0]); [javac] ^ [javac] /x1/gump/public/workspace/lucene-java/contrib/db/bdb/src/java/org/apache/lucene/store/db/DbDirectory.java:171: cannot find symbol [javac] symbol : class DatabaseEntry [javac] location: class org.apache.lucene.store.db.DbDirectory [javac] DatabaseEntry data = new DatabaseEntry((byte[]) null); [javac] ^ [javac] /x1/gump/public/workspace/lucene-java/contrib/db/bdb/src/java/org/apache/lucene/store/db/DbDirectory.java:171: cannot find symbol [javac] symbol : class DatabaseEntry [javac] location: class org.apache.lucene.store.db.DbDirectory [javac] DatabaseEntry data = new DatabaseEntry((byte[]) null); [javac] ^ [javac] /x1/gump/public/workspace/lucene-java/contrib/db/bdb/src/java/org/apache/lucene/store/db/DbDirectory.java:178: cannot find symbol [javac] symbol : variable DbConstants [javac] location: class org.apache.lucene.store.db.DbDirectory [javac]DbConstants.DB_SET_RANGE | flags) != DbConstants.DB_NOTFOUND) [javac]^ [javac] /x1/gump/public/workspace/lucene-java/contrib/db/bdb/src/java/org/apache/lucene/store/db/DbDirectory.java:178: cannot find symbol [javac] symbol : variable DbConstants [javac] location: class org.apache.lucene.store.db.DbDirectory
[EMAIL PROTECTED]: Project lucene-java (in module lucene-java) failed
To whom it may engage... This is an automated request, but not an unsolicited one. For more information please visit http://gump.apache.org/nagged.html, and/or contact the folk at [EMAIL PROTECTED] Project lucene-java has an issue affecting its community integration. This issue affects 4 projects, and has been outstanding for 6 runs. The current state of this project is 'Failed', with reason 'Build Failed'. For reference only, the following projects are affected by this: - eyebrowse : Web-based mail archive browsing - jakarta-lucene : Java Based Search Engine - jakarta-slide : Content Management System based on WebDAV technology - lucene-java : Java Based Search Engine Full details are available at: http://vmgump.apache.org/gump/public/lucene-java/lucene-java/index.html That said, some information snippets are provided here. The following annotations (debug/informational/warning/error messages) were provided: -DEBUG- Sole output [lucene-core-03062007.jar] identifier set to project name -DEBUG- Dependency on javacc exists, no need to add for property javacc.home. -INFO- Failed with reason build failed -INFO- Failed to extract fallback artifacts from Gump Repository The following work was performed: http://vmgump.apache.org/gump/public/lucene-java/lucene-java/gump_work/build_lucene-java_lucene-java.html Work Name: build_lucene-java_lucene-java (Type: Build) Work ended in a state of : Failed Elapsed: 20 secs Command Line: /opt/jdk1.5/bin/java -Djava.awt.headless=true -Xbootclasspath/p:/usr/local/gump/public/workspace/xml-commons/java/external/build/xml-apis.jar:/usr/local/gump/public/workspace/xml-xerces2/build/xercesImpl.jar org.apache.tools.ant.Main -Dgump.merge=/x1/gump/public/gump/work/merge.xml -Dbuild.sysclasspath=only -Dversion=03062007 -Djavacc.home=/usr/local/gump/packages/javacc-3.1 package [Working Directory: /usr/local/gump/public/workspace/lucene-java] CLASSPATH: /opt/jdk1.5/lib/tools.jar:/usr/local/gump/public/workspace/lucene-java/build/classes/java:/usr/local/gump/public/workspace/lucene-java/build/classes/demo:/usr/local/gump/public/workspace/lucene-java/build/classes/test:/usr/local/gump/public/workspace/ant/dist/lib/ant-jmf.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-swing.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-apache-resolver.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-trax.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-junit.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-launcher.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-nodeps.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant.jar:/usr/local/gump/packages/junit3.8.1/junit.jar:/usr/local/gump/public/workspace/xml-commons/java/build/resolver.jar:/usr/local/gump/packages/je-1.7.1/lib/je.jar:/usr/local/gump/public/workspace/jakarta-commons/digester/dist/commons-digester.jar:/usr/local/gump/public/workspace/jakarta-regexp/build/jakarta-regexp-03062007.jar:/usr/local/gump/packages/javacc-3.1/bin/lib/javacc.jar:/usr/local/gump/public/workspace/jline/target/jline-0.9.92-SNAPSHOT.jar:/usr/local/gump/packages/jtidy-04aug2000r7-dev/build/Tidy.jar:/usr/local/gump/public/workspace/junit/dist/junit-03062007.jar:/usr/local/gump/public/workspace/xml-commons/java/external/build/xml-apis-ext.jar - [javac] location: class org.apache.lucene.store.db.DbDirectory [javac] DatabaseEntry key = new DatabaseEntry(new byte[0]); [javac] ^ [javac] /x1/gump/public/workspace/lucene-java/contrib/db/bdb/src/java/org/apache/lucene/store/db/DbDirectory.java:171: cannot find symbol [javac] symbol : class DatabaseEntry [javac] location: class org.apache.lucene.store.db.DbDirectory [javac] DatabaseEntry data = new DatabaseEntry((byte[]) null); [javac] ^ [javac] /x1/gump/public/workspace/lucene-java/contrib/db/bdb/src/java/org/apache/lucene/store/db/DbDirectory.java:171: cannot find symbol [javac] symbol : class DatabaseEntry [javac] location: class org.apache.lucene.store.db.DbDirectory [javac] DatabaseEntry data = new DatabaseEntry((byte[]) null); [javac] ^ [javac] /x1/gump/public/workspace/lucene-java/contrib/db/bdb/src/java/org/apache/lucene/store/db/DbDirectory.java:178: cannot find symbol [javac] symbol : variable DbConstants [javac] location: class org.apache.lucene.store.db.DbDirectory [javac]DbConstants.DB_SET_RANGE | flags) != DbConstants.DB_NOTFOUND) [javac]^ [javac] /x1/gump/public/workspace/lucene-java/contrib/db/bdb/src/java/org/apache/lucene/store/db/DbDirectory.java:178: cannot find symbol [javac] symbol : variable DbConstants [javac] location: class org.apache.lucene.store.db.DbDirectory
Re: [EMAIL PROTECTED]: Project lucene-java (in module lucene-java) failed
Greetings Gumpers, The Lucene-Java gump build seems to been failing consistently for the past few days. The problem seems to relate to some cleanup we did recently to our build files to help ensure that the various contrib sub projects could be cleanly built/tested on their own, or as part of hte master build. currently gump is reporting some javac errors when compiling the contrib/db/bdb subproject relating to packages and symbols which can't be found -- these are defined in the db-4.3.29.jar which is automaticly downloaded in the get-db-jar task executed just prior to compilation. This seems to work fine for develoeprs locally, as well as for our own nightly build system running on Hudson... ...If someone could take a look at the GUMP setup for Lucene-Java and either: a) fix whatever problem it is currently having or b) let us know if there is something wrong with out build process that we're not noticing becuase we are too close to the problem. ...we would greatly appreciate it. -Hoss : Date: Sun, 03 Jun 2007 03:20:44 PDT : Subject: [EMAIL PROTECTED]: Project lucene-java (in module lucene-java) failed : : To whom it may engage... : : This is an automated request, but not an unsolicited one. For : more information please visit http://gump.apache.org/nagged.html, : and/or contact the folk at [EMAIL PROTECTED] : : Project lucene-java has an issue affecting its community integration. : This issue affects 4 projects, : and has been outstanding for 6 runs. : The current state of this project is 'Failed', with reason 'Build Failed'. : For reference only, the following projects are affected by this: : - eyebrowse : Web-based mail archive browsing : - jakarta-lucene : Java Based Search Engine : - jakarta-slide : Content Management System based on WebDAV technology : - lucene-java : Java Based Search Engine : : : Full details are available at: : http://vmgump.apache.org/gump/public/lucene-java/lucene-java/index.html : : That said, some information snippets are provided here. : : The following annotations (debug/informational/warning/error messages) were provided: : -DEBUG- Sole output [lucene-core-03062007.jar] identifier set to project name : -DEBUG- Dependency on javacc exists, no need to add for property javacc.home. : -INFO- Failed with reason build failed : -INFO- Failed to extract fallback artifacts from Gump Repository : : : : The following work was performed: : http://vmgump.apache.org/gump/public/lucene-java/lucene-java/gump_work/build_lucene-java_lucene-java.html : Work Name: build_lucene-java_lucene-java (Type: Build) : Work ended in a state of : Failed : Elapsed: 20 secs : Command Line: /opt/jdk1.5/bin/java -Djava.awt.headless=true -Xbootclasspath/p:/usr/local/gump/public/workspace/xml-commons/java/external/build/xml-apis.jar:/usr/local/gump/public/workspace/xml-xerces2/build/xercesImpl.jar org.apache.tools.ant.Main -Dgump.merge=/x1/gump/public/gump/work/merge.xml -Dbuild.sysclasspath=only -Dversion=03062007 -Djavacc.home=/usr/local/gump/packages/javacc-3.1 package : [Working Directory: /usr/local/gump/public/workspace/lucene-java] : CLASSPATH: /opt/jdk1.5/lib/tools.jar:/usr/local/gump/public/workspace/lucene-java/build/classes/java:/usr/local/gump/public/workspace/lucene-java/build/classes/demo:/usr/local/gump/public/workspace/lucene-java/build/classes/test:/usr/local/gump/public/workspace/ant/dist/lib/ant-jmf.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-swing.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-apache-resolver.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-trax.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-junit.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-launcher.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-nodeps.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant.jar:/usr/local/gump/packages/junit3.8.1/junit.jar:/usr/local/gump/public/workspace/xml-commons/java/build/resolver.jar:/usr/local/gump/packages/je-1.7.1/lib/je.jar:/usr/local/gump/public/workspace/jakarta-commons/digester/dist/commons-digester.jar:/usr/local/gump/public/workspa ce : /jakarta-regexp/build/jakarta-regexp-03062007.jar:/usr/local/gump/packages/javacc-3.1/bin/lib/javacc.jar:/usr/local/gump/public/workspace/jline/target/jline-0.9.92-SNAPSHOT.jar:/usr/local/gump/packages/jtidy-04aug2000r7-dev/build/Tidy.jar:/usr/local/gump/public/workspace/junit/dist/junit-03062007.jar:/usr/local/gump/public/workspace/xml-commons/java/external/build/xml-apis-ext.jar : - : [javac] location: class org.apache.lucene.store.db.DbDirectory : [javac] DatabaseEntry key = new DatabaseEntry(new byte[0]); : [javac] ^ : [javac] /x1/gump/public/workspace/lucene-java/contrib/db/bdb/src/java/org/apache/lucene/store/db/DbDirectory.java:171: cannot find symbol : [javac] symbol : class DatabaseEntry : [javac]
[jira] Updated: (LUCENE-894) Custom build.xml for binary distributions
[ https://issues.apache.org/jira/browse/LUCENE-894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hoss Man updated LUCENE-894: Attachment: lucene-894.patch Michael: this approach seems great. I couldn't help but make a few tweaks... 1) used a filterset to do the token replacements as part of hte copy (isntead of after the fact) 2) since the demo build file is now just a template (and can't be run until it's pacakged) it now seems more like source then part of the build system, so i moved it into src/demo (which has the added benefit of keeping the root dir a little cleaner) Custom build.xml for binary distributions - Key: LUCENE-894 URL: https://issues.apache.org/jira/browse/LUCENE-894 Project: Lucene - Java Issue Type: Bug Components: Build Affects Versions: 2.1 Reporter: Michael Busch Assignee: Michael Busch Priority: Minor Fix For: 2.2 Attachments: lucene-894.patch, lucene-894.patch, lucene-894.patch, lucene-894.patch The binary files of a distribution come with the demo sources and a build.xml file. However, the build.xml doesn't work for the binary distribution, so it can't be used to build the demos. This problem was notices the first time when release 2.1 was made. Before we ship 2.2 we should fix this. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Updated: (LUCENE-894) Custom build.xml for binary distributions
[ https://issues.apache.org/jira/browse/LUCENE-894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hoss Man updated LUCENE-894: Attachment: lucene-894.patch whoops ... forgot to svn add src/demo/demo-build.template after i moved it .. it has some minor changes because of how i setup the filterset Custom build.xml for binary distributions - Key: LUCENE-894 URL: https://issues.apache.org/jira/browse/LUCENE-894 Project: Lucene - Java Issue Type: Bug Components: Build Affects Versions: 2.1 Reporter: Michael Busch Assignee: Michael Busch Priority: Minor Fix For: 2.2 Attachments: lucene-894.patch, lucene-894.patch, lucene-894.patch, lucene-894.patch, lucene-894.patch The binary files of a distribution come with the demo sources and a build.xml file. However, the build.xml doesn't work for the binary distribution, so it can't be used to build the demos. This problem was notices the first time when release 2.1 was made. Before we ship 2.2 we should fix this. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Commented: (LUCENE-903) FilteredQuery explanation inaccuracy with boost
[ https://issues.apache.org/jira/browse/LUCENE-903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501067 ] Hoss Man commented on LUCENE-903: - I have some reservations about this patch. I think it's fine to set as a goal that all core query classes should have Explanations where the description accurately describes a mathematical calculation that can be performed on the details to arrive at the value of the Explanation -- which is easy to do since we have the luxury of changing the CHeckHits class to suit our needs anytime we add a new class that has a more interesting mathematical calculation then we can current account for. But we should also try to maintain CheckHIts as a reusable class that clients can use to run basic sanity tests on their own custom Query classes, and holding them to the same standard (when they can't easily modify the string pattern rules in CheckHits) doesn't seem fair. lemme try to refactor the current patch a bit so that the deep Explanation testing is optional (and used by the core tests) FilteredQuery explanation inaccuracy with boost --- Key: LUCENE-903 URL: https://issues.apache.org/jira/browse/LUCENE-903 Project: Lucene - Java Issue Type: Bug Components: Search Affects Versions: 2.2 Reporter: Doron Cohen Assignee: Doron Cohen Priority: Minor Fix For: 2.2 Attachments: lucene-903.patch The value of explanation is different than the product of its part if boost 1. This is exposed after tightening the explanation check (part of LUCENE-446). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Commented: (LUCENE-903) FilteredQuery explanation inaccuracy with boost
[ https://issues.apache.org/jira/browse/LUCENE-903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501074 ] Doron Cohen commented on LUCENE-903: holding them to the same standard (when they can't easily modify the string pattern rules in CheckHits) doesn't seem fair. lemme try to refactor the current patch a bit so that the deep Explanation testing is optional (and used by the core tests) I agree (and pointed that) the deep explanation test is imperfect in this regard. But it seems to me that it is the benefit of new query writers to have their new queries explanations tested as thorough as possible. So I am not sure that disabling this test by default (for non core queries) is the right thing to do. How do you feel about adding an escape way, allowing one to specify that a certain (part of an) explanation is not to be checked in this manner? I see two immediate ways to add this: - by the explanation text: if the description starts with other: the explanation is not subject to this deep check - by api: adding a property to Explanation class - disabledDeepCheck - false by default - allows new unique queries to deliberately disable this deep test. FilteredQuery explanation inaccuracy with boost --- Key: LUCENE-903 URL: https://issues.apache.org/jira/browse/LUCENE-903 Project: Lucene - Java Issue Type: Bug Components: Search Affects Versions: 2.2 Reporter: Doron Cohen Assignee: Doron Cohen Priority: Minor Fix For: 2.2 Attachments: lucene-903.patch The value of explanation is different than the product of its part if boost 1. This is exposed after tightening the explanation check (part of LUCENE-446). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Updated: (LUCENE-903) FilteredQuery explanation inaccuracy with boost
[ https://issues.apache.org/jira/browse/LUCENE-903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hoss Man updated LUCENE-903: Attachment: lucene-903.patch in general i'm not fond of the inspecting the string to determine whether the math is correct ... in an ideal world every query would have it's own unit test class and it would have a testExplanation method that would know exactly what structure (ie: how many sub-details) to expect from an artificially constructed query instance. the current approach of having big explanation test classes that validate an explanation by comparing them to the scores is a fairly big hack (and i say that as the guy that added it). But i'm okay with the string comparisons as a Convenience to make it easier for us to test the explanations of the core queries (since so many of them follow very clear patterns) ... but i think we should avoid having any sort of expectation on the string for custom queries (either that they match product of or that they match other:) my main concern is purely with giving people a way to do the same kind of Explanation testing they could do before (that the root value equals the score) even if the details of their explanation don't fit any of the patterns of the existing core queries. (i'd hate for people to not even do shallow testing of their Explanations just because we don't give them a method for doing so) this refactoring... - makes verifyExplanations a public method in CheckHits that anyone can call - adds a deep boolean option to verifyExplanations - adds variants of CheckHits.checkExplanations and the ExplanationAsserter to make the deep testing optional - makes QueryUtils.checkExplanations use the deep option - makes TestExplanations explicitly use deep testing ...at the moment, the old APIs default to deep==false but i'm on board with changing those to true if you want to (in theory it's a non-backawards compatible API change, but since it's in src/test we can probably get away with it). FilteredQuery explanation inaccuracy with boost --- Key: LUCENE-903 URL: https://issues.apache.org/jira/browse/LUCENE-903 Project: Lucene - Java Issue Type: Bug Components: Search Affects Versions: 2.2 Reporter: Doron Cohen Assignee: Doron Cohen Priority: Minor Fix For: 2.2 Attachments: lucene-903.patch, lucene-903.patch The value of explanation is different than the product of its part if boost 1. This is exposed after tightening the explanation check (part of LUCENE-446). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Commented: (LUCENE-902) Check on PositionIncrement with StopFilter.
[ https://issues.apache.org/jira/browse/LUCENE-902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501078 ] Toru Matsuzawa commented on LUCENE-902: --- My patch was consideration shortage as shown in the point. However, I think that processing positionIncrements with StopFilter is necessary. Check on PositionIncrement with StopFilter. Key: LUCENE-902 URL: https://issues.apache.org/jira/browse/LUCENE-902 Project: Lucene - Java Issue Type: Bug Components: Analysis Affects Versions: 2.2 Reporter: Toru Matsuzawa Attachments: stopfilter.patch PositionIncrement set with Tokenizer is not considered with StopFilter. When PositionIncrement of Token is 1, it is deleted by StopFilter. However, when PositionIncrement of Token following afterwards is 0, it is not deleted. I think that it is necessary to be deleted. Because it is thought same Token when PositionIncrement is 0. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Commented: (LUCENE-622) Provide More of Lucene For Maven
[ https://issues.apache.org/jira/browse/LUCENE-622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501079 ] Hoss Man commented on LUCENE-622: - i just skimmed Sami's patch (did not test it) ... i can't help but think that the enumerated list of all contribs in the generate-maven-artifacts task seems like it will easily become just as hard to maintain as the related list in the javadocs macro. it seems like a more maintainable approach would be to... 1) put the maven files for each contrib in the respective contrib dirs. 2) put this new m2-deploy macro in common-build.xml 3) add a dist-maven task to contrib/contrib-build.xml where it can be inherited by each contrib and calls m2-deploy using hte project name as the artifactId 4) change generate-maven-artifacts to use contrib-crawl to take care of the pom's for the contribs. some other questions: a) it doesn't seem like all contribs are accounted for ... were some excluded intentionally? b) if we're going to start officially releasing POMs for lucene and the contribs, we should probably have a way of validating they are correct ... is there an easy mechanism for doing that? (my worry is that overtime a contrib changes, it's POM doesn't get updated, and we have a release in which the maven artifacts are incorrect (which in my opinion is worse then not having any maven artifacts at all) final minor note: using copy immediately followed by a replace on the file that was just copied can be done more cleanly using a filterset on the copy Provide More of Lucene For Maven Key: LUCENE-622 URL: https://issues.apache.org/jira/browse/LUCENE-622 Project: Lucene - Java Issue Type: Task Affects Versions: 2.0.0 Reporter: Stephen Duncan Jr Assignee: Michael Busch Fix For: 2.2 Attachments: lucene-622.txt, lucene-core.pom, lucene-highlighter-2.0.0.pom, lucene-maven.patch, lucene-maven.tar.bz2 Please provide javadoc source jars for lucene-core. Also, please provide the rest of lucene (the jars inside of contrib in the download bundle) if possible. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Commented: (LUCENE-902) Check on PositionIncrement with StopFilter.
[ https://issues.apache.org/jira/browse/LUCENE-902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501080 ] Hoss Man commented on LUCENE-902: - without a unit test demonstrating an actual problem, i'm having a hard time udnerstanding what exactly the bug is in this issue. from what i can tell based on the comments and my reading of the patch, Toru is concerned about cumulative positionIncrements of tokens being lost when one of those tokens is a stop word. (ie: if indexing multiple names of movies in a Document about an actor, and using a positionIncriment of 10 between each Field value (ie: movie name), indexing the values Dirty Harry and The Good the bad and the Ugly could result in no gap between the tokens harry and good since the is a stop word. is my understanding of the problem correct? if so, then i'm not sure how this patch really addresses the problem ... besides the fact that it treats 1 as a special case (the problem can come up with any positionIncrement) it doesn't seem to make any allowance for the situation where multiple stop words appear in sequence. i'm also not clear on why non stop words immediately following stop words (ie: the else if(flag) case) are not returned unless their positionIncriment is 1. Check on PositionIncrement with StopFilter. Key: LUCENE-902 URL: https://issues.apache.org/jira/browse/LUCENE-902 Project: Lucene - Java Issue Type: Bug Components: Analysis Affects Versions: 2.2 Reporter: Toru Matsuzawa Attachments: stopfilter.patch PositionIncrement set with Tokenizer is not considered with StopFilter. When PositionIncrement of Token is 1, it is deleted by StopFilter. However, when PositionIncrement of Token following afterwards is 0, it is not deleted. I think that it is necessary to be deleted. Because it is thought same Token when PositionIncrement is 0. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Commented: (LUCENE-894) Custom build.xml for binary distributions
[ https://issues.apache.org/jira/browse/LUCENE-894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501081 ] Michael Busch commented on LUCENE-894: -- I couldn't help but make a few tweaks... Thanks for reviewing! 1) used a filterset to do the token replacements as part of hte copy (isntead of after the fact) Neat! 2) since the demo build file is now just a template (and can't be run until it's pacakged) it now seems more like source then part of the build system, so i moved it into src/demo (which has the added benefit of keeping the root dir a little cleaner) Good! Yes I agree that src/demo is a better place for the template file. Custom build.xml for binary distributions - Key: LUCENE-894 URL: https://issues.apache.org/jira/browse/LUCENE-894 Project: Lucene - Java Issue Type: Bug Components: Build Affects Versions: 2.1 Reporter: Michael Busch Assignee: Michael Busch Priority: Minor Fix For: 2.2 Attachments: lucene-894.patch, lucene-894.patch, lucene-894.patch, lucene-894.patch, lucene-894.patch The binary files of a distribution come with the demo sources and a build.xml file. However, the build.xml doesn't work for the binary distribution, so it can't be used to build the demos. This problem was notices the first time when release 2.1 was made. Before we ship 2.2 we should fix this. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Commented: (LUCENE-446) search.function - (1) score based on field value, (2) simple score customizability
[ https://issues.apache.org/jira/browse/LUCENE-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501082 ] Hoss Man commented on LUCENE-446: - Doron: I haven't really been able to keep up with the way this issue has evolved, or dig into your new patches, but to answer your question about the Ord functions: yes they are very useful, and it active use in Solr. I believe the warning about MultiSearcher mainly has to do with the fact that the MultiSearcher/FieldCache APIs give us know way to know the lowest of highest value in a field cache across an entire logical index, so the Ord functions can't really be queried against a MultiSearcher. search.function - (1) score based on field value, (2) simple score customizability -- Key: LUCENE-446 URL: https://issues.apache.org/jira/browse/LUCENE-446 Project: Lucene - Java Issue Type: New Feature Components: Search Reporter: Yonik Seeley Assignee: Doron Cohen Priority: Minor Fix For: 2.2 Attachments: function.patch.txt, function.patch.txt, function.zip, function.zip FunctionQuery can return a score based on a field's value or on it's ordinal value. FunctionFactory subclasses define the details of the function. There is currently a LinearFloatFunction (a line specified by slope and intercept). Field values are typically obtained from FieldValueSourceFactory. Implementations include FloatFieldSource, IntFieldSource, and OrdFieldSource. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Commented: (LUCENE-902) Check on PositionIncrement with StopFilter.
[ https://issues.apache.org/jira/browse/LUCENE-902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501085 ] Toru Matsuzawa commented on LUCENE-902: --- for stop word only is. sample words A is B. For instance,When Tokenizer on StopFilter returns the following as a result. termText positionIncrement A1 is 1 are 0 be 0 B1 The result of StopFilter. termText positionIncrement A1 are 0 be 0 B1 A and are and be become the same positions. When thinking that it will process the result of a Japanese morphological analysis with StopFilter, it becomes a problem. Check on PositionIncrement with StopFilter. Key: LUCENE-902 URL: https://issues.apache.org/jira/browse/LUCENE-902 Project: Lucene - Java Issue Type: Bug Components: Analysis Affects Versions: 2.2 Reporter: Toru Matsuzawa Attachments: stopfilter.patch PositionIncrement set with Tokenizer is not considered with StopFilter. When PositionIncrement of Token is 1, it is deleted by StopFilter. However, when PositionIncrement of Token following afterwards is 0, it is not deleted. I think that it is necessary to be deleted. Because it is thought same Token when PositionIncrement is 0. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Updated: (LUCENE-831) Complete overhaul of FieldCache API/Implementation
[ https://issues.apache.org/jira/browse/LUCENE-831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hoss Man updated LUCENE-831: Attachment: fieldcache-overhaul.diff i haven't had any time to do further work on this issue ... partly because i haven't had a lot of time, but mainly because i'm hoping to get some feedback on the overall approach before any more serious effort investment. updated patch to work against the trunk (r544035) Complete overhaul of FieldCache API/Implementation -- Key: LUCENE-831 URL: https://issues.apache.org/jira/browse/LUCENE-831 Project: Lucene - Java Issue Type: Improvement Components: Search Reporter: Hoss Man Attachments: fieldcache-overhaul.diff, fieldcache-overhaul.diff Motivation: 1) Complete overhaul the API/implementation of FieldCache type things... a) eliminate global static map keyed on IndexReader (thus eliminating synch block between completley independent IndexReaders) b) allow more customization of cache management (ie: use expiration/replacement strategies, disk backed caches, etc) c) allow people to define custom cache data logic (ie: custom parsers, complex datatypes, etc... anything tied to a reader) d) allow people to inspect what's in a cache (list of CacheKeys) for an IndexReader so a new IndexReader can be likewise warmed. e) Lend support for smarter cache management if/when IndexReader.reopen is added (merging of cached data from subReaders). 2) Provide backwards compatibility to support existing FieldCache API with the new implementation, so there is no redundent caching as client code migrades to new API. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Lucene 2.2 soon?
: The part I am not sure of in this regard are my changes : to FieldCache and FieldcacheImpl, while LUCENE-831 is : ongoing too. (though btw 831 doesn't apply cleanly on : current trunk). (It is my plan to get into LUCENE-831 : but I haven't got to it yet.) 1) i updated 831 to work on the trunk 2) the spirt of 831 is to change the underlying impl but still be backwards compatible with the FieldCache API ... i skimmed the FieldCache API changes in the latest patch on 446, and they seem to just be adding new get methods (getShort, and getBytes) on the existing underlying impl .. that should be easily supported by the stuff in 831 (and easy to convert/migrate the same way i did the getInt/getLong/getString methods) -Hoss - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Resolved: (LUCENE-894) Custom build.xml for binary distributions
[ https://issues.apache.org/jira/browse/LUCENE-894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Busch resolved LUCENE-894. -- Resolution: Fixed Works great, Hoss! Thanks again. I just committed this. Custom build.xml for binary distributions - Key: LUCENE-894 URL: https://issues.apache.org/jira/browse/LUCENE-894 Project: Lucene - Java Issue Type: Bug Components: Build Affects Versions: 2.1 Reporter: Michael Busch Assignee: Michael Busch Priority: Minor Fix For: 2.2 Attachments: lucene-894.patch, lucene-894.patch, lucene-894.patch, lucene-894.patch, lucene-894.patch The binary files of a distribution come with the demo sources and a build.xml file. However, the build.xml doesn't work for the binary distribution, so it can't be used to build the demos. This problem was notices the first time when release 2.1 was made. Before we ship 2.2 we should fix this. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Created: (LUCENE-904) Calculate MD5 checksums in target dist-all
Calculate MD5 checksums in target dist-all Key: LUCENE-904 URL: https://issues.apache.org/jira/browse/LUCENE-904 Project: Lucene - Java Issue Type: Improvement Components: Build Reporter: Michael Busch Assignee: Michael Busch Priority: Trivial Fix For: 2.2 Trivial patch that extends the ant target dist-all to calculate the MD5 checksums for the dist files. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Updated: (LUCENE-904) Calculate MD5 checksums in target dist-all
[ https://issues.apache.org/jira/browse/LUCENE-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Busch updated LUCENE-904: - Attachment: lucene-904.patch Calculate MD5 checksums in target dist-all Key: LUCENE-904 URL: https://issues.apache.org/jira/browse/LUCENE-904 Project: Lucene - Java Issue Type: Improvement Components: Build Reporter: Michael Busch Assignee: Michael Busch Priority: Trivial Fix For: 2.2 Attachments: lucene-904.patch Trivial patch that extends the ant target dist-all to calculate the MD5 checksums for the dist files. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Commented: (LUCENE-904) Calculate MD5 checksums in target dist-all
[ https://issues.apache.org/jira/browse/LUCENE-904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501097 ] Hoss Man commented on LUCENE-904: - FWIW ... i put something like this in the solr build.xml a while back, but discovered the format ant's checksum task uses by default is really trivial and doesn't work when people use md5sum -c to try and check the sums. (checksum has a format attribute now, but that wasn't added until ~Dec2006, so I think it requires ant 1.7) the way we solved the problem was with the solr-checksum macro added to our build.xml in this commit... http://svn.apache.org/viewvc/lucene/solr/trunk/build.xml?r1=483882r2=484780 background... http://www.nabble.com/%22correct%22-format-for-the-md5-files--tf2779495.html#a7754724 Calculate MD5 checksums in target dist-all Key: LUCENE-904 URL: https://issues.apache.org/jira/browse/LUCENE-904 Project: Lucene - Java Issue Type: Improvement Components: Build Reporter: Michael Busch Assignee: Michael Busch Priority: Trivial Fix For: 2.2 Attachments: lucene-904.patch Trivial patch that extends the ant target dist-all to calculate the MD5 checksums for the dist files. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
