i was wondering if there was a specific test for SpanPayloadCheckQuery method
matches = payloadToMatch.get(upto).bytesEquals(payload); the only implementation i could locate was in collectLeaf of SpanPayloadCheckQuery I will submit JIRA with Patch as a CS *dinosaur* I am more familiar with LISP for dissecting sentence fragments (what we called phenomes) than current SEO implementations Can you suggest a book to read to better understand Lucenes pattern dissection and match algorithms? Many Thanks for helpful guidance Martin ______________________________________________ ________________________________ From: Erik Hatcher <erik.hatc...@gmail.com> Sent: Sunday, April 30, 2017 2:06 PM To: dev@lucene.apache.org Subject: Re: Release 6.6 Martin - I have to admit to still being unsure what the gist is here - is there a bug? Apologies for not catching what you’re saying/showing here. Again, as far as I can tell SpanPayloadCheckQuery is working as expected now. I’m going to focus purely on SOLR-1485 this week to get it committed for 6.6. If there is an issue to address with your work would you please open a JIRA and include your patch there? Thanks, Erik On Apr 30, 2017, at 7:47 AM, Martin Gainty <mgai...@hotmail.com<mailto:mgai...@hotmail.com>> wrote: Mornin' Erik there is a collectLeaf override in org.apache.lucene.queries.payloads.TestPayloadSpans ..but its never called: static class VerifyingCollector implements SpanCollector { List<BytesRef> payloads = new ArrayList<>(); @Override public void collectLeaf(PostingsEnum postings, int position, Term term) throws IOException { .... } } the modification in org.apache.lucene.queries.payloads.TestPayloadCheckQuery tests collectLeaf for query //initialise term log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 231 before term1=new org.apache.lucene.index.Term('field','withPayload')"); org.apache.lucene.index.Term term1=new org.apache.lucene.index.Term("field", "withPayload"); log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 233 term1="+term1); //assume position is 5 int position=5; log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 235 position="+position); BytesRef pay = new BytesRef("pos: " + position); log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 237 pay="+pay); //build spanQuery with term parameter org.apache.lucene.search.spans.SpanQuery spanQuery1 = new SpanTermQuery(term1); log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 239 spanQuery1="+spanQuery1); //add BytesRef to payloadToMatch list java.util.List<org.apache.lucene.util.BytesRef> payloadToMatch=new java.util.ArrayList<org.apache.lucene.util.BytesRef>(); log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 241 payloadToMatch="+payloadToMatch); payloadToMatch.add(pay); //build SpanPayloadCheckQuery query=new org.apache.lucene.queries.payloads.SpanPayloadCheckQuery( (org.apache.lucene.search.spans.SpanQuery)query, (java.util.List<org.apache.lucene.util.BytesRef>)payloadToMatch); log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 249 query="+query); //build lucene Directory (TODO: we should use an existing directory with REAL test-data) org.apache.lucene.store.Directory ram = newDirectory(com.carrotsearch.randomizedtesting.RandomizedContext.current().getRandom()); //build SegmentReader from Directory log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 251 ram="+ram); SegmentReader reader = getOnlySegmentReader(org.apache.lucene.index.DirectoryReader.open(ram)); log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 253 reader="+reader); //populate SlowCompositeReaderWrapper with SegmentReader org.apache.lucene.index.LeafReader sr = org.apache.lucene.index.SlowCompositeReaderWrapper.wrap(reader); log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 255 sr="+sr); //add term to SegmentReader postings org.apache.lucene.index.PostingsEnum postings = sr.postings(term1, PostingsEnum.PAYLOADS); log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 257 before query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings, (int)position,(org.apache.lucene.index.Term)term1) where postings="+postings); //display the parameters for collectLeaf method for query: log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 258 before query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings, (int)position,(org.apache.lucene.index.Term)term1) where position="+position); log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 259 before query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings, (int)position,(org.apache.lucene.index.Term)term1) where term1="+term1); try { //public void collectLeaf(org.apache.lucene.index.PostingsEnum postings, int position, //org.apache.lucene.index.Term term) throws java.io.IOException { query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings, (int)position,(org.apache.lucene.index.Term)term1); } catch(java.io.IOException ioe) { log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 264 query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings, (int)position,(org.apache.lucene.index.Term)term1) LINE 106 throws IOException ="+ioe.getMessage()); } collectLeaf is the only method that compares matches=payloadToMatch.get(upto).bytesEquals(payload) to determine "match" unless of course match between individual payload and payloadToMatch is tested elsewhere ? WDYT? Martin ______________________________________________ ________________________________ From: Erik Hatcher <erik.hatc...@gmail.com<mailto:erik.hatc...@gmail.com>> Sent: Saturday, April 29, 2017 7:58 PM To: Lucene/Solr dev Subject: Re: Release 6.6 Martin - Interesting test but I couldn’t copy/paste it in to try it out as it uses some logging and APIs that aren’t already in the project and classpath of these lucene tests within that queries module (within IntelliJ, mind you). I was able to resolve the misunderstanding (and .equals/.hashCode bugs) so everything is working as expected with SpanPayloadCheckQuery for me now. Do your tests point out an issue? Or confirming that it is working properly at a lower level? Erik On Apr 29, 2017, at 9:08 AM, Martin Gainty <mgai...@hotmail.com<mailto:mgai...@hotmail.com>> wrote: Martin Gainty has shared a OneDrive file with you. To view it, click the link below. <https://1drv.ms/u/s!AkpuiYcNg4cSgWRHc6DadiCIYaFN> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<https://1drv.ms/u/s!AkpuiYcNg4cSgWRHc6DadiCIYaFN> TestPayloadCheckQuery.java<https://1drv.ms/u/s!AkpuiYcNg4cSgWRHc6DadiCIYaFN> attached I coded this last nite so it is "quick and dirty" in any case let me know if testSpanPayloadCheck() method modification properly addresses your situation Thanks! Martin ______________________________________________ ________________________________ From: Erik Hatcher <erik.hatc...@gmail.com<mailto:erik.hatc...@gmail.com>> Sent: Saturday, April 29, 2017 8:58 AM To: dev@lucene.apache.org<mailto:dev@lucene.apache.org> Subject: Re: Release 6.6 Martin - there is a test class specifically for this query. See the recent commits I've made to test it further fixing .equals/.hashCode and rewrite. Can you send your full test code? Erik On Apr 29, 2017, at 07:32, Martin Gainty <mgai...@hotmail.com<mailto:mgai...@hotmail.com>> wrote: when Erik mentioned he couldnt get SpanPayloadCheckQuery to work I was about to reply just run that TestCase (until i discovered there wasnt any!) i'll start the bidding at 1 dinar for TestCase org.apache.lucene.queries.payloads.TestPayloadCheckQuery mod: @org.junit.Test public void testSpanPayloadCheck() throws Exception //now lets test the collectLeaf for query //of course calling Base Class SpanPayloadCheckQuery to construct query log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 257 before query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings, (int)position,(org.apache.lucene.index.Term)term1) where postings="+postings); log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 258 before query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings, (int)position,(org.apache.lucene.index.Term)term1) where position="+position); log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 259 before query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings, (int)position,(org.apache.lucene.index.Term)term1) where term1="+term1); try { //test public void collectLeaf(org.apache.lucene.index.PostingsEnum postings, int position, //org.apache.lucene.index.Term term) throws java.io.IOException { query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings, (int)position,(org.apache.lucene.index.Term)term1); } catch(java.io.IOException ioe) { log.error("TestPayloadCheckQuery::testSpanPayloadCheck LINE 264 query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings, (int)position,(org.apache.lucene.index.Term)term1) LINE 106 throws IOException ="+ioe.getMessage()); } unless of course anyone has a better idea to test collectLeaf ? Martin ______________________________________________ ________________________________ From: Erik Hatcher <erik.hatc...@gmail.com<mailto:erik.hatc...@gmail.com>> Sent: Tuesday, April 25, 2017 7:57 PM To: dev@lucene.apache.org<mailto:dev@lucene.apache.org> Subject: Re: Release 6.6 Probably no bribe needed, but an updated patch would be a good start (or will the 2.5 year old patch still apply and be acceptable as-is?) Erik On Apr 25, 2017, at 7:52 PM, Walter Underwood <wun...@wunderwood.org<mailto:wun...@wunderwood.org>> wrote: Who do I have to bribe to get SOLR-629 included? https://issues.apache.org/jira/browse/SOLR-629 wunder Walter Underwood wun...@wunderwood.org<mailto:wun...@wunderwood.org> http://observer.wunderwood.org/ (my blog) On Apr 25, 2017, at 10:46 AM, Ishan Chattopadhyaya <ichattopadhy...@gmail.com<mailto:ichattopadhy...@gmail.com>> wrote: Hi, We have lots of changes, optimizations, bug fixes for 6.6. I'd like to propose a 6.6 release (perhaps the last 6x non-bug-fix release before 7.0 release). I can volunteer to release this, and I can cut the release branch around 4th May, in order to let some time for devs to finish current issues. What do you think? Regards, Ishan
