Planning to cut the release branch in another 10-12 hours.
On Mon, May 1, 2017 at 4:00 PM, Martin Gainty <mgai...@hotmail.com> wrote:
> i was wondering if there was a specific test for SpanPayloadCheckQuery
> method
>
> matches = payloadToMatch.get(upto).bytesEquals(payload);
>
>
> the only implementation i could locate was in collectLeaf of
> SpanPayloadCheckQuery
>
>
> I will submit JIRA with Patch
>
>
> as a CS *dinosaur* I am more familiar with LISP for dissecting sentence
> fragments (what we called phenomes) than current SEO implementations
>
>
> Can you suggest a book to read to better understand Lucenes pattern
> dissection and match algorithms?
>
>
> Many Thanks for helpful guidance
> Martin
> ______________________________________________
>
>
>
>
> ------------------------------
> *From:* Erik Hatcher <erik.hatc...@gmail.com>
> *Sent:* Sunday, April 30, 2017 2:06 PM
>
> *To:* dev@lucene.apache.org
> *Subject:* Re: Release 6.6
>
> Martin -
>
> I have to admit to still being unsure what the gist is here - is there a
> bug? Apologies for not catching what you’re saying/showing here. Again,
> as far as I can tell SpanPayloadCheckQuery is working as expected now.
>
> I’m going to focus purely on SOLR-1485 this week to get it committed for
> 6.6. If there is an issue to address with your work would you please open
> a JIRA and include your patch there?
>
> Thanks,
> Erik
>
>
> On Apr 30, 2017, at 7:47 AM, Martin Gainty <mgai...@hotmail.com> wrote:
>
> Mornin' Erik
>
> there is a collectLeaf override in org.apache.lucene.queries.payloads.
> TestPayloadSpans ..but its never called:
>
> static class VerifyingCollector implements SpanCollector {
> List<BytesRef> payloads = new ArrayList<>();
> @Override
> public void collectLeaf(PostingsEnum postings, int position, Term
> term) throws IOException {
> ....
> }
> }
>
>
> the modification in org.apache.lucene.queries.payloads.TestPayloadCheckQuery
> tests
> collectLeaf for query
>
> //initialise term
>
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 231 before
> term1=new org.apache.lucene.index.Term('field','withPayload')");
> org.apache.lucene.index.Term term1=new org.apache.lucene.index.Term("field",
> "withPayload");
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 233
> term1="+term1);
>
> //assume position is 5
> int position=5;
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 235
> position="+position);
>
> BytesRef pay = new BytesRef("pos: " + position);
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 237
> pay="+pay);
>
> //build spanQuery with term parameter
> org.apache.lucene.search.spans.SpanQuery spanQuery1 = new
> SpanTermQuery(term1);
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 239
> spanQuery1="+spanQuery1);
>
> //add BytesRef to payloadToMatch list
> java.util.List<org.apache.lucene.util.BytesRef> payloadToMatch=new
> java.util.ArrayList<org.apache.lucene.util.BytesRef>();
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 241
> payloadToMatch="+payloadToMatch);
> payloadToMatch.add(pay);
>
> //build SpanPayloadCheckQuery
>
> query=new org.apache.lucene.queries.payloads.SpanPayloadCheckQuery(
> (org.apache.lucene.search.spans.SpanQuery)query,
> (java.util.List<org.apache.lucene.util.BytesRef>)payloadToMatch);
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 249
> query="+query);
>
> //build lucene Directory (TODO: we should use an existing directory with
> REAL test-data)
> org.apache.lucene.store.Directory ram = newDirectory(com.carrotsearch.
> randomizedtesting.RandomizedContext.current().getRandom());
>
> //build SegmentReader from Directory
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 251
> ram="+ram);
> SegmentReader reader = getOnlySegmentReader(org.apache.lucene.index.
> DirectoryReader.open(ram));
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 253
> reader="+reader);
>
> //populate SlowCompositeReaderWrapper with SegmentReader
> org.apache.lucene.index.LeafReader sr = org.apache.lucene.index.
> SlowCompositeReaderWrapper.wrap(reader);
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 255
> sr="+sr);
>
> //add term to SegmentReader postings
> org.apache.lucene.index.PostingsEnum postings = sr.postings(term1,
> PostingsEnum.PAYLOADS);
>
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 257 before
> query.getPayloadChecker().collectLeaf((org.apache.
> lucene.index.PostingsEnum)postings,
> (int)position,(org.apache.lucene.index.Term)term1)
> where postings="+postings);
>
> //display the parameters for collectLeaf method for query:
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 258 before
> query.getPayloadChecker().collectLeaf((org.apache.
> lucene.index.PostingsEnum)postings,
> (int)position,(org.apache.lucene.index.Term)term1)
> where position="+position);
>
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 259 before
> query.getPayloadChecker().collectLeaf((org.apache.
> lucene.index.PostingsEnum)postings,
> (int)position,(org.apache.lucene.index.Term)term1)
> where term1="+term1);
> try
> { //public void collectLeaf(org.apache.lucene.index.PostingsEnum
> postings, int position, //org.apache.lucene.index.Term term) throws
> java.io.IOException {
> query.getPayloadChecker().collectLeaf((org.apache.
> lucene.index.PostingsEnum)postings, (int)position,(org.apache.
> lucene.index.Term)term1);
> }
> catch(java.io.IOException ioe) {
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck
> LINE 264 query.getPayloadChecker().collectLeaf((org.apache.
> lucene.index.PostingsEnum)postings,
> (int)position,(org.apache.lucene.index.Term)term1)
> LINE 106 throws IOException ="+ioe.getMessage()); }
>
> collectLeaf is the only method that compares matches=payloadToMatch.get(
> upto).bytesEquals(payload)
>
> to determine "match"
>
> unless of course match between individual payload and payloadToMatch is
> tested elsewhere ?
>
> WDYT?
> Martin
> ______________________________________________
>
>
>
>
> ------------------------------
> *From:* Erik Hatcher <erik.hatc...@gmail.com>
> *Sent:* Saturday, April 29, 2017 7:58 PM
> *To:* Lucene/Solr dev
> *Subject:* Re: Release 6.6
>
> Martin -
>
> Interesting test but I couldn’t copy/paste it in to try it out as it uses
> some logging and APIs that aren’t already in the project and classpath of
> these lucene tests within that queries module (within IntelliJ, mind you).
>
> I was able to resolve the misunderstanding (and .equals/.hashCode bugs) so
> everything is working as expected with SpanPayloadCheckQuery for me now.
> Do your tests point out an issue? Or confirming that it is working
> properly at a lower level?
>
> Erik
>
>
> On Apr 29, 2017, at 9:08 AM, Martin Gainty <mgai...@hotmail.com> wrote:
>
> Martin Gainty has shared a OneDrive file with you. To view it, click the
> link below.
> <https://1drv.ms/u/s!AkpuiYcNg4cSgWRHc6DadiCIYaFN>
> TestPayloadCheckQuery.java
> <https://1drv.ms/u/s!AkpuiYcNg4cSgWRHc6DadiCIYaFN>
> <https://1drv.ms/u/s!AkpuiYcNg4cSgWRHc6DadiCIYaFN>
> attached
>
> I coded this last nite so it is "quick and dirty"
>
> in any case let me know if testSpanPayloadCheck() method modification
> properly addresses your situation
>
> Thanks!
> Martin
> ______________________________________________
>
>
>
>
> ------------------------------
> *From:* Erik Hatcher <erik.hatc...@gmail.com>
> *Sent:* Saturday, April 29, 2017 8:58 AM
> *To:* dev@lucene.apache.org
> *Subject:* Re: Release 6.6
>
> Martin - there is a test class specifically for this query. See the
> recent commits I've made to test it further fixing .equals/.hashCode and
> rewrite.
>
> Can you send your full test code?
>
> Erik
>
> On Apr 29, 2017, at 07:32, Martin Gainty <mgai...@hotmail.com> wrote:
>
> when Erik mentioned he couldnt get SpanPayloadCheckQuery to work I was
> about to reply just run that TestCase
> (until i discovered there wasnt any!)
>
> i'll start the bidding at 1 dinar for TestCase org.apache.
> lucene.queries.payloads.TestPayloadCheckQuery mod:
> @org.junit.Test
> public void testSpanPayloadCheck() throws Exception
>
>
> //now lets test the collectLeaf for query
> //of course calling Base Class SpanPayloadCheckQuery to construct query
>
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 257 before
> query.getPayloadChecker().collectLeaf((org.apache.
> lucene.index.PostingsEnum)postings,
> (int)position,(org.apache.lucene.index.Term)term1)
> where postings="+postings);
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 258 before
> query.getPayloadChecker().collectLeaf((org.apache.
> lucene.index.PostingsEnum)postings,
> (int)position,(org.apache.lucene.index.Term)term1)
> where position="+position);
> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 259 before
> query.getPayloadChecker().collectLeaf((org.apache.
> lucene.index.PostingsEnum)postings,
> (int)position,(org.apache.lucene.index.Term)term1)
> where term1="+term1);
>
> try
> { //test public void collectLeaf(org.apache.lucene.index.PostingsEnum
> postings, int position, //org.apache.lucene.index.Term term)
> throws java.io.IOException {
> query.getPayloadChecker().collectLeaf((org.apache.
> lucene.index.PostingsEnum)postings, (int)position,(org.apache.
> lucene.index.Term)term1);
> }
> catch(java.io.IOException ioe) {
> log.error("TestPayloadCheckQuery::testSpanPayloadCheck
> LINE 264 query.getPayloadChecker().collectLeaf((org.apache.
> lucene.index.PostingsEnum)postings,
> (int)position,(org.apache.lucene.index.Term)term1)
> LINE 106 throws IOException ="+ioe.getMessage()); }
>
> unless of course anyone has a better idea to test collectLeaf ?
> Martin
> ______________________________________________
>
>
>
>
> ------------------------------
> *From:* Erik Hatcher <erik.hatc...@gmail.com>
> *Sent:* Tuesday, April 25, 2017 7:57 PM
> *To:* dev@lucene.apache.org
> *Subject:* Re: Release 6.6
>
> Probably no bribe needed, but an updated patch would be a good start (or
> will the 2.5 year old patch still apply and be acceptable as-is?)
>
> Erik
>
> On Apr 25, 2017, at 7:52 PM, Walter Underwood <wun...@wunderwood.org>
> wrote:
>
> Who do I have to bribe to get SOLR-629 included?
>
> https://issues.apache.org/jira/browse/SOLR-629
>
> wunder
> Walter Underwood
> wun...@wunderwood.org
> http://observer.wunderwood.org/ (my blog)
>
>
> On Apr 25, 2017, at 10:46 AM, Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
> Hi,
> We have lots of changes, optimizations, bug fixes for 6.6. I'd like to
> propose a 6.6 release (perhaps the last 6x non-bug-fix release before 7.0
> release).
>
> I can volunteer to release this, and I can cut the release branch around
> 4th May, in order to let some time for devs to finish current issues.
>
> What do you think?
>
> Regards,
> Ishan
>
>
>
