Sure, Jim. Please go ahead! On Fri, May 12, 2017 at 10:01 PM, jim ferenczi <jim.feren...@gmail.com> wrote:
> Hi, > Would be great to have https://issues.apache.org/jira/browse/LUCENE-7824 > included for 6.6. Can I merge the fix this week end Ishan ? > > 2017-05-11 16:45 GMT+02:00 Ishan Chattopadhyaya <ichattopadhy...@gmail.com > >: > >> Done, Adrien. Thanks! >> >> On Thu, May 11, 2017 at 7:34 PM, Adrien Grand <jpou...@gmail.com> wrote: >> >>> Ishan, wdyt about running addVersion on the branch_6x/master as well? I >>> think it would help realize that the 6.6 branch has been cut when looking >>> at the CHANGES.txt file and not forget about backporting? >>> >>> Le mar. 9 mai 2017 à 17:58, Ishan Chattopadhyaya < >>> ichattopadhy...@gmail.com> a écrit : >>> >>>> I've cut the branch for 6.6. (branch_6_6). Please feel free to add >>>> bugfixes to the branch, if needed. >>>> Planning to build the first RC on 15 May. Please let me know if there >>>> are any objections. >>>> >>>> Thanks, >>>> Ishan >>>> >>>> On Tue, May 9, 2017 at 11:10 AM, Ishan Chattopadhyaya < >>>> ichattopadhy...@gmail.com> wrote: >>>> >>>>> Planning to cut the release branch in another 10-12 hours. >>>>> >>>>> On Mon, May 1, 2017 at 4:00 PM, Martin Gainty <mgai...@hotmail.com> >>>>> wrote: >>>>> >>>>>> i was wondering if there was a specific test for >>>>>> SpanPayloadCheckQuery method >>>>>> >>>>>> matches = payloadToMatch.get(upto).bytesEquals(payload); >>>>>> >>>>>> >>>>>> the only implementation i could locate was in collectLeaf of >>>>>> SpanPayloadCheckQuery >>>>>> >>>>>> >>>>>> I will submit JIRA with Patch >>>>>> >>>>>> >>>>>> as a CS *dinosaur* I am more familiar with LISP for dissecting >>>>>> sentence fragments (what we called phenomes) than current SEO >>>>>> implementations >>>>>> >>>>>> >>>>>> Can you suggest a book to read to better understand Lucenes pattern >>>>>> dissection and match algorithms? >>>>>> >>>>>> >>>>>> Many Thanks for helpful guidance >>>>>> Martin >>>>>> ______________________________________________ >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ------------------------------ >>>>>> *From:* Erik Hatcher <erik.hatc...@gmail.com> >>>>>> *Sent:* Sunday, April 30, 2017 2:06 PM >>>>>> >>>>>> *To:* dev@lucene.apache.org >>>>>> *Subject:* Re: Release 6.6 >>>>>> >>>>>> Martin - >>>>>> >>>>>> I have to admit to still being unsure what the gist is here - is >>>>>> there a bug? Apologies for not catching what you’re saying/showing >>>>>> here. >>>>>> Again, as far as I can tell SpanPayloadCheckQuery is working as expected >>>>>> now. >>>>>> >>>>>> I’m going to focus purely on SOLR-1485 this week to get it committed >>>>>> for 6.6. If there is an issue to address with your work would you please >>>>>> open a JIRA and include your patch there? >>>>>> >>>>>> Thanks, >>>>>> Erik >>>>>> >>>>>> >>>>>> On Apr 30, 2017, at 7:47 AM, Martin Gainty <mgai...@hotmail.com> >>>>>> wrote: >>>>>> >>>>>> Mornin' Erik >>>>>> >>>>>> there is a collectLeaf override in org.apache.lucene >>>>>> .queries.payloads.TestPayloadSpans ..but its never called: >>>>>> >>>>>> static class VerifyingCollector implements SpanCollector { >>>>>> List<BytesRef> payloads = new ArrayList<>(); >>>>>> @Override >>>>>> public void collectLeaf(PostingsEnum postings, int position, Term >>>>>> term) throws IOException { >>>>>> .... >>>>>> } >>>>>> } >>>>>> >>>>>> >>>>>> the modification in org.apache.lucene.queries.payl >>>>>> oads.TestPayloadCheckQuery tests collectLeaf for query >>>>>> >>>>>> //initialise term >>>>>> >>>>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 231 >>>>>> before term1=new org.apache.lucene.index.Term(' >>>>>> field','withPayload')"); >>>>>> org.apache.lucene.index.Term term1=new >>>>>> org.apache.lucene.index.Term("field", >>>>>> "withPayload"); >>>>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 233 >>>>>> term1="+term1); >>>>>> >>>>>> //assume position is 5 >>>>>> int position=5; >>>>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 235 >>>>>> position="+position); >>>>>> >>>>>> BytesRef pay = new BytesRef("pos: " + position); >>>>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 237 >>>>>> pay="+pay); >>>>>> >>>>>> //build spanQuery with term parameter >>>>>> org.apache.lucene.search.spans.SpanQuery spanQuery1 = new >>>>>> SpanTermQuery(term1); >>>>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 239 >>>>>> spanQuery1="+spanQuery1); >>>>>> >>>>>> //add BytesRef to payloadToMatch list >>>>>> java.util.List<org.apache.lucene.util.BytesRef> >>>>>> payloadToMatch=new java.util.ArrayList<org.apache >>>>>> .lucene.util.BytesRef>(); >>>>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 241 >>>>>> payloadToMatch="+payloadToMatch); >>>>>> payloadToMatch.add(pay); >>>>>> >>>>>> //build SpanPayloadCheckQuery >>>>>> >>>>>> query=new org.apache.lucene.queries.payloads.SpanPayloadCheckQuery( >>>>>> (org.apache.lucene.search.spans.SpanQuery)query, >>>>>> (java.util.List<org.apache.lucene.util.BytesRef>)payloadToMatch); >>>>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 249 >>>>>> query="+query); >>>>>> >>>>>> //build lucene Directory (TODO: we should use an existing directory >>>>>> with REAL test-data) >>>>>> org.apache.lucene.store.Directory ram = >>>>>> newDirectory(com.carrotsearch.randomizedtesting.RandomizedCo >>>>>> ntext.current().getRandom()); >>>>>> >>>>>> //build SegmentReader from Directory >>>>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 251 >>>>>> ram="+ram); >>>>>> SegmentReader reader = getOnlySegmentReader(org.apach >>>>>> e.lucene.index.DirectoryReader.open(ram)); >>>>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 253 >>>>>> reader="+reader); >>>>>> >>>>>> //populate SlowCompositeReaderWrapper with SegmentReader >>>>>> org.apache.lucene.index.LeafReader sr = >>>>>> org.apache.lucene.index.SlowCompositeReaderWrapper.wrap(reader); >>>>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 255 >>>>>> sr="+sr); >>>>>> >>>>>> //add term to SegmentReader postings >>>>>> org.apache.lucene.index.PostingsEnum postings = sr.postings(term1, >>>>>> PostingsEnum.PAYLOADS); >>>>>> >>>>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 257 >>>>>> before >>>>>> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings, >>>>>> (int)position,(org.apache.lucene.index.Term)term1) where >>>>>> postings="+postings); >>>>>> >>>>>> //display the parameters for collectLeaf method for query: >>>>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 258 >>>>>> before >>>>>> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings, >>>>>> (int)position,(org.apache.lucene.index.Term)term1) where >>>>>> position="+position); >>>>>> >>>>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 259 >>>>>> before >>>>>> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings, >>>>>> (int)position,(org.apache.lucene.index.Term)term1) where >>>>>> term1="+term1); >>>>>> try >>>>>> { //public void collectLeaf(org.apache.lucene.index.PostingsEnum >>>>>> postings, int position, //org.apache.lucene.index.Term term) throws >>>>>> java.io.IOException { >>>>>> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings, >>>>>> (int)position,(org.apache.lucene.index.Term)term1); >>>>>> } >>>>>> catch(java.io.IOException ioe) { >>>>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck >>>>>> LINE 264 >>>>>> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings, >>>>>> (int)position,(org.apache.lucene.index.Term)term1) LINE 106 throws >>>>>> IOException ="+ioe.getMessage()); } >>>>>> >>>>>> collectLeaf is the only method that compares >>>>>> matches=payloadToMatch.get(upto).bytesEquals(payload) >>>>>> >>>>>> to determine "match" >>>>>> >>>>>> unless of course match between individual payload and payloadToMatch >>>>>> is tested elsewhere ? >>>>>> >>>>>> WDYT? >>>>>> Martin >>>>>> ______________________________________________ >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ------------------------------ >>>>>> *From:* Erik Hatcher <erik.hatc...@gmail.com> >>>>>> *Sent:* Saturday, April 29, 2017 7:58 PM >>>>>> *To:* Lucene/Solr dev >>>>>> *Subject:* Re: Release 6.6 >>>>>> >>>>>> Martin - >>>>>> >>>>>> Interesting test but I couldn’t copy/paste it in to try it out as it >>>>>> uses some logging and APIs that aren’t already in the project and >>>>>> classpath >>>>>> of these lucene tests within that queries module (within IntelliJ, mind >>>>>> you). >>>>>> >>>>>> I was able to resolve the misunderstanding (and .equals/.hashCode >>>>>> bugs) so everything is working as expected with SpanPayloadCheckQuery for >>>>>> me now. Do your tests point out an issue? Or confirming that it is >>>>>> working properly at a lower level? >>>>>> >>>>>> Erik >>>>>> >>>>>> >>>>>> On Apr 29, 2017, at 9:08 AM, Martin Gainty <mgai...@hotmail.com> >>>>>> wrote: >>>>>> >>>>>> Martin Gainty has shared a OneDrive file with you. To view it, click >>>>>> the link below. >>>>>> <https://1drv.ms/u/s!AkpuiYcNg4cSgWRHc6DadiCIYaFN> >>>>>> TestPayloadCheckQuery.java >>>>>> <https://1drv.ms/u/s!AkpuiYcNg4cSgWRHc6DadiCIYaFN> >>>>>> <https://1drv.ms/u/s!AkpuiYcNg4cSgWRHc6DadiCIYaFN> >>>>>> attached >>>>>> >>>>>> I coded this last nite so it is "quick and dirty" >>>>>> >>>>>> in any case let me know if testSpanPayloadCheck() >>>>>> method modification properly addresses your situation >>>>>> >>>>>> Thanks! >>>>>> Martin >>>>>> ______________________________________________ >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ------------------------------ >>>>>> *From:* Erik Hatcher <erik.hatc...@gmail.com> >>>>>> *Sent:* Saturday, April 29, 2017 8:58 AM >>>>>> *To:* dev@lucene.apache.org >>>>>> *Subject:* Re: Release 6.6 >>>>>> >>>>>> Martin - there is a test class specifically for this query. See the >>>>>> recent commits I've made to test it further fixing .equals/.hashCode and >>>>>> rewrite. >>>>>> >>>>>> Can you send your full test code? >>>>>> >>>>>> Erik >>>>>> >>>>>> On Apr 29, 2017, at 07:32, Martin Gainty <mgai...@hotmail.com> wrote: >>>>>> >>>>>> when Erik mentioned he couldnt get SpanPayloadCheckQuery to work I >>>>>> was about to reply just run that TestCase >>>>>> (until i discovered there wasnt any!) >>>>>> >>>>>> i'll start the bidding at 1 dinar for TestCase org.apache.lucene >>>>>> .queries.payloads.TestPayloadCheckQuery mod: >>>>>> @org.junit.Test >>>>>> public void testSpanPayloadCheck() throws Exception >>>>>> >>>>>> >>>>>> //now lets test the collectLeaf for query >>>>>> //of course calling Base Class SpanPayloadCheckQuery to construct >>>>>> query >>>>>> >>>>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 257 >>>>>> before >>>>>> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings, >>>>>> (int)position,(org.apache.lucene.index.Term)term1) where >>>>>> postings="+postings); >>>>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 258 >>>>>> before >>>>>> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings, >>>>>> (int)position,(org.apache.lucene.index.Term)term1) where >>>>>> position="+position); >>>>>> log.debug("TestPayloadCheckQuery::testSpanPayloadCheck LINE 259 >>>>>> before >>>>>> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings, >>>>>> (int)position,(org.apache.lucene.index.Term)term1) where >>>>>> term1="+term1); >>>>>> >>>>>> try >>>>>> { //test public void collectLeaf(org.apache.lucene.index.PostingsEnum >>>>>> postings, int position, //org.apache.lucene.index.Term >>>>>> term) throws java.io.IOException { >>>>>> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings, >>>>>> (int)position,(org.apache.lucene.index.Term)term1); >>>>>> } >>>>>> catch(java.io.IOException ioe) { >>>>>> log.error("TestPayloadCheckQuery::testSpanPayloadCheck >>>>>> LINE 264 >>>>>> query.getPayloadChecker().collectLeaf((org.apache.lucene.index.PostingsEnum)postings, >>>>>> (int)position,(org.apache.lucene.index.Term)term1) LINE 106 throws >>>>>> IOException ="+ioe.getMessage()); } >>>>>> >>>>>> unless of course anyone has a better idea to test collectLeaf ? >>>>>> Martin >>>>>> ______________________________________________ >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ------------------------------ >>>>>> *From:* Erik Hatcher <erik.hatc...@gmail.com> >>>>>> *Sent:* Tuesday, April 25, 2017 7:57 PM >>>>>> *To:* dev@lucene.apache.org >>>>>> *Subject:* Re: Release 6.6 >>>>>> >>>>>> Probably no bribe needed, but an updated patch would be a good start >>>>>> (or will the 2.5 year old patch still apply and be acceptable as-is?) >>>>>> >>>>>> Erik >>>>>> >>>>>> On Apr 25, 2017, at 7:52 PM, Walter Underwood <wun...@wunderwood.org> >>>>>> wrote: >>>>>> >>>>>> Who do I have to bribe to get SOLR-629 included? >>>>>> >>>>>> https://issues.apache.org/jira/browse/SOLR-629 >>>>>> >>>>>> wunder >>>>>> Walter Underwood >>>>>> wun...@wunderwood.org >>>>>> http://observer.wunderwood.org/ (my blog) >>>>>> >>>>>> >>>>>> On Apr 25, 2017, at 10:46 AM, Ishan Chattopadhyaya < >>>>>> ichattopadhy...@gmail.com> wrote: >>>>>> >>>>>> Hi, >>>>>> We have lots of changes, optimizations, bug fixes for 6.6. I'd like >>>>>> to propose a 6.6 release (perhaps the last 6x non-bug-fix release before >>>>>> 7.0 release). >>>>>> >>>>>> I can volunteer to release this, and I can cut the release branch >>>>>> around 4th May, in order to let some time for devs to finish current >>>>>> issues. >>>>>> >>>>>> What do you think? >>>>>> >>>>>> Regards, >>>>>> Ishan >>>>>> >>>>>> >>>>>> >>>>> >>>> >> >
