Hi, Should we report this as a bug in Solr?
Regards, Edwin On Fri, 8 Feb 2019 at 22:18, Zheng Lin Edwin Yeo <edwinye...@gmail.com> wrote: > Hi Paul, > > Regarding the regex (\n\s*){2,} that we are using, when we try in on > https://regex101.com/, it is able to give us the correct result for all > the examples (ie: All of them will only have <br><br>, and not more than > that like what we are getting in Solr in our earlier examples). > > Could there be a possibility of a bug in Solr? > > Regards, > Edwin > > On Fri, 8 Feb 2019 at 00:33, Zheng Lin Edwin Yeo <edwinye...@gmail.com> > wrote: > >> Hi Paul, >> >> We have tried it with the space preceeding the \n i.e. <str >> name="pattern">(\s*\n){2,}</str>, with the following regex pattern: >> >> <processor class="solr.RegexReplaceProcessorFactory"> >> <str name="fieldName">content</str> >> <str name="pattern">(\s*\n){2,}</str> >> <str name="replacement"><br><br></str> >> </processor> >> >> However, we are also getting the exact same results as the earlier >> Example 1, 2 and 3. >> >> As for your point 2 on perhaps in the data you have other (non printing) >> characters than \n, we have find that there are no non printing characters. >> It is just next line with a space. You can refer to the original content in >> the same examples below. >> >> >> Example 1: The sentence that the above regex pattern is working correctly >> *Original content in EML file:* >> Dear Sir, >> >> >> I am terminating >> *Original content:* Dear Sir, \n\n \n \n\n I am terminating >> *Index content: * Dear Sir, <br><br>I am terminating >> >> Example 2: The sentence that the above regex pattern is partially working >> (as you can see, instead of 2 <br>, there are 4 <br>) >> *Original content in EML file:* >> >> *exalted* >> >> *Psalm 89:17* >> >> >> 3 Choa Chu Kang Avenue 4 >> *Original content:* exalted \n \n\n Psalm 89:17 \n\n \n\n 3 Choa >> Chu Kang Avenue 4, Singapore >> *Index content: *exalted <br><br>Psalm 89:17 <br><br> <br><br>3 Choa >> Chu Kang Avenue 4, Singapore >> >> Example 3: The sentence that the above regex pattern is partially working >> (as you can see, instead of 2 <br>, there are 4 <br>) >> *Original content in EML file:* >> >> http://www.concordpri.moe.edu.sg/ >> >> >> >> >> >> >> >> >> On Tue, Dec 18, 2018 at 10:07 AM >> *Original content:* http://www.concordpri.moe.edu.sg/ \n\n \n\n \n >> \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n\n \n\n\n On Tue, Dec 18, >> 2018 at 10:07 AM >> *Index content: *http://www.concordpri.moe.edu.sg/ <br><br> >> <br><br>On Tue, Dec 18, 2018 at 10:07 AM >> >> >> Appreciate any other ideas or suggestions that you may have. >> >> Thank you. >> >> Regards, >> Edwin >> >> On Thu, 7 Feb 2019 at 22:49, <paul.d...@ub.unibe.ch> wrote: >> >>> Hi Edwin >>> >>> >>> >>> 1. Sorry, the pattern was wrong, the space should preceed the \n i.e. >>> <str name="pattern">(\s*\n){2,}</str> >>> 2. Perhaps in the data you have other (non printing) characters than >>> \n? >>> >>> >>> >>> Gesendet von Mail<https://go.microsoft.com/fwlink/?LinkId=550986> für >>> Windows 10 >>> >>> >>> >>> Von: Zheng Lin Edwin Yeo<mailto:edwinye...@gmail.com> >>> Gesendet: Donnerstag, 7. Februar 2019 15:23 >>> An: solr-user@lucene.apache.org<mailto:solr-user@lucene.apache.org> >>> Betreff: Re: RegexReplaceProcessorFactory pattern to detect multiple \n >>> >>> >>> >>> Hi Paul, >>> >>> We have tried this suggested regex pattern as follow: >>> <processor class="solr.RegexReplaceProcessorFactory"> >>> <str name="fieldName">content</str> >>> <str name="pattern">(\n\s*){2,}</str> >>> <str name="replacement"><br><br></str> >>> </processor> >>> >>> But we still have exactly the same problem of Example 1,2 and 3 below. >>> >>> Example 1: The sentence that the above regex pattern is working correctly >>> *Original content:* Dear Sir, \n\n \n \n\n I am terminating >>> *Index content: * Dear Sir, <br><br>I am terminating >>> >>> Example 2: The sentence that the above regex pattern is partially working >>> (as you can see, instead of 2 <br>, there are 4 <br>) >>> *Original content:* exalted \n \n\n Psalm 89:17 \n\n \n\n 3 Choa >>> Chu Kang Avenue 4, Singapore >>> *Index content: *exalted <br><br>Psalm 89:17 <br><br> <br><br>3 Choa >>> Chu Kang Avenue 4, Singapore >>> >>> Example 3: The sentence that the above regex pattern is partially working >>> (as you can see, instead of 2 <br>, there are 4 <br>) >>> *Original content:* http://www.concordpri.moe.edu.sg/ \n\n \n\n \n >>> \n\n >>> \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n\n \n\n\n On Tue, Dec 18, >>> 2018 >>> at 10:07 AM >>> *Index content: *http://www.concordpri.moe.edu.sg/ <br><br> >>> <br><br>On >>> Tue, Dec 18, 2018 at 10:07 AM >>> >>> Any further suggestion? >>> >>> Thank you. >>> >>> Regards, >>> Edwin >>> >>> On Thu, 7 Feb 2019 at 22:20, <paul.d...@ub.unibe.ch> wrote: >>> >>> > To avoid the «\n+\s*» matching too many \n and then failing on the {2,} >>> > part you could try >>> > >>> > >>> > >>> > <str name="pattern">(\n\s*){2,}</str> >>> > >>> > >>> > >>> > If you also want to match CRLF then >>> > >>> > <str name="pattern">(\r?\n\s*){2,}</str> >>> > >>> > >>> > >>> > >>> > >>> > Gesendet von Mail<https://go.microsoft.com/fwlink/?LinkId=550986> für >>> > Windows 10 >>> > >>> > >>> > >>> > Von: Zheng Lin Edwin Yeo<mailto:edwinye...@gmail.com> >>> > Gesendet: Donnerstag, 7. Februar 2019 15:10 >>> > An: solr-user@lucene.apache.org<mailto:solr-user@lucene.apache.org> >>> > Betreff: Re: RegexReplaceProcessorFactory pattern to detect multiple \n >>> > >>> > >>> > >>> > Hi Paul, >>> > >>> > Thanks for your reply. >>> > >>> > When I use this pattern: >>> > <processor class="solr.RegexReplaceProcessorFactory"> >>> > <str name="fieldName">content</str> >>> > <str name="pattern">(\n+\s*){2,}</str> >>> > <str name="replacement"><br><br></str> >>> > </processor> >>> > >>> > It is working for some sentence within the same content and not >>> working for >>> > some sentences. Please see below for the one that is working and >>> another >>> > that is not working (partially working): >>> > >>> > Example 1: The sentence that the above regex pattern is working >>> correctly >>> > *Original content:* Dear Sir, \n\n \n \n\n I am terminating >>> > *Index content: * Dear Sir, <br><br>I am terminating >>> > >>> > Example 2: The sentence that the above regex pattern is partially >>> working >>> > (as you can see, instead of 2 <br>, there are 4 <br>) >>> > *Original content:* exalted \n \n\n Psalm 89:17 \n\n \n\n 3 >>> Choa >>> > Chu Kang Avenue 4, Singapore >>> > *Index content: *exalted <br><br>Psalm 89:17 <br><br> <br><br>3 >>> Choa >>> > Chu Kang Avenue 4, Singapore >>> > >>> > Example 3: The sentence that the above regex pattern is partially >>> working >>> > (as you can see, instead of 2 <br>, there are 4 <br>) >>> > *Original content:* http://www.concordpri.moe.edu.sg/ \n\n \n\n \n >>> > \n\n >>> > \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n\n \n\n\n On Tue, Dec 18, >>> 2018 >>> > at 10:07 AM >>> > *Index content: *http://www.concordpri.moe.edu.sg/ <br><br> >>> <br><br>On >>> > Tue, Dec 18, 2018 at 10:07 AM >>> > >>> > We would appreciate your help to see what is wrong? >>> > >>> > Thank you. >>> > >>> > Regards, >>> > Edwin >>> > >>> > On Thu, 7 Feb 2019 at 21:24, <paul.d...@ub.unibe.ch> wrote: >>> > >>> > > You don’t say what happens, just that it is not working. I assume >>> nothing >>> > > is replaced? Perhaps the pattern should be >>> > > >>> > > >>> > > >>> > > <str name="pattern">"(\n\s*){2,}"</str> >>> > > >>> > > >>> > > >>> > > ?? >>> > > >>> > > >>> > > >>> > > Gesendet von Mail<https://go.microsoft.com/fwlink/?LinkId=550986> >>> für >>> > > Windows 10 >>> > > >>> > > >>> > > >>> > > Von: Zheng Lin Edwin Yeo<mailto:edwinye...@gmail.com> >>> > > Gesendet: Donnerstag, 7. Februar 2019 14:08 >>> > > An: solr-user@lucene.apache.org<mailto:solr-user@lucene.apache.org> >>> > > Betreff: RegexReplaceProcessorFactory pattern to detect multiple \n >>> > > >>> > > >>> > > >>> > > Hi, >>> > > >>> > > I am trying to use the RegexReplaceProcessorFactory to remove more >>> than >>> > two >>> > > \n with any number of spaces between them (Eg: \n\n, \n \n, \n \n \n >>> > \n), >>> > > and replace it with two <br>. >>> > > >>> > > I use the following regex pattern and it is working when I test it in >>> > > regex101.com. But it is not working when I put it inside the >>> > > RegexReplaceProcessorFactory as below: >>> > > >>> > > <updateRequestProcessorChain name="removeCode"> >>> > > <processor class="solr.RegexReplaceProcessorFactory"> >>> > > <str name="fieldName">content</str> >>> > > <str name="pattern">"(\\n\s*){2,}"</str> >>> > > <str name="replacement"><br><br></str> >>> > > </processor> >>> > > </updateRequestProcessorChain> >>> > > >>> > > To explain further about my regex pattern, \s* is instructing the >>> regex >>> > to >>> > > match any \n that have space after and {2,} is instructing the regex >>> to >>> > > match 2 or more occurrence of such pattern (\n). >>> > > >>> > > Please kindly let me know what is wrong and how should I do it? >>> > > >>> > > I am using Solr 7.6.0. >>> > > >>> > > Regards, >>> > > Edwin >>> > > >>> > >>> >>