You're right, I was able to reproduce it too without highlighting. Regarding the existing bug, I think there might be an additional issue here because it happens only when id field contains an underscore (didn't check for other special characters). Currently I have no other choice but to use enableLazyFieldLoading=false. I hope it wouldn't have a significant performance impact.
-----Original Message----- From: David Smiley <dsmi...@apache.org> Sent: יום ה 18 פברואר 2021 01:03 To: solr-user <solr-user@lucene.apache.org> Subject: Re: Atomic Update (nested), Unified Highlighter and Lazy Field Loading => Invalid Index I think the issue is this existing bug, but needs to refer to toSolrInputDocument instead of toSolrDoc: https://issues.apache.org/jira/browse/SOLR-13034 Highlighting isn't involved; you just need to somehow get a document cached with lazy fields. In a test I was able to do this simply by doing a query that only returns the "id" field. No highlighting. ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Wed, Feb 17, 2021 at 10:28 AM David Smiley <dsmi...@apache.org> wrote: > Thanks for more details. I was able to reproduce this locally! I > hacked a test to look similar to what you are doing. BTW it's okay to > fill out a JIRA imperfectly; they can always be edited :-). Once I > better understand the nature of the bug today, I'll file an issue and respond > with it here. > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley > > > On Wed, Feb 17, 2021 at 6:36 AM Nussbaum, Ronen > <ronen.nussb...@verint.com> > wrote: > >> Hello David, >> >> Thank you for your reply. >> It was very hard but finally I discovered how to reproduce it. I >> thought of issuing an issue but wasn't sure about the components and >> priority. >> I used the "tech products" configset, with the following changes: >> 1. Added <field name="_nest_path_" type="_nest_path_" /><fieldType >> name="_nest_path_" class="solr.NestPathField" /> 2. Added <field >> name="text_en" type="text_en" indexed="true" >> stored="true" termVectors="true" termOffsets="true" termPositions="true" >> required="false" multiValued="true" /> Than I inserted one document >> with a nested child e.g. >> {id:"abc_1", utterances:{id:"abc_1-1", text_en:"Solr is great"}} >> >> To reproduce: >> Do a search with surround and unified highlighter: >> >> hl.fl=text_en&hl.method=unified&hl=on&q=%7B!surround%7Dtext_en%3A4W(" >> solr"%2C"great") >> >> Now, try to update the parent e.g. {id:"abc_1", categories_i:{add:1}} >> >> Important: it happens only when "id" contains underscore characters! >> If you'll use "abc-1" it would work. >> >> Thanks in advance, >> Ronen. >> >> -----Original Message----- >> From: David Smiley <dsmi...@apache.org> >> Sent: יום א 14 פברואר 2021 19:17 >> To: solr-user <solr-user@lucene.apache.org> >> Subject: Re: Atomic Update (nested), Unified Highlighter and Lazy >> Field Loading => Invalid Index >> >> Hello Ronen, >> >> Can you please file a JIRA issue? Some quick searches did not turn >> anything up. It would be super helpful to me if you could list a >> series of steps with Solr out-of-the-box in 8.8 including what data >> to index and query. Solr already includes the "tech products" sample >> data; maybe that can illustrate the problem? It's not clear if >> nested schema or nested docs are actually required in your example. >> If you share the JIRA issue with me, I'll chase this one down. >> >> ~ David Smiley >> Apache Lucene/Solr Search Developer >> http://www.linkedin.com/in/davidwsmiley >> >> >> On Sun, Feb 14, 2021 at 11:16 AM Ronen Nussbaum <rone...@gmail.com> >> wrote: >> >> > Hi All, >> > >> > I discovered a strange behaviour with this combination. >> > Not only the atomic update fails, the child documents are not >> > properly indexed, and you can't use highlights on their text >> > fields. Currently there is no workaround other than reindex. >> > >> > Checked on 8.3.0, 8.6.1 and 8.8.0. >> > 1. Configure nested schema. >> > 2. enableLazyFieldLoading is true (default). >> > 3. Run a search with hl.method=unified and hl.fl=<one of child text >> > fields> 4. Trying to do an atomic update on some of the *parents* >> > fields> of >> > the returned documents from #3. >> > >> > You get an error: "TransactionLog doesn't know how to serialize >> > class org.apache.lucene.document.LazyDocument$LazyField". >> > >> > Now trying to run #3 again yields an error message that the text >> > field is indexed without positions. >> > >> > If enableLazyFieldLoading is false or if using the default >> > highlighter this doesn't happen. >> > >> > Ronen. >> > >> >> >> This electronic message may contain proprietary and confidential >> information of Verint Systems Inc., its affiliates and/or >> subsidiaries. The information is intended to be for the use of the >> individual(s) or >> entity(ies) named above. If you are not the intended recipient (or >> authorized to receive this e-mail for the intended recipient), you >> may not use, copy, disclose or distribute to anyone this message or >> any information contained in this message. If you have received this >> electronic message in error, please notify us by replying to this e-mail. >> > This electronic message may contain proprietary and confidential information of Verint Systems Inc., its affiliates and/or subsidiaries. The information is intended to be for the use of the individual(s) or entity(ies) named above. If you are not the intended recipient (or authorized to receive this e-mail for the intended recipient), you may not use, copy, disclose or distribute to anyone this message or any information contained in this message. If you have received this electronic message in error, please notify us by replying to this e-mail.
