Partial update bug on solr 8.8.0
Any idea about this post? https://stackoverflow.com/q/66335803/141438 Regards.
Re: Getting SOE after a clean import and restart on Solr 7.2.1
Any idea? Can someone direct me where the error come from? As I see, Noggit is a streaming JSON parser, but where is the JSON data? I can only see three tiny files in my core which are in JSON fromat. All of them are valid: 1. _schema_analysis_stopwords_english { "initArgs":{"ignoreCase":false}, "managedList":[]} 2. configoverlay.json {"userProps":{"update.autoCreateFields":"false"}} 3. params.json {"params":{ "query":{ "defType":"edismax", "q.alt":"*:*", "rows":"10", "fl":"*,score", "":{"v":0} }, "facets":{ "facet":"on", "facet.mincount": "1", "":{"v":0} }, "velocity":{ "wt": "velocity", "v.template":"browse", "v.layout": "layout", "":{"v":0} } }} More stack is here: Caused by: java.lang.StackOverflowError at org.noggit.CharArr.reserve(CharArr.java:116) at org.apache.solr.common.util.ByteUtils.UTF8toUTF16(ByteUtils.java:68) at org.apache.solr.common.util.JavaBinCodec.readStr(JavaBinCodec.java:868) at org.apache.solr.common.util.JavaBinCodec.readStr(JavaBinCodec.java:857) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:266) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256) at org.apache.solr.common.util.JavaBinCodec.readOrderedMap(JavaBinCodec.java:199) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:274) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256) at org.apache.solr.common.util.JavaBinCodec.readNamedList(JavaBinCodec.java:210) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:276) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256) at org.apache.solr.common.util.JavaBinCodec.readArray(JavaBinCodec.java:747) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:272) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256) at org.apache.solr.common.util.JavaBinCodec.readNamedList(JavaBinCodec.java:210) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:276) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256) at org.apache.solr.common.util.JavaBinCodec.readOrderedMap(JavaBinCodec.java:199) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:274) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256) at org.apache.solr.common.util.JavaBinCodec.readNamedList(JavaBinCodec.java:210) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:276) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256) at org.apache.solr.common.util.JavaBinCodec.readArray(JavaBinCodec.java:747) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:272) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256) at org.apache.solr.common.util.JavaBinCodec.readArray(JavaBinCodec.java:747) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:272) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256) at org.apache.solr.common.util.JavaBinCodec.readNamedList(JavaBinCodec.java:210) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:276) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256) at org.apache.solr.common.util.JavaBinCodec.readArray(JavaBinCodec.java:747) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:272) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256) at org.apache.solr.common.util.JavaBinCodec.readNamedList(JavaBinCodec.java:210) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:276) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256) at org.apache.solr.common.util.JavaBinCodec.readOrderedMap(JavaBinCodec.java:199) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:274) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256) at org.apache.solr.common.util.JavaBinCodec.readNamedList(JavaBinCodec.java:210) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:276) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256) at org.apache.solr.common.util.JavaBinCodec.readOrderedMap(JavaBinCodec.java:199) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:274) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256) On Tue, Feb 13, 2018 at 3:39 PM, Mohsen Saboorian wrote: > After importing 3 million records from DB into Solr 7.2.1 on a CentOS 7, > OpenJDK 8, I just restarted Solr and the c
Getting SOE after a clean import and restart on Solr 7.2.1
After importing 3 million records from DB into Solr 7.2.1 on a CentOS 7, OpenJDK 8, I just restarted Solr and the core (mycore) cannot instantiate. data/index is 36GB and tlog folder contains a file named tlog.0001862 which is about 20MB. Here is the full log: INFO (main) [ ] o.e.j.s.Server jetty-9.3.20.v20170531 INFO (main) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.2.1 INFO (main) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port 8983 INFO (main) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: /opt/solr INFO (main) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|Start time: 2018-02-13T11:09:55.143Z INFO (main) [ ] o.a.s.u.StartupLoggingUtils Property solr.log.muteconsole given. Muting ConsoleAppender named CONSOLE INFO (main) [ ] o.a.s.c.SolrResourceLoader Using system property solr.solr.home: /var/solr/data INFO (main) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /var/solr/data/solr.xml INFO (main) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /var/solr/data INFO (main) [ ] o.a.s.c.CorePropertiesLocator Cores are: [mycore] INFO (coreLoadExecutor-6-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.c.SolrResourceLoader [null] Added 5 libs to classloader, from paths: [/var/solr/data/mycore/lib] INFO (main) [ ] o.e.j.s.Server Started @4565ms INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.c.SolrResourceLoader [mycore] Added 55 libs to classloader, from paths: [/opt/solr/contrib/clustering/lib, /opt/solr/contrib/extraction/lib, /opt/solr/contrib/langid/lib, /opt/solr/contrib/velocity/lib, /opt/solr/dist] INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.2.1 INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.s.IndexSchema [mycore] Schema name=solr721-mycore WARN (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.LatLonType]. Please consult documentation how to replace it accordingly. INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.s.IndexSchema Loaded schema solr721-mycore/1.6 with uniqueid field id INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.c.CoreContainer Creating SolrCore 'mycore' using configuration from instancedir /var/solr/data/mycore, trusted=true INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.c.SolrCore [[mycore] ] Opening new SolrCore at [/var/solr/data/mycore], dataDir=[/var/solr/data/mycore/data/] INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.r.XSLTResponseWriter xsltCacheLifetimeSeconds=5 INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.c.SolrCore [mycore] CLOSING SolrCore org.apache.solr.core.SolrCore@735e855b INFO (coreLoadExecutor-6-thread-1) [ x:mycore] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.mycore, tag=1935574363 ERROR (coreContainerWorkExecutor-2-thread-1) [ ] o.a.s.c.CoreContainer Error waiting for SolrCore to be created java.util.concurrent.ExecutionException: org.apache.solr.common.SolrException: Unable to create core [mycore] at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.apache.solr.core.CoreContainer.lambda$load$14(CoreContainer.java:671) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.solr.common.SolrException: Unable to create core [mycore] at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1053) at org.apache.solr.core.CoreContainer.lambda$load$13(CoreContainer.java:642) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197) ... 5 more Caused by: org.apache.solr.common.SolrException: Error Instantiating Update Handler, solr.DirectUpdateHandler2 failed to in
problem with boolaen queries after upgrading solr to 6.2
I recently upgraded from Solr 4.10 to 6.2 and fully re-indexed my data. There is something strange with boolean queries against edismax. When searching for term abc, it returns 100 results, while when I add another term, say bcd, with OR it returns 10 results. Is there any change in boolean queries in edismax? Regards, Mohsen
How to handle line breaks for quoted queries
How can I recognize line breaks and do not allow matching of a quoted query in the following example. I have two documents with just one text field: 1. AAA BBB CCC DDD 2. BBB CCC DDD AAA User enters query "BBB CCC". How can I configure tokenizers so that Solr only returns doc #2? Thanks, Mohsen
Re: Remove operation of partial update doesn't work
Solved as I updated to latest solr 5.2.1. I seems that solr 4.10 is buggy with partial remove operation. On Wed, Jul 8, 2015 at 8:44 PM, Mohsen Saboorian wrote: > Can you post your solrj code? > در تاریخ 8 ژوئیهٔ 2015 19:32، "Alessandro Benedetti" < > benedetti.ale...@gmail.com> نوشت: > >> I just tried on my own, and it is working perfectly. >> >> Stupid question, have you committed after your update? >> >> Cheers >> >> 2015-07-08 15:41 GMT+01:00 Mohsen Saboorian : >> >> > >> > I use add and remove both on a multivalue field (think of tags on a blog >> > post). For this, set null won't work because I want only one value >> (tag) to >> > be removed >> > , >> > and set null neither remove one nor all of values (all tags here). >> > >> > So I use some >> > S >> > olr >> > J >> > code which would translate to something like this: >> > >> > { >> > "id": "docId", >> > "tagId": {"remove": "someTagId"} >> > } >> > >> > After commit, there is still taId: someTagId in my document. >> > >> > Here is my schema part for tagId: >> > >> > >> > Thanks, >> > Mohsen >> > >> > On Wed, Jul 8, 2015 at 3:26 PM, Alessandro Benedetti < >> > benedetti.ale...@gmail.com> wrote: >> > >> > > In this scenarios, Documentation is key : >> > > >> > > Modifier >> > > >> > > Usage >> > > >> > > set >> > > >> > > Set or replace the field value(s) with the specified value(s), or >> *remove >> > > the values if 'null' or empty list is specified as the new value.* >> > > >> > > May be specified as a single value, or as a list for multivalued >> fields >> > > >> > > add >> > > >> > > Adds the specified values to a multivalued field. >> > > >> > > May be specified as a single value, or as a list. >> > > remove >> > > >> > > Removes (all occurrences of) the specified values from a multivalued >> > field. >> > > >> > > May be specified as a single value, or as a list. >> > > removeregex >> > > >> > > Removes all occurrences of the specified regex from a multiValued >> field. >> > > >> > > May be specified as a single value, or as a list. >> > > >> > > inc >> > > >> > > Increments a numeric value by a specific amount. >> > > >> > > Must be specified as a single numeric value. >> > > >> > > In my opinion "set" is the right direction to look into. >> > > Not sure what happens if you use the remove to remove only a single >> > valued >> > > field value. >> > > Can you explain us what you noticed ? An empty value remain for that >> > field >> > > ? >> > > It is kind of weird, I would expect the field to become null. >> > > >> > > Cheers >> > > >> > > >> > > 2015-07-08 10:34 GMT+01:00 Mohsen Saboorian : >> > > >> > > > In my code when operation is "add" it works correctly on a >> multivalue >> > > > field. But no multivalue field can be deleted with "remove" >> operation. >> > > > The add operation adds a value to a multivaled field. The remove >> > > operation >> > > > removes a value from a multivalued field. >> > > > >> > > > If you believe that something is not working, please state clearly >> why >> > > you >> > > > believe that something is not working. Start by describing the >> symptom. >> > > > >> > > > >> > > > -- Jack Krupansky >> > > > >> > > > On Mon, Jul 6, 2015 at 9:22 PM, Mohsen Saboorian > > >> > > > wrote: >> > > > >> > > > > I can partially 'add' fields to my Solr index, but 'remove' >> operation >> > > > seems >> > > > > not working. I'm on Solr 4.10. Here is my SolrJ snippet: >> > > > > >> > > > > SolrInputDocument doc = new SolrInputDocument(); >> > > > > Map partialUpdate = new HashMap<>(); >> > > > > partialUpdate.put(operation, value); // value can be >> > object >> > > > > (string, number, etc) or list. operation can be add, set or >> remove. >> > > > > doc.addField("id", id); // document id >> > > > > doc.addField(fieldName, partialUpdate); >> > > > > >> > > > > getSolrServer().add(doc, commitWithin); >> > > > > >> > > > > Is there anything wrong with my code? >> > > > > >> > > > >> > > >> > > >> > > >> > > -- >> > > -- >> > > >> > > Benedetti Alessandro >> > > Visiting card : http://about.me/alessandro_benedetti >> > > >> > > "Tyger, tyger burning bright >> > > In the forests of the night, >> > > What immortal hand or eye >> > > Could frame thy fearful symmetry?" >> > > >> > > William Blake - Songs of Experience -1794 England >> > > >> > >> >> >> >> -- >> -- >> >> Benedetti Alessandro >> Visiting card : http://about.me/alessandro_benedetti >> >> "Tyger, tyger burning bright >> In the forests of the night, >> What immortal hand or eye >> Could frame thy fearful symmetry?" >> >> William Blake - Songs of Experience -1794 England >> >
Re: Remove operation of partial update doesn't work
Can you post your solrj code? در تاریخ 8 ژوئیهٔ 2015 19:32، "Alessandro Benedetti" < benedetti.ale...@gmail.com> نوشت: > I just tried on my own, and it is working perfectly. > Stupid question, have you committed after your update? > > Cheers > > 2015-07-08 15:41 GMT+01:00 Mohsen Saboorian : > > > > > I use add and remove both on a multivalue field (think of tags on a blog > > post). For this, set null won't work because I want only one value (tag) > to > > be removed > > , > > and set null neither remove one nor all of values (all tags here). > > > > So I use some > > S > > olr > > J > > code which would translate to something like this: > > > > { > > "id": "docId", > > "tagId": {"remove": "someTagId"} > > } > > > > After commit, there is still taId: someTagId in my document. > > > > Here is my schema part for tagId: > > > > > > Thanks, > > Mohsen > > > > On Wed, Jul 8, 2015 at 3:26 PM, Alessandro Benedetti < > > benedetti.ale...@gmail.com> wrote: > > > > > In this scenarios, Documentation is key : > > > > > > Modifier > > > > > > Usage > > > > > > set > > > > > > Set or replace the field value(s) with the specified value(s), or > *remove > > > the values if 'null' or empty list is specified as the new value.* > > > > > > May be specified as a single value, or as a list for multivalued fields > > > > > > add > > > > > > Adds the specified values to a multivalued field. > > > > > > May be specified as a single value, or as a list. > > > remove > > > > > > Removes (all occurrences of) the specified values from a multivalued > > field. > > > > > > May be specified as a single value, or as a list. > > > removeregex > > > > > > Removes all occurrences of the specified regex from a multiValued > field. > > > > > > May be specified as a single value, or as a list. > > > > > > inc > > > > > > Increments a numeric value by a specific amount. > > > > > > Must be specified as a single numeric value. > > > > > > In my opinion "set" is the right direction to look into. > > > Not sure what happens if you use the remove to remove only a single > > valued > > > field value. > > > Can you explain us what you noticed ? An empty value remain for that > > field > > > ? > > > It is kind of weird, I would expect the field to become null. > > > > > > Cheers > > > > > > > > > 2015-07-08 10:34 GMT+01:00 Mohsen Saboorian : > > > > > > > In my code when operation is "add" it works correctly on a multivalue > > > > field. But no multivalue field can be deleted with "remove" > operation. > > > > The add operation adds a value to a multivaled field. The remove > > > operation > > > > removes a value from a multivalued field. > > > > > > > > If you believe that something is not working, please state clearly > why > > > you > > > > believe that something is not working. Start by describing the > symptom. > > > > > > > > > > > > -- Jack Krupansky > > > > > > > > On Mon, Jul 6, 2015 at 9:22 PM, Mohsen Saboorian > > > > wrote: > > > > > > > > > I can partially 'add' fields to my Solr index, but 'remove' > operation > > > > seems > > > > > not working. I'm on Solr 4.10. Here is my SolrJ snippet: > > > > > > > > > > SolrInputDocument doc = new SolrInputDocument(); > > > > > Map partialUpdate = new HashMap<>(); > > > > > partialUpdate.put(operation, value); // value can be > > object > > > > > (string, number, etc) or list. operation can be add, set or remove. > > > > > doc.addField("id", id); // document id > > > > > doc.addField(fieldName, partialUpdate); > > > > > > > > > > getSolrServer().add(doc, commitWithin); > > > > > > > > > > Is there anything wrong with my code? > > > > > > > > > > > > > > > > > > > > > -- > > > -- > > > > > > Benedetti Alessandro > > > Visiting card : http://about.me/alessandro_benedetti > > > > > > "Tyger, tyger burning bright > > > In the forests of the night, > > > What immortal hand or eye > > > Could frame thy fearful symmetry?" > > > > > > William Blake - Songs of Experience -1794 England > > > > > > > > > -- > -- > > Benedetti Alessandro > Visiting card : http://about.me/alessandro_benedetti > > "Tyger, tyger burning bright > In the forests of the night, > What immortal hand or eye > Could frame thy fearful symmetry?" > > William Blake - Songs of Experience -1794 England >
Re: Remove operation of partial update doesn't work
Yes I did. I use commitWithin to commit after a fixed timeout. Moreover my add operation works! در تاریخ 8 ژوئیهٔ 2015 19:32، "Alessandro Benedetti" < benedetti.ale...@gmail.com> نوشت: > I just tried on my own, and it is working perfectly. > Stupid question, have you committed after your update? > > Cheers > > 2015-07-08 15:41 GMT+01:00 Mohsen Saboorian : > > > > > I use add and remove both on a multivalue field (think of tags on a blog > > post). For this, set null won't work because I want only one value (tag) > to > > be removed > > , > > and set null neither remove one nor all of values (all tags here). > > > > So I use some > > S > > olr > > J > > code which would translate to something like this: > > > > { > > "id": "docId", > > "tagId": {"remove": "someTagId"} > > } > > > > After commit, there is still taId: someTagId in my document. > > > > Here is my schema part for tagId: > > > > > > Thanks, > > Mohsen > > > > On Wed, Jul 8, 2015 at 3:26 PM, Alessandro Benedetti < > > benedetti.ale...@gmail.com> wrote: > > > > > In this scenarios, Documentation is key : > > > > > > Modifier > > > > > > Usage > > > > > > set > > > > > > Set or replace the field value(s) with the specified value(s), or > *remove > > > the values if 'null' or empty list is specified as the new value.* > > > > > > May be specified as a single value, or as a list for multivalued fields > > > > > > add > > > > > > Adds the specified values to a multivalued field. > > > > > > May be specified as a single value, or as a list. > > > remove > > > > > > Removes (all occurrences of) the specified values from a multivalued > > field. > > > > > > May be specified as a single value, or as a list. > > > removeregex > > > > > > Removes all occurrences of the specified regex from a multiValued > field. > > > > > > May be specified as a single value, or as a list. > > > > > > inc > > > > > > Increments a numeric value by a specific amount. > > > > > > Must be specified as a single numeric value. > > > > > > In my opinion "set" is the right direction to look into. > > > Not sure what happens if you use the remove to remove only a single > > valued > > > field value. > > > Can you explain us what you noticed ? An empty value remain for that > > field > > > ? > > > It is kind of weird, I would expect the field to become null. > > > > > > Cheers > > > > > > > > > 2015-07-08 10:34 GMT+01:00 Mohsen Saboorian : > > > > > > > In my code when operation is "add" it works correctly on a multivalue > > > > field. But no multivalue field can be deleted with "remove" > operation. > > > > The add operation adds a value to a multivaled field. The remove > > > operation > > > > removes a value from a multivalued field. > > > > > > > > If you believe that something is not working, please state clearly > why > > > you > > > > believe that something is not working. Start by describing the > symptom. > > > > > > > > > > > > -- Jack Krupansky > > > > > > > > On Mon, Jul 6, 2015 at 9:22 PM, Mohsen Saboorian > > > > wrote: > > > > > > > > > I can partially 'add' fields to my Solr index, but 'remove' > operation > > > > seems > > > > > not working. I'm on Solr 4.10. Here is my SolrJ snippet: > > > > > > > > > > SolrInputDocument doc = new SolrInputDocument(); > > > > > Map partialUpdate = new HashMap<>(); > > > > > partialUpdate.put(operation, value); // value can be > > object > > > > > (string, number, etc) or list. operation can be add, set or remove. > > > > > doc.addField("id", id); // document id > > > > > doc.addField(fieldName, partialUpdate); > > > > > > > > > > getSolrServer().add(doc, commitWithin); > > > > > > > > > > Is there anything wrong with my code? > > > > > > > > > > > > > > > > > > > > > -- > > > -- > > > > > > Benedetti Alessandro > > > Visiting card : http://about.me/alessandro_benedetti > > > > > > "Tyger, tyger burning bright > > > In the forests of the night, > > > What immortal hand or eye > > > Could frame thy fearful symmetry?" > > > > > > William Blake - Songs of Experience -1794 England > > > > > > > > > -- > -- > > Benedetti Alessandro > Visiting card : http://about.me/alessandro_benedetti > > "Tyger, tyger burning bright > In the forests of the night, > What immortal hand or eye > Could frame thy fearful symmetry?" > > William Blake - Songs of Experience -1794 England >
Re: Remove operation of partial update doesn't work
I use add and remove both on a multivalue field (think of tags on a blog post). For this, set null won't work because I want only one value (tag) to be removed , and set null neither remove one nor all of values (all tags here). So I use some S olr J code which would translate to something like this: { "id": "docId", "tagId": {"remove": "someTagId"} } After commit, there is still taId: someTagId in my document. Here is my schema part for tagId: Thanks, Mohsen On Wed, Jul 8, 2015 at 3:26 PM, Alessandro Benedetti < benedetti.ale...@gmail.com> wrote: > In this scenarios, Documentation is key : > > Modifier > > Usage > > set > > Set or replace the field value(s) with the specified value(s), or *remove > the values if 'null' or empty list is specified as the new value.* > > May be specified as a single value, or as a list for multivalued fields > > add > > Adds the specified values to a multivalued field. > > May be specified as a single value, or as a list. > remove > > Removes (all occurrences of) the specified values from a multivalued field. > > May be specified as a single value, or as a list. > removeregex > > Removes all occurrences of the specified regex from a multiValued field. > > May be specified as a single value, or as a list. > > inc > > Increments a numeric value by a specific amount. > > Must be specified as a single numeric value. > > In my opinion "set" is the right direction to look into. > Not sure what happens if you use the remove to remove only a single valued > field value. > Can you explain us what you noticed ? An empty value remain for that field > ? > It is kind of weird, I would expect the field to become null. > > Cheers > > > 2015-07-08 10:34 GMT+01:00 Mohsen Saboorian : > > > In my code when operation is "add" it works correctly on a multivalue > > field. But no multivalue field can be deleted with "remove" operation. > > The add operation adds a value to a multivaled field. The remove > operation > > removes a value from a multivalued field. > > > > If you believe that something is not working, please state clearly why > you > > believe that something is not working. Start by describing the symptom. > > > > > > -- Jack Krupansky > > > > On Mon, Jul 6, 2015 at 9:22 PM, Mohsen Saboorian > > wrote: > > > > > I can partially 'add' fields to my Solr index, but 'remove' operation > > seems > > > not working. I'm on Solr 4.10. Here is my SolrJ snippet: > > > > > > SolrInputDocument doc = new SolrInputDocument(); > > > Map partialUpdate = new HashMap<>(); > > > partialUpdate.put(operation, value); // value can be object > > > (string, number, etc) or list. operation can be add, set or remove. > > > doc.addField("id", id); // document id > > > doc.addField(fieldName, partialUpdate); > > > > > > getSolrServer().add(doc, commitWithin); > > > > > > Is there anything wrong with my code? > > > > > > > > > -- > -- > > Benedetti Alessandro > Visiting card : http://about.me/alessandro_benedetti > > "Tyger, tyger burning bright > In the forests of the night, > What immortal hand or eye > Could frame thy fearful symmetry?" > > William Blake - Songs of Experience -1794 England >
Re: Remove operation of partial update doesn't work
In my code when operation is "add" it works correctly on a multivalue field. But no multivalue field can be deleted with "remove" operation. The add operation adds a value to a multivaled field. The remove operation removes a value from a multivalued field. If you believe that something is not working, please state clearly why you believe that something is not working. Start by describing the symptom. -- Jack Krupansky On Mon, Jul 6, 2015 at 9:22 PM, Mohsen Saboorian wrote: > I can partially 'add' fields to my Solr index, but 'remove' operation seems > not working. I'm on Solr 4.10. Here is my SolrJ snippet: > > SolrInputDocument doc = new SolrInputDocument(); > Map partialUpdate = new HashMap<>(); > partialUpdate.put(operation, value); // value can be object > (string, number, etc) or list. operation can be add, set or remove. > doc.addField("id", id); // document id > doc.addField(fieldName, partialUpdate); > > getSolrServer().add(doc, commitWithin); > > Is there anything wrong with my code? >
Remove operation of partial update doesn't work
I can partially 'add' fields to my Solr index, but 'remove' operation seems not working. I'm on Solr 4.10. Here is my SolrJ snippet: SolrInputDocument doc = new SolrInputDocument(); Map partialUpdate = new HashMap<>(); partialUpdate.put(operation, value); // value can be object (string, number, etc) or list. operation can be add, set or remove. doc.addField("id", id); // document id doc.addField(fieldName, partialUpdate); getSolrServer().add(doc, commitWithin); Is there anything wrong with my code?
Re: Solr 4.10 very slow on build()
Thanks for the answer. Voted for the issue. On Sat, Nov 8, 2014 at 11:38 PM, Yonik Seeley-5 [via Lucene] < ml-node+s472066n4168375...@n3.nabble.com> wrote: > Try commenting out the suggester component & handler in solrconfig.xml: > https://issues.apache.org/jira/browse/SOLR-6679 > > -Yonik > http://heliosearch.org - native code faceting, facet functions, > sub-facets, off-heap data > > > On Sat, Nov 8, 2014 at 2:03 PM, Mohsen Saboorian <[hidden email] > <http://user/SendEmail.jtp?type=node&node=4168375&i=0>> wrote: > > > I have a ~4GB index which takes a minute (or over) to /build()/ when > starting > > server. I noticed that this happens when I upgrade from solr 4.0 to > 4.10. > > The index was fully rebuilt with solr 4.10 (using DIH). How can I speed > up > > startup time?Here is the slow part of the starting log:INFO > > 141101-23:48:18.239 Loading spell index for spellchecker: wordbreakINFO > > 141101-23:48:18.239 Loading suggester index for: mySuggesterINFO > > 141101-23:48:18.239 reload()INFO *141101-23:48:18.239* build()INFO > > *141101-23:49:15.270* [admin] webapp=null path=/admin/cores > > params={_=1414873135659&wt=json} status=0 QTime=11INFO > 141101-23:49:22.503 > > [news] Registered new searcher Searcher@28195344[news] > > main{StandardDirectoryReader(segments_1b6:65731:nrt _fgm(4.10.1):C244111 > > _1pw(4.10.1):C191483/156:delGen=140 _1wg(4.10.1):C174054/11:delGen=11 > > _236(4.10.1):C1920/1:delGen=1 _23h(4.10.1):C1756 > > _67x(4.10.1):C2120/144:delGen=126 _23l(4.10.1):C2185/2:delGen=2 > > _4ch(4.10.1):C784/145:delGen=126 _3b5(4.10.1):C758/80:delGen=79 > > _23q(4.10.1):C3391 _97s(4.10.1):C1218/136:delGen=127 > > _buo(4.10.1):C1096/86:delGen=84 _eh8(4.10.1):C819/73:delGen=69 > > _fg8(4.10.1):C413/94:delGen=81 _geb(4.10.1):C229/5:delGen=5 > > _g4b(4.10.1):C130/24:delGen=23 _g6c(4.10.1):C144/15:delGen=14 > > _ghj(4.10.1):C21/2:delGen=2 _gj6(4.10.1):C25/3:delGen=3 > > _gfz(4.10.1):C10/1:delGen=1 _ghe(4.10.1):C1 _gir(4.10.1):C3/2:delGen=1 > > _gis(4.10.1):C2/1:delGen=1 _gja(4.10.1):C1 _gjb(4.10.1):C2/1:delGen=1 > > _gjd(4.10.1):C1 _gjj(4.10.1):C1 _gjo(4.10.1):C1 _gjp(4.10.1):C1 > > _gjq(4.10.1):C1 _gjs(4.10.1):C1)}INFO 141101-23:49:22.505 Creating new > > IndexWriter...INFO 141101-23:49:22.506 Waiting until IndexWriter is > > unused... core=newsINFO 141101-23:49:22.506 Closing old IndexWriter... > > core=newsINFO 141101-23:49:22.650 SolrDeletionPolicy.onInit: commits: > > num=1 > > commit{dir=NRTCachingDirectory(MMapDirectory@/app/solr/solrhome/news/data/index > > > lockFactory=NativeFSLockFactory@/app/solr/solrhome/news/data/index; > > maxCacheMB=48.0 > maxMergeSizeMB=4.0),segFN=segments_bpm,generation=15178}INFO > > 141101-23:49:22.650 newest commit generation = 15178 > > > > > > > > -- > > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-4-10-very-slow-on-build-tp4168368.html > > Sent from the Solr - User mailing list archive at Nabble.com. > > > -- > If you reply to this email, your message will be added to the discussion > below: > > http://lucene.472066.n3.nabble.com/Solr-4-10-very-slow-on-build-tp4168368p4168375.html > To unsubscribe from Solr 4.10 very slow on build(), click here > <http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4168368&code=bW9oc2Vuc0BnbWFpbC5jb218NDE2ODM2OHw5MTY2NjQ3NTA=> > . > NAML > <http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-4-10-very-slow-on-build-tp4168368p4168406.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr 4.10 very slow on build()
I have a ~4GB index which takes a minute (or over) to /build()/ when starting server. I noticed that this happens when I upgrade from solr 4.0 to 4.10. The index was fully rebuilt with solr 4.10 (using DIH). How can I speed up startup time?Here is the slow part of the starting log:INFO 141101-23:48:18.239 Loading spell index for spellchecker: wordbreakINFO 141101-23:48:18.239 Loading suggester index for: mySuggesterINFO 141101-23:48:18.239 reload()INFO *141101-23:48:18.239* build()INFO *141101-23:49:15.270* [admin] webapp=null path=/admin/cores params={_=1414873135659&wt=json} status=0 QTime=11INFO 141101-23:49:22.503 [news] Registered new searcher Searcher@28195344[news] main{StandardDirectoryReader(segments_1b6:65731:nrt _fgm(4.10.1):C244111 _1pw(4.10.1):C191483/156:delGen=140 _1wg(4.10.1):C174054/11:delGen=11 _236(4.10.1):C1920/1:delGen=1 _23h(4.10.1):C1756 _67x(4.10.1):C2120/144:delGen=126 _23l(4.10.1):C2185/2:delGen=2 _4ch(4.10.1):C784/145:delGen=126 _3b5(4.10.1):C758/80:delGen=79 _23q(4.10.1):C3391 _97s(4.10.1):C1218/136:delGen=127 _buo(4.10.1):C1096/86:delGen=84 _eh8(4.10.1):C819/73:delGen=69 _fg8(4.10.1):C413/94:delGen=81 _geb(4.10.1):C229/5:delGen=5 _g4b(4.10.1):C130/24:delGen=23 _g6c(4.10.1):C144/15:delGen=14 _ghj(4.10.1):C21/2:delGen=2 _gj6(4.10.1):C25/3:delGen=3 _gfz(4.10.1):C10/1:delGen=1 _ghe(4.10.1):C1 _gir(4.10.1):C3/2:delGen=1 _gis(4.10.1):C2/1:delGen=1 _gja(4.10.1):C1 _gjb(4.10.1):C2/1:delGen=1 _gjd(4.10.1):C1 _gjj(4.10.1):C1 _gjo(4.10.1):C1 _gjp(4.10.1):C1 _gjq(4.10.1):C1 _gjs(4.10.1):C1)}INFO 141101-23:49:22.505 Creating new IndexWriter...INFO 141101-23:49:22.506 Waiting until IndexWriter is unused... core=newsINFO 141101-23:49:22.506 Closing old IndexWriter... core=newsINFO 141101-23:49:22.650 SolrDeletionPolicy.onInit: commits: num=1 commit{dir=NRTCachingDirectory(MMapDirectory@/app/solr/solrhome/news/data/index lockFactory=NativeFSLockFactory@/app/solr/solrhome/news/data/index; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_bpm,generation=15178}INFO 141101-23:49:22.650 newest commit generation = 15178 -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-4-10-very-slow-on-build-tp4168368.html Sent from the Solr - User mailing list archive at Nabble.com.
Search for certain keywords fail: java.io.IOException: Input/output error: NIOFSIndexInput
I have a Solr 4.2 server setup on a CentOS 6.4 x86 and Java: OpenJDK Runtime Environment (rhel-2.3.8.0.el6_4-i386) OpenJDK Server VM (build 23.7-b01, mixed mode) Currently it has a 4.6GB index with ~400k records. When I search for certain keywords, Solr fails with the following message. Any idea why this happens? Can I repair indices? 1. I didn't specify any SolrDeletionPolicy. It's commented in solrconfig.xml as default. 2. is now LUCENE_42 (but it was LUCNE_41 before I upgrade to solr 4.2) Thanks, Mohsen SEVERE: null:java.io.IOException: Input/output error: NIOFSIndexInput(path="/app/solr/tomcat/solr/core1/data/index/_2xmx.tvd") at org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:191) at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:272) at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:51) at org.apache.lucene.util.packed.BlockPackedReaderIterator.skip(BlockPackedReaderIterator.java:127) at org.apache.lucene.codecs.compressing.CompressingTermVectorsReader.readPositions(CompressingTermVectorsReader.java:586) at org.apache.lucene.codecs.compressing.CompressingTermVectorsReader.get(CompressingTermVectorsReader.java:381) at org.apache.lucene.index.SegmentReader.getTermVectors(SegmentReader.java:175) at org.apache.lucene.index.BaseCompositeReader.getTermVectors(BaseCompositeReader.java:97) at org.apache.lucene.search.highlight.TokenSources.getTokenStreamWithOffsets(TokenSources.java:280) at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:453) at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:391) at org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:139) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1817) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:639) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:947) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1009) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) Caused by: java.io.IOException: Input/output error at sun.nio.ch.FileDispatcherImpl.pread0(Native Method) at sun.nio.ch.FileDispatcherImpl.pread(FileDispatcherImpl.java:51) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:222) at sun.nio.ch.IOUtil.read(IOUtil.java:198) at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:674) at org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:176) ... 32 more
SEVERE: null:java.io.IOException: Input/output error: NIOFSIndexInput
I have a Solr 4.2 (with Lucene 4.1 index version) server setup on a CentOS 6.4 x86 and Java: OpenJDK Runtime Environment (rhel-2.3.8.0.el6_4-i386) OpenJDK Server VM (build 23.7-b01, mixed mode) Currently it has a 4.6GB index with ~400k records. When I serach for certain keywords, Solr fails with the following message: Any idea why this happens? Can I repair indices? Thanks. Mohsen SEVERE: null:java.io.IOException: Input/output error: NIOFSIndexInput(path="/app/solr/tomcat/solr/core1/data/index/_2xmx.tvd") at org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:191) at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:272) at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:51) at org.apache.lucene.util.packed.BlockPackedReaderIterator.skip(BlockPackedReaderIterator.java:127) at org.apache.lucene.codecs.compressing.CompressingTermVectorsReader.readPositions(CompressingTermVectorsReader.java:586) at org.apache.lucene.codecs.compressing.CompressingTermVectorsReader.get(CompressingTermVectorsReader.java:381) at org.apache.lucene.index.SegmentReader.getTermVectors(SegmentReader.java:175) at org.apache.lucene.index.BaseCompositeReader.getTermVectors(BaseCompositeReader.java:97) at org.apache.lucene.search.highlight.TokenSources.getTokenStreamWithOffsets(TokenSources.java:280) at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:453) at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:391) at org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:139) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1817) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:639) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:947) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1009) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) Caused by: java.io.IOException: Input/output error at sun.nio.ch.FileDispatcherImpl.pread0(Native Method) at sun.nio.ch.FileDispatcherImpl.pread(FileDispatcherImpl.java:51) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:222) at sun.nio.ch.IOUtil.read(IOUtil.java:198) at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:674) at org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:176) ... 32 more -- View this message in context: http://lucene.472066.n3.nabble.com/SEVERE-null-java-io-IOException-Input-output-error-NIOFSIndexInput-tp4059478.html Sent from the Solr - User mailing list archive at Nabble.com.
How to boost score of documents with a field equal to a special value
Hi, I have documents with id, type, date and body fields. type has values from 1 to 10. I want to boos results with type=1,2 at query time. Is this possible using function query or similar solr tricks? I want something similar to this but for specific integer values: http://wiki.apache.org/solr/SolrRelevancyFAQ#How_can_I_boost_the_score_of_newer_documents Thanks, Mohsen -- View this message in context: http://lucene.472066.n3.nabble.com/How-to-boost-score-of-documents-with-a-field-equal-to-a-special-value-tp3577477p3577477.html Sent from the Solr - User mailing list archive at Nabble.com.