bin/solr auth enable

[David Glick]

When I issue “bin/solr auth enable -prompt true -blockUnknown true”, I get a 
Null Pointer Exception.  I’m using the 8.5.1 release.  Am I doing something 
wrong?

Thanks.

Sent from my iPhone

Re: Cybersecurity Incident Report

[Shawn Heisey]

On 7/24/2020 2:35 PM, Man with No Name wrote:
This version of jackson is pulled in as a shadow jar. Also solr is using 
io.netty version 4.1.29.Final which has critical vulnerabilities which 
are fixed in 4.1.44.


It looks like that shaded jackson library is included in the jar for 
htrace.  I looked through the commit history and learned that htrace is 
included for the HDFS support in Solr.  Which means that if you are not 
using the HDFS capability, then htrace will not be used, so the older 
jackson library will not be used either.


If you are not using TLS connections from SolrCloud to ZooKeeper, then 
your install of Solr will not be using the netty library, and 
vulnerabilities in netty will not apply.


The older version of Guava is pulled in with a jar from carrot2.  If 
your Solr install does not use carrot2 clustering, then that version of 
Guava will never be called.


The commons-compress and tika libraries are only used if you have 
configured the extraction contrib, also known as SolrCell.  This contrib 
module is used to index rich-text documents, such as PDF and Word. 
Because it makes Solr unstable, we strongly recommend that nobody should 
use SolrCell in production.  When rich-text documents need to be 
indexed, it should be accomplished by using Tika outside of Solr... and 
if that recommendation is followed, you can control the version used so 
that the well-known vulnerabilities will not be present.


We have always recommended that Solr should be located in a network 
place that can only be reached by systems and people who are authorized. 
 If that is done, then nobody will be able to exploit any 
vulnerabilities that might exist in Solr unless they first successfully 
break into an authorized system.


We do take these reports of vulnerabilities seriously and close them as 
quickly as we can.


Thanks,
Shawn

Re: Loading JSON docs into Solr with Streaming Expressions?

[Joel Bernstein]

It's probably time to add JSON loading support to streaming
expressions, but nothing yet. This ticket is almost done and paves the way
for a suite of parseXYZ functions:

https://issues.apache.org/jira/browse/SOLR-14673



Joel Bernstein
http://joelsolr.blogspot.com/


On Fri, Jul 24, 2020 at 1:00 PM Eric Pugh 
wrote:

> Hey all,   I wanted to load some JSON docs into Solr and as I load them,
> do some manipulations to the documents as they go in.   I looked at
> https://lucene.apache.org/solr/guide/8_6/transforming-and-indexing-custom-json.html
> <
> https://lucene.apache.org/solr/guide/8_6/transforming-and-indexing-custom-json.html>,
> however I also wanted to see if Streaming would help.
>
> I’ve used the combination of cat and parseCSV streaming functions
> successfully to load data into Solr, so I looked a bit at what we could do
> with JSON source format.
>
> I didn’t see an obvious path for taking a .json file and loading it, so I
> played around and made this JSON w/ Lines formatted file streaming
> expression:
> https://github.com/epugh/playing-with-solr-streaming-expressions/pull/3 <
> https://github.com/epugh/playing-with-solr-streaming-expressions/pull/3>
>
> The expression looks like
> commit(icecat,
>   update(icecat,
> parseJSONL(
>   cat('two_docs.jsonl')
> )
>   )
> )
> I was curious what other folks have done?  I saw that there is a
> JSONTupleStream, but it didn’t quite seem to fit the need.
>
> Eric
>
> ___
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com <
> http://www.opensourceconnections.com/> | My Free/Busy <
> http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
>
>

Loading JSON docs into Solr with Streaming Expressions?

[Eric Pugh]

Hey all,   I wanted to load some JSON docs into Solr and as I load them, do 
some manipulations to the documents as they go in.   I looked at 
https://lucene.apache.org/solr/guide/8_6/transforming-and-indexing-custom-json.html
 
,
 however I also wanted to see if Streaming would help.

I’ve used the combination of cat and parseCSV streaming functions successfully 
to load data into Solr, so I looked a bit at what we could do with JSON source 
format.

I didn’t see an obvious path for taking a .json file and loading it, so I 
played around and made this JSON w/ Lines formatted file streaming expression: 
https://github.com/epugh/playing-with-solr-streaming-expressions/pull/3 


The expression looks like
commit(icecat,
  update(icecat,
parseJSONL(
  cat('two_docs.jsonl')
)
  )
)
I was curious what other folks have done?  I saw that there is a 
JSONTupleStream, but it didn’t quite seem to fit the need.

Eric

___
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | 
http://www.opensourceconnections.com  | 
My Free/Busy   
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed 


This e-mail and all contents, including attachments, is considered to be 
Company Confidential unless explicitly stated otherwise, regardless of whether 
attachments are marked as such.

Re: solr suggester.rebuild takes forever and eventually runs out of memory on production

[Anthony Groves]

Hi Sebastian,
I saw similar undesirable results with the standard SuggestComponent when
we upgraded multiple Solr versions. For our index of 2.8M docs, it takes
6-7 minutes to build a suggestion index using SuggestComponent out of the
box, so I wouldn't be surprised if your 7.2M doc index takes a good while.

Do you have any custom Solr plugins? Suggestions are an area where
customization really seems to come in handy. We handle that "suggester not
built" exception here, by just returning an empty suggest results so the
clients do not blow up:
https://github.com/oreillymedia/ifpress-solr-plugin/blob/bf3b07c5be32fbcfa7b6fdfd439d511ef60dab68/src/main/java/com/ifactory/press/db/solr/spelling/suggest/SafariInfixSuggester.java#L141
I was also able to speed up the build slightly by excluding specific
suggest contexts at build time rather than query time (a specific type of
documents we didn't care to include in suggestions).

We did way more sophisticated things with the older SpellCheckComponent
(see MultiSuggester in that same repo) like progress logging and returning
the existing suggest index while a new one is building. If you do choose to
write a custom plugin, feel free to re-use any code there that may help
(Apache License 2.0).


*Anthony Groves*  | Technical Lead, Search
O'Reilly Media, Inc.  | 724.255.7323 <724-255-7323>  |  oreilly.com



On Fri, Jul 24, 2020 at 4:52 AM Sebastian Riemer 
wrote:

> Oh, I am sorry, I totally forgot to mention our solr version, it's 7.7.3.
>
> -Ursprüngliche Nachricht-
> Von: Sebastian Riemer [mailto:s.rie...@littera.eu]
> Gesendet: Freitag, 24. Juli 2020 09:53
> An: solr-user@lucene.apache.org
> Betreff: solr suggester.rebuild takes forever and eventually runs out of
> memory on production
>
> Dear mailing list community,
>
> we have troubles when starting the Suggester-Build on one of our
> production servers.
>
>
> 1.   We execute the required query with the suggest.build parameter
>
> 2.   It seems solr is taking up the task to recreate the suggester
> index (we see that the CPU rises significantly)
>
> 3.   It takes forever to build (and seems to never finish!)
>
> 4.   Sometimes  the linux OOM killer strikes and usually picks the
> solr process and kills it
> 5.   During the rebuild calling the suggester results in "suggester
> not built" exception
> 6.   Restarting the solr-Service has no effect. It just continues the
> rebuild
>
> How long should it take for that task, given that our index currently
> holds approximately 7,2 Mio  documents in a parent/child structure?
> Is it possible, to query the progress of the suggester.build task after it
> was started?
> How can we be sure, whether the suggester.build task is still running or
> whether it is finished?
>
> Which factors have the most significant impact on the duration of the
> rebuild process, given that we use the config below? (Let me now, if you
> need additional information) Can we speed up the process somehow?
>
> Best regards,
> Sebastian
> Solrconfig.xml
> 
>
> name="name">infixSuggester
> name="lookupImpl">AnalyzingInfixLookupFactory
> name="indexPath">infix_suggestions
>
> name="dictionaryImpl">DocumentDictionaryFactory
> name="field">SUGGEST
> name="suggestAnalyzerFieldType">textSuggest
> name="buildOnStartup">false
> name="buildOnCommit">false
>
> 
> 
>
> name="suggest">true
> name="suggest.dictionary">infixSuggester
> name="suggest.onlyMorePopular">true
> name="suggest.count">500
> name="suggest.collate">true
>
>
>suggest
>
> 
>
>
> Mit freundlichen Grüßen
> Sebastian Riemer, BSc
>

Re: Cybersecurity Incident Report

[matthew sporleder]

docker pull solr:8.4.1-slim

docker run -it --rm solr:8.4.1-slim /bin/bash

solr@223042112be5:/opt/solr-8.4.1$ find ./ -name "*jackson*"
./server/solr-webapp/webapp/WEB-INF/lib/jackson-core-2.10.0.jar
./server/solr-webapp/webapp/WEB-INF/lib/jackson-annotations-2.10.0.jar
./server/solr-webapp/webapp/WEB-INF/lib/jackson-dataformat-smile-2.10.0.jar
./server/solr-webapp/webapp/WEB-INF/lib/jackson-databind-2.10.0.jar
./contrib/prometheus-exporter/lib/jackson-jq-0.0.8.jar
./contrib/prometheus-exporter/lib/jackson-core-2.10.0.jar
./contrib/prometheus-exporter/lib/jackson-annotations-2.10.0.jar
./contrib/prometheus-exporter/lib/jackson-databind-2.10.0.jar
./contrib/clustering/lib/jackson-annotations-2.10.0.jar
./contrib/clustering/lib/jackson-databind-2.10.0.jar

How does the scanner work?

On Thu, Jul 23, 2020 at 11:23 PM Man with No Name
 wrote:
>
> Any help on this.?
>
> On Wed, Jul 22, 2020 at 4:25 PM Man with No Name 
> wrote:
>
> > The image is pulled from docker hub. After scanning the image from docker
> > hub, without any modification, this is the list of CVE we're getting.
> >
> >
> > Image  ID  CVE Package  
> >   Version SeverityStatus
> >CVSS
> > -  --  --- ---  
> >   --- --
> >
> > solr:8.4.1-slim57561b4889690532CVE-2019-16335  
> > com.fasterxml.jackson.core_jackson-databind2.4.0   critical 
> >fixed in 2.9.10  9.8
> > solr:8.4.1-slim57561b4889690532CVE-2020-8840   
> > com.fasterxml.jackson.core_jackson-databind2.4.0   critical 
> > 9.8
> > solr:8.4.1-slim57561b4889690532CVE-2020-11620  
> > com.fasterxml.jackson.core_jackson-databind2.4.0   critical 
> >fixed in 2.9.10.49.8
> > solr:8.4.1-slim57561b4889690532CVE-2020-9546   
> > com.fasterxml.jackson.core_jackson-databind2.4.0   critical 
> >fixed in 2.9.10.49.8
> > solr:8.4.1-slim57561b4889690532CVE-2020-9547   
> > com.fasterxml.jackson.core_jackson-databind2.4.0   critical 
> >fixed in 2.9.10.49.8
> > solr:8.4.1-slim57561b4889690532CVE-2019-20445  
> > io.netty_netty-codec   4.1.29.Finalcritical 
> >fixed in 4.1.44  9.1
> > solr:8.4.1-slim57561b4889690532CVE-2020-9548   
> > com.fasterxml.jackson.core_jackson-databind2.4.0   critical 
> >fixed in 2.9.10.49.8
> > solr:8.4.1-slim57561b4889690532CVE-2017-15095  
> > com.fasterxml.jackson.core_jackson-databind2.4.0   critical 
> >fixed in 2.9.1, 2.8.10   9.8
> > solr:8.4.1-slim57561b4889690532CVE-2018-14718  
> > com.fasterxml.jackson.core_jackson-databind2.4.0   critical 
> >fixed in 2.9.7   9.8
> > solr:8.4.1-slim57561b4889690532CVE-2019-16942  
> > com.fasterxml.jackson.core_jackson-databind2.4.0   critical 
> > 9.8
> > solr:8.4.1-slim57561b4889690532CVE-2019-14893  
> > com.fasterxml.jackson.core_jackson-databind2.4.0   critical 
> >fixed in 2.10.0, 2.9.10  9.8
> > solr:8.4.1-slim57561b4889690532CVE-2018-7489   
> > com.fasterxml.jackson.core_jackson-databind2.4.0   critical 
> >fixed in 2.9.5, 2.8.11.1, 2.7.9.39.8
> > solr:8.4.1-slim57561b4889690532CVE-2019-20444  
> > io.netty_netty-codec   4.1.29.Finalcritical 
> >fixed in 4.1.44  9.1
> > solr:8.4.1-slim57561b4889690532CVE-2019-14540  
> > com.fasterxml.jackson.core_jackson-databind2.4.0   critical 
> >fixed in 2.9.10  9.8
> > solr:8.4.1-slim57561b4889690532CVE-2019-16943  
> > com.fasterxml.jackson.core_jackson-databind2.4.0   critical 
> > 9.8
> > solr:8.4.1-slim57561b4889690532CVE-2020-11612  
> > io.netty_netty-codec   4.1.29.Finalcritical 
> >fixed in 4.1.46  9.8
> > solr:8.4.1-slim57561b4889690532CVE-2019-20330  
> > com.fasterxml.jackson.core_jackson-databind2.4.0   critical 
> >fixed in 2.9.10.29.8
> > solr:8.4.1-slim57561b4889690532CVE-2019-17267  
> > com.fasterxml.jackson.core_jackson-databind2.4.0   critical 
> >fixed in 2.9.10  9.8
> >
> >
> > On Tue, Jul 21, 2020 at 

Re: Replication of Solr Model and feature store

[Christine Poerschke (BLOOMBERG/ LONDON)]

Hi Krishan,

Could you share what version of Solr you are using?

And I wonder if the observed behaviour could be reproduced e.g. with the 
techproducts example, changes not applying after reload [1] sounds like a bug 
if so.

Hope that helps.

Regards,

Christine

[1] 
https://lucene.apache.org/solr/guide/8_6/learning-to-rank.html#applying-changes

From: solr-user@lucene.apache.org At: 07/22/20 14:00:59To:  
solr-user@lucene.apache.org
Subject: Re: Replication of Solr Model and feature store

Adding more details here

I need some help on how to enable the solr LTR model and features on all
nodes of a solr cluster.

I am unable to replicate the model and the feature store though from any
master to its slaves with the replication API ? And unable to find any
documentation for the same. Is replication possible?

Without replication, would I have to individually update all nodes of a
cluster ? Or can the feature and model files be read as a resource (like
config or schema) so that I can replicate the file or add the file to my
deployments.


On Wed, Jul 22, 2020 at 5:53 PM krishan goyal  wrote:

> Bump. Any one has an idea how to proceed here ?
>
> On Wed, Jul 8, 2020 at 5:41 PM krishan goyal 
> wrote:
>
>> Hi,
>>
>> How do I enable replication of the model and feature store ?
>>
>> Thanks
>> Krishan
>>
>

Re: How to measure search performance

[Erick Erickson]

Oh, call it a brain fart on my part. docValues have nothing to with 
pseudo-fields, sorry for the confusion. If function queries are used for 
_scoring_, then they must be docValues but that’s not the case here since 
you’re just dealing with pseudo fields.

As far as whether edismax adds work, not really since you’re specifying the 
field to evaluate against. It’s not doing you any good either though…

Best,
Erick

> On Jul 23, 2020, at 3:12 PM, Webster Homer  
> wrote:
> 
> I forgot to mention, the fields being used in the function query are indexed 
> fields. They are mostly text fields that cannot have DocValues
> 
> -Original Message-
> From: Webster Homer 
> Sent: Thursday, July 23, 2020 2:07 PM
> To: solr-user@lucene.apache.org
> Subject: RE: How to measure search performance
> 
> Hi Erick,
> 
> This is an example of a pseudo field: wdim_pno_:if(gt(query({!edismax 
> qf=searchmv_pno v=$q}),0),1,0) I get your point that it would only be applied 
> to the results returned and not to all the results. The intent is to be able 
> to identify which of the fields matched the search. Our business people are 
> keen to know, for internal reasons.
> 
> I have not done a lot of function queries like this, does using edismax make 
> this less performant? My tests have a lot of variability but I do see an 
> effect on the QTime for adding these, but it is hard to quantify. It could be 
> as much as 10%
> 
> Thank you for your quick response.
> Webster
> 
> -Original Message-
> From: Erick Erickson 
> Sent: Thursday, July 23, 2020 12:52 PM
> To: solr-user@lucene.apache.org
> Subject: Re: How to measure search performance
> 
> This isn’t usually a cause for concern. Clearing the caches doesn’t 
> necessarily clear the OS caches for instance. I think you’re already aware 
> that Lucene uses MMapDirectory, meaning the index pages are mapped to OS 
> memory space. Whether those pages are actually _in_ the OS physical memory or 
> not is anyone’s guess so depending on when they’re needed they might have to 
> be read from disk. This is entirely independent of Solr’s caches, and could 
> come into play even if you restarted Solr.
> 
> Then there’s your function queries for the pseudo fields. This is read from 
> the docValues sections of the index. Once again the relevant parts of the 
> index may or may not be in the OS memory.
> 
> So comparing individual queries is “fraught” with uncertainties. I suppose 
> you could reboot the machines each time ;) I’ve only ever had luck averaging 
> a bunch of unique queries when trying to measure perf differences.
> 
> Do note that function queries for pseudo fields is not something I’d expect 
> to add much overhead at all. The reason is that they’re only called for the 
> top N docs that you’re returning, not part of the search at all. Consider a 
> function query involved in scoring. That one must be called for every 
> document that matches. But a function query for a pseudo field is only called 
> for the docs returned in the packet, i.e. the “rows” parameter.
> 
> Best,
> Erick
> 
>> On Jul 23, 2020, at 11:49 AM, Webster Homer 
>>  wrote:
>> 
>> I'm trying to determine the overhead of adding some pseudo fields to one of 
>> our standard searches. The pseudo fields are simply function queries to 
>> report if certain fields matched the query or not. I had thought that I 
>> could run the search without the change and then re-run the searches with 
>> the fields added.
>> I had assumed that the QTime in the query response would be a good metric to 
>> use when comparing the performance of the two search queries. However I see 
>> that the QTime for a query can vary by more than 10%. When testing I cleared 
>> the query cache between tests. Usually the QTime would be within a few 
>> milliseconds of each other, however in some cases there was a 10X or more 
>> difference between them.
>> Even cached queries vary in their QTime, though much less.
>> 
>> I am running Solr 7.7.2 in a solrcloud configuration with 2 shards and 2 
>> replicas/shard. Our nodes have 32Gb memory and 16GB of heap allocated to 
>> solr.
>> 
>> I am concerned that these discrepancies indicate that our system is not 
>> tuned well enough.
>> Should I expect that a query's QTime really is a measure of the query's 
>> inherent performance? Is there a better way to measure query performance?
>> 
>> 
>> 
>> 
>> 
>> This message and any attachment are confidential and may be privileged or 
>> otherwise protected from disclosure. If you are not the intended recipient, 
>> you must not copy this message or attachment or disclose the contents to any 
>> other person. If you have received this transmission in error, please notify 
>> the sender immediately and delete the message and any attachment from your 
>> system. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not 
>> accept liability for any omissions or errors in this message which may arise 
>> as a result of 

Re: tlog keeps growing

[Erick Erickson]

Gaël:

Many thanks for your writeup. I preserved your and Carlos’ comments in a JIRA: 
https://issues.apache.org/jira/browse/SOLR-14679.

How fast you bring the nodes up and down shouldn’t really matter, but if 
pausing between bouncing nodes when doing a rolling upgrade keeps you from 
having operational problems, then it’s the lesser of two evils.

Carlos:

Killing Solr forcefully can be problematic, that makes it much more likely that 
you’ll replay tlogs and trigger this problem. Mind you this is a real problem 
and being patient on shutdown is a band-aid… If it fits for you operationally 
and can 

1> stop ingesting
2> insure a hard commit happens before shutdown

then you should avoid a lot of this and shutdown should be very quick. We 
changed bin/solr to wait up to 180 seconds rather than 10 because “kill -9" was 
causing problems. I realize it’s not always possible to control. It’s kind of a 
pay-me-now-or-pay-me-later situation, the time saved killing Solr may be more 
than used up buy startup later. IIRC, failed leader elections were also more 
frequent with SIGKILL

Best,
Erick

> On Jul 24, 2020, at 5:52 AM, Gael Jourdan-Weil 
>  wrote:
> 
> I think I've come down to the root cause of this mess in our case.
> 
> Everything is confirming that the TLOG state is "BUFFERING" rather than 
> "ACTIVE".
> 1/ This can be seen with the metrics API as well where we observe:
> "TLOG.replay.remaining.bytes":48997506,
> "TLOG.replay.remaining.logs":1,
> "TLOG.state":1,
> 2/ When a hard commit occurs, we can see it in the logs and as the index 
> files are updated ; but we can also see that postCommit and preCommit 
> UpdateLog methods are called but exits immediately which looking at the code 
> indicates the state is "BUFFERING".
> 
> So, why is this TLOG still in "BUFFERING" state?
> 
> From the code, the only place where state is set to "BUFFERING" seems to be 
> UpdateLog.bufferUpdates.
> From the logs, in our case it comes from recovery process. We see the message 
> "Begin buffering updates. core=[col_blue_shard1]".
> Just after we can see "Publishing state of core [col_blue_shard1] as 
> recovering, leader is [http://srv2/solr/col_blue_shard1/] and I am 
> [http://srv1/solr/col_blue_shard1/];.
> 
> Until here, everything is expected I guess but why the TLOG state is not set 
> to "ACTIVE" a bit later?
> 
> Well, the "Begin buffering updates" occured and 500ms later we can see:
> - "Updated live nodes from ZooKeeper... (2) -> (1)" (I think at this time we 
> shut down srv2, this is our main cause of problem)
> - "I am going to be the leader srv1"
> - "Stopping recovery for core=[col_blue_shard1] coreNodeName=[core_node1]"
> And 2s later:
> - "Attempting to PeerSync from [http://srv2/solr/es_blue_shard1/] - 
> recoveringAfterStartup=[true]"
> - "Error while trying to recover. 
> core=es_blue_shard1:org.apache.solr.common.SolrException: Failed to get 
> fingerprint from leader"
> - "Finished recovery process, successful=[false]"
> 
> At this point, I think the root cause on our side is a rolling update that we 
> did too quickly: we stopped node2 while node1 while recovering from it.
> 
> It's still not clear how everything went back to "active" state after such a 
> failed recovery and a TLOG still in "BUFFERING".
> 
> We shouldn't have been in recovery in the first place and I think we know 
> why, this is a first thing that we have adressed.
> Then we need to add some pauses in our rolling update strategy.
> 
> Does it makes sense? Can you think of something else to check/improve?
> 
> Best Regards,
> Gaël

Re: JsonLayout breaks logging?

[t spam]

Hi Naz,

Could you give me some directions in where or how I should provide these
dependencies? I can see these dependencies are already in various places by
default:

[tijmen@solr-1 solr-7.7.3]$ find . -name jackson*
./contrib/clustering/lib/jackson-annotations-2.9.8.jar
./contrib/clustering/lib/jackson-databind-2.9.8.jar
./contrib/prometheus-exporter/lib/jackson-annotations-2.9.8.jar
./contrib/prometheus-exporter/lib/jackson-core-2.9.8.jar
./contrib/prometheus-exporter/lib/jackson-databind-2.9.8.jar
./contrib/prometheus-exporter/lib/jackson-jq-0.0.8.jar
./licenses/jackson-annotations-2.9.8.jar.sha1
./licenses/jackson-annotations-LICENSE-ASL.txt
./licenses/jackson-annotations-NOTICE.txt
./licenses/jackson-core-2.9.8.jar.sha1
./licenses/jackson-core-LICENSE-ASL.txt
./licenses/jackson-core-NOTICE.txt
./licenses/jackson-core-asl-1.9.13.jar.sha1
./licenses/jackson-core-asl-LICENSE-ASL.txt
./licenses/jackson-core-asl-NOTICE.txt
./licenses/jackson-databind-2.9.8.jar.sha1
./licenses/jackson-databind-LICENSE-ASL.txt
./licenses/jackson-databind-NOTICE.txt
./licenses/jackson-dataformat-smile-2.9.8.jar.sha1
./licenses/jackson-dataformat-smile-LICENSE-ASL.txt
./licenses/jackson-dataformat-smile-NOTICE.txt
./licenses/jackson-jq-0.0.8.jar.sha1
./licenses/jackson-jq-LICENSE-ASL.txt
./licenses/jackson-jq-NOTICE.txt
./licenses/jackson-mapper-asl-1.9.13.jar.sha1
./licenses/jackson-mapper-asl-LICENSE-ASL.txt
./licenses/jackson-mapper-asl-NOTICE.txt
./server/solr-webapp/webapp/WEB-INF/lib/jackson-annotations-2.9.8.jar
./server/solr-webapp/webapp/WEB-INF/lib/jackson-core-2.9.8.jar
./server/solr-webapp/webapp/WEB-INF/lib/jackson-core-asl-1.9.13.jar
./server/solr-webapp/webapp/WEB-INF/lib/jackson-databind-2.9.8.jar
./server/solr-webapp/webapp/WEB-INF/lib/jackson-dataformat-smile-2.9.8.jar
./server/solr-webapp/webapp/WEB-INF/lib/jackson-mapper-asl-1.9.13.jar

Thanks for your time.

Tijmen

On Fri, Jul 24, 2020 at 1:16 PM Naz S  wrote:

>
> You should explicitly provide the jackson dependencies: jackson-core,
> jackson-databind and/or jackson-annotations.
>
> On Fri, Jul 24, 2020 at 8:24 AM t spam  wrote:
>
>> Hi,
>>
>> I'm having difficulty configuring JsonLayout for appenders. I have the
>> following in my log4j2.xml:
>>
>> 
>> 
>>   
>>
>> 
>>   
>> 
>>   %d{-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard}
>> %X{replica} %X{core}] %c{1.} %m%n
>> 
>>   
>> 
>>
>> > name="RollingFile"
>> fileName="${sys:solr.log.dir}/solr.log"
>> filePattern="${sys:solr.log.dir}/solr.log.%i" >
>>   
>>   
>> 
>> 
>>   
>>   
>> 
>>
>> > name="SlowFile"
>> fileName="${sys:solr.log.dir}/solr_slow_requests.log"
>> filePattern="${sys:solr.log.dir}/solr_slow_requests.log.%i" >
>>   
>> 
>>   %d{-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard}
>> %X{replica} %X{core}] %c{1.} %m%n
>> 
>>   
>>   
>> 
>> 
>>   
>>   
>> 
>>
>>   
>>   
>> 
>> 
>> 
>> > additivity="false">
>>   
>> 
>>
>> 
>>   
>>   
>> 
>>   
>> 
>>
>> For some reason it just stops logging anything. I only get the solr_gc.log
>> and not the expected solr.log. I see an old thread mentioning the exact
>> same issue (except that solr.log isn't even created in my case) but it
>> wasn't resolved there. See here:
>>
>> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201809.mbox/%3cdb5pr0101mb211949c76c5c39472ecdff87f4...@db5pr0101mb2119.eurprd01.prod.exchangelabs.com%3e
>>
>> I see this behaviour with both SOLR 7 and 8.
>>
>> Am I missing something? Help is much appreciated.
>>
>> - Tijmen
>>
>> This was the original message:
>>
>> -Ursprüngliche Nachricht-
>> Von: Shawn Heisey 
>> Gesendet: Freitag, 7. September 2018 17:58
>> An: solr-user@lucene.apache.org
>> Betreff: Re: Solr 7.4 and log4j2 JSONLayout
>>
>> On 9/6/2018 7:46 AM, Michael Aleythe, Sternwald wrote:
>> > I'm trying to edit the log4j2 logging configuration for solr. The goal
>> is to get a log
>> file in json format. I configured the the JSONLayout for this purpose
>> inside the rollingFile
>> appender in the log4j2.xml. After this solr stops logging entirely.
>> Solr.log file is empty.
>> Only the solr-8983-console.log file contains 10 lines. The line
>> "2018-09-06 13:22:25.378:INFO:oejs.Server:main:
>> Started @2814ms" is the last one.
>> > My first guess was that the jackson-core and jackson-databind jars were
>> missing, but
>> that did not fix the problem.
>>
>> As Varun said, jackson is already included in Solr.  You won't need to
>> add any jars for that.
>>
>> > Does anyone know where to find error-messages or exceptions that point
>> me towards whats
>> going wrong here?
>>
>> Start Solr in the foreground, with the -f option.  This will cause
>> Solr to log to the console.
>> When Solr is started in the background, it suppresses console 

Re: JsonLayout breaks logging?

[Naz S]

You should explicitly provide the jackson dependencies: jackson-core,
jackson-databind and/or jackson-annotations.

On Fri, Jul 24, 2020 at 8:24 AM t spam  wrote:

> Hi,
>
> I'm having difficulty configuring JsonLayout for appenders. I have the
> following in my log4j2.xml:
>
> 
> 
>   
>
> 
>   
> 
>   %d{-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard}
> %X{replica} %X{core}] %c{1.} %m%n
> 
>   
> 
>
>  name="RollingFile"
> fileName="${sys:solr.log.dir}/solr.log"
> filePattern="${sys:solr.log.dir}/solr.log.%i" >
>   
>   
> 
> 
>   
>   
> 
>
>  name="SlowFile"
> fileName="${sys:solr.log.dir}/solr_slow_requests.log"
> filePattern="${sys:solr.log.dir}/solr_slow_requests.log.%i" >
>   
> 
>   %d{-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard}
> %X{replica} %X{core}] %c{1.} %m%n
> 
>   
>   
> 
> 
>   
>   
> 
>
>   
>   
> 
> 
> 
>  additivity="false">
>   
> 
>
> 
>   
>   
> 
>   
> 
>
> For some reason it just stops logging anything. I only get the solr_gc.log
> and not the expected solr.log. I see an old thread mentioning the exact
> same issue (except that solr.log isn't even created in my case) but it
> wasn't resolved there. See here:
>
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201809.mbox/%3cdb5pr0101mb211949c76c5c39472ecdff87f4...@db5pr0101mb2119.eurprd01.prod.exchangelabs.com%3e
>
> I see this behaviour with both SOLR 7 and 8.
>
> Am I missing something? Help is much appreciated.
>
> - Tijmen
>
> This was the original message:
>
> -Ursprüngliche Nachricht-
> Von: Shawn Heisey 
> Gesendet: Freitag, 7. September 2018 17:58
> An: solr-user@lucene.apache.org
> Betreff: Re: Solr 7.4 and log4j2 JSONLayout
>
> On 9/6/2018 7:46 AM, Michael Aleythe, Sternwald wrote:
> > I'm trying to edit the log4j2 logging configuration for solr. The goal
> is to get a log
> file in json format. I configured the the JSONLayout for this purpose
> inside the rollingFile
> appender in the log4j2.xml. After this solr stops logging entirely.
> Solr.log file is empty.
> Only the solr-8983-console.log file contains 10 lines. The line
> "2018-09-06 13:22:25.378:INFO:oejs.Server:main:
> Started @2814ms" is the last one.
> > My first guess was that the jackson-core and jackson-databind jars were
> missing, but
> that did not fix the problem.
>
> As Varun said, jackson is already included in Solr.  You won't need to
> add any jars for that.
>
> > Does anyone know where to find error-messages or exceptions that point
> me towards whats
> going wrong here?
>
> Start Solr in the foreground, with the -f option.  This will cause
> Solr to log to the console.
> When Solr is started in the background, it suppresses console logging.
>
> I see that you have changed the logfile rollover size to 1MB.  If your
> Solr server sees much
> traffic, this is going to result in an extremely fast rollover, which
> may mean that you lose
> access to logged events VERY quickly.  This will especially be the
> case with JSON logging
> -- each event will take up a lot more space.
>
> Thanks,
> Shawn
>

RE: tlog keeps growing

[Gael Jourdan-Weil]

I think I've come down to the root cause of this mess in our case.

Everything is confirming that the TLOG state is "BUFFERING" rather than 
"ACTIVE".
1/ This can be seen with the metrics API as well where we observe:
"TLOG.replay.remaining.bytes":48997506,
"TLOG.replay.remaining.logs":1,
"TLOG.state":1,
2/ When a hard commit occurs, we can see it in the logs and as the index files 
are updated ; but we can also see that postCommit and preCommit UpdateLog 
methods are called but exits immediately which looking at the code indicates 
the state is "BUFFERING".

So, why is this TLOG still in "BUFFERING" state?

>From the code, the only place where state is set to "BUFFERING" seems to be 
>UpdateLog.bufferUpdates.
>From the logs, in our case it comes from recovery process. We see the message 
>"Begin buffering updates. core=[col_blue_shard1]".
Just after we can see "Publishing state of core [col_blue_shard1] as 
recovering, leader is [http://srv2/solr/col_blue_shard1/] and I am 
[http://srv1/solr/col_blue_shard1/];.

Until here, everything is expected I guess but why the TLOG state is not set to 
"ACTIVE" a bit later?

Well, the "Begin buffering updates" occured and 500ms later we can see:
- "Updated live nodes from ZooKeeper... (2) -> (1)" (I think at this time we 
shut down srv2, this is our main cause of problem)
- "I am going to be the leader srv1"
- "Stopping recovery for core=[col_blue_shard1] coreNodeName=[core_node1]"
And 2s later:
- "Attempting to PeerSync from [http://srv2/solr/es_blue_shard1/] - 
recoveringAfterStartup=[true]"
- "Error while trying to recover. 
core=es_blue_shard1:org.apache.solr.common.SolrException: Failed to get 
fingerprint from leader"
- "Finished recovery process, successful=[false]"

At this point, I think the root cause on our side is a rolling update that we 
did too quickly: we stopped node2 while node1 while recovering from it.

It's still not clear how everything went back to "active" state after such a 
failed recovery and a TLOG still in "BUFFERING".

We shouldn't have been in recovery in the first place and I think we know why, 
this is a first thing that we have adressed.
Then we need to add some pauses in our rolling update strategy.

Does it makes sense? Can you think of something else to check/improve?

Best Regards,
Gaël

AW: solr suggester.rebuild takes forever and eventually runs out of memory on production

[Sebastian Riemer]

Oh, I am sorry, I totally forgot to mention our solr version, it's 7.7.3.

-Ursprüngliche Nachricht-
Von: Sebastian Riemer [mailto:s.rie...@littera.eu] 
Gesendet: Freitag, 24. Juli 2020 09:53
An: solr-user@lucene.apache.org
Betreff: solr suggester.rebuild takes forever and eventually runs out of memory 
on production

Dear mailing list community,

we have troubles when starting the Suggester-Build on one of our production 
servers.


1.   We execute the required query with the suggest.build parameter

2.   It seems solr is taking up the task to recreate the suggester index 
(we see that the CPU rises significantly)

3.   It takes forever to build (and seems to never finish!)

4.   Sometimes  the linux OOM killer strikes and usually picks the solr 
process and kills it
5.   During the rebuild calling the suggester results in "suggester not 
built" exception
6.   Restarting the solr-Service has no effect. It just continues the 
rebuild

How long should it take for that task, given that our index currently holds 
approximately 7,2 Mio  documents in a parent/child structure?
Is it possible, to query the progress of the suggester.build task after it was 
started?
How can we be sure, whether the suggester.build task is still running or 
whether it is finished?

Which factors have the most significant impact on the duration of the rebuild 
process, given that we use the config below? (Let me now, if you need 
additional information) Can we speed up the process somehow?

Best regards,
Sebastian
Solrconfig.xml

   
   infixSuggester
   AnalyzingInfixLookupFactory
   infix_suggestions
   
   DocumentDictionaryFactory
   SUGGEST
   textSuggest
   false
   false
   


   
   true
   infixSuggester
   true
   500
   true
   
   
   suggest
   



Mit freundlichen Grüßen
Sebastian Riemer, BSc

solr suggester.rebuild takes forever and eventually runs out of memory on production

[Sebastian Riemer]

Dear mailing list community,

we have troubles when starting the Suggester-Build on one of our production 
servers.


1.   We execute the required query with the suggest.build parameter

2.   It seems solr is taking up the task to recreate the suggester index 
(we see that the CPU rises significantly)

3.   It takes forever to build (and seems to never finish!)

4.   Sometimes  the linux OOM killer strikes and usually picks the solr 
process and kills it
5.   During the rebuild calling the suggester results in "suggester not 
built" exception
6.   Restarting the solr-Service has no effect. It just continues the 
rebuild

How long should it take for that task, given that our index currently holds 
approximately 7,2 Mio  documents in a parent/child structure?
Is it possible, to query the progress of the suggester.build task after it was 
started?
How can we be sure, whether the suggester.build task is still running or 
whether it is finished?

Which factors have the most significant impact on the duration of the rebuild 
process, given that we use the config below? (Let me now, if you need 
additional information)
Can we speed up the process somehow?

Best regards,
Sebastian
Solrconfig.xml

   
   infixSuggester
   AnalyzingInfixLookupFactory
   infix_suggestions
   
   DocumentDictionaryFactory
   SUGGEST
   textSuggest
   false
   false
   


   
   true
   infixSuggester
   true
   500
   true
   
   
   suggest
   



Mit freundlichen Grüßen
Sebastian Riemer, BSc

JsonLayout breaks logging?

[t spam]

Hi,

I'm having difficulty configuring JsonLayout for appenders. I have the
following in my log4j2.xml:



  


  

  %d{-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard}
%X{replica} %X{core}] %c{1.} %m%n

  



  
  


  
  



  

  %d{-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard}
%X{replica} %X{core}] %c{1.} %m%n

  
  


  
  


  
  




  



  
  

  


For some reason it just stops logging anything. I only get the solr_gc.log
and not the expected solr.log. I see an old thread mentioning the exact
same issue (except that solr.log isn't even created in my case) but it
wasn't resolved there. See here:
http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201809.mbox/%3cdb5pr0101mb211949c76c5c39472ecdff87f4...@db5pr0101mb2119.eurprd01.prod.exchangelabs.com%3e

I see this behaviour with both SOLR 7 and 8.

Am I missing something? Help is much appreciated.

- Tijmen

This was the original message:

-Ursprüngliche Nachricht-
Von: Shawn Heisey 
Gesendet: Freitag, 7. September 2018 17:58
An: solr-user@lucene.apache.org
Betreff: Re: Solr 7.4 and log4j2 JSONLayout

On 9/6/2018 7:46 AM, Michael Aleythe, Sternwald wrote:
> I'm trying to edit the log4j2 logging configuration for solr. The goal is to 
> get a log
file in json format. I configured the the JSONLayout for this purpose
inside the rollingFile
appender in the log4j2.xml. After this solr stops logging entirely.
Solr.log file is empty.
Only the solr-8983-console.log file contains 10 lines. The line
"2018-09-06 13:22:25.378:INFO:oejs.Server:main:
Started @2814ms" is the last one.
> My first guess was that the jackson-core and jackson-databind jars were 
> missing, but
that did not fix the problem.

As Varun said, jackson is already included in Solr.  You won't need to
add any jars for that.

> Does anyone know where to find error-messages or exceptions that point me 
> towards whats
going wrong here?

Start Solr in the foreground, with the -f option.  This will cause
Solr to log to the console.
When Solr is started in the background, it suppresses console logging.

I see that you have changed the logfile rollover size to 1MB.  If your
Solr server sees much
traffic, this is going to result in an extremely fast rollover, which
may mean that you lose
access to logged events VERY quickly.  This will especially be the
case with JSON logging
-- each event will take up a lot more space.

Thanks,
Shawn