Re: My SOLR read and write is very slow

2022-03-17 Thread Shawn Heisey
On 3/17/2022 6:42 PM, Shawn Heisey wrote: Uncached filter queries in general can be slow ... executing up to 128 of them is likely to be VERY slow. A question for my fellow devs:  I wonder if we can have Solr execute multiple warming queries in parallel.  Perhaps a "warmingThreads" or similar

Re: My SOLR read and write is very slow

2022-03-17 Thread Shawn Heisey
On 3/17/2022 10:18 AM, Baptiste Delbary wrote: I have a problem with my Solr database. I have a core with around 50 millions documents (split in 4 shards) and it's extremely slow. That right there (specifically the autoWarmCount) is likely the reason for the slow commits.  You've told Sol

Re: Return formatted dates from solr query

2022-03-17 Thread Thomas Corthals
If the desired output format is the same for every query, I would store the formatted date in a separate string field when indexing. You can query the date field, but return the string field instead. Thomas Op do 17 mrt. 2022 om 22:08 schreef Shawn Heisey : > On 3/17/22 14:46, Teresa McMains wro

Re: Return formatted dates from solr query

2022-03-17 Thread Shawn Heisey
On 3/17/22 14:46, Teresa McMains wrote: I would like to return the transaction date as a date without the time stamp or time zone but instead it looks like this: Do you know how I might have it strip the timestamp and time zone using a function in the query? Or do I have to change schema.xml (

Re: [EXTERNAL] Re: [EXT] Re: Looking for expertise on comparing Solr search to Postgres full-text search

2022-03-17 Thread dmitri maziuk
On 2022-03-17 3:41 PM, Dave wrote: I’m a big believer in the right tool for the job. Like what said before if you’re doing just a field:value query or four and no complications, sure use a standard rdbms. But if you inform the client that something like Leaves And whitm* title^3 with bf:title^

Return formatted dates from solr query

2022-03-17 Thread Teresa McMains
Client has a date field which is stored in database as a date/time. The field is defined as follows in schema.xml: Where tdate is: Query is formatted as follows (this is shortened for ease of reading): '/solr/aml/select?q=TRANSACTION_REFERENCE_NUMBER%253A%2522${transactionReferenceNumb

how can i get the filename of an uploaded file for indexing?

2022-03-17 Thread Solange Silva Ajuria
hello, your collaboration please. I'm using solr 8.11 I need to configure the requestHandler (/update/extract) to add a field that stores the name of the uploaded file, but I can't find how to do it. in mager-schema.xml, I define a field : FieldType string is defined. But I don't know how to

Re: [EXTERNAL] Re: [EXT] Re: Looking for expertise on comparing Solr search to Postgres full-text search

2022-03-17 Thread Dave
I’m a big believer in the right tool for the job. Like what said before if you’re doing just a field:value query or four and no complications, sure use a standard rdbms. But if you inform the client that something like Leaves And whitm* title^3 with bf:title^3 author ^2 Is possible, the conver

Re: [EXTERNAL] Re: [EXT] Re: Looking for expertise on comparing Solr search to Postgres full-text search

2022-03-17 Thread Davis, Daniel (NIH/NLM) [C]
This is really a question of how big the haystack is and what sort of search task users are trying to accomplish. If there is no IDF (a mistake I did *not* make at https://www.indexengines.com/ despite using home-grown search BTW), then there is an assumption both on the size of the documents b

Re: [EXT] Re: Looking for expertise on comparing Solr search to Postgres full-text search

2022-03-17 Thread Alessandro Benedetti
Ok Charlie, Eric, we are on the same page. I agree it's definitely possible with some custom proxy work on both Quepid and RRE, I meant it's not possible to directly point to the DB (for example via JDBC). Thanks! Cheers -- Alessandro Benedetti Apache Lucene/Solr PMC member

Re: [EXT] Re: Looking for expertise on comparing Solr search to Postgres full-text search

2022-03-17 Thread Bayer, Samuel
You are, indeed :-). What appears to be the problem - and I'm not sure yet, but it sure seems like a good culprit - is that Postgres search, for reasons that mystify me, was implemented with TF but no notion of IDF. There are various extensions that add IDF-like properties to Postgres search.

Re: Looking for expertise on comparing Solr search to Postgres full-text search

2022-03-17 Thread Charlie Hull
Hi, Sort of. You can make Quepid talk to other search engines with a 'shim' layer to make the response look like a Solr or Elasticsearch response. Quepid can send pretty much anything to a HTTP API as the query. There's a project called iSpy that is a prototype for this but I think it's curre

Re: Looking for expertise on comparing Solr search to Postgres full-text search

2022-03-17 Thread Eric Pugh
What I’ve done to compare other search engines with RRE and Quepid is to put a proxy in the middle that converts your query into what looks like a Solr request/response ;-). This works great for custom Search API’s, and I *guess* you could do it with database backed search? Now we are probably

My SOLR read and write is very slow

2022-03-17 Thread Baptiste Delbary
Hy, I have a problem with my Solr database. I have a core with around 50 millions documents (split in 4 shards) and it's extremely slow. It takes around 50 seconds for 1000 new documents to be committed and I frequently get this error: Task queue processing has stalled for 20195ms with 0 remaini

Re: Looking for expertise on comparing Solr search to Postgres full-text search

2022-03-17 Thread Alessandro Benedetti
This is an interesting question. I second both comments so far (from Eric and David), but I am afraid at the moment the open-source tools for search quality evaluation can't really compare Postgres to Solr. As far as I know, both Quepid(Eric correct me if I am wrong) and RRE( https://github.com/Sea

RE: How to run on ip address and not on localhost?

2022-03-17 Thread Solange Silva Ajuria
Thanks, I'll try everything you suggest. Saludos Cordiales, Solange Silva Ajuria               -Mensaje original- De: Shawn Heisey Enviado el: jueves, 17 de marzo de 2022 10:10 Para: users@solr.apache.org Asunto: Re: How to run on ip address and not on localhost? On 3/17/22 07

Re: How to run on ip address and not on localhost?

2022-03-17 Thread Shawn Heisey
On 3/17/22 07:48, Shawn Heisey wrote: You tried to start Solr as root. The start script detected this and refused to start.  As far as I know it won't be possible to start the cloud example as root because a -force parameter on the main script execution will not be transferred to the later call

Re: How to run on ip address and not on localhost?

2022-03-17 Thread Shawn Heisey
On 3/17/22 07:22, Solange Silva Ajuria wrote: Error 404 Not Found This would seem to indicate that the techproducts index doesn't exist.  The fact that you got a 404 means something is listening on the port. According to the official installation documentation /etc/default/solr.in.sh SOLR

How to run on ip address and not on localhost?

2022-03-17 Thread Solange Silva Ajuria
Hello, I have solar installed on my server which is accessed via vpn (putty) to the service installed in linux centos eg: 172.16.100.75:8983 but when I need to execute the indexing command line as shown in the examples of the reference guide, the following error appears: [root@centos-sgd sol

Re: Solr dashboard - number of CPUs available

2022-03-17 Thread Thomas Matthijs
I don't know how it affects solr, but if you're interested in java's support to detect cgroup/container limits on cpu/memory etc, you can use these links as starting points to investigate. It affect some jvm configuration, like initial GC selection & settings that can affect performance. It was

Re: Question regarding the MoreLikeThis features

2022-03-17 Thread Alessandro Benedetti
Hi Marco, I have been working for a long time on the Apache Lucene More Like This component and its integration in Apache Solr as a committer. Let me try to summarise a bit how it works to help you with your use case. You can find benefits from a presentation I gave in Tokyo for the Open Source sum

Re: Solr 8 edismax issue with mm and q.op

2022-03-17 Thread Bernd Fehling
Because your query is "q=Champion Croissance" there is another parameter to take care of. It is "sow" which could be true or false. Just search through the jira issues with edismax, mm and sow to get any reports about it. Regards Bernd Am 16.03.22 um 18:40 schrieb Dominique Bejean: Hi Shawn,