Re: Need an advice for architecture.

2018-07-19 Thread Chris Hostetter
: FWIW: I used the script below to build myself 3.8 million documents, with : 300 "text fields" consisting of anywhere from 1-10 "words" (integers : between 1 and 200) Whoops ... forgot to post the script... #!/usr/bin/perl use strict; use warnings; my $num_docs = 3_800_000; my

Re: Need an advice for architecture.

2018-07-19 Thread Chris Hostetter
: SQL DB 4M documents with up to 5000 metadata fields each document [2xXeon : 2.1Ghz, 32GB RAM] : Actual Solr: 1 Core version 4.6, 3.8M documents, schema has 300 metadata : fields to import, size 3.6GB [2xXeon 2.4Ghz, 32GB RAM] : (atm we need 35h to build the index and about 24h for a mass

Re: Document Count Difference Between Solr Versions 4.7 and 7.3

2018-07-19 Thread Chris Hostetter
: I performed a bulk reindex against one of our larger databases for the first : time using solr 7.3. The document count was substantially less (like at : least 15% less) than our most recent bulk reindex from th previous solr 4.7 : server. I will perform a more careful analysis, but I am

RE: SOLR 7.2.1 on SLES 11?

2018-07-19 Thread Lichte, Lucas R - DHS (Tek Systems)
Welp, that didn't go spectacularly. All the OpenSuSE SLES 11 downloads are RPM, both source and compiled. Non-relocatable. I did attempt to rebuild, but it choked on the following dependencies: audit-devel is needed by bash-4.3-286.1.x86_64 fdupes is needed by bash-4.3-286.1.x86_64

Re: CDCR documentation typo

2018-07-19 Thread Erick Erickson
Thanks, but I think that section has been reworked, that typo isn't in the current documentation. It's doubtful that we'll re-release that reference guide. Best, Erick On Thu, Jul 19, 2018 at 3:14 AM, Yair Yotam wrote: > Hi, > > CDCR documentation page for v 7.1: >

Re: CDCR documentation typo

2018-07-19 Thread Alexandre Rafalovitch
Thank you for sharing this with others. For documentation, it looks like it had been refactored and fixed already: https://lucene.apache.org/solr/guide/7_4/cdcr-config.html Regards, Alex. On 19 July 2018 at 06:14, Yair Yotam wrote: > Hi, > > CDCR documentation page for v 7.1: >

Re: Need an advice for architecture.

2018-07-19 Thread Walter Underwood
Are you doing a commit after every document? Is the index on local disk? That is very slow indexing. With four shards and smaller documents, we can index about a million documents per minute. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Jul 19,

Re: Document Count Difference Between Solr Versions 4.7 and 7.3

2018-07-19 Thread David Hastings
monitor the logging on the admin interface while indexing. also make sure to add a commit when done to get the docs in the collection before comparing the document counts On Thu, Jul 19, 2018 at 10:30 AM, THADC wrote: > Hi, > > I performed a bulk reindex against one of our larger databases for

Document Count Difference Between Solr Versions 4.7 and 7.3

2018-07-19 Thread THADC
Hi, I performed a bulk reindex against one of our larger databases for the first time using solr 7.3. The document count was substantially less (like at least 15% less) than our most recent bulk reindex from th previous solr 4.7 server. I will perform a more careful analysis, but I am assuming

Re: Solr Nodes Killed During a ReIndexing Process on New VMs Out of Memory Error

2018-07-19 Thread THADC
Thanks, made heap size considerably larger and its fine now. Thank you -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

CDCR documentation typo

2018-07-19 Thread Yair Yotam
Hi, CDCR documentation page for v 7.1: https://lucene.apache.org/solr/guide/7_1/cross-data-center-replication-cdcr.html Contains a typo in "real world" scenario section - solrconfig.xml: Target & Source should be lowercase. Using this configuration as reference will result in a generic none

Problem in QueryElevationComponent with solr 7.4.0

2018-07-19 Thread nc-tech-user
Hello. We are using solr 6.6.2 and want to upgrade it to version 7.4.0. But we have a problem with QueryElevationComponent when adding parameter "elevateIds=..." and "fl=[elevated]" Expample of query

Re: Need an advice for architecture.

2018-07-19 Thread Emir Arnautović
Hi Francois, If I got your numbers right, you are indexing on a single server and indexing rate is ~31 doc/s. I would first check if something is wrong with indexing logic. You check where the bottleneck is: do you read documents from DB fast enough, do you batch documents… Assuming you cannot

Need an advice for architecture.

2018-07-19 Thread servus01
Would like to ask what your recommendations are for a new performant Solr architecture. SQL DB 4M documents with up to 5000 metadata fields each document [2xXeon 2.1Ghz, 32GB RAM] Actual Solr: 1 Core version 4.6, 3.8M documents, schema has 300 metadata fields to import, size 3.6GB [2xXeon