Re: Indexed Data Size

2019-08-13 Thread Greg Harris
floor > Charlotte, NC 28263 > Tel: 704.988.4508 > Fax: 704.988.4907 > bmo...@tiaa.org > > -Original Message- > From: Shawn Heisey > Sent: Friday, August 9, 2019 2:25 PM > To: solr-user@lucene.apache.org > Subject: Re: Indexed Data Size > > O

RE: Indexed Data Size

2019-08-13 Thread Moyer, Brett
very 8625 Andrew Carnegie Blvd | 4th floor Charlotte, NC 28263 Tel: 704.988.4508 Fax: 704.988.4907 bmo...@tiaa.org -Original Message- From: Shawn Heisey Sent: Friday, August 9, 2019 2:25 PM To: solr-user@lucene.apache.org Subject: Re: Indexed Data Size On 8/9/2019 12:17 PM, Moyer, Brett w

Re: Indexed Data Size

2019-08-09 Thread Shawn Heisey
On 8/9/2019 12:17 PM, Moyer, Brett wrote: The biggest is /data/solr/system_logs_shard1_replica_n1/data/index, files with the extensions I stated previously. Each is 5gb and there are a few hundred. Dated by to last 3 months. I don’t understand why there are so many files with such small

RE: Indexed Data Size

2019-08-09 Thread Moyer, Brett
and there are a few hundred. Dated by to last 3 months. I don’t understand why there are so many files with such small indexes. Not sure how to clean them up. -Original Message- From: Shawn Heisey Sent: Friday, August 9, 2019 9:11 AM To: solr-user@lucene.apache.org Subject: Re: Indexed Data Size On 8/9

Re: Indexed Data Size

2019-08-09 Thread Shawn Heisey
On 8/9/2019 6:12 AM, Moyer, Brett wrote: Thanks! We update each index nightly, we don’t clear, but bring in New and Deltas, delete expired/404. All our data are basically webpages, so none are very large. Some PDFs but again not too large. We are running Solr 7.5, hopefully you can access the

RE: Indexed Data Size

2019-08-09 Thread Moyer, Brett
/lzd6hkoikhagujs/CoreOne.png?dl=0 https://www.dropbox.com/s/ae6rayb38q39u9c/CoreTwo.png?dl=0 Brett -Original Message- From: Erick Erickson Sent: Thursday, August 8, 2019 5:49 PM To: solr-user@lucene.apache.org Subject: Re: Indexed Data Size On the surface, this makes no sense at all, so there’s

Re: Indexed Data Size

2019-08-08 Thread Shawn Heisey
On 8/8/2019 3:17 PM, Moyer, Brett wrote: In our data/solr//data/index on the filesystem, we have files that go back 1 year. I don’t understand why and I doubt they are in use. Files with extensions like fdx,cfe,doc,pos,tip,dvm etc. Some of these are very large and running us out of server

Re: Indexed Data Size

2019-08-08 Thread Erick Erickson
On the surface, this makes no sense at all, so there’s something I don’t understand here ;). How often do you update your index? Having files from a long time ago is perfectly reasonable if you’re not updating regularly. But your statement that some of these are huge for just a 50K document

Indexed Data Size

2019-08-08 Thread Moyer, Brett
In our data/solr//data/index on the filesystem, we have files that go back 1 year. I don’t understand why and I doubt they are in use. Files with extensions like fdx,cfe,doc,pos,tip,dvm etc. Some of these are very large and running us out of server space. Our search indexes themselves are not