Correct our indexes are small document wise, but for some ready we have a years' worth of files in the data/solr folders. There are no index.<timestamp> files.
The biggest is /data/solr/system_logs_shard1_replica_n1/data/index, files with the extensions I stated previously. Each is 5gb and there are a few hundred. Dated by to last 3 months. I don’t understand why there are so many files with such small indexes. Not sure how to clean them up. -----Original Message----- From: Shawn Heisey <apa...@elyograg.org> Sent: Friday, August 9, 2019 9:11 AM To: solr-user@lucene.apache.org Subject: Re: Indexed Data Size On 8/9/2019 6:12 AM, Moyer, Brett wrote: > Thanks! We update each index nightly, we don’t clear, but bring in New and > Deltas, delete expired/404. All our data are basically webpages, so none are > very large. Some PDFs but again not too large. We are running Solr 7.5, > hopefully you can access the links. Solr is saying that the entire size of the index directory is 95 MB for one of those indexes and the other is 30 MB. Those sound to me like very small indexes, not very large like you indicated. You were saying that the large files were in data/index, and did not mention anything about index.<timestamp> directories. If you do have a bunch of index.<timestamp> directories in the "Data" directory mentioned on the Core overview page, you can safely delete all of the index and/or index.* directories under that directory EXCEPT the one that is indicated as the "Index" directory. If you delete that one, you're deleting the actual live index ... and since you're not on Windows, the OS will let you delete it without complaining. The directory locations are cut off on both screenshots, so I can't confirm anything there. The larger core has about 2000 deleted docs and the smaller one has 40. Doing an optimize will not save much disk space or take very long. Thanks, Shawn ************************************************************************* This e-mail may contain confidential or privileged information. If you are not the intended recipient, please notify the sender immediately and then delete it. TIAA *************************************************************************
