Re: [Dspace-tech] Updating stats with new spider IP addresses

2013-01-03 Thread Andrea Schweer
Hi Steve, On 18/12/12 19:15, Steve Swinsburg wrote: We have identified a number of new spider IP addresses from Google and other indexers being responsible for vastly inflating our stats. I've created a local spider filter list with the IP addresses and I am running the stats updater: dspace

Re: [Dspace-tech] Updating stats with new spider IP addresses

2012-12-20 Thread Steve Swinsburg
Thanks Bram, I've added a note to the Jira about our additional spider list. We are rolling out 1.8.2 early in 2013 so I'll just hold off until then. regards, Steve On 20/12/2012, at 6:14 PM, Bram Luyten b...@mire.bemailto:b...@mire.be wrote: Hi Steve, already apologies in advance for the

Re: [Dspace-tech] Updating stats with new spider IP addresses

2012-12-19 Thread Steve Swinsburg
Does anyone ever update their solr stats? Does anyone know about the performance issue I am seeing here? thanks, Steve On 18/12/2012, at 5:15 PM, Steve Swinsburg steve.swinsb...@anu.edu.aumailto:steve.swinsb...@anu.edu.au wrote: Hi all, We have identified a number of new spider IP addresses

Re: [Dspace-tech] Updating stats with new spider IP addresses

2012-12-19 Thread Ian Boston
Hi, I was having a problem recently with stats in ds3, caused by excessive SQL queries building parent collections. There was a patch shared on list about a week ago by Andrea. It might help ? Ian On Thursday, December 20, 2012, Steve Swinsburg wrote: Does anyone ever update their solr

Re: [Dspace-tech] Updating stats with new spider IP addresses

2012-12-19 Thread Bram Luyten
Hi Steve, already apologies in advance for the vagueness of this answer but there have been several performance related optimizations to the stats between 1.6.2 and 3.0. The latest one, SOLR sharding by year, was added in 3.0. This is especially useful for those institutions who have accumulated

[Dspace-tech] Updating stats with new spider IP addresses

2012-12-17 Thread Steve Swinsburg
Hi all, We have identified a number of new spider IP addresses from Google and other indexers being responsible for vastly inflating our stats. I've created a local spider filter list with the IP addresses and I am running the stats updater: dspace stats-util -m to reprocess the stats and mark