Hi Steve,
already apologies in advance for the vagueness of this answer but there
have been several performance related optimizations to the stats between
1.6.2 and 3.0.
The latest one, SOLR sharding by year, was added in 3.0. This is especially
useful for those institutions who have accumulated multiple years of SOLR
stats.
https://wiki.duraspace.org/display/DSDOC3x/Managing+Usage+Statistics#ManagingUsageStatistics-Solr
ShardingByYear<https://wiki.duraspace.org/display/DSDOC3x/Managing+Usage+Statistics#ManagingUsageStatistics-Solr
ShardingByYear>
Enabling auto commit was also something not included in the DSpace 1.6
version of the stats by default:
https://atmire.com/website/?q=content/increasing-dspace-performance
Hope this helps! 200,000 usage events is indeed not a huge number so there
should be a way to optimize.
Can I also ask you to share your findings about the spiders here?
https://jira.duraspace.org/browse/DS-790
best regards,
Bram
--
[image: logo]
*Bram Luyten* *@mire*
*2888 Loker Avenue East, Suite 315, Carlsbad, CA. 92010*
*Esperantolaan 4, Heverlee 3001, Belgium*
<http://www.atmire.com/>www.atmire.com<http://atmire.com/website/?q=services&utm_source=emailfooter&utm_medium=email&utm_campaign=braml>
On Thu, Dec 20, 2012 at 3:36 AM, Ian Boston <ib...@cam.ac.uk> wrote:
> Hi,
>
> I was having a problem recently with stats in ds3, caused by excessive SQL
> queries building parent collections. There was a patch shared on list about
> a week ago by Andrea. It might help ?
>
> Ian
>
>
> On Thursday, December 20, 2012, Steve Swinsburg wrote:
>
>> Does anyone ever update their solr stats? Does anyone know about the
>> performance issue I am seeing here?
>>
>> thanks,
>> Steve
>>
>>
>> On 18/12/2012, at 5:15 PM, Steve Swinsburg <steve.swinsb...@anu.edu.au>
>> wrote:
>>
>> Hi all,
>>
>> We have identified a number of new spider IP addresses from Google and
>> other indexers being responsible for vastly inflating our stats. I've
>> created a local spider filter list with the IP addresses and I am running
>> the stats updater:
>> dspace stats-util -m
>>
>> to reprocess the stats and mark them appropriately, then will remove
>> them via:
>> dspace stats-util -f
>>
>> However the mark is taking hours. Likewise if I go ahead and just
>> delete them based on the new rules, via:
>> dspace stats-util -i
>>
>> Is that normal? We only have about 200,000 views to process.
>>
>> Version 1.6.2 but about to rollout an upgrade. If the performance has
>> improved in 1.8.2 we can wait a week or so.
>>
>>
>> regards,
>> Steve
>>
>>
>> ------------------------------------------------------------------------------
>> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
>> Remotely access PCs and mobile devices and provide instant support
>> Improve your efficiency, and focus on delivering more value-add services
>> Discover what IT Professionals Know. Rescue delivers
>>
>> http://p.sf.net/sfu/logmein_12329d2d_______________________________________________
>> DSpace-tech mailing list
>> DSpace-tech@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>> List Etiquette:
>> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
>>
>>
>>
>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
> List Etiquette:
> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
>
------------------------------------------------------------------------------
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
_______________________________________________
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
