[
http://jira.dspace.org/jira/browse/DS-440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=11148#action_11148
]
Mark Diggory commented on DS-440:
---------------------------------
Latest commits to svn include features of above patch plus a few others...
1.) Multiple IP list files downloaded and placed in to spiders from iplist.com
2.) director to store load multiple spider files.
3.) New Data structure IPTable, to store sparse table of spider IPs.
4.) configurable preemptive filtering of bots so they never get into solr.
5.) configurable elimination of bots stored in solr from statistics views via
list of IPs or "isBot" field.
7.) new storage of User Agents and isBot fields in solr.
8.) CLI functions to mark and delete bots from solr.
> spiders.txt empty
> -----------------
>
> Key: DS-440
> URL: http://jira.dspace.org/jira/browse/DS-440
> Project: DSpace 1.x
> Issue Type: Bug
> Affects Versions: 1.6.0
> Reporter: Stuart Lewis
> Assignee: Mark Diggory
> Fix For: 1.6.0
>
> Attachments: [DS-440]_spiders_txt_is_empty.patch.txt
>
>
> spiders.txt is currently empty, so search engine robots are not being
> excluded from solr stats.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.dspace.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com
_______________________________________________
Dspace-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dspace-devel
