Way back when, in the harvester guidelines, we suggested using both the
User-Agent (along the lines proposed but without URI suggestion) and the
From (for email contact) headers:
http://www.openarchives.org/OAI/2.0/guidelines-harvester.htm#AgentInfo
So, a long winded +1 to a URI being a useful thing to put in the
User-Agent string.
Cheers,
Simeon
On 11/17/14, 3:50 AM, Stuart A. Yeates wrote:
I've been looking at the logs for our OAI server and I'd like to appeal to
those harvesting over OAI to put URLs into the user agent string. Putting
the name of your project into the user agent string seems like a great way
to build profile. It also avoids the situation where the easiest way to
contact you is via the contacts associated with your DNS block.
For reference, these are some of the user agent strings I'm seeing
(standard browser strings removed):
"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
"Net::OAI::Harvester"
"Googlebot/2.1 (+http://www.google.com/bot.html)"
"OAIHarvester/2.0"
"Jakarta Commons-HttpClient/3.1"
"Mozilla/5.0 (compatible; Baiduspider/2.0; +
http://www.baidu.com/search/spider.html)"
"Celestial/3.02"
"WorldCat Digital Collection Gateway from OCLC.org"
"Apache-HttpClient/4.0.1 (java 1.5)"
"lwp-trivial/1.41"
"OAIGet-1.12"
"DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; Googlebot-Mobile/2.1; +
http://www.google.com/bot.html)"
"Mozilla/5.0 (compatible; Sosospider/2.0; +
http://help.soso.com/webspider.htm)"
"yacybot (freeworld/global; amd64 Linux 3.2.0-36-generic; java 1.6.0_27;
"OAIHarvesterObj 31 University of Illinois Library"
"PKPHarvester/2.x"
"OAI Harvester/1.0; FS Consulting, Inc."
"Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
"Typhoeus - https://github.com/typhoeus/typhoeus";
....
cheers
stuart
