Re: [Bibdesk-users] All ISI WOS search seems disrupted - How about importing from Scopus?

2012-09-11 Thread Adam R. Maxwell

On Sep 11, 2012, at 03:27 , Fischlin Andreas  
wrote:

> Dear Adam,
> 
> Many thanks. All the links to the source code you gave me helps me a lot to 
> understand the situation much better. A pity this is so elaborate.
> 
> In the meantime our case with Thomson Reuters resulted in directing us to a 
> web site, of which the answer e-mail claimed, it would supersede the sites we 
> discussed previously.

I did a few quick tests with Python and suds today to see how hard this would 
be.  

First, using the old WSDL gives an error saying that the service is no longer 
supported; this confirms what we suspected.  Not sure why BibDesk isn't showing 
that error, but we probably need to manually check for a fault.

Second, I'm not sanguine about getting this to work with Apple's framework, as 
TR has started using complex objects instead of simple (albeit messy) string 
parameters.  Explicit authentication is also required, and you have to pass an 
HTTP cookie back with each search/retrieve request.  I've not dealt with 
serializing and deserializing objects in Apple's Web Services framework.

The easiest option at this point is shipping suds inside BibDesk, then writing 
a command-line tool that would do the Web Service stuff and echo back XML.  
Unfortunately, it's LGPL v3 and I'm not sure that's compatible with BibDesk's 
BSD license.  It's also 500 kbytes of files, though that's not a deal breaker.  
Not sure that other devs think of it.  Using gSOAP is also an option, but not 
as easy.

> However, would it not be easier not to work with Mountain Lion first, in 
> particular to overcome the caveats of the new protocol? Say, using the 
> reliable Snow Leopard (or Lion, but I wouldn't use the word reliable in this 
> case ;-) ). Once understood it might work even under Mountain Lion?

OS version is irrelevant.  Apple's Web Service code likely won't work any 
differently under 10.8.  Code that is deprecated isn't necessarily going away 
right now (though the jerks at Apple broke DataTank by removing 
XgridFoundation.framework in 10.8, after deprecating it in 10.7).

> One further thought: Such SOAP like protocols 
> (http://en.wikipedia.org/wiki/SOAP_(protocol)) may be of interest also in 
> other cases.

BibDesk's DBLP search group also uses it.

> Would it not be more robust for BibDesk to develop a generic module that can 
> handle such queries while the syntax specifics of any given protocol are 
> described with a meta language, e.g. similar as attributed EBNF 
> (http://en.wikipedia.org/wiki/EBNF) this can do for programing languages. 
> Then BibDesk could be extended by any query by simply providing the protocol 
> description in form of that language?

You want a meta language on top of SOAP?  Forget it.  I'm not an expert, but 
IMO there's too much variety in Web Service interfaces to do something like 
that.  Generic isn't always better, and neither are layers of abstraction 
against a time when something might possibly be added (though I've perpetrated 
some of those).

Now, a workable solution might be to define an API for writing search group 
plugins in Python, much as we did with script groups (yes, I'm heavily biased 
towards Python).  In the case of Web Service clients, it would be easier for a 
3rd party to add/maintain/modify a search group.  When we talked about this 
several years ago, I think we concluded that it would be cool, but a waste of 
time because it's not likely that 3rd parties will go to that effort.

> What is your experience with these interfaces? Are they clean? Formally well 
> designed or are they all just tedious, cumbersome patchwork full of 
> idiosyncrasies?

Web Services interfaces are all over the map; some are good, some are bad, and 
others are just weird.  None of the services I've used has performed exactly as 
documented, but that's a fairly small sample size.

Adam


--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Bibdesk-users mailing list
Bibdesk-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bibdesk-users


Re: [Bibdesk-users] All ISI WOS search seems disrupted - How about importing from Scopus?

2012-09-11 Thread Fischlin Andreas
Dear Adam,

Many thanks. All the links to the source code you gave me helps me a lot to 
understand the situation much better. A pity this is so elaborate.

In the meantime our case with Thomson Reuters resulted in directing us to a web 
site, of which the answer e-mail claimed, it would supersede the sites we 
discussed previously. Here is that URL, since it may also be of interest:

http://wokinfo.com/products_tools/products/related/webservices/

Despite all being very helpful what you wrote, it is certainly not good news. 
And I share several of your concerns only too easily. I am willing to help. My 
first step along this lines is currently to continue insisting that the company 
really helps us. My institution is paying AFAIK huge amounts of subscription 
fees, yet nobody seems to have informed us of the consequences of the 
transition as end of August nor do they really help at the moment to overcome 
the difficulties. The information they have provided so far are all very 
conflicting and no real progress can be seen. Obviously as an end-user one has 
to insist they help, given the way they have handled us during this recent 
transition, or we will never get anywhere. The case only exists, because I 
complained insistingly, which finally did trigger this case and so far a few 
reactions.

However, would it not be easier not to work with Mountain Lion first, in 
particular to overcome the caveats of the new protocol? Say, using the reliable 
Snow Leopard (or Lion, but I wouldn't use the word reliable in this case ;-) ). 
Once understood it might work even under Mountain Lion?

One further thought: Such SOAP like protocols 
(http://en.wikipedia.org/wiki/SOAP_(protocol)) may be of interest also in other 
cases. Would it not be more robust for BibDesk to develop a generic module that 
can handle such queries while the syntax specifics of any given protocol are 
described with a meta language, e.g. similar as attributed EBNF 
(http://en.wikipedia.org/wiki/EBNF) this can do for programing languages. Then 
BibDesk could be extended by any query by simply providing the protocol 
description in form of that language? What is your experience with these 
interfaces? Are they clean? Formally well designed or are they all just 
tedious, cumbersome patchwork full of idiosyncrasies?

Regards,
Andreas





On 11/09/2012, at 01:17 , Adam R. Maxwell wrote:


On Sep 10, 2012, at 14:48 , Fischlin Andreas 
mailto:andreas.fisch...@env.ethz.ch>> wrote:

Dear Adam,

Many thanks for this most valuable information. Indeed, in the meantime I got a 
confirmation from company Thomson Reuters that they have switched from version 
1.0 to 2.0 of their web services, including the SOAP access as of 31st August 
2012.

Well, that's a pretty clear indication of what has happened.

Unfortunately I need very urgently ISI WOS in BibDesk back.

Any idea what could be done?

New WS stub code [1] has to be generated with WSMakeStubs, it has to be checked 
for bugs, and then glue code has to be written to integrate the new service 
into BibDesk [2].  This last is the only significant task, and based on my 
cursory glance last week, it'll be a fairly extensive rewrite.  
BDSKISIWebServices.[hm] will be replace entirely, and BDSKISIGroupServer.m will 
have to be rewritten to use the new methods and error checking.

[1] 
http://bibdesk.svn.sourceforge.net/viewvc/bibdesk/trunk/bibdesk/BDSKISIWebServices.m?revision=18776&view=markup

[2] 
http://bibdesk.svn.sourceforge.net/viewvc/bibdesk/trunk/bibdesk/BDSKISIGroupServer.m?revision=18776&view=markup

I'd guess 4-8 hours work, assuming the new API is straightforward and not too 
different from the old one, but I have very low confidence in those 
assumptions, and I have no time to look at this myself before October.

Notably I have for sure following questions for the time being:

1) Where are search details as contained in a .bdsksearch file actually stored 
in BibDesk once such a search has been installed?
I have consulted the help pages, and examples of .bdsksearch files are also not 
that difficult to construct from scratch. But this all did not answer me this 
question. Have I overlooked something or is it really missing.

Those files are largely irrelevant for WoS searches; they just store a database 
name and human-readable counterpart.  Local additions should end up in 
~/Library/Application Support/BibDesk/SearchGroupServers.

2) What URL is "Searches -> Web of Science SCI" using?
In the current case I opened with Thomson Reuters, they are asking me this 
question, but I do not know what to answer.

The default endpoint is

http://wok-ws.isiknowledge.com/esti/soap/SearchRetrieve

See link [1] for details.

3) Can I edit or at least see Web of Science SCI search specs at the XML level?
Is there a way to edit a search once its .bdsksearch file has been installed? I 
looked for plist files, but neither inside the application BibDesk nor in 
~/Library/Application Support could I find the wante