Re: [pu jsonalchemy] Aggregation of several fields into now
On 27.03.2014 15:38, Esteban Gabancho wrote: Hi! I think the second solution is the closest one to reality, the `None` express that the record doesn’t have a first author. And I also think that we could apply this solution for other cases where we have this kind of situation (like with the `110__` and `710__`). What do you think? If I may: as a librarian you have a 100. You may not have a 700, but in case you have only one author it is 100 by definition. You always have to read Marc with the cataloging rulebook in mind. (AACR*, RAK, what have you.) Still one should handle a cataloguing error. -- Kind regards, Alexander Wagner Scientific Services / Scientific Publishing Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 http://www.fz-juelich.de/zb/wp Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: Invenio Developer Forum - Ticketing and Merging Tools - today at 16:45 CET
On 13.03.2014 11:43, Tibor Simko wrote: Hi! You mean Application Program Interface? Yes. Basically I was wondering what your primary use case for Z39.50 would be, to see whether it may be an option to use some other alternative ways of accessing the data. Invenio offers other APIs, but not Z39.50 unfortunately. One common use case for Z39.50 (execpt if you /are/ the union catalogue) is hooking up with client side literature management tools like EndNote, ReferenceManager, Citavi and friends. They all know Z39.50 to ingest data from library catalogues. Especially in humanities and social sciences, where you've a lot of book and/or gray literature, it's quite common to hook them up with your catalogue. People then use Z39.50 linkups to search for literature instead of using the real database and donwload the links, which is the far more common procedure in STM. Sad thing is, that they (AFAIK!) /only/ know Z39.50 and not more modern protocols like SRU. :S jm2ct/ -- Kind regards, Alexander Wagner Scientific Services / Scientific Publishing Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 http://www.fz-juelich.de/zb/wp Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: Search Bracketing
On 27.02.2014 14:58, Wagner, Alexander wrote: Hi! About bracket reduction: this is what I added to my code. It is not that simple as I generate them from othet ops. I agree, that it would be a good idea if the query parser removes unnecessary brackets. Would avoid trouble. I'll try to build a sample against the demo site. It seems to be fixed on the demo site, at least my trivial use cases work there as expected. So, we'll see if all is well and gone after an eventual upgrade. Still it might be a good idea to add some simplify() to the query parser. After thinking about it: probably interactive input will not trigger this sort of error easily. It's more if you have something like query calculations as in our backend use case, where I calculate the effective query from intbitset-intersections/differences/combinations, that one might end up with superflous brackets. - Reply message - From: Tibor Simko tibor.si...@cern.ch Date: Thu, Feb 27, 2014 14:28 Subject: Search Bracketing To: Wagner, Alexander a.wag...@fz-juelich.de Cc: project-invenio-devel@cern.ch project-invenio-devel@cern.ch On Thu, 27 Feb 2014, Wagner, Alexander wrote: The ind:val-example I gave, where you noticed it is perfrctly ok, is actually broken on JuSER. Would you have an example that fails on our demo site? (ind:val1 and ind:val2) and ((ind:val3 or ind:val4) or ind:val5) If you use this query literally, then please note that several pairs of parentheses are not needed here, as they are chaining within the same AND or OR sequence. You can reduce the above to: ind:val1 and ind:val2 and (ind:val3 or ind:val4 or ind:val5) The logic being: (A and B) and ((C or D) or E) == A and B and (C or D or E) It may be helpful to the query parser to always reduce the number of incoming parentheses. Best regards -- Tibor Simko -- Kind regards, Alexander Wagner Scientific Services / Scientific Publishing Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 http://www.fz-juelich.de/zb/wp Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Search Bracketing
Hi! The only ticket I found on the issue is the quite old http://invenio-software.org/ticket/131 which seems to be closed ages ago. However, there persists an issue with bracketing. I just checked on CDS: 041__a:eng vs. (041__a:eng) The first one yields (without need for exact numbers ;) 1,093,692 records, the latter 776, so there's a difference. I think there shouldn't be one. Semantically both queries are the same, right? Just the second one has a superfluous bracket. In the above case it is an inconvenience, and additionally something which will be barely noticed as nobody uses the second form. However, I just came across this while calculating queries from intbitset-operations. There I get structures of the form (ind:val1 and ind:val2) and ((ind:val3 or ind:val4) or ind:val5) At the moment I simplify this before output by investing some knowledge about our queries, but it is definitely a hack and will not work in general. I fear there's still a bug in the in bracket handling. -- Kind regards, Alexander Wagner Scientific Services / Scientific Publishing Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 http://www.fz-juelich.de/zb/wp Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: RFC unifying phrase search behaviour
On 24.02.2014 11:30, Tibor Simko wrote: Hi! People don't easily distinguish between the following queries: title:'some phrase' substring title:some phrase exact search [...] 245:'some phrase' 245:some phrase so that single-quoted and double-quoted phrase queries would always return the same result. Which is then an exact match, right? So to get '' matches one would use *bla*, right? What this change means for you: 1. The end users can use single-quoted or double-quoted queries to express phrase search, in all indexes. There would be no difference. 2. The phrase search would be done by default via word pair matching, unless indexes are tokenised in a special manner (e.g. exact author name) or unless users search inside physical MARC tags (when no word pair index exists). 3. If you have relied on partial phrase matching, please switch to regular expression queries like: 245:/some phrase/ 245:/[[:blank:]]some phrase[[:blank:]]/ This should be 245:*some phrase*... 4. If you have relied on exact phrase matching, please switch to regular expression queries like: 245:/^Exact title.$/ This should be 245:Exact title. Sorry, if I ask here. If I get this correctly, every /exact/ search, in old world bla (no substring) would be a regular expression now, in this new scheme, right? This would IMHO /not/ be sensible at all. First of all if I place bla explicitly in quotes I /expect/ it to be an exact match and not a substring, so it is contraintuitive. See G (and friends): the only way to switch off their intelligence is to put things explicitly in quotes. Secondly, it would mean that all our ID searches which are ID:(src)Number-type things end up in really /expensive/ regexp searches. I.e. we would regexp something like this: http://juser.fz-juelich.de/search?p=%28collection%3A%22VDB%22+and+web%3A%222013%22%29+and+%28id%3A%22WOS%3A000%2A%22+or+sid%3A%22StatID%3A%28DE-HGF%290100%22+or+sid%3A%22StatID%3A%28DE-HGF%290110%22+or+sid%3A%22StatID%3A%28DE-HGF%290111%22+or+sid%3A%22StatID%3A%28DE-HGF%290120%22+or+sid%3A%22StatID%3A%28DE-HGF%290130%22%29+and+pof%3A%22G%3A%28DE-HGF%29POF2-110%22 Please holler if this change could badly break some of your workflows. If I get it correctly, it breaks almost all our bean counting. IDs are something like sid:(DE-HGF)1 or sid:(DE-HGF)11 if you map sid:(DE-HGF)1 to the old 'sid:(DE-HGF)1' it matches also sid:(DE-HGF)11, which is wrong and not intended. So, if one wants to unify quotes (I agree that distinguishing '' vs is difficult to explain and especially needs explanation) then one should unify it to use quotes always for exact matches. One could then have substring search by either leaving out the quotes or putting explicit * oprators like *bla* ie. let the '' behave like the in invenio logic but not to have call sub string matches. This is also something I can explain to the Normal User(tm), while I think your regexps above are a bit beyond their common language ;) -- Kind regards, Alexander Wagner Scientific Services / Scientific Publishing Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 http://www.fz-juelich.de/zb/wp Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: authority indexes to gather data from authority records only
On 03.02.2014 12:27, Theodoros Theodoropoulos wrote: Hi! Hello everyone and apologies if this has been discussed (or even tackled). The issue I'm having is that authority-related indexes (ie authority author) are getting too large because some of the 'authority' fields that should be indexed coincide with some popular bibliographic fields (such as 500__a which is see also/personal name in authority records and note field in bibliographic records) and thus the relative index fills up with 'garbage'. Sortof in our discussions with Tibor while ago. Tibor might remember his musings whether the auth-index should be completely separated or not With that in mind, I think it would be nice to be able to restrict indexing to certain collections only. If this seems like a bad policy (and maybe it is), a check button could instead exist in the bibindexadmin page that could be used to indicate an authority-related index (and thus restrict the data gathered to authority-related records only). Does this seem reasonable? For me it definitely sounds sensible. In a way I think even the tag already exists as Authority Records should belong to the collection AUTHORITY (980__a). BTW: we used 400 instead of 500 for synonyms. Did we do this one wrong? -- Kind regards, Alexander Wagner Scientific Services / Scientific Publishing Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 http://www.fz-juelich.de/zb/wp Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: authority indexes to gather data from authority records only
On 03.02.2014 13:36, Theodoros Theodoropoulos wrote: Hello Theodoros! BTW: we used 400 instead of 500 for synonyms. Did we do this one wrong? I don't think you did anything wrong there. 400 is 'see' and 500 is 'see also' for authorities. For alternative names 400 seems more proper, although both can be used without problem. (I think that 400 is mainly used if you want to directly point to another authority that deals with the same person -ie a name variant-, where 500 is used when you want to point to a relative or a more generic term/name). Anyway, I still have 500 because this is the default value for the 'demo' site :) Ah. It could be that when Chris did this stuff there were some peding issues in discussion or just a typo as well. I think this is where demo records come from. Additionally, in the discussion with Tibor/Chris back then we droped indicators from the demo records as usually CDS is ignorant about them. However, we use them a bit (if 999 fields would not be enough one could easily go for 9, right, and even allow for alphabethic indicators in case of lacking space ;) So the marcup we use at JuSER and friends is actually 1001_ for people using Last, First notation and actually telling this in the marcup via the indicators. This goes with 5001_ (and 4001_ in case). Still you're right, the idea that Marc Authority uses the same tags/indicators as Bibliographic and that there is acutally (at least per see) no difference between the type of records is not so nice. The 980__aAUTHORITY notion is IMHO one way to go. -- Kind regards, Alexander Wagner Scientific Services / Scientific Publishing Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 http://www.fz-juelich.de/zb/wp Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: chatroom future?
On 06.01.2014 17:56, Tibor Simko wrote:
Hi!
Heck yeah!
OK, we now have experimental #invenio channel on freenode.
Ok, as it took me a while to fiddle it out how to connect to IRC, here
is a short description/howto.
Problem: IRC is blocked in Jülich due to some regulations, so I can not
use a regular IRC client as all connections get silently dropped.
Solution: Use Jabber + an IRC transport.
Actually, I admit that, though it didn't really take off as it should,
transports are a really cool feature of Jabber, allowing to connect to
whatever network from within a single chat client.
Now step by step for Pidgin (available for *nix, Win, Mac, whathaveyou
On Mac it's called Addium)
1. Tools / Plugins
- Enable XMPP Service Discovery
By default this is off, but it required to register with a transport.
2. Find a suitable transport. This one does NOT need to live on your own
Jabber server, it could be any other. Discover showed me, that CERNs
Jabber Server offers an IRC transport, but I did not get this one in
cooperative mode. Therefore, I checked the services of
jabber.freenet.de and registered with IRC transport there.
Parameters: irc-name, well a string ;) and some funny connection parameters:
[{irc.freenode.net,utf-8}].
Note the dot at the end.
3. Add a group chat with the following parameters:
Room : invenio%irc.freenode.net
Server: irc.jabber.freenet.de
Handle: your_irc_name
Note not to add a hash (#) in front of the channel. The transport seems
to do this service for you.
Opening the group chat (takes a second)... tadaa
Other clients should work similarly. Also note: in principle, Pidgin
could also act as a native irc client. But this doesn't solve the
firewall issue like the transport does.
--
Kind regards,
Alexander Wagner
Scientific Services / Scientific Publishing
Central Library
52425 Juelich
mail : a.wag...@fz-juelich.de
phone: +49 2461 61-1586
Fax : +49 2461 61-6103
http://www.fz-juelich.de/zb/wp
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt
Re: Invenio i18n in crowdin?
On 19.12.2013 11:11, Ferran Jorba wrote: Hi! On Wed, 18 Dec 2013, th...@physics.auth.gr wrote: Crowdin seems nice with lots of features. For 'open' opensource projects (and also for academic/research(?) institutions) its completely free. [...] Personally I prefer Emacs's po-mode. Offline, ultra fast, auto-completion, one-key access to the phrase context, language syntax checking, etc. I also appreciate Emacs po-mode, among those reasons Tibor observes, to switch between the for or five languages I use: I translate into two, but, after the English original, I often read the French and Italian translations to look for inspiration or alternatives, just pressing the 'a' key. Match that! Just two cents from a recent discussion I had this morning. Depending on who is doing translations unfortunately emacs is a bit out of the question, though I personally understand that once you got the keys hard wired into your fingertips it's surely as convenient as I'm with vim. However, if it comes to users manuals, or in general the further away from the API docs you get, the more you'd like non-techies to write the docs... Anyway, one of the main problems a professional translator mentioned to me and one should keep in mind is, that not the word by word translation is an issue or how this is done. The main issue in a larger project is to keep things consistent across the project. Ie. always use the same wording and concepts. If just one person does this it is difficult, but may be possible. Though I know from another project where I wrote a whole manual once (rewriting/updating the English version and then translating the whole beast to German; I'm not sure that I'll ever do this kind of work again ;) it is really difficult to stay consistent even in this setting. It's easier if you do in in one chunk, but if you have to interrupt your work and take it up some weeks later... Or if you're working in unavoidable updates. So, I'm not sure that I managed it that well. Having several translators working on the same language, but most likely in completely different areas of the whole thing, this surely doesn't get easier. Though our translators use different tools, and due to their work a public storage is out of the question, they told me that their toolchain is actually somewhat similar to the crowdin Theodoros mentioned. They stressed the point that this kind of software keeps suggesting the /same/ translations for the same words/concepts throughout the project and that this is a huge help when it comes to a consistent translation, which they feel is the main challenge. I don't want to vote for something specific here, but just to mention this point as I admit that I see the problem of consistency as well. Up to the point that I always vote for dropping i18n entirely in favour of (good) English only and be done. Yes, I know that this is most likely not the favourable solution for the majority. ;) -- Kind regards, Alexander Wagner Scientific Services / Scientific Publishing Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 http://www.fz-juelich.de/zb/wp Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: Invenio Developer Forum - JavaScript toolkit frameworks - today at 16:45 CET
On 11.11.2013 09:49, Tibor Simko wrote: Hi! As usual 16:45 is not possible for me. Anyway some small input from HGF... In http://invenio-software.org/wiki/Talk/UIFrameworks you mention about jQueryUI that Invenio doesn't use it too much but you seem (to my reading) generally quite open to drop it. I may say, that cause Invenio started using jQueryUI we at HGF adopted jQuery for all our local additions. Mainly tokeninput in an debugged and adopted version. As every unwritten line of code (especially JS) is a good line of code we tried to minimize it. However, for a rich web GUI, unfortunately... Anyway, our project (and myself as well ;) would be quite happy if we'd not need to redo everything again to adopt a common look and feel. Probably there's some option available in your musings. -- Kind regards, Alexander Wagner Scientific Services / Scientific Publishing Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 http://www.fz-juelich.de/zb/wp Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: chatroom future?
On 29.10.2013 03:51, Tibor Simko wrote: Hi! in my case I simply find the current Hangout experience flawless. It is a good experience indeed, So, where do you hang out and I'll join in. I'll have to leave this other browser open, but well. politics However, personally, I strongly prefer /open/ in all regards. Open Access, Open Source, Open Protocols... Openness is the foundation of the Net. I don't like to throw that away and I do not understand why people throw our Net at G, M$, F and the like. I do not see a point in giving up the free Internet to company control just cause I'd have to read the manual to keep it free. So I really don't like nor use unsocial networks at all. /politics however there is always room for improvement, for instance in the client configurability department. Well its passed to G, M$, F whoever. That where it ends. It's not in your control anymore. Take what they give you and pay for it. For me, Jabber integrates better Agree. Though not on emacs as you ;) in my Emacs oriented workflow. See an incoming IM, press a key, answer message, press the key again, and voilà, back in the original work buffer. The Hangout client requires a bit more key presses and/or mouse movements... Agree. In any case its a matter of taste. We tend to use hangouts for video conferencing however I'd prefer something more open there as well. (For what it's worth German DFNs configs were not understood even by our geeks. So this has clearly some room for improvement. Evo went commercial and doesn't really like guests anymore, so...) The main point for me in a chat is to have a low footprint and fast way to communicate. Our experience at the hgf-project with Jabber is pretty good. Lengthy things to the list, short stuff in chat. (BTW: Sam, I can't imagine that a geek like you has trouble setting up a Jabber client ;) Anyway, I miss you in the chat room. Was always very helpful and fast.) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: chatroom future?
On 29.10.2013 09:45, Samuele Kaplun wrote: Hi! politics However, personally, I strongly prefer /open/ in all regards. Open [...] networks at all. /politics Well, sometimes one need to balance extreme opinions with pragmatism :-) Well, however I feel giving up the free net is a bit to less pragmatic ;) Nothing prevents for an open client to become better and having a smaller barrier than Hangout (OK I guess Hangout being so integrated - a la IE6 - is sort of a barrier in this sense, It is (IMHO) a bit careless to move infrastructure to closed protocols. Could you imagine that G just closes GMail? Actually, why not? Their share feature integrates so much better and circles are so much easier than an address book. Plug chrome on top of that and ... but in the end you have seen that even if Firefox/Mozilla was not coming out-of-the-box on Win machine it won the browser battle, simply because it was better). Currently, it seems to go to chrome it integrates so much better... In this very moment there are plenty of features that makes Hangout better (of course these are just opinions), and therefore in the evolutionary schema it has its advantages. I raise the question of better compared to they set it up for me I don't care. (BTW: Sam, I can't imagine that a geek like you has trouble setting up a Jabber client ;) Anyway, I miss you in the chat room. Was always very helpful and fast.) Well my geeky religion is with KDE Ok, I admit I never really liked that one and ran on WindowMaker for the better part of the last years. and there, the Jabber integration was horrible :-( Hm. Ic. I don't need your level of integration. OTH Psi isn't so bad a client. and there it goes (and that explains also why at some point, after certain upgrades, I was no longer in the chatroom). :( Besides, some years ago I was daily recompiling my OS (with Gentoo) :-) Ok, I admit that I never reached that stage of geekiness. I tried it once and aborted that silly game after 3 days make world. ;) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: chatroom future?
On 28.10.2013 20:41, Samuele Kaplun wrote: Hi! [...] in my case I simply find the current Hangout experience flawless. Hm, not my world ;) I simply have chat opening in a gentle way, it's integrated immediately with smartphone and tablet, I use xabber on Android. (e.g. in KDE). It's sad Google dropped support for XMPP, When did they do that? Or when are they supposed to do that? -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
ORCID in Invenio
Hi! In http://invenio-software.org/ticket/1379 ORCID is specified for integration in Author profiles / Websearch. Unfortunatley, the ticket is a bit short in words ;) Anyway, what are the plans/current status of Invenio wrt ORCiD? I'd not be so much interested in OAuth-Like logins or the like but more in data exchange of Invenio - ORCiD. Say: autoregistration to ORCiD as soon as a paper is submitted, and authors don't have an ORCiD or we don't know it yet. (In our workflow, finally approved by the library). Pushing bibliographic information from Invenio - ORCiD, probably get publications from an ORCiD record into Invenio, stuff like that. So something like the member-api of http://support.orcid.org/knowledgebase/articles/116874-orcid-api-guide Anyone working on that and/or interested in joining forces? -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Das Forschungszentrum oeffnet seine Tueren am Sonntag, 29. September, von 10:00 bis 17:00 Uhr: http://www.tagderneugier.de
Re: ORCID in Invenio
On 09.08.2013 09:30, Laura Rueda wrote: Hello Laura! Indeed, the ticket does not offer much information :) I can tell you that we are working on the ORCID integration under the EU Project ODIN (http://odin-project.eu/) and Lars will be doing something similar for Zenodo. Already imagined something like that :) We will cover the OAuth login (it can be important to have access to the 'limited' publications) and both push and pull. In fact, we are also working with ORCID for a quick push/pull, based on DOIs. That sounds very promising. How much do you already have here? On what invenio version are you implementing it? I guess it is next/ in case, could this be (easily) backportable to the 1.x series? What is not in our current use cases is the autoregistration on submission. But I understand it will be as simple as to plug the OAuth step. This sounds promising. It could be that we could come up with this plugin, given we know the framework around. I will be happy to keep you updated! I'd be very interested, indeed. ORCiD integration is quite a hot topic here and I'd definitely not want to redo what's already done. We should probably check out how to team up e.g. if registration of ORCiDs is not that much of interest on your end we could work on that as we definitely would need that. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Das Forschungszentrum oeffnet seine Tueren am Sonntag, 29. September, von 10:00 bis 17:00 Uhr: http://www.tagderneugier.de
Re: ORCID in Invenio
On 09.08.2013 10:40, Laura Rueda wrote: Hi! That sounds very promising. How much do you already have here? On what invenio version are you implementing it? I guess it is next/ in case, could this be (easily) backportable to the 1.x series? We are working on 1.x as well, but our developments are a bit tailored to the peculiarities of the INSPIRE installation. If the tayloring is not to much hardcoded I think we could well live with our friends from inspire. We could check out if/where/how we need adoptions/configs. I'd be very interested, indeed. ORCiD integration is quite a hot topic here and I'd definitely not want to redo what's already done. We should probably check out how to team up e.g. if registration of ORCiDs is not that much of interest on your end we could work on that as we definitely would need that. Sounds excellent! I will send you more information so we can see how to get the best from the effort. So I get it we'll go that way :) At the moment we have some smaller issues to sort out first, but ORCiD is then one of our very next topics. Thanks! De rien :) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Das Forschungszentrum oeffnet seine Tueren am Sonntag, 29. September, von 10:00 bis 17:00 Uhr: http://www.tagderneugier.de
Re: Baskets Templates
On 12.07.2013 17:29, Tibor Simko wrote: Hi! Yes we could add it in that structure as we discussed at CERN. One point, however: we should (here at HGF but also with you) discuss a bit how to do it decently cause it is IMHO not helfpul if I clutter our docs amongst ot many wikis. Yes, indeed. You may perhaps want to maintain extensive documentation on your own wiki (in German?) and to create just a few pages on Project/HGF that would describe your workflow in 10-20 lines just to express the basket/message/workflow/holding pen idea briefly for other Invenio sites so that they could discover more easily your specifics. This sounds of course reasonable. Probably using some links to the more in depth parts. However, I was also thinking about the wiki as such. Point is: when this guy at CERN proposed a system for information management, he proposed HTML to have only /one/ language for it. This wiki-things, unfortunately, weren't really designed like this. It is a mess with just to many definitions and dialects, so you can't even just copypaste easily. (A point, why I don't really like wikis...) Or to make it a bit more explicit: our dev-wiki is currently mediawiki-based. The users manual/docs/online help is twiki, invenio-software.org is trac. So if some redmine or moinmoin comes along we start translating from one dialect to the other and have all major players on board. Nothing I'd like to opt for ;) Anyway, I'll think about some text in this direction. Probably, in preparation of IUGM/2. You remembered out discussion about a ticketing system, resuing TRAC and so on. Did you, by any chance, know already how TRAC behaves with multiple instances in one host? Unfortunately no advancements on this front... Ok, please drop me a note if there are any. TIA :) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Das Forschungszentrum oeffnet seine Tueren am Sonntag, 29. September, von 10:00 bis 17:00 Uhr: http://www.tagderneugier.de
Re: Baskets Templates
On 11.07.2013 21:34, Nikos Kasioumis wrote: Hi! [...] As soon as some department here stops to be creative with new stat-requests and our gang is on board again, this is the very next thing we're about to do. In the meantime, you might also want to have a look at the MySQL group_concat_max_len [...] function page. GROUP_CONCAT's result is in fact truncated to group_concat_max_len's value, which by default is 1024 bytes. Ic. Thanks for the pointer, I'll keep that in mind. Increasing it, would alleviate the issue for now, As mentioned I got a working version by some other magic (dropping some stuff). I just wondered about the genral behaviour and as it transpired when I was at CERN that most likely our HGF instance are one of the heavier users of baskets, alerts, webmessage etc. it might have been that the issue didn't show up yet. but as Tibor suggested, updating to the v1.1.x series would fix this issue along with a many other improvements. On top of the list, especially for the necessary fixes in OAI server. Just some slight bottleneck of resources around here, that's delaying the transition. :S -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Das Forschungszentrum oeffnet seine Tueren am Sonntag, 29. September, von 10:00 bis 17:00 Uhr: http://www.tagderneugier.de
Re: Baskets Templates
On 12.07.2013 10:39, Tibor Simko wrote: Hello Tibor! I just wondered about the genral behaviour and as it transpired when I was at CERN that most likely our HGF instance are one of the heavier users of baskets, alerts, webmessage etc. it might have been that the issue didn't show up yet. WebMessage is rarely used indeed. BTW you make a very nice use of it in your cataloguing approval workflows. It may be good to describe it in a new HowTo document, perhaps other Invenio sites would get inspired and be interested in using something similar? I could do some description of our workflow in such a document. Would be nice for our internal documentation anyway. However, at the moment I can give you no date for that to be finished. Would mediawiki a suitable format? Or would twiki be preferable? WebBasket and WebAlert are used quite heavily here, e.g. on the CDS instance there are several thousands of them. But we keep the CDS code base very close to the bleeding-edge master branch, so we rarely see `old' issues in production. Sure. (Naturally it would not be good for you to track our bleeding edge master branch for production purposes; Well, we'd just need to hire you then we could do that. :) Surely, your views to the Mont Blanc and Lac Leman is not comparable at all to our nice vistas at the Sophienhöhe. You'll agree immediately when we meet here, I'm sure ;) [...] I should add that the old maint-x.y branches do get love, especially for bug fixes and security improvements; but for some feature updates, an upgrade to higher maint-x.y branches is really necessary.) As mentioned we'll most likely jump in the 1.2 direction when our gang is back on board again. Currently there's some kind of holiday season. Its very understandable that new stuff does not get backportet to main/ -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Das Forschungszentrum oeffnet seine Tueren am Sonntag, 29. September, von 10:00 bis 17:00 Uhr: http://www.tagderneugier.de
Re: Baskets Templates
On 12.07.2013 11:32, Tibor Simko wrote: Hi! I could do some description of our workflow in such a document. Would be nice for our internal documentation anyway. However, at the moment I can give you no date for that to be finished. I guess you can make a nice demo at the next Invenio User Group Workshop meeting. Yep. This is actually one point on the schedule about our HGF based project. Would mediawiki a suitable format? Or would twiki be preferable? Trac wiki format would be preferable, in the form like: Ok. http://invenio-software.org/wiki/HowTo I thought of something like ``How To Use Messaging and Baskets In Cataloguing Workflows'' -- but perhaps this would be better off living under `Project/HGF/CataloguingWorkflow' somewhere indeed, since the topic is wider in scope than the usual Invenio HowTo QA recipes. (that are typically kept short, general, narrow-targeted) Ah, ic! Yes we could add it in that structure as we discussed at CERN. One point, however: we should (here at HGF but also with you) discuss a bit how to do it decently cause it is IMHO not helfpul if I clutter our docs amongst ot many wikis. You remembered out discussion about a ticketing system, resuing TRAC and so on. Did you, by any chance, know already how TRAC behaves with multiple instances in one host? -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Das Forschungszentrum oeffnet seine Tueren am Sonntag, 29. September, von 10:00 bis 17:00 Uhr: http://www.tagderneugier.de
Re: Baskets Templates
On 10.07.2013 18:02, Tibor Simko wrote: Hi! However, I wonder if there isn't something wrong here in principle. As I can call all baskets nicely on the backend (luckily, cause I have to delete them to get the system going again ;) it should work on the frontend as well, right? Seems like you are using v1.0.x. Well... As we discussed at CERN... The eval() code in the basket templates is not pretty, it has been removed in maint-1.1, together with many improvements, see below. Any chance you could upgrade to v1.1.x? As soon as some department here stops to be creative with new stat-requests and our gang is on board again, this is the very next thing we're about to do. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Das Forschungszentrum oeffnet seine Tueren am Sonntag, 29. September, von 10:00 bis 17:00 Uhr: http://www.tagderneugier.de
Baskets Templates
Hi! It seems that I do not understand a certain part of basket handling. As we need to sort out a bunch of papers by hand the idea was to set up baskets for that so people can just add them to the baskets and we can do the magic by reading those in some pythonic format. Now it happens, that I need a bunch of them. (Luckily, each of our victims have to take care of only about 10 or so.) However, if I get it correctly I hit some string length barrier in webbasket_templates.py, l.293 ff. It fails in l. 304 baskets = eval(topic_and_baskets[2] + ',') seemingly if the lenght of the basket list surpasses a certain amount. Looks like 1024 chars? At least it complains about SyntaxError: EOL while scanning single-quoted string (line 1) seemingly it cuts my baskets name at a point where it is not yet finished. I can work around it by using shorter basket names, and I get rid of the 500 dump. However, I wonder if there isn't something wrong here in principle. As I can call all baskets nicely on the backend (luckily, cause I have to delete them to get the system going again ;) it should work on the frontend as well, right? -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Das Forschungszentrum oeffnet seine Tueren am Sonntag, 29. September, von 10:00 bis 17:00 Uhr: http://www.tagderneugier.de
Re: RFC: enable bibsched scripting capabilities [draft patch]
On 27.06.2013 16:28, Ferran Jorba wrote: Hi! [...] One very curious behaviour is that the first task after midnight switches to SCHEDULED state but doesn't run. Strange indeed. However, the guys at RWTH had something similar with a cron job that seemed to put bibsched to manual operation. I'm not sure if it still persists, but it's nasty as well. I get the friendly daily mail as such: Emergency from http://ddd.uab.cat: BibSched halted: Process bibsort (task_id: 140997) was launched but seems not to be able to reach RUNNING status. Anyhow, I needed a mechanism to automate my daily manual task to put bibsched into manual mode, know which is the task in SCHEDULED state, run it and put bibsched back to automatic mode. Out of sheer curiosity: does putting bibsched to manual and run the task really put to running status? I wonder how that could work. Sounds like a deeper problem, cause (without knowing the internals of bibsched) this is exactly what bibsched is supposed to do automagically, right? It shounds a bit strange, that hooking up a scheduler with another scheduler (cron, I guess ;) makes the first one working. I have been patching bibsched to allow, at least, those basic scripting capabilities. I don't know how many more tasks do I need (acKnowledge errors, maybe?). I'm unsure on the names I have choosen. How compares bibsched --mode to bibsched stop/start? Do you know how much of your stuff could be accomplished by these bibtasklet stuff? -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: Question about document migration to new server
On 28.05.2013 17:05, Theodoros Theodoropoulos wrote: Hi! On 28/5/2013 3:23 μμ, Tibor Simko wrote: Yes, I think this is the fastest way to go. Don't forget all bibdoc related tables such as `bibdocmoreinfo'. Whoops! I didn't see that coming! Not in this specific case but in general, something like that is points to an IMHO very serious problem. The supply of Tibors and friends that know all those interals ran a bit low ;) Though my collegues in our project now, after some two years, have some strong feelings about my consistent nagging that they should /never ever even consider/ to do something beyond the highest level API (beware within the database, filesystem and the like) I think Theodoros post just shows nicely why. There are one or two points where we need to call SQL ourselves, indeed but I admit: I consider these cases /bugs/ in Invenio. One point I have in mind is an API that only spits out HTML, but I need it as a backend. So sort of a design flawn as the SQL is glued to HTML generation. Should be two functions, and I could get rid of the SQL code copy. The issue in websubmit (Tomek is our expert there) seems to be even a bit darker. But anyway there should be some pythonic way to handle what Theodoros want's to do, right? And IMHO pointers should go that way. Maybe it's not the highest level bibdoc api, but some api should exist. IMHO, it would IMHO be great if all those Atlantis setup routines wouldn't use sql statements, but the APIs instead so one could easily see how one should do it decently. IMHO dbexec with INSERT INTO is not the correct way. Same for cloning an instance. Uploading sql is bad there has to be a way to move documents from one setup to another without this. Concerning overlay-idea to roll out Invenio (which is as far as I understood it from a quick glance almost exactly what we're doing at HGF, though split a bit differently) even those setups shound avoid INSERT INTO like the devil avoids Holy Water and go for the real stuff (API) as well. Actually, we have pythonic routines for all stuff like generation of collections, groups, firerols, permissions, OAI, what have you for this very reason to avoid the SQL. (Or in my simple mind: I don't want to and don't want to /need to know/ the db structure of Invenio ;) JM2CT -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: Copyright notice per docfile
On 29.05.2013 10:23, Theodoros Theodoropoulos wrote: Hi! Depending on the Invenio version, the BibDoc objects have `more info' property store that basically serve as a storage for any key-value combination. We use it e.g. to store technical metadata about pictures: width, height, position on page. It could be used to store per-docfile license/copyright information. Alternatively, in the past, there was also an approach to enrich BibDoc independently with licensing information and then represent it in MARC accordingly. We did not commit this part, but we may perhaps revive it with Jerome. Licensing Information per docfile is a useful thing to have in Invenio. Agree. Having said that, using a seperate 'field' in bibdocfile -like comment or description- to keep it (although one possible way to do it) will probably require a lot of changes in several files. I would say that a cleaner way would be to have some link to a license. Some sort of license as authority where I can add a link to a persistent ID within the system that refers to this very license. Makes it easier to maintain, as you need to keep the relevant licences only in one place and you don't need individual texts per record. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: Access rights for files
On 26.04.2013 16:35, Theodoros Theodoropoulos wrote: Hello Theodoros! I too would be interested in such a functionality! :) Personally, I don't think there is an easy way to do it. :( As i see it, a new Action (similar to SRV) should be created (along with relevant websubmit functions and elements) that would read and display the current files along with their restrictions. Important note: an update to relevant bibdocfile.py functions would be needed as 'get-restrictions' is not supported! I think one has to be a bit careful here. In general I'm not sure that one can set proper permissions upon initial websubmit. In our case websubmit is done by our scientists and/or their secretaries. They do not want to fiddle with copyright-issues here. On the other hand we'd like to get the full texts anyway as, you know there is a limited period of only 70 years that applies in the worst case. So we encourage all our users to upload a full text and only give access at the library if we are allowed to. So this might be very well two distinct tasks. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: Access rights for files
On 26.04.2013 16:50, Samuele Kaplun wrote:
Hello Sam!
A common use case arises here due to embargoed open access. We get the
file for OA from our authors, but it has an embargo for say 12 months.
Afterwards, we can enable access to the file. Easy to accomplish, by use
of firerules in principle.
In maint-1.1, the Document File Manager (which is actually the same interface
that can be displayed in WebSubmit with the combo
Create_Upload_Files_Interface + Move_Uploaded_Files_to_Storage) can be
actually tuned to many use cases by playing with the config variables
CFG_BIBDOCFILE_DOCUMENT_FILE_MANAGER_MISC,
CFG_BIBDOCFILE_DOCUMENT_FILE_MANAGER_DOCTYPES and in particular
CFG_BIBDOCFILE_DOCUMENT_FILE_MANAGER_RESTRICTIONS:
Ah. This will solve most of our issues already.
## Eg:
## [('', 'No restriction'), ('restr', 'Restricted')]
CFG_BIBDOCFILE_DOCUMENT_FILE_MANAGER_RESTRICTIONS = [
('', 'Public'),
('restricted', 'Restricted')]
If this could learn something like 6-month embargo and 12-month
embargo e.g. by reading a field like Lars' OpenAIRE setting we would
already have a 99% solution and the rest could be done easily by hand.
It would fall back to some time calculations, but well. Probably one
could merge it with the 942-aproach of OpenAIRE?
Concerning our 1.0.2 vs. 1.1: yes we are working on that upgrade. I
understand that we should definitely go upstream. But as I mentioned
already we are a bit tighter in resources than CERN ;)
--
Kind regards,
Alexander Wagner
Subject Specialist
Central Library
52425 Juelich
mail : a.wag...@fz-juelich.de
phone: +49 2461 61-1586
Fax : +49 2461 61-6103
www.fz-juelich.de/zb/DE/zb-fi
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt
Re: Access rights for files
On 29.04.2013 08:53, Theodoros Theodoropoulos wrote:
Hi!
So we encourage all our users to upload a full text and only give access
at the library if we are allowed to. So this might be very well two
distinct tasks.
Your scenario is not uncommon i think...
Definitely not. I have at least 5 identical cases already. But I also
know that it's common procedure at most universities here in Germany.
And it seems not that difficult to implement!
Even with the existing Create_Upload_Files_Interface, you could define a
custom 'static' restriction -as Sam said- that allows librarian group
and submitter and denies everyone else.
This is done upon websubmit. It's a bit more involved but in principle
that's what's happening. A semistatic rule who can access the file upon
inital upload.
IF user chooses to 'restrict'
the document this will be applied. The added bonus is that with invenio
1.1 you could CHANGE this restriction afterwards with Document File
Management. OR, even with earlier versions you could probably implement
the same functionality with Submit Revised Version ('SRV') action.
As I mentioned, I can solve many areas with the static solution. E.g.
free on campus and stuff like that. I just wasn't aware what this
restriced refers to. The only thing I think that's still lacking is
embargoed stuff.
I even know how to do that in principle but I didn't want to invent
something that exists already. IMHO, as pointed out in my other mail, if
one marries the OpenAIRE 942 fielded solution for embargos with the
invenio 1.1 static stuff 99% of all (our) cases are covered.
--
Kind regards,
Alexander Wagner
Subject Specialist
Central Library
52425 Juelich
mail : a.wag...@fz-juelich.de
phone: +49 2461 61-1586
Fax : +49 2461 61-6103
www.fz-juelich.de/zb/DE/zb-fi
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt
Access rights for files
Hi! Upon upload, I can easily set up permissions of a file by means of the fft-tags according to our needs. However, till now I did not find a way to change the rights of a full text file on a higher level than an fft tag in marc xml. Or in other words: where is the interface to adopt access rights that could be accessible for our library staff? It seems that managedocfile does not do the trick. I can revise the file therem but the only sensible option for access is public. However, sensible might clash with jurisdiction (as usual ;) This dialogue offers restricted as well, but I have no idea what restricited actually sets. Nor do I see a away to define which restrictions should be applied. (Not even if I allow free form firerule writing, which would probably be a bit technical in some cases as well.) A common use case arises here due to embargoed open access. We get the file for OA from our authors, but it has an embargo for say 12 months. Afterwards, we can enable access to the file. Easy to accomplish, by use of firerules in principle. But: to get the content in the first place and for some other (legal) reasons within websubmit we set the access permissions for the files quite tightly. This allows us to tell our users: upload the file in case of doubt we check anyway. Restrictions here are only staff and the uploader can see the file itself. Then the library checks the submission and if the file in question can be released to the world our librarian would like to set permissions properly. Probably we would like to set something like deny until 2014-01-01. But at the moment I see no way how we can do that if the final permission is not free for world. Thanks for your help! :) Currently still invenio 1.0.2x -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: Access rights for files
On 26.04.2013 14:09, Lars Holm Nielsen wrote: Hi! Unfortunately I don't know of any librarian accessible interfaces to properly set the file. In Orphan repository I have a bibtasklet running in the background which based on what's in the MARC, will ensure that the files has proper access right. I can share if you are interested, but I'm not sure it applies in your specific case. This would be definitely of interest. How do you store the permissions in marc? IMHO even FFT should probably be exposed to bibedit anyway like a usual MARC field. (My librarians can usually easily handle everything that is in Marc but python backend is another story.) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: perform_request_search() vs. search_pattern()
On 26.02.2013 18:31, Tibor Simko wrote: Hi! On Wed, 20 Feb 2013, Alexander Wagner wrote: I understood this but didn't understand why the high level api returned more results for query that went basically exact match on logical field x. I guess I'll understand that from the docs, and if not I'll ask again :) Usually it is the other way round, i.e. p_r_s() returns less hits than s_p(). This is because p_r_s() filters collections while s_p() searches regardless of collection. Hence the latter may get restricted records, deleted records, and whatnot. Was it really the other way round, p_r_s() giving more records than s_p() in your case? If so, we should have closer look. It is indeed as I described. But as far as I understand it (here you're the experts) this stems from the fact that search_pattern() does not really like a complex query like pub:2008-2012, right? I admit I used search_pattern() mainly cause it returned an intbitset straight away. (I've to do bean counting at the moment it is convenient to use intersect, difference, union and friends.) However, I feel I really want something like intbitset(perform_request_search(p=searchstring)) instead, allowing for the more advanced parameters of p_r_s() and the identical results compared to the usual websearch, ie. I don't have to take care of deleted records and such. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: perform_request_search() vs. search_pattern()
On 20.02.2013 14:31, Ludmila Marian wrote: Hello Ludmila! Some info about this can be found here: http://invenio-demo.cern.ch/help/hacking/search-engine-api The search engine internals are here: http://invenio-demo.cern.ch/help/hacking/search-engine-internals Thanks for the pointers. I'll have a read. :) In a few words the difference is that perform_request_search() is the high-level API (same results as the ones displayed) while search_pattern() is mid-level API (does not know about restrictions, collections, various washing of parameters, etc..). I understood this but didn't understand why the high level api returned more results for query that went basically exact match on logical field x. I guess I'll understand that from the docs, and if not I'll ask again :) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: : handling in search engine
On 29.01.2013 11:00, Ludmila Marian wrote: Hello Ludmila! Thanks for pointing this out! De rien. :) Thank YOU for having a look at it :) The error that you pointed out (and actually similar ones, like searching for 'foo, bar') can be fixed by washing away the punctuation characters once we are sure we are doing simple word search. This sounds good. However, stripping usually makes me thinking about possible side effects if the char that is stripped of is part of say an identifier. But I guess that will not strip anything. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: Results of Advanced Search: Missing Message
On 29.01.2013 16:32, Ludmila Marian wrote:
Hello Ludmila!
[...]
Similar use case: JuSER feeds our web pages. Usually, these are searches
like: (cid:inst-ID and typ:doctypeID and pub:year) and require
return exactly. However, at the beginning of the year most of these
searches will yield empty results for some time and if the search
algorithm throws in a did you mean and returns say the results from
institute A but those for institute B just cause there is something
(did you mean?) in A but not in B this will cause trouble.
Indeed, problems might arise due to nearest search terms, but the
suggestions are mostly for web-interface users, to guide them to a
better search. Any other outputs, except the html-based ones, will
return an empty list.
Actually, there are some ways you can instruct the search engine to do
only exact search, and no nearest searches. One way is to use the the
'ap' parameter - alternative pattern (you can test it's behaviour by
adding ap=0 or ap=1)
Ah! :) I didn't know that one. Thanks for this pointer. Seems that I'm
on the safe side if I add it as ap=0 in case we need exact matches.
In any case, the patch that I proposed for the advanced search issue, is
not proposing nearest searches ('did you mean') nor is doing any query
manipulation in order to get some results.
I understood this intention. I just wanted to point out that stripping
of seemingly useless puctuations could cause trouble in case of exact
matching. Google shows this problem e.g. if I search for identifier like
terms (specs, rep-no, stuff like that).
--
Kind regards,
Alexander Wagner
Subject Specialist
Central Library
52425 Juelich
mail : a.wag...@fz-juelich.de
phone: +49 2461 61-1586
Fax : +49 2461 61-6103
www.fz-juelich.de/zb/DE/zb-fi
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt
Results of Advanced Search: Missing Message
Hi! Happy new year to all! :) Unfortunately, I fear I have to start the year with a bug report. One of our users just noted that if you use the advanced search and enter a query without any results you do not get any notification in certain circumstances. E.g. http://goo.gl/gsJ3p (searching müller and wert in the author index of JuSER) triggers this behaviour. However, if I fill in only one field it seems to report properly. I think this should be fixed. At this occasion I wonder if not the advanced search should just create a simple search with proper fields instead of using it's own code. As far as I can see the above example (at least should ;) be identical to author:müller and author:wert in simple search. And as far as I can see, there is, (taking /, and ' modifiers into account) nothing that adv. search can perform that simple search can't. At this occasion I'd like to mention that I got the feedback from the users that they actually prefer the advanced search in many cases instead of just keying in the above term. Not that I understand it, it's just users feedback. Even though we added a lot of just 3 keys logical fields. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
: handling in search engine
Hi! Invenio uses the colon : to tell the search engine the field to search. Per see this is nice, but unfortunately, the colon is not really a good separator in bibliographic data search. E.g. it usually separates the main title from the subtitle as well. An example would be: http://juser.fz-juelich.de/record/21825 reading The Mont-Blanc Project: European Summit of Exascale System Architectures Now, if you have the exact title you can copypaste it into the search box but it will yield no result as project: seems to trigger something here. If I give the title in quotes ( or ') everything is fine. If I remove the colon, everything is fine again. Just a literal copypaste fails. From this I would suggest to improve the search engine a bit. It seems reasonable, that - if the term in front of a colon is NOT a defined index nor a marc field, it should not trigger a field search. Probably it is safe in those occasions to strip the colon. - if the letter following the colon is a blank it does not refer to a field but is a literal colon. I may add that at least for books there is also a separation common (default RAK behaviour, ie. all german speaking library catalogues) that is defined as : ie a colon enclosed in spaces. Might be that the search engine doesn't like that either. At least if I add a space to the above title I get some funny comment from invenio. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: Default full text search with SOLR
On 04.12.2012 14:33, Patrick Oliver Glauner wrote: Hi! Awesome! And a Google-like search field should also definitely query the fulltext index. I may mention that it is IMHO not advisable at all to include full texts _by default_ in all index. You just get to much garbage in the results. Think of someone searching the papers of Smith, John. Searching full text by default will give you all papers mentioning a Smith, John anywhere. Be it authorship, citations within the text as it is just a sample name, what have you... Results will be bad enough if you just search metadata. Or think of stuff like mentioned in the book of Smith or we handle x y z but not your search term. For some time a huge union catalogue did something like full text indexing of reviews only. From one day to the next you were not even able to find Gerthsens Physik anymore just as all discussions of physics books that compared book X with Gerthsen floated to the top of the list. You can check this easily in some of those commercially available discovery systems. You can discover there a lot you just don't find your papers anymore ; IMHO full text indices/searches are WAY overrated. But that's my personal opinion of course ;) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: Default full text search with SOLR
On 04.12.2012 15:05, Patrick Oliver Glauner wrote: Hi! Thanks for your message! I absolutely agree, that fulltext should not be included by default in all index. Nevertheless, many people love a Google-like search and subconsciously expect fulltext to be queried automatically. You're right, if you ask people they want it that way. Well, the gods will fulfil your wishes... if they want to punish you ;) The solution will be configurable. We will see how it is going to look exactly as soon as I implement it. I'll stay tuned. :) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: (pseudo)authorities: what is supported in Invenio/Inspire?
On 30.11.2012 11:09, Theodoros Theodoropoulos wrote: Hi! In order to be able to plan ahead in my installation, I would like to know if there is any way to deal with authorities in Invenio. The BibAuthority module (described in detail here: https://twiki.cern.ch/twiki/bin/view/CDS/BibAuthority) is EXACLY what i would like to have, but i cannot find it in my installation nor in the invenio/inspire repo. Was the module abandoned/postponed/replaced by something even better? We currently use some sort of authority control in JuSER where we mainly implemented the functionality within websubmit type of things. This however does not (yet?) use bibauthority. Unfortunately, the searchengine does not yet know about authorities. If we can be of some help here, drop me a note. You can check out what we did at http://juser.fz-juelich.de/ in the Authorities-collections. You might also want to check out the Marc we used. E.g. for an institute http://juser.fz-juelich.de/record/98380 You may use the arrow links to go to predecessors, successors or top. On the other hand, inspirehep.net seems to have such functionality (HepNames, Institutions, Conferences) seem to be 'authority' records that are considered when performing a search. We would definitely also want to know how they can be used within search. I mean not simple stuff like searching for an ID, we have this already but searching for an ID x and find all records for ID x and y as the authority record of x list's y as addional identifier. We work around our current limitations e.g. using the index cid (contributing institutes also gathering all former forms of the institute) but this is not really done in a nice way. Bascially a record here just has enough collection entires, but it's no real authority lookup. ps. I apologize for sending a similar mail in the past too. It's quite urgent that i know at least some general information and the best practices for the current/next Invenio version in order to make the right decisions for my next milestone. I'll do the reading/implementation/testing myself. When we started to implement this stuff I checked closely with Annette Holtkamp, so I think we should indeed use the same or at least a very similar authority record layout as Inspire. HTH :) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: Add all search results to a basket?
On 30.11.2012 13:32, Samuele Kaplun wrote: Hi! In data venerdì 30 novembre 2012 13:30:00, Alexander Wagner ha scritto: This looks like what I'm missing, right. So I understand not yet but will come in 1.2 or 1.3. well, next branch, being based on SQLAlchemy+Werkzeug+Flask+Jinja2+Bootstrap is more likely to become 2.0 ;-) Ups, this will then take a bit longer to adopt here, I fear. Do you have a very rough idea of the time frame for this major transition yet? -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: (pseudo)authorities: what is supported in Invenio/Inspire?
for all that, and I know that GSI, DESY and RWTH are with me here as well. Plus our upcoming friends as Munich and at JARA will also require it. For the time being we can work around it, but I hope that some ressources are available at CERN here as we discussed with Tibor while ago. I fear it is a bit too much inside of Invenio to get it done with our ressources. :( (Probably those things are not as funny as bottles and tools and what ever though ;) - Is it planned for 1.2 1.x or 2.x series? I admit that we (as mentioned above) very strongly hope that it is a 1.x issue. We actually hoped for 1.0 to have it working. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Add all search results to a basket?
Hi! Is there currently a way to add all records retrieved by a search to a basket? I mean without hooking on every hit and then add per page? Probably even some hook all records on this page function? I just got this question from our users and at the moment have to admit that the only answer I know would be to do it per record. (Note: I do not want to store the query in the basket but really the resulting records.) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
CFG_WEBSEARCH_MAX_RECORDS_IN_GROUPS
Hi! In invenio.conf there is a statement ## CFG_WEBSEARCH_MAX_RECORDS_IN_GROUPS -- in order to limit denial of ## service attacks the total number of records per group displayed as a ## result of a search query will be limited to this number. Only the ## superuser queries will not be affected by this limit. CFG_WEBSEARCH_MAX_RECORDS_IN_GROUPS = 200 This should effectively limit the number of hits returned to 200, except if I'm superuser. First of all: this works. However, if I check search engine code, there is also a parameter for rg to return _all_ records in a collection. This disregards any max settings, and I think it's existence is also sensible. But if I actually set it invenio does NOT check if I have any special rights, it seems. At least on our test system guest just happily dumped out 15.000 records in hb. It's sort of a bug-feature. In fact I was searching for such a possibility, or at least a way to get a larger chunk of data (you know those guys from the bibliometrics department always have funny ideas...) Nevertheless, a feeling tells me that this behaviour is not intended. I think, that dumping down a whole collection might be quite usefull. Even in http given one might use a format like hx, xe or other structured stuff for post processing. (Sure, probably a simple fetching loop is better.) But I think one should probably be able to limit it to a certain group of people or the like. Firerules come into mind, say if user = bibliometryfreak and timeofday = 2:00-3:00am and requestingIP in range Ah, Invenio 1.0.2 maint. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
JuSER online
Hi! Since tomorrow morning you may acutally see our JuSER instance in Jülich. Just for those curious what we did in the last, hm, almost two years. And/or those who were quite interested at the user meeting earlier this year where I was not yet able to show you the stuff hands on. http://juser.fz-juelich.de Many thanks for all the help from the folks at CERN, especially Tibor, Jerome, Samuele, Ludmila, Christopher, Annette just to name a few n place of the whole team and of course all members of our project team: Martin Köhler (a) Zaven Akopov (a;b) Tomasz Pazera (a) Katrin Große (c) Stefan Hesselbach (d) Bernhard Mittermaier (e) Anna Fründ (e) Heike Lexis (e) Cornelia Plott (e) Christoph Holzke (e) Ulrike Eich (f) Louai Barake (f) Abdoulaye Diallo (f) Roland Rappmann (f) Dominik Schmitz (f) Edmund Wollgarten (f) a) DESY Library and Documentation, b) Project Inspire, c) GSI Library, d) GSI Core IT, e) Forschungszentrum Jülich, Zentralbibliothek, f) RWTH Aachen, Hochschulbibliothek Now we'll start fixing details and polishing stuff. ;) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: Fwd: Invenio 1.0.2: Exception (webjournaladminlib.py:34:?)
On 26.10.2012 10:02, Jerome Caffaro wrote: Hello! On 10/26/2012 07:51 AM, Alexander Wagner wrote: Stefan pointed out the following issue and I was able to reproduce it on our freshly installed Invenio box @1.0.2 maint as well. Just hit Admin/Configure WebJournal - 500. It seems that there's some new variable that is missing in the confs. thanks for the feedback. De rien :) I will fix it. [...] Thanks :) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Kennen Sie schon unsere app? http://www.fz-juelich.de/app
Re: Output format / mime type
On 02.10.2012 14:07, Alexander Wagner wrote: Hello Jerome! On 10/02/2012 11:24 AM, Jerome Caffaro wrote: [...] if I change the mime type in the output formats definition to say appplications/x-endnote it just gets ignored. This looks like a bug. I will have a look at it. Ticketized at http://invenio-software.org/ticket/1181. Could it be that your fix for this issue didn't make it into 1.0.2? I played with this yesterday and our new box showed the same behaviour, that's why I wonder and I didn't find a notice in the git log. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Kennen Sie schon unsere app? http://www.fz-juelich.de/app
Re: Output format / mime type
On 26.10.2012 11:13, Jerome Caffaro wrote: Hi Alexander, On 10/26/2012 10:30 AM, Alexander Wagner wrote: Could it be that your fix for this issue didn't make it into 1.0.2? I played with this yesterday and our new box showed the same behaviour, that's why I wonder and I didn't find a notice in the git log. Indeed this fix is not part of 1.0.2. Ok, then I understand of course that it's not working as expected. So it will surely be in 1.0.3 some day. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Kennen Sie schon unsere app? http://www.fz-juelich.de/app
Fwd: Invenio 1.0.2: Exception (webjournaladminlib.py:34:?)
/webjournal/webjournaladmin.py'
Frame ? in string at line 24
__builtins__ = '{\'IndexError\': class exceptions.IndexError
at 0x2b3407f0, \'help\': Type help() for interactive help, or
help(object) for help about object., \'vars\': built-in function vars,
\'SyntaxError\': class exceptions.SyntaxError at 0x2b3407888710,
\'unicode\': type \'unicode\', \'UnicodeDecodeError\': class
exceptions.UnicodeDecodeError at 0x2b3407888c50, \'isinstance\':
built-in function isinstance, \'copyright\': Copyright (c) 2001-2006
Python Software Foundation.\nAll Rights Reser [...]
__revision__ = '$Id$'
__doc__ = 'Invenio WebJournal Administrator Interface.'
__lastupdated__ = '$Date$'
Frame ? in /usr/lib/python2.4/site-packages/invenio/webjournaladminlib.py at
line 34
***
31 # pylint: enable=W0622
32
33 from invenio.errorlib import register_exception
34 from invenio.config import \
35 CFG_SITE_URL, \
36 CFG_SITE_LANG, \
37 CFG_SITE_NAME, \
***
cPickle = 'None'
__revision__ = 'None'
CFG_SITE_URL = 'None'
__builtins__ = '{\'IndexError\': class exceptions.IndexError
at 0x2b3407f0, \'help\': Type help() for interactive help, or
help(object) for help about object., \'vars\': built-in function vars,
\'SyntaxError\': class exceptions.SyntaxError at 0x2b3407888710,
\'unicode\': type \'unicode\', \'UnicodeDecodeError\': class
exceptions.UnicodeDecodeError at 0x2b3407888c50, \'isinstance\':
built-in function isinstance, \'copyright\': Copyright (c) 2001-2006
Python Software Foundation.\nAll Rights Reser [...]
CFG_ETCDIR = 'None'
__file__ = 'None'
CFG_CACHEDIR = 'None'
CFG_SITE_NAME = 'None'
sys = 'None'
re = 'None'
__name__ = 'None'
CFG_SITE_LANG = 'None'
register_exception = 'None'
os = 'None'
__doc__ = 'None'
urlopen = 'None'
Best regards
--
JuSER http://zb0035.zb.kfa-juelich.de
Need human intervention? Contact ju...@fz-juelich.de
--
Kind regards,
Alexander Wagner
Subject Specialist
Central Library
52425 Juelich
mail : a.wag...@fz-juelich.de
phone: +49 2461 61-1586
Fax : +49 2461 61-6103
www.fz-juelich.de/zb/DE/zb-fi
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt
Kennen Sie schon unsere app? http://www.fz-juelich.de/app
Search and logical indices
Hi! To ease up the searches for our users I set up several additional indices that should allow to search some strange Marc-fields in an easy way. Now, I have e.g. an index called CID, that uses the logical field [CID] contributing institute ID, set up as follows: Field Code: CID Name (en) : [CID] contributing institute ID Marc tags : 980__a 9201_0 bibindex, have cup of coffee or tea, Tada! Now I can indeed select [CID] contributing institute ID from the search box, give one of our funny IDs like I:(DE-Juel1)ZB-20090406 as search term an. Heureka! This tells me that the index creation and indexing itself proceeded and I didn't do something wrong, right? Now, I thought I should be able to use this index also for searches within the search box just like a marc field, e.g.: CID:I:(DE-Juel1)ZB-20090406 IMHO this should accomplish the same thing. However, it seems that there's some conversion going on... verbose=9 tells me that invenio is searching in the field cid which does not exist. (?) * Could it be that the name of the logical fields have to be in lower case? At least the above doesn't give any results. However, if I change the field code to cid it seems to work fine. If this is intended I'd have a line for the bibindex-admin-guide on the wish list ;) * Just for my understanding: if I'm searching like the above I'm effectively searching the logical field, not the index CID, right? Ie. an index could be made up of many logical fields. However, as far as I understand it, I can not search it. Just for my understanding, I can live with defined fields. I'm just wondering what the index as such is doing. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Kennen Sie schon unsere app? http://www.fz-juelich.de/app
Output format / mime type
Hello! I'm searching for a seemingly simple solution: how do I specify the mime type Invenio returns for a given output format? E.g. endnote being an XML format returns text/xml. This causes the browser to open it and (especially the average endnote user) to be confused. They just don't understand how to proceed and usually don't have a technical background to notice that they got xml that they should just save. However, if I change the mime type in the output formats definition to say appplications/x-endnote it just gets ignored. I found that I could get at least text/plain if I change the name of the of from XE to something like EN. (There seems to be some file name checking still active in Invenio 1.0 maint, that AFAIK should have gone.) However, it's seems not possible to get something like applications/* which triggers the web browser to just fire up endnote against the download. Any hints? TIA! :) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Kennen Sie schon unsere app? http://www.fz-juelich.de/app
Re: Output format / mime type
On 02.10.2012 11:59, Jerome Caffaro wrote: Hello Jerome! On 10/02/2012 11:24 AM, Jerome Caffaro wrote: [...] if I change the mime type in the output formats definition to say appplications/x-endnote it just gets ignored. This looks like a bug. I will have a look at it. Ticketized at http://invenio-software.org/ticket/1181. Thanks a lot :) This confirms that I was not completely stupid when fiddling with this. :) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Kennen Sie schon unsere app? http://www.fz-juelich.de/app
Re: JSON for MARC
\ :
\StatID:(DE-HGF)1050\, \2\ : \StatID\, \b\ : \BIOSIS
Previews\, }, ],
SHORTTITLE : NMR separation of intra- and extracellular
compounds based on intermolecular coherences. / Hoerr, Verena ;
Biophysical journal 99 2336 - 2343 ; [u. a.] : Biophysical Society,
2010 ; 10.1016/j.bpj.2010.06.068 ; ,
DUPES : 124524, 130115,
}
Note: this does not include the reserved tag label which
is what the user usually gets displayed, as this output is
really meant to just fill in the full mask without further
intervention. However, it contains SHORTTITLE and DUPES
which signify the result in a short ISBD-like notation and
possible DUPES in the database by their record ID. (Simple
dupe matching based on values in 0247_a) so we can prevent
the user from erronously add a second record with the same
content or the wrong record (made a typo in the PMID eg.)
The above is actually the output of
$ ./GenMetadata.pl mode=full pmid=20923669 format=JSON
GenMetadata being the fater of all imports calling itself
various backends to collect data.
Now, we have several bibformat_templates for various sets of
our data and one format JS that calls them appropriately, so
you can usually always specify of=js for whatever we have.
E.g. for an authority record linkage if you search some name
you'd get something like
{
label : Wagner, G. A. (VDB1731),
I1001_0 : P:(DE-Juel1)VDB1731,
I1001_a : Wagner, G. A.,
},
{
label : Wagner, A. (VDB13547),
I1001_0 : P:(DE-Juel1)VDB13547,
I1001_a : Wagner, A.,
},
{
label : Wagner, Alexander (ZB / x...@fz-juelich.de),
I1001_0 : P:(DE-Juel1)133832,
I1001_a : Wagner, Alexander,
I371__0 : I:(DE-Juel1)ZB-20090406,
I371__m : x...@fz-juelich.de,
I371__c : ZB,
I373__0 : I:(DE-Juel1)ZB-20090406,
},
label allows the user to select the correct one, the rest
of this hash gets stored to the record.
I think this gives some impression on what we do in the
backend here and how we use JSON. Modulo, of course, of some
technicalities which are not quite clear from the above, but
stem mainly from the use in Websubmit and Ajax itself.
Surely we didn't find the very best way, but one that was
pretty simple to implement and matches the needs. (E.g. most
of the JSONs are actually plain format templates of invenio,
that's why I have a superflous comma in the above output: I
found no way to drop it on the last record in a list of
results without the need to resort to python which in the
above case is not that convenient for some reason.)
HTH :)
--
Kind regards,
Alexander Wagner
Subject Specialist
Central Library
52425 Juelich
mail : a.wag...@fz-juelich.de
phone: +49 2461 61-1586
Fax : +49 2461 61-6103
www.fz-juelich.de/zb/DE/zb-fi
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt
Kennen Sie schon unsere app? http://www.fz-juelich.de/app
Re: hgf-invenio submission forms
On 24.08.2012 13:40, Pazera, Tomasz wrote: Hi! Perl : If you have trouble with those please drop me a note as I'm the one who seems to be responsible for fixing what's broken in the Perl area. Most of it is old code I use for ages and a day but, well... you never know. You'll need it only for foreign data import (DOI/ArXiv/Pubmed/ISBN). All other parts should work without it. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Kennen Sie schon unsere app? http://www.fz-juelich.de/app
Re: Restricting fulltext access
On 23.08.2012 09:10, Samuele Kaplun wrote: Hi! So, if I understand it correctly, I would have to restrict C such that guest have access (sort of a restriction but not hard to meet). Then the bibliographic records are viewable by guests even though they are furthermore part of A and B. Good point yes. In that case though it's like A and B are not restricted at all. For this single record that is in C, right? Say: A: allow group A recid 1 recid 2 recid 3 B: allow group B recid 2 recid 3 recid 4 C: allow any recid 3 recid 4 This results in guest could see the records 3 and 4 by means of the collection C, but not 1 and 2 as they are not member of C. A could see 1-4, as 1-3 are members of A and 4 is member of C. B could see 2-4 as they are either member of B or C but not 1 which is only in A. And to allow to guest, just have a role with firewall definition allow any. Yes. But(!) then I again need to restrict the docs as mentioned above to be viewable for A and B only (e.g. if there is a legal restriction on them) or otherwise they'd be open to world. Yes. Mmh to my undestanding of your complex situation, I believe in the end, if what you need is only to restrict download off fulltext (but not the metadata), For the public collections it refers to full texts, for private collections also to metadata. is to relay on FFT$r and set there a firerole rule dynamically generated based on all the complex rules you described... Ok. This is what we originally intended as well. I fear this is a bit over simplified, but actually what we try to do. ;) Sure... over simplified... I see... :-) . o O ( lucky one that you don't really have too ;) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Kennen Sie schon unsere app? http://www.fz-juelich.de/app
Re: Restricting fulltext access
On 22.08.2012 11:46, Samuele Kaplun wrote: Hi! In data lunedì, 20 agosto 2012 14.48:27, Alexander Wagner ha scritto: Guessing that there were some holidays involved, Indeed! Hope you enjoyed those well deserved days off :) However, as far as I can see, it is not possible to have two roles here, nor can I combine this with the allow groups 'STAFF' rule mentioned in the first example. That's exact. If you go for role/firerole/group/email etc. only one value can be currently specified. Ok. Also prefixing with firerole: like datafield tag=FFT ind1= ind2= subfield code=a/tmp/arwagner/fulltext.pdf/subfield subfield code=rrole: 'Institutes-Role-ID' firerole:allow groups 'STAFF' /subfield /datafield doesnt work. Yep. The involved parser does not expect this situation. Ok. And I understand that I can not have role: 'Institues-Role-ID' role: 'hgfstaff' either as only one stanza is allowed and role does not allow the comma syntax you describe below for firerules, ie. role: 'Institutes-Role-ID','hgfstaff' Right? Alternatively, I could set up a new role by combining our Institute-Role-ID with hgfstaff (matching 'STAFF' above), but I do not see a way to accomplish this in firerole language. Unfortunately, since Firerole language is used mainly to directly define roles, it is not possible to use roles as rules, because this might lead to definition loop (and would actually be quite complex to implement in an efficient way).. Understood. Maybe, this however is just thinking out loud now, our main issue stems from the connection of roles and the concepts of a group of people that should get this role. Together with certain rights on documents in a collection. It is a similar to the thesis-example except that we do not have document types specified (noting that I didn't really understand how this is done in your thesis example, it seems a bit special) but on collection membership. Say, we use external auth via LDAP, I get people belonging to group X. On our systems this should trigger to have the right to see a certain restricted collection and have access to the (sometimes otherwise locked) full texts there. Now a document could belong to the collections of group X, Y, Z, so we have the necessity for specifying several groups with logical OR. In other words we want to have personal collections for groups of people who log in via external auth and who should be able to see everything in their collection and other open collections. Could it be, that this is just not possible? Or do I just miss the obvious? Indeed, what you are aiming at is currently not possible. The first solution, i.e. to have one firerole rule: [...] firerole: allow groups 'InstitutesID' allow groups 'STAFF' [...] is, currently the most flexible one. Note that you can also put it in one line as in: firerole: allow groups 'InstitutesID','STAFF' Ok, sounds the way to go. As I understan it this would allow all users belonging to the group InstitutesID or the group STAFF access to the full text. Right? [...] role: 'Institutes-Role-ID' firerole:allow groups 'STAFF' [...] I'll ticketize it. Thanks :) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Kennen Sie schon unsere app? http://www.fz-juelich.de/app
Re: Restricting fulltext access
the collection, and all the inner records and documents. Documents are automatically protected to anybody not authorized to the collection - but the owner of the record (who always can access attached documents). This would suffice indeed if we didn't have an open collection that our records belong to in many cases. You might remember that you prepared a patch for us that allows for if any rule matches in the firerules block. In order to enforce this you might wish to create some script that be run regularly Done :) I think we have everything in place except the the doc restrictions (which wouln't be necessary in the first place if Science would really be free scnr ;) (e.g. via BibTasklet in case of recent Invenio), that looks for all the groups in the group table (e.g. via invenio.webgroup_dblayer.get_all_groups_description API) and populate automatically new roles (e.g. via invenio.access_control_admin.acc_add_role API). This is a bit more involved here as we acutally need to harvest the LDAP (or a similar) system and also need to generate a bunch of bibliographic records thereof. You probably remember our discussion @CERN about authority control where we use authority records living in MARC instead of knowledge bases. (I think we should definitely take a cup of tea over these issues in the near future, ie. as soon as our stuff went publicly online.) Anyway, basically we need to set up those records in this step as well. For those following this far and wondering why the h... does he use so complex naming schemes, this is the very reason: mapping institute histories, hierarchies and rights with a bunch of collections living in t=now and not valid for t=tomorrow in a managable way. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Kennen Sie schon unsere app? http://www.fz-juelich.de/app
Re: Restricting fulltext access
does the BEFORE/AFTER syntax in FireRole help you in any way? (i.e. you can have time-based authorizations too). They will be very helpful for embargoed documents, indeed. But this before/after stuff is not an issue of rights but of institutional history and hierarchy. Something like: Institute a is renamed to b. a is part of A and b is part of B. Institute c, part of C is split to at some point in time in d (being part of D) and e (being part of E) and later d and e is merged to f still being part of E. In short: Just think of the worst ticket you can imagine as a starting point, stick a name on any leaf, branch and twig. Then let this grow (yes, they can grow together and split apart again, over time branches die and new siblings coming up...). Imagine several of those (on the level of institutes, grants and programs) ie. a wood and then spice it up with an ever changing group of carbon based life forms called people who insist on silly entities called real names, that change for some reason even in various directions while they insist that they are still the same. Once done try to map that in Marc to answer stuff like I just have a very simple question. I only need to know: what did e publish on grant g in time t [much larger than the life time of any name involved] without f when X was leader of subdivision y. As this is so simple, you can surely answer it right on the spot, I need it NOW. Plus: in this clearly structured universe every entity like f wants to have her unique, own, private collection with their records in it for easy access. Some of them being public (WE did this!) while other are private (censored X did this already we need to...) And only then comes some lawyer and the restriction stuff. I fear this is a bit over simplified, but actually what we try to do. ;) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Kennen Sie schon unsere app? http://www.fz-juelich.de/app
Restricting fulltext access
Hi! For some reasons we need to restrict access to fulltext files after initial upload and have to verify first if we can allow free access. I found that I can accomplish this by using the $r subfield of fft e.g. by means of datafield tag=FFT ind1= ind2= subfield code=a/tmp/arwagner/fulltext.pdf/subfield subfield code=rfirerole:allow groups 'InstitutesID' allow groups 'STAFF' /subfield /datafield to give access to members of the group 'InstitutesID' and members of the group 'STAFF'. However, it would be more convenient from an administrators point of view to actually (re-)use an already existing role instead of copying it here, and indeed I can accomplish this by datafield tag=FFT ind1= ind2= subfield code=a/tmp/arwagner/fulltext.pdf/subfield subfield code=rrole: 'Institutes-Role-ID'/subfield /datafield However, as far as I can see, it is not possible to have two roles here, nor can I combine this with the allow groups 'STAFF' rule mentioned in the first example. Also prefixing with firerole: like datafield tag=FFT ind1= ind2= subfield code=a/tmp/arwagner/fulltext.pdf/subfield subfield code=rrole: 'Institutes-Role-ID' firerole:allow groups 'STAFF' /subfield /datafield doesnt work. Alternatively, I could set up a new role by combining our Institute-Role-ID with hgfstaff (matching 'STAFF' above), but I do not see a way to accomplish this in firerole language. Could it be, that this is just not possible? Or do I just miss the obvious? -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Kennen Sie schon unseren neuen Film? http://www.fz-juelich.de/film Kennen Sie schon unsere app? http://www.fz-juelich.de/app
Re: [hgf-dev] Re: CFG_WEBSTYLE_TEMPLATE_SKIN
On 20.06.2012 17:44, Tibor Simko wrote: Hi! The changes sound useful. However, can you please submit a clean patch? Sure. Attached you`ll find the patch. It was created with diff -Naur. I have tested it with the current version of invenio master (commit: 7dbf1735d4742551678dd1ea0e1e0ccd4cf342cb). Looks good. Since this would be the first time the skin variable is used for something else than, err, skinning the output, we have to document it carefully. Can you please add a short paragraph going like: I may add just as a side note that we use this variable in some more places. Usually it also serves as some sort of identification shortcut for the installation in question. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Kennen Sie schon unseren neuen Film? http://www.fz-juelich.de/film Kennen Sie schon unsere app? http://www.fz-juelich.de/app
Re: [hgf-dev] Re: CFG_WEBSTYLE_TEMPLATE_SKIN
On 21.06.2012 07:15, Jerome Caffaro wrote: Hi! [...] if we start from WebSubmit elements, we can spread this technique to say BibFormat elements and elsewhere [...] If so, please take note of the following ticket: http://invenio-software.org/ticket/333 [...] Thanks for this pointer. Definitely points in the same direction. I would also add that I would not go as far as to allow overriding of of BibFormat elements (bfe_*.py) and templates (*.bft), as these can already be very well handled in the case of Invenio upgrades as well as installs from an overlay repo. Actually, bfe_* and *.bft cause some trouble at the moment if you want to handle them cleanly. At least as far as I understood it. In our current code base we tried not to really overwrite default stuff but actually create new ones, but some changes here are unavoidable, I fear. At least I did some minor but for us essential changes to the Default_HTML_*. Some of it might hold for the general public some of it might not. E.g. in HB I enabled the title to be click able and handled like the Details link as it is common in almost all databases. This would be portable. In HD I added some actions like Request correction which generates a message to a certain group. This is again portable within our installations but might not be suitable for the general public, as incorporates some parts of our workflow. Additionally, I admit that we won about 25 new bfe_*. Some of them are IMHO very generic (bfe_sfx, bfe_dblinkup e.g. that hook up other library services like an OpenURL resolver or gateways to other databases or something like bfe_idnumber to extract a given ID from a dataset) I think they'd translate well to general installations. Those I also tried to implement with protability in mind. E.g. bfe_idnumber has default settings for working on 0247_ but you can change the fields and subfields, criteria, transforms and so on to your liking via parameters. Then we have something like bfe_authors_hgf.py or bfe_bibtex_hgf.py which are generic but. We use this _hgf suffix for stuff that is common for all our installations which might rely on some local conventions. IMHO all are sensible conventions and to the best of our knowledge none of them breaks or even bends Marc. (If it does I fear I'm to be blamed...) E.g. bfe_authors_hgf.py is a direct descendent of your own style, except that we use indicators for 100/700 to be 1_ not __ plus we use authority control via 1001_$0 which allows us to build up a better URL in case we want to search all articles from Ms. Smith: ie. we check if $0 is set, if it is build the search-url to use this number, if not use her name. This still works in general, but currently e.g. uses the convention that every author (though she may have several IDs) is linked in $0 by only one of them in the records. Say if Ms. Smith has an ORCHID-ID, a VIAF, an ID from DESY and one from us our current scheme allows to store them in her authority record, but we assume (silently) for this procedure to work that the linkup is done by ONE of those IDs only. Otherwise the list will not be complete. (Hint: here we are relying here on authority controled search to enter the invenio search engine eventually as discussed with Tibor while ago. Ie. it should acutally take the number given, look up the authority record and then search for all possible Ids. I could add this to the bfe_ but it doesn't sound to sensible to live there.) BTW: this authors-thing is actually using CFG_WEBSTYLE_TEMPLATE_SKIN to notify internal authors. Ie. authors from our own institute. It would e.g. print Wagner, Alexander (FZJ), where the FZJ is uppercase(CFG_WEBSTYLE_TEMPLATE_SKIN). I fear this intermediate layer is not easy to catch via CFG_WEBSTYLE_TEMPLATE_SKIN postfixing. Finally, we have some friends that might be really depending on a specific instance of one of our installations. Those would finally get the CFG_WEBSTYLE_TEMPLATE_SKIN postfix. I think, as soon as we got this beast flying we should set up some meeting to discuss in a small group how we can contribute our stuff to invenio-git and how to fiddle things apart due to these minor issues. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: How to deal with change of syntax of an invenio.conf variable
On 05.04.2012 14:23, Samuele Kaplun wrote: Hello Samuele! as part ofhttp://invenio-software.org/ticket/997 I would like to change the syntax of a config variable in invenio.conf, namely: CFG_BATCHUPLOADER_WEB_ROBOT_AGENT This is currently considered as a comma-separated list of authorized user agent strings while I would like to change it into a regular expression that will be used to match incoming user-agent strings. The problem of course is for Invenio users who have customized this variable to know that they will need to update this value. Wouldn't it be cleaner to use a new variable and let invenio throw a complaint if it finds the old one? E.g. an exception that notifies root about the issue? Or similarly if it finds the wrong syntax... -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Kennen Sie schon unsere app? http://www.fz-juelich.de/app
Re: Bibupload #721
On 26.03.2012 08:16, Johnny Mariéthoz wrote: Hi! nobody really? Is it a dummy question? Do I miss something in the documentation or in the mailing list? AFAIK neither. I can not be of particular help here, though especially from the scenario you outline here you strengthen my feeling that it's not wise to stick to record IDs, but use other mechanisms to identify individual records. (Ie. real persistent identifers, authority stuff and so on.) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Kennen Sie schon unsere app? http://www.fz-juelich.de/app
Re: ldap authentication
On 27.01.2012 06:40, Diwaker Ghimire wrote: Hi! I need to authenticate certain users from ldap server. in our case there [...] 1. Extending invenio's ldap authentication: Basically, configuring LDAP is almost that. I think you can well accomplish what you want by following the instructions on how to enable ldap in the first place. I'm not clear, however, if you want to have some automatic fallback ie. no select box on the log in page that allows the user to select ldap or internal (you're free to name them as you like, so it can be a lot less technical like email address @ ... and foreighn users). I'm not sure how you'd handle this automatic fall back but if you're comfortable with a small select box that is defaulted to LDAP (however you call it) and allows for the internal method all is there. If i have to extend this feature how and where i have to change. access_control_config.py needs to be adopted to allow ldap as another login method. You'll find helpful commentary right in there. external_authentication_ldap.py contains the details for your ldap config. It's modeled after an existing organisation (EPFL), documented in the source itself. EPFL's structure however doesn't entirely match what I found in talking to our ldap server. Theirs is a bit simpler. Some time ago I sent a refurbished version of a howto to Samuele I think it should have made it to the git by now. Otherwise I could send you the file by PM. I found another helpful page was http://www.leccionespracticas.com/cds-invenio/cds-invenio-configuring-ldap-to-login-into-repository I basically followed that one plus the src commentaries with some mild educated guessing. 2. Hooking invenio login mechanism: I think you do not need to do that beyond the above mentioned which is a pretty clean way. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi --- --- Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt --- --- Kennen Sie schon unsere app? http://www.fz-juelich.de/app
Re: Slow MySQL queries with large data structures in memory
On 11.01.2012 14:24, Tibor Simko wrote: Hi! Given that many major Linux _server_ distributions are still on 2.6 (one might argue for a reason), it would be better to solve the issues for 2.6 instead of moving up the requirements, IMHO. Yes, that's the GC on/off workaround we are going to commit in any case. :) The patch may be actually interesting to your site conditions as well, since IIRC you may have around 2M of records or thereabouts? The very big box would even be quite a bit beyond that, yes. But this is currently postponed till we get our current project finshed which is a bit smaller. But you're right, this very thread was read here with quite some interest. Personally, I even do not like base requirements beyond what Debian _stable_ delivers. Yes, it is one of our design goals that Invenio should run out of the box there. One of our build boxes is running Debian/Stable for this very reason. Perfect :) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Kennen Sie schon unsere app? http://www.fz-juelich.de/app
Re: Slow MySQL queries with large data structures in memory
On 05.01.2012 13:54, Tibor Simko wrote: Hi! Reading this thread only now... Interesting findings. I confirm I can see the problem with Python-2.6 but not with Python-2.7. Hence the first option: - What about compiling /opt/python-2.7 on your CentOS servers? This can take care of all `hidden' similar GC-related issues in a natural way. Given that many major Linux _server_ distributions are still on 2.6 (one might argue for a reason), it would be better to solve the issues for 2.6 instead of moving up the requirements, IMHO. I admit that for a productive server I don't really like to compile any of the requirements myself as it involves tracking all sec and bug issues for all those packages in detail by hand and with the compiler as well. To avoid this is surely one of the main reason to run a server distribution and not Debian unstable. Personally, I even do not like base requirements beyond what Debian _stable_ delivers. This is not my personal preference for Debian speaking, but more the observation that one can assume that almost any other decent Linux server distribution will meet these as well. This eases up adoption of a product by new users a lot, ie. geek mode=off Would you start compiling n libs and stuff that come at to low a version with your distribution packages to just check out if a certain product does what you want and is to your liking or wouldn't you just check those products first that are only an aptitue install away? /geek ym2c -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Kennen Sie schon unsere app? http://www.fz-juelich.de/app
Re: Editting Python Scripts
On 14.12.2011 10:22, Guenay, Yigit wrote: Hi! I’ve been trying to supply the web site with the ability to open local links on the server. Some browsers (i.e. Firefox) don’t really permit linking on local files. After having implemented some changes by using JavaScript, I could not still get the last updated version of the page. The original one was still there. Although I disabled caching on the browser, it still caches somehow. I would appreciate your information if you’ve got an idea on this issue. Did you try to restart the apache? Sounds like wsgi is getting in your way and actually caching the code. Just a guess, however, as I don't really understand what your usecase is. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Author handling bfe_authors.py et al
Hi!
Currently, bfe_authors.py uses
authors = []
authors_1 = bfo.fields('100__')
authors_2 = bfo.fields('700__')
This enforces authors to be considered only if both indicators are
blank. However, you may notice that from the authoritative definition at
http://www.loc.gov/marc/bibliographic/bd100.html
One should actually set at least indicator 1 like
100 0_ $aWinston Churchill
100 1_ $aChurchill, Winston
to distingish storage of firstname lastname vs. lastname, firstname,
not to metion stuff like
100 3_ $aFarquhar family
Given the fact that if we upload foreign data we win a lot of indicators
here I suggest to change code for author handling to use
authors = []
authors_1 = bfo.fields('100%%')
authors_2 = bfo.fields('700%%')
wherever it is applicable. This is also relevant for indexing, where the
default definition is __ as well.
Just got some 50.000 anonymous papers which I'll right now give back to
their authors ;)
--
Kind regards,
Alexander Wagner
Subject Specialist
Central Library
52425 Juelich
mail : a.wag...@fz-juelich.de
phone: +49 2461 61-1586
Fax : +49 2461 61-6103
www.fz-juelich.de/zb/DE/zb-fi
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt
Re: Author handling bfe_authors.py et al
On 25.11.2011 11:59, Ferran Jorba wrote: Hi! this is an example of the default values of Invenio not being Marc21 compliant. Right. And then these are bad defaults. I've complained about this a few times, and I should have filled a task about those defaults, and sent a few patches, although I haven't yet ;-(. The reasons why I haven't done it myself, besides the lack-of-time issue (bad excuse) are that on my instances I have a mix of better-than-default values and local ones; Well, it would be great if you could drop me some sort of list in case your previous post was not complete. We're about to roll out some installation here based on recent Invenio so we might work that in if it's not already done. So the suggestion would be: give me what you have and I'll check against current git master (probably some weeks back). [...] But idealy one should be able to go, for example, to http://www.archive.org/details/ol_data and get and load all University of Toronto Library catalog in the local Invenio and use it, maybe just adjusting some valid collection field value. Agree. But I don't need to go to Toronto I'd just start out with out own catalogue. Still, it's more cumbersome to fiddle out everything again you might already have found in your (local) patches. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 www.fz-juelich.de/zb/DE/zb-fi Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: auto-complete shortcut for BibEdit ???
Am 27.06.2011 13:59, schrieb Tibor Simko: Hi! I know your're all die hard EMACsers, but maybe one could come up with something easier to type and remember. Definitely. The ultimate goal is to make a fast enough auto-suggestion facility that it can be triggered fully automatically, say when BibEdit detects a small pause in user typing, the suggestions will pop up automatically by themselves, without having to go via any hot key. Until we have this implemented, we'd have some hot key to help reduce the load; currently it is hooked on both C-S-a and on C-9, but this is just a demo. Sounds reasonable. Probably one should have a threshold for number of characters. (E.g. Journal of Physics if you search already if the user enters j...) SmithC-S-a Smith, John (j.sm...@fz-juelich.de : 01.01.2000--) Smith, John (j.sm...@fz-juelich.de : 01.01.1990--01.01.1998) Smith, John (john.sm...@fz-juelich.de : 01.01.2000--) Ah, we think alike. :) :) BTW: it makes sense at some point to extend dup checking to 024 fields as they accumulate a number of identifiers. Putting them all to 035 isn't good compared with regard to the Marc Standard and repeating them isn't either. Yes, please submit a ticket. Done. We could either introduce 024 as yet another standard field that is to be checked for uniqueness, or we can have a new variable that will list all the supposedly unique MARC tags that BibUpload would check against. (Together with the provenance.) Both sounds reasonable. The second idea with a list might be more flexible but also probably more complex. Especially if you need to check two subfields for provenance. -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 http://www.fz-juelich.de/zb/mitarbeiter/fachinformation#wagner Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt Besuchen Sie uns auf unserem neuen Webauftritt unter www.fz-juelich.de
Re: [task #19001] Web of Science integration with CDS
Am 27.01.2011 19:47, schrieb noreply [Ludmila Marian]: Hi! == OVERVIEW of task #19001: == Request from Jens: Integrate the WoS services to a maximum so that CERN users (and others with access to WoS), can easily access the WoS data (and thus, take advantage of the CERN WoS subscription). Getting the WoS data can be done using the Links Article Match Retrieval Service. What might be done: - We could start by adding a link to the detailed page of each record, and discuss after about integrating this data in the metadata of each record (e.g citation counts - comparing some citation counts between WoS and Inspire showed, in some cases, better numbers in Inspire, but (citing Jens) the Inspire data are not recognized outside of HEP, while WoS is considered as the authoritative source across all disciplines). It seems not wise to integrate WoS Citation counts into the record as this data changes, sometimes freequently. Therefore it should be fetchted upon request. Using UT-code seems the procedure of choice as it restricts the number of requests necessary. It seems advisable to store UT-codes e.g. like 024 7_ $2ISI$0utcode for each record that is covered in WoS and where it is available. Then creation of queries against WoS are pretty simple as you can search for (UT=$0) obtaining one single record only. - We will probably need to put in place a caching mechanism (some data is uploaded only once per week on their side, and the links need to be generated only once at the creation of a record). - We need to test the response time from their side as a first step. Response time from WoS: slow to very slow. Reliability: depends upon load, sometimes frequent timeouts. (Sample: definitely representative with good statistics ;) Based on this, we can know at which step this information should be requested (either at the creation of a record, or offline for the already submitted records, and in this case, keep the info in the database, or maybe the response time is very fast, and in this case, we can do the request online, at the display time). For single records you can fetch the data upon request. You have to handle timeouts there as it isn't the most reliable service either. Therefore, it seems advisable enrich records by such data offline, unless you want to enable a copy from wos feature for record ingest. (For the latter it might be desirable to check out PUMA's bookmarklets which enable record feeding from quite a bunch of sites on the Net (cf. http://puma.uni-kassel.de/) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 http://www.fz-juelich.de/zb/mitarbeiter/fachinformation#wagner Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: Protection of bibdocfiles
Am 22.07.2010 09:11, schrieb Samuele Kaplun: Hi! Il giorno gio, 22/07/2010 alle 08.32 +0200, Alexander Wagner ha scritto: Hm, WHY would those documents live WITHIN the original document? Say I [...] report to the refereed papers MARC data.) thank you very much for the answer. Indeed that sound very reasonable to me. So by simply implement such feature of always allowing authors to access their documents, we would implicitly guide repository manager to create improved workflows, which might be a bit more difficult to implement at the beginning but would pay off in the long term. Sounds like a win-win situation :-) Agree. :) One might even considering workflows that are set up at the repository management but help out users locally in the institutes to get their things done. Imagine e.g. a setup like: - A central repository for all publications of the whole institution - Small areas within that repository, private to individual groups where they can store locally needed publications and their ongoing work. Now, in the latter child repositories so to say one might want to implement a workflow that maps essentially the publication workflow within such a group, from I would want to publish my results till the final submission to the publisher, arXiv, and the main repository. From the idea one could think in the direction of integrating the repository into the institutional workflows. IMHO this would solve some issues repositories face in acceptance these days and allow to further OpenAccess (beyond the HEP-community ;). Also I had quite some users feedback that those are actually functionalities the scientists long for. Some even started to implement this on their own. (But I fear for submission of the data to the publication database they'll have to rework their stuff quite a bit to get decent bibliographic records...) -- Kind regards, Alexander Wagner Subject Specialist Central Library 52425 Juelich mail : a.wag...@fz-juelich.de phone: +49 2461 61-1586 Fax : +49 2461 61-6103 http://www.fz-juelich.de/zb/mitarbeiter/fachinformation#wagner Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
[bug #60507] bibupload fails on valid MarcXML
This is an automated notification sent by LCG Savannah. It relates to: bugs #60507, project CDS Invenio == OVERVIEW of bugs #60507: == URL: http://savannah.cern.ch/bugs/?60507 Summary: bibupload fails on valid MarcXML Project: CDS Invenio Submitted by: arwagner Submitted on: 2009-12-11 12:12 Category: None Severity: 3 - Normal Priority: 5 - Normal Item Group: None Status: None Privacy: Public Assigned to: None Originator Email: Open/Closed: Open Discussion Lock: Any ___ Hi! To upload some items to our current test system I had to do some small adjustments of the incoming data. I did these by a simple XSLT that added some fields and moved others and while I'm at it I also added 980__ with the collection id. Running throuhg the XSLT-processor produced new datafields like this: marc:datafield xmlns:marc=http://www.loc.gov/MARC21/slim; tag=980 ind1= ind2= marc:subfield code=aBibliothekskatalog/marc:subfield /marc:datafield These fields, however, are not recognised by bibupload, that is I ended up with a bunch of records not containing the 980 field. Additionally, all other fiels I reworked/added via XSLT being of the same form were silently lost. For a work arround one could either process the records trough a simple Perl script to remove the explict namespaces or transform them by means of LoC's MarcXML-Toolkit once to mrc and back to MarcXML. Note, however, that LoC considers records with missing 001 as invalid and is not able to process them while on the other end bibupload will not -i records that have a 001 attached. Therefore, the processing twice trough the MarcXML-Toolkit requires the removal of 001 as a last step. cu Alexander Wagner ___ Carbon-Copy List: CC Address | Comment +- 6193| -SUB- == This item URL is: http://savannah.cern.ch/bugs/?60507 ___ Message sent via/by LCG Savannah http://savannah.cern.ch/
